DataLoader and the N+1 Problem

In this tutorial, you will learn about DataLoader and the N+1 Problem. We cover key concepts, practical examples, and best practices to help you master this topic.

The N+1 problem occurs when a resolver makes one query for the parent and N queries for each child. DataLoader solves this by batching individual requests into a single query and caching results within a request.

What You'll Learn

• The N+1 problem and why it matters
• DataLoader batch functions and scheduling
• Per-request caching Strategy
• Creating and using DataLoader instances
• Composing multiple loaders

Why It Matters

Without DataLoader, a query for 100 books with authors would make 101 database queries (1 for books + 100 for authors). With DataLoader, this becomes 2 queries (1 for books + 1 for authors with batched IDs).

Real-World Use

Facebook developed DataLoader internally to solve N+1 in their Graphql API. Apollo Server documentation recommends DataLoader for every relational field. GitHub uses a similar batching strategy for all list fields.

flowchart TD
    Query[Query: books { title, author { name } }] --> BooksResolver[books resolver]
    BooksResolver --> DB1[SELECT * FROM books]
    DB1 --> Books[100 books returned]
    Books --> AuthorField[author field resolver x100]
    AuthorField --> NoLoader[Without DataLoader: 100 queries]
    AuthorField --> WithLoader[With DataLoader: 1 batch query]
    NoLoader --> Slow[101 total queries]
    WithLoader --> Fast[2 total queries]

Teacher Mindset

DataLoader turns N+1 queries into 1+1 queries. Create loaders per request and add them to context. Each resolver field that loads related data uses the same loader, which batches and caches automatically.

Code Examples

// Example 1: Basic DataLoader setup
const DataLoader = require('dataloader');

const createAuthorLoader = () => {
  return new DataLoader(async (ids) => {
    const authors = await db.authors.findByIds(ids);
    return ids.map(id => authors.find(a => a.id === id));
  });
};

// Example 2: DataLoader in context
const server = new ApolloServer({
  typeDefs,
  resolvers,
  context: () => ({
    loaders: {
      author: createAuthorLoader(),
      book: createBookLoader(),
      review: createReviewLoader()
    }
  })
});

// Example 3: Using DataLoader in resolvers
const resolvers = {
  Book: {
    author: (parent, args, { loaders }) => {
      return loaders.author.load(parent.authorId);
    }
  },
  Author: {
    books: (parent, args, { loaders }) => {
      return loaders.book.loadMany(parent.bookIds);
    }
  }
};

Common Mistakes

• Creating DataLoader instances globally instead of per-request
• Not returning results in the same order as the input IDs
• Returning undefined for missing keys instead of null
• Using loadMany without handling mixed results
• Nesting DataLoader calls synchronously without awaiting

Practice

1. Create a DataLoader for loading books by author ID.
2. Apply the loader to the Book.author resolver field.
3. Create a second loader for reviews and compose it with book loading.
4. Test the N+1 problem by logging query counts with and without loaders.
5. Challenge: Implement a loader that handles Composite keys for a join table.

FAQ

Mini Project

Add DataLoader to your library schema. Create loaders for author, book, and review. Apply them to the author resolver on Book and the books resolver on Author. Measure the query reduction.

What's Next

Next, you will learn about pagination and connections using the Relay Connection specification.