C# LINQ Chunk — Complete Guide

In this tutorial, you'll learn about C# LINQ Chunk. We cover key concepts, practical examples, and best practices.

You need to process a large list in batches of 100 — sending items to an API that accepts at most 100 per request. You write a loop with Skip and Take, or manually slice arrays. The LINQ Chunk method (C# 9+) splits a sequence into fixed-size arrays.

Wrong

int batchSize = 100;
for (int i = 0; i < items.Count; i += batchSize)
{
    var batch = items.Skip(i).Take(batchSize).ToList();
    ProcessBatch(batch);
}

Output: Works. But Skip/Take on each iteration is O(n^2) — Skip re-iterates from the start each time.

Right

int batchSize = 100;
foreach (var batch in items.Chunk(batchSize))
{
    ProcessBatch(batch); // batch is T[]
}

Output: Same batching, single pass. Chunk produces arrays of at most batchSize elements.

Chunk materializes each batch as a T[]:

var batches = Enumerable.Range(1, 10).Chunk(3).ToList();
// [[1,2,3], [4,5,6], [7,8,9], [10]]
// Last batch may be smaller than the chunk size

Prevention

• Use Chunk for batch processing (API calls, database inserts, parallel processing).
• Use Chunk instead of Skip/Take loops for correctness and performance.
• Use Chunk with Parallel.ForEach for batch-parallel processing.
• Use Chunk on IQueryable carefully — it may materialize the entire query first.
• Use SelectMany with Chunk for inverse (flattening batches).
• Remember the last batch may be smaller than the chunk size — handle that in your processing logic.

Common Mistakes with linq chunk

1. Non-exhaustive pattern matches that compile with warnings then crash at runtime
2. Misunderstanding that String is [Char] with poor performance for large text operations
3. Using foldl instead of foldl' causing stack overflow on large lists

These mistakes appear frequently in real-world CSHARP code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

Chunk is used in DodaTech's data import pipelines for batch-processing records. For more LINQ, visit DodaTech.