Executors — Thread Pools, ScheduledExecutorService, invokeAll, and invokeAny

In this tutorial, you will learn about Executors. We cover key concepts, practical examples, and best practices to help you master this topic.

Java's ExecutorService framework decouples task submission from execution, providing thread pools that manage worker threads efficiently. Creating a new thread for every task is expensive — thread startup overhead, context switching, and memory consumption degrade performance. Thread pools reuse a fixed number of threads, improving throughput and resource management.

What You'll Learn

• Thread pool types: fixed, cached, single, scheduled
• ExecutorService: submit, shutdown, awaitTermination
• ScheduledExecutorService: schedule, scheduleAtFixedRate
• invokeAll and invokeAny for batch operations

Why It Matters

The ExecutorService is the standard way to manage threads in production. Spring uses it for async methods, HTTP servers use it for request handling, and batch processors use it for parallel execution.

Real-World Use

Web servers maintain a thread pool of 200 worker threads. Batch jobs use fixed thread pools for parallel processing. Scheduled services run periodic cleanup tasks.

Thread Pool Types

FixedThreadPool

ExecutorService executor = Executors.newFixedThreadPool(4);
// 4 threads, shared work queue

Best for: known number of tasks, predictable load.

CachedThreadPool

ExecutorService executor = Executors.newCachedThreadPool();
// Creates threads as needed, reuses idle threads

Best for: many short-lived tasks, variable load. Threads idle for 60 seconds and are then removed.

SingleThreadExecutor

ExecutorService executor = Executors.newSingleThreadExecutor();
// Single worker thread, tasks execute sequentially

Best for: tasks that must run in order (file writes, database transactions).

WorkStealingPool

ExecutorService executor = Executors.newWorkStealingPool();
// ForkJoinPool-based, work-stealing, uses all processors

Best for: CPU-intensive tasks with recursive decomposition.

Submitting Tasks

ExecutorService executor = Executors.newFixedThreadPool(4);

// Runnable — no return value
executor.execute(() -> System.out.println("Fire and forget"));

// submit() returns Future
Future<String> future = executor.submit(() -> {
    Thread.sleep(1000);
    return "Result";
});

// Future provides get(), isDone(), cancel()
String result = future.get(); // blocks until done

Shutting Down

executor.shutdown(); // no new tasks, completes existing tasks
// OR
executor.shutdownNow(); // attempts to stop running tasks

try {
    boolean terminated = executor.awaitTermination(10, TimeUnit.SECONDS);
    if (!terminated) {
        executor.shutdownNow();
    }
} catch (InterruptedException e) {
    executor.shutdownNow();
    Thread.currentThread().interrupt();
}

invokeAll and invokeAny

invokeAll — Execute all tasks

ExecutorService executor = Executors.newFixedThreadPool(4);

List<Callable<Integer>> tasks = IntStream.range(0, 10)
    .mapToObj(i -> (Callable<Integer>) () -> {
        Thread.sleep(500);
        return i * i;
    })
    .toList();

List<Future<Integer>> futures = executor.invokeAll(tasks);

for (Future<Integer> f : futures) {
    System.out.println(f.get()); // blocks but all are done
}

invokeAny — First successful result

String result = executor.invokeAny(List.of(
    () -> fetchFromCache("key"),
    () -> fetchFromDatabase("key"),
    () -> fetchFromAPI("key")
));
// Returns as soon as one Callable completes successfully

Useful for redundant operations — query multiple data sources and use the first response.

ScheduledExecutorService

ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

// Run once after 1 second
ScheduledFuture<?> future = scheduler.schedule(
    () -> System.out.println("Delayed task"),
    1, TimeUnit.SECONDS
);

// Run repeatedly with fixed rate (every 5 seconds)
scheduler.scheduleAtFixedRate(
    () -> System.out.println("Periodic task"),
    0, 5, TimeUnit.SECONDS
);

// Run repeatedly with fixed delay (5 seconds after previous completes)
scheduler.scheduleWithFixedDelay(
    () -> {
        System.out.println("Task starting");
        Thread.sleep(1000);
        System.out.println("Task done");
    },
    0, 5, TimeUnit.SECONDS
);

scheduleAtFixedRate vs scheduleWithFixedDelay

• Fixed rate — starts next execution at period intervals regardless of execution time
• Fixed delay — waits for completion plus the delay before starting next execution

Custom Thread Pool Factory

Name threads for debugging and monitoring:

ExecutorService executor = Executors.newFixedThreadPool(4, new ThreadFactory() {
    private final AtomicInteger counter = new AtomicInteger(1);

    @Override
    public Thread newThread(Runnable r) {
        Thread t = new Thread(r, "worker-" + counter.getAndIncrement());
        t.setDaemon(false);
        t.setPriority(Thread.NORM_PRIORITY);
        return t;
    }
});

Common Mistakes

1. Not shutting down the executor. Thread pools have non-daemon threads — the JVM will not exit until they are stopped.
2. Using Executors.newCachedThreadPool() for CPU-bound tasks. Unlimited thread creation leads to thread thrashing. Use fixed pool sized to CPU cores.
3. Calling future.get() without timeout. Blocks indefinitely. Use future.get(1, TimeUnit.SECONDS) with timeout.
4. Submitting tasks but never checking for errors. If a task throws an exception, future.get() throws ExecutionException. Unchecked exceptions in execute() go to the uncaught exception handler.
5. Using scheduleAtFixedRate when scheduleWithFixedDelay is appropriate. If tasks take longer than the period, they pile up. Fixed delay ensures no overlap.

Practice Questions

1. What is the difference between newFixedThreadPool(4) and newCachedThreadPool()?
Fixed pool maintains 4 threads regardless of load. Cached pool creates threads on demand and reuses idle ones (60s timeout).

2. What does shutdown() vs shutdownNow() do?
shutdown() prevents new tasks but completes existing ones. shutdownNow() interrupts running tasks and returns a list of waiting tasks.

3. What is the difference between invokeAll and invokeAny?
invokeAll waits for all tasks to complete and returns a list of Futures. invokeAny returns the result of the first successfully completed task.

4. What is the difference between scheduleAtFixedRate and scheduleWithFixedDelay?
Fixed rate starts next execution at fixed intervals regardless of execution time. Fixed delay waits for completion plus the delay.

5. How do you safely shut down an executor?
Call shutdown(), then awaitTermination(timeout), then shutdownNow() if not terminated.

Challenge Question:
Create a batch processor that reads 1000 URLs, fetches each concurrently (max 20 threads), and processes responses. Use ExecutorService.invokeAll with timeout. Handle partial failures — if some URLs fail, continue processing the successful ones. Collect all results into a list.

FAQ

Mini Project

Write a program ExecutorDemo.java that:

1. Compares performance of single-threaded vs fixed pool execution (compute factorial of many numbers)
2. Uses invokeAll to fetch prices from 5 redundant data sources
3. Uses ScheduledExecutorService to run a periodic health check every 10 seconds
4. Demonstrates proper shutdown with awaitTermination
5. Creates a custom thread factory with named threads
6. Handles a task that times out — uses future.get(2, TimeUnit.SECONDS) and cancels
7. Shows the difference between fixed rate and fixed delay scheduling

What's Next

Executors manage threads, but concurrent data structures are needed for safe sharing. Lesson 55 covers concurrent collections — ConcurrentHashMap, CopyOnWriteArrayList, and BlockingQueue implementations for thread-safe data sharing.