Java Threads

Threads or virtual threads?

A thread is a line of execution: a piece of code running with its own call stack. Having many threads is Java’s classic way of dealing with many things at once, which is concurrency, not necessarily parallelism.

Since Java 21 there are two kinds: the platform thread, the thread we always had, and the virtual thread. Both use the same Thread class and the same API; the difference is who manages them and how much they cost.

The traditional model: platform threads

• 1:1 mapping: each Java Thread maps to an operating system thread, and the OS decides when it runs.
• High cost: each one reserves a fixed stack (1 MB by default on Linux x64, adjustable with -Xss), and creating or switching threads (context switch) goes through the kernel.
• Scaling limit: in practice, a few thousand per process. That is why we have always used pools with a fixed number of threads.

ExecutorService pool = Executors.newFixedThreadPool(200);
pool.submit(() -> handle(request)); // at most 200 tasks at the same time

The blocking I/O problem

In classic Java, when a thread makes a network or database call, it sits idle waiting for the response, but it keeps holding the OS thread and its memory.

Little’s law shows why this limits the server: concurrent requests = arrival rate × response time. If 10,000 requests arrive per second and each spends 0.5 s waiting for the database, there are 5,000 requests in flight, that is, 5,000 busy threads, almost all of them just waiting. With a pool of 200, the rest goes into a queue; with 5,000 platform threads, memory and the OS become the bottleneck.

The old way out was asynchronous code (CompletableFuture, reactive frameworks): it scales well, but it is hard to read, to debug and to follow in a stack trace.

Virtual threads (Java 21+)

Virtual threads are threads managed by the JVM, not by the OS.

• M:N mapping: thousands (or millions) of virtual threads run on top of a few platform threads, called carrier threads. By default, one per CPU core, in a JVM-owned ForkJoinPool.
• Lightweight: a virtual thread’s stack is an object on the heap that grows and shrinks as needed. Creating 1 million of them is feasible.
• Unblocking: when a virtual thread blocks on I/O, the JVM unmounts it from the carrier, which is free to run another virtual thread. When the response arrives, it is mounted again, on any free carrier, and continues where it left off.

What happens when a virtual thread waits for the database:

Creating virtual threads

// a single thread
Thread.startVirtualThread(() -> handle(request));
Thread.ofVirtual().name("worker").start(() -> handle(request));

// one virtual thread per task (adapted from JEP 444)
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 10_000).forEach(i -> executor.submit(() -> {
        Thread.sleep(Duration.ofSeconds(1));
        return i;
    }));
} // close() waits for all of them to finish

The 10,000 tasks finish in a little over 1 second. With a fixed pool of 200 platform threads, they would take about 50.

What changed in practice

Simplicity: you go back to writing synchronous code, easy to read and debug, with the scalability that used to require asynchronous code.

Dropping the pool: we used to cap the number of threads so the server wouldn’t fall over. Virtual threads are cheap, so the recommendation is one thread per task, discarded at the end. Don’t pool virtual threads.

Limit the resource, not the thread: if the database can handle 50 connections, the limit comes from a Semaphore (or the connection pool), not from the number of threads.

private static final Semaphore DB_PERMITS = new Semaphore(50);

String query(String sql) throws InterruptedException {
    DB_PERMITS.acquire();
    try {
        return database.execute(sql);
    } finally {
        DB_PERMITS.release();
    }
}

In the frameworks

Almost nobody creates threads by hand in a web application: the framework decides which thread each request runs on. That is why adopting virtual threads is usually a setting, not a rewrite.

Spring Boot

One property switches to virtual threads the Tomcat and Jetty request processing, the @Async executor, the @Scheduled scheduler and the messaging listeners (Kafka, RabbitMQ):

spring.threads.virtual.enabled=true
# virtual threads are daemon threads: without this, an app with only @Scheduled may shut itself down
spring.main.keep-alive=true

The controller stays the same. Two things to watch: with virtual threads, pool size properties (spring.task.execution.pool.*, server.tomcat.threads.max) no longer have any effect, so the limits move to the database and connections; and the documentation itself recommends Java 24+, because of pinning (see below).

Quarkus

Quarkus was born reactive: by default, code runs on the Vert.x event loop or on a worker pool. Virtual threads are opt-in per endpoint, and the three models coexist in the same application:

@Path("/orders")
public class OrderResource {

    @GET
    @RunOnVirtualThread // this method runs on a new virtual thread on every call
    public List<Order> list() {
        return repository.listAll(); // blocking, and that's fine
    }
}

The annotation also works on messaging consumers, @Scheduled tasks and gRPC services. The Quarkus documentation warns about libraries that keep objects in a ThreadLocal as a cache (it names Jackson and Netty): with a new thread per call, the cache is rebuilt every time and memory usage goes up.

Helidon 4

Helidon 4’s web server was rewritten on top of virtual threads: each request already runs on its own, with nothing to configure. It is the example of a framework designed for this model from the start.

Pitfalls

• They are not faster threads. They increase throughput (more tasks at the same time), not the speed of each one. For CPU work (calculations, compression) there is no gain: use platform threads or a pool sized to the number of cores.
• Pinning: in some cases the virtual thread cannot unmount and “pins” the carrier while it waits.
  • In Java 21 to 23, this happened when blocking inside a synchronized block. The recommendation was to switch to ReentrantLock.
  • In Java 24+ (JEP 491), synchronized no longer pins. It still pins when blocking happens inside native code (JNI or the Foreign Function API) that called back into Java.
  • To investigate: the jdk.VirtualThreadPinned event in Java Flight Recorder.
• Heavy ThreadLocal: with millions of threads, a large ThreadLocal cache becomes millions of copies. Prefer ScopedValue (below) or pass the value explicitly.

Beyond virtual threads

Scoped Values (final in Java 25)

A safer replacement for ThreadLocal: the value is immutable and only exists inside the scope where it was bound.

static final ScopedValue<User> CURRENT_USER = ScopedValue.newInstance();

ScopedValue.where(CURRENT_USER, user).run(() -> handle(request));

// anywhere down the call chain inside handle():
User user = CURRENT_USER.get();

Structured Concurrency (preview)

Treats a group of subtasks as one unit: they start and finish inside a block, and if one fails, the others are cancelled.

// JDK 27: java --enable-preview --source 27 ...
Response handle() throws InterruptedException, ExecutionException {
    try (var scope = StructuredTaskScope.open()) {
        Subtask<User> user = scope.fork(() -> findUser());
        Subtask<Order> order = scope.fork(() -> fetchOrder());
        scope.join(); // waits for both; if one fails, cancels the other
        return new Response(user.get(), order.get());
    }
}

The API still changes with every release: in Java 27 it is in its seventh preview (JEP 533), and tutorials from earlier versions may not compile.

When to use each

Videos

• Project Loom and Virtual Threads: Next Phases (Alan Bateman, Oracle, Devoxx Belgium 2024): best practices, the pinning fix and Structured Concurrency, from someone who works on the project (the video embedded at the top).

References

• JEP 444: Virtual Threads: the specification, with the motivation and usage recommendations.
• JEP 491: Synchronize Virtual Threads without Pinning: the synchronized fix.
• JEP 533: Structured Concurrency (seventh preview): the Java 27 changes (InfoQ).
• Java Structured Concurrency in JDK 27: a complete example, with the repository (Dan Vega).
• Spring Boot: virtual threads and task execution and scheduling: what the property changes and the keep-alive warning.
• Quarkus: virtual thread support: the three execution models and the pitfalls.
• Helidon 4: what’s new: the server built on virtual threads.