The service is slow. Someone raises the thread pool from 50 to 500. It gets worse.
Threads aren't workers. They're claims on a CPU that only has so many cores. Beyond that, they don't run in parallel — they take turns, and every turn costs a context switch: registers saved, cache lines evicted, the next thread starting cold.
At 500 threads on 8 cores, the machine spends its time changing its mind.
Two different jobs, two different answers:
| Workload | Right number of threads |
|---|---|
| CPU-bound | ≈ cores. More just adds switching |
| I/O-bound | ≈ cores × (1 + wait/compute). They're asleep, not competing |
And the real ceiling is usually downstream. 500 threads hitting a 100-connection database pool means 400 threads queuing — you moved the queue, you didn't remove it.