Cards/ConcurrencySystem Design · Day 53Sep 24, 2026

One Thread, Ten Thousand Connections

Ten thousand threads holding a megabyte of stack each while asleep, against one event loop asking the kernel which sockets have data ready, and the rule that nothing in a handler may ever block

The old model gave every connection its own thread. Ten thousand users meant ten thousand threads, each holding about a megabyte of stack, each mostly asleep, waiting on a network read that hadn't arrived. The machine spent its memory on waiting and its CPU on switching between waits.

The event loop asks a different question. One thread asks the kernel which of those ten thousand sockets has something ready right now, handles those, and asks again. The kernel was tracking every socket anyway — we stopped guessing and started asking.

Idle connections then cost almost nothing, which is how Node, nginx and Redis hold connection counts that would flatten a thread-per-request server.

The price is a rule with no exceptions. Nothing in a handler may block. One long calculation, one synchronous file read, and all ten thousand connections wait together.

Enormous efficiency is often available — to whoever is willing to give up the freedom to be sloppy.

Enormous efficiency is often available — to whoever is willing to give up the freedom to be sloppy.