p99 latency spikes to 800ms. No slow query, no network blip, no code change. The garbage collector ran.
Memory gets freed when the collector decides to look, rather than when we stop using an object — and classically that meant stopping every application thread while it did.
Two ideas took most of the pain out. Generational collection leans on the observation that most objects die young: collect the nursery often and cheaply, promote the survivors, and visit the old region rarely. Concurrent collection does most of its marking while the application runs, leaving only a tiny stop. ZGC gets this down to sub-millisecond pauses on heaps of any size.
None of it is free. Concurrent collectors spend real CPU cycles to avoid stopping, so a latency-tuned service does measurably less work per core.
And the biggest lever sits outside the collector entirely: allocating less.
Automatic memory management didn't remove the cost. It moved it somewhere we don't control.