Member-only story
Stop using Worker Pools for everything in Go — Goroutines aren’t the problem
Your worker pool might be hiding the bottleneck instead of fixing it
Every Go code review has a moment like this:
“We’re spawning a goroutine per item here. Shouldn’t we put a worker pool in front of it?”
And to be honest, it does sound “responsible”.
Bounded concurrency, predictable memory, no runaway goroutines.
Some of you might argue that, worker pools have become the “safe” answer to anything concurrent at scale. But “safe” and “correct” aren’t the same thing.
In a lot of systems, a worker pool doesn’t fix the bottleneck. It just gives it somewhere to hide.
Goroutines are not ‘threads’
The worker pool habit comes from other ecosystems, where threads cost around a megabyte of stack and the scheduler is the OS.
Spawning 100,000 of them would take your process down. A pool was the only sane option.
A goroutine starts with a stack of a few KB, grows on demand, and is multiplexed by the Go runtime onto a handful of OS threads.
Spawning 100,000 of them is routine. The fear we inherited doesn’t apply the same way.
