Member-only story
Cloudflare Reclaimed 100TB of RAM — Not With New Hardware, With Math
A struct that didn’t need to be 8 bytes and a hash-point count nobody had actually calculated added up to 100 terabytes, globally
Overview
Cloudflare’s internal load balancer, Pingora Backend Router, uses consistent hashing to decide which server handles each cacheable request. It was using far more memory than it needed to, and the reason wasn’t a leak, a bug, or old hardware — it was two “obviously fine” default decisions that nobody had actually sat down and calculated: a data structure with padding nobody noticed, and a number of hash points per server that had been copied forward as a sensible default without anyone working out what the correct number actually was. Fixing both, published by Cloudflare’s own engineering blog, reclaimed about 100TB of RAM globally, on top of a separate 100TB the DNS team had already freed the month before.
What Pingora Backend Router actually does
PBR routes cacheable requests by URL so that each of Cloudflare’s data centers keeps exactly one copy of a given file, and each file has one stable, predictable location. It does this with consistent hashing: both the servers and the cache keys get hashed onto a 32-bit number line, and each key is handled by whichever server sits nearest to…
