Member-only story
Database Connection Pooling — The 50ms Tax You’re Paying on Every Request
Every request opens a connection. Every connection costs time. You’re paying for both.
We deployed a FastAPI service behind a load balancer with four replicas. Each replica ran four Uvicorn workers. Traffic was moderate — about 300 requests per second total. PostgreSQL was on a separate server with max_connections = 100.
On the second day of production, users started seeing intermittent 500 errors. The logs showed: sqlalchemy.exc.OperationalError: connection to server at "10.0.1.5", port 5432 failed: FATAL: too many connections for role "apiuser".
I checked pg_stat_activity. There were 97 active connections. Four replicas × four workers = sixteen processes, each opening and closing connections for every request. Under sustained load, the connection teardown on one request overlapped with the connection setup on the next. Connections piled up. PostgreSQL refused new ones.
The fix was connection pooling — reusing database connections instead of creating a new one for every single request. It took 15 minutes to configure and solved the problem permanently. But the connection exhaustion was just the obvious symptom. The subtler problem was the 50–70ms of latency we’d been paying on every…
