Member-only story
Load Balancing in System Design: How Applications Handle Millions of Requests
How Load Balancing Distributes Traffic, Detects Failures, Supports Deployments, and Helps Applications Scale
In my previous article on how the internet works, we followed a request as it travelled across networks, routers, DNS, and servers.
But there is an important detail we did not focus on then.
What happens when the application is no longer running on just one server?
At the beginning, deploying a web application on a single machine feels perfectly reasonable. A user sends a request, the server processes it, talks to the database, and returns a response. For a small application, this setup may work for quite some time.
Then the application becomes popular.
More users start sending requests. CPU usage rises, memory pressure increases, queues begin to form, and response times slowly get worse. Eventually, some requests start timing out.
At that point, the obvious thought is:
Let’s add another server.
That is usually the right direction. But adding servers creates a new problem:
How does each request know which server it should go to?
