Scaling From Zero: How a Simple Website Becomes a Distributed System
How a simple website evolves as more people start using it.
I’ve been learning system design through Alex Xu’s System Design Interview, and Chapter 1 inspired this post. I’ve found it easier to understand concepts like load balancers, caching, and sharding by imagining how a system grows as more people use it. So let’s build one together.
Imagine we’re building a small e-commerce website. For now, it only displays products, lets users browse a catalog, and stores some basic information in a database.
We’ll start simple. Then, whenever we hit a problem, we’ll figure out what to change.
Let’s see how far we can take it.
Step 1: Let’s Start With One Server
Our online store is ready. It has HTML, CSS, some backend logic, and a database. We’ve deployed our application on a single server, and everything works.
Not many people visit our website yet, so we don’t need anything complicated.
But how does someone actually reach our website when they type its domain name into their browser?
When a user enters mysite.com, the browser needs to find the IP address associated with that domain. It does this through DNS (Domain Name System).
Think of DNS as the internet’s phonebook. It translates human-friendly domain names into the addresses computers use to locate services.
Once the browser has the IP address, it connects to the server, typically over HTTPS, and requests the page.
Our application processes the request, retrieves product information from the database if needed, and sends a response back to the browser.
And just like that, our website loads.
Everything is peaceful.
Until someone shares our store on social media.
Suddenly, hundreds of people visit. Then thousands. Our server starts struggling to keep up.
What do we do? We scale.
Step 2: Scaling Up vs. Scaling Out
Our first problem is that a single server has limited resources. As traffic grows, it can run out of CPU, memory, or other resources.
We have two options.
Vertical scaling (scaling up)
We upgrade our existing server with more CPU power, RAM, or faster storage. But there’s a ceiling on how big one machine can get, and it remains a single point of failure.
Horizontal scaling (scaling out)
Instead of upgrading one server, we add more servers to share the workload.
Now, instead of one application server, we have three.
Each server can handle requests, giving us more capacity than before.
This is one of the foundations of distributed systems: multiple computers working together to provide a service.
But there’s a problem.
If users are still sending requests to our original server, how do the new servers help?
We need something to distribute incoming traffic.
Step 3: Enter the Load Balancer
A load balancer receives incoming requests and distributes them across our available servers.
Our architecture now looks like this:
Instead of pointing our domain at one application server, we configure DNS to resolve to the load balancer’s public IP address. The application servers sit in a private network, so users can’t reach them directly from the public internet.
Now imagine Server 2 suddenly crashes.
A load balancer can detect that the server is unhealthy and stop routing requests to it. The remaining healthy servers can continue handling traffic.
However, distributing traffic introduces another question.
What happens if a user’s first request goes to Server 1, but their next request goes to Server 3?
Will Server 3 know who that user is?
Step 4: Making Our Application Stateless
Let’s say a customer logs in to our store and adds a product to their shopping cart. We need to remember who they are and what they’ve added.
If Server 1 stores the customer’s session only in its own memory, another request sent to Server 3 might fail because Server 3 doesn’t know about that session.
We could try to keep sending that customer to Server 1 using session affinity (also called sticky sessions, where a user is always sent to the same server). But what happens if Server 1 crashes?
We need a better approach.
This is where stateless architecture helps. In a stateless design, an application server doesn’t keep session information only in its own memory. Instead, session data moves somewhere every server can reach, like a shared data store (Redis, for example) or a signed authentication token. Shopping cart data can live in a shared database.
Now any healthy server can handle the customer’s next request, as long as it can reach the shared data. If Server 1 fails, Server 2 or Server 3 picks up where it left off.
One important point: stateless doesn’t mean our application has no data. It means a user’s session isn’t tied to one particular server. We’ve made our application easier to scale.
But our store is getting busier, and customers keep requesting the same product listings. Every one of those requests can still hit the database.
Can we avoid repeating that work?
Step 5: Let’s Add a Cache
Picture a popular product category with hundreds of items. Thousands of customers browse it every minute. The product information hasn’t changed, but our application keeps asking the database for the same data.
That’s unnecessary work.
What if we could temporarily store frequently requested data somewhere faster to access? That’s the idea behind caching.
One tool we can use is Redis, an in-memory data store commonly used for caching.
Here’s what happens when a customer requests a popular product listing:
- The application checks Redis for the cached product data.
- If the data is there, we return it. This is a cache hit.
- If the data isn’t there, we retrieve it from the database. This is a cache miss.
- We store a copy in Redis so future requests can reuse it.
Instead of repeatedly querying the database for the same information, we serve many requests straight from the cache. This reduces database load and improves response times.
But what happens when a product’s price changes?
We don’t want to keep showing customers an outdated price just because an old copy is sitting in the cache.
We need a caching strategy. We might set expiration times, invalidate cached entries when data changes, or refresh the cache periodically.
Not every piece of data should be cached for the same amount of time. Public product descriptions are good candidates, while account balances or checkout prices need more careful handling.
Caching is useful, but it doesn’t remove the need for a reliable database.
Speaking of which, our database is growing too.
Step 6: Our Database Is Getting Bigger
Our store is growing. More customers are browsing products, updating their profiles, and placing orders.
Get Aysha’s stories in your inbox
Join Medium for free to get updates from this writer.
Our application servers are scaling, and caching has reduced some of the database workload. But our database can still become a bottleneck.
Before introducing anything complicated, we need to understand what’s going wrong.
Is our database a single point of failure? Are read requests consuming too many resources? Or have we reached the limits of what one database server can handle?
Let’s tackle these problems one at a time.
6.1 Replication: Keeping Copies of Our Data
Say our database server suddenly crashes. If it holds the only copy of our data, the application can’t reach that data until the database is recovered.
We need copies of our data on other database servers.
This is where replication comes in.
We set up a Master database and one or more Slave databases (also called primary and replicas) that maintain copies of its data.
If a Slave fails, the Master keeps operating. If the Master fails, a Slave can be promoted to take its place, provided we’ve set up failover. How much protection we get depends on our replication and recovery setup.
But keeping copies of our data opens up another possibility.
What if those Slaves could handle some of our database traffic too?
6.2 Separating Reads and Writes
Let’s look at how customers use our store.
Most are browsing product listings. A smaller share are updating information or placing orders.
Right now, our Master handles both reads and writes. As traffic grows, reads can eat up a large share of its resources.
Since we already have Slaves, we can send read requests to them and keep writes on the Master.
For example, when an administrator updates a product description, the application writes the change to the Master. When customers browse a product category, the application reads from a Slave.
This spreads read traffic across multiple database instances and reduces the load on the Master.
There’s a catch, though: replication might not happen instantly. If a customer updates their profile and immediately reads it from a Slave that hasn’t received the change yet, they might see outdated information.
For operations that need the latest data, we can read from the Master or use another consistency strategy.
We’ve improved resilience and given our database more room to handle reads.
But what if the problem isn’t read traffic at all? What if the amount of data itself is too much for one database server?
6.3 Sharding: Splitting the Data
Now our store has millions of customer records, and the number keeps growing. We’ve optimized our queries, added indexes, and spread reads across Slaves. But the Master still has to handle every write and store all of the data.
Adding more Slaves doesn’t fix that. They spread out reads, but they don’t spread out writes or shrink the Master’s storage load.
We need to consider another approach: sharding.
Instead of storing every customer’s record on one database server, we divide the data across several. Shard 1 stores some customers, Shard 2 stores others, and Shard 3 stores the rest. Each shard holds only a portion of the dataset.
For example, we might distribute customer records using the customer ID as the shard key. Our application then needs a way to work out which shard holds a particular customer.
Sharding spreads both storage and workload across servers, which can improve write capacity too. But choosing a good shard key matters. Poor distribution can leave one shard overloaded while the others sit idle.
It also brings new challenges: routing queries, redistributing data as the system grows, and handling operations that span multiple shards.
And remember those Slaves? We can replicate each shard, so a single database failure doesn’t take out a slice of our customers.
Here’s the distinction worth remembering:
- Replication copies data to improve resilience and potentially distribute reads.
- Sharding splits data across multiple database servers to distribute storage and workload.
We can combine both techniques: shard the data, then replicate each shard.
Our database is in much better shape now. But customers aren’t all sitting near our servers.
Step 7: Going Global With a CDN
Say our application is hosted in one region, but customers are browsing our store from Malaysia, Japan, Europe, and the United States.
They all want to view product images.
If every image request travels back to our origin server, we create unnecessary network traffic and add latency.
What if we served those images from locations closer to our customers?
A CDN (Content Delivery Network) is a distributed network of servers that cache and deliver content closer to users.
Picture a customer in Tokyo requesting a product image. If no cached copy exists at a nearby CDN edge location, the CDN fetches the image from our origin server and caches it. When another customer nearby requests the same image, the CDN serves the cached copy instead of going back to the origin.
This reduces repeated requests to our origin server and speeds up loading.
CDNs are especially useful for static assets such as images, CSS, JavaScript, and videos. They can also cache certain public API responses when configured for it.
However, we shouldn’t blindly cache private customer information. Personalized and sensitive content needs careful caching rules.
But what if our entire application runs in one geographic region and that region has an outage?
Step 8: What If an Entire Region Goes Down?
We’ve added application servers, a load balancer, caching, replication, sharding, and a CDN. Our system is much more capable than the one we started with.
But if all our application servers and our Master database depend on the same cloud region, a major regional outage could still take much of our service offline.
For stronger resilience, we can deploy across multiple regions. For example, we might run application servers in two regions and route traffic so the other region can serve users if one becomes unavailable.
Our database needs a replication and recovery strategy that works across regions too. This brings new challenges: cross-region replication lag, consistency, failover, network latency, and recovery objectives.
Multi-region architecture improves disaster recovery and availability, but it costs more and adds complexity.
We don’t need multiple regions simply because our store has become popular. We need them when our availability, latency, or disaster-recovery requirements justify the cost.
Wrapping Up: Building One Piece at a Time
Let’s look back at how our little online store evolved.
- One application server and one database: enough to get started.
- Horizontal scaling and a load balancer: more application capacity and better handling of individual server failures.
- Stateless application design: requests can be handled by any healthy server.
- Caching with Redis: reduce repeated database queries for frequently requested data.
- Database replication: maintain copies of data to improve resilience.
- Separating reads and writes: distribute suitable read requests across database instances.
- Database sharding: distribute data and workload across multiple database servers.
- CDN: serve cacheable content closer to users.
- Multi-region deployment: improve geographical resilience when the requirements justify it.
Every component solves a different problem, and none is automatically necessary for every application.
The goal isn’t to build the biggest architecture possible. It’s to understand the problem, find the bottleneck, and choose the simplest solution that meets our requirements.
If you’re learning system design too, I hope building this imaginary store piece by piece helps you as much as it helped me.
