Rate Limiting in Backend Systems: From Basic API Protection to Production-Ready Architecture

Suneel Kumar

21 min read

·

Oct 28, 2026

--

1

Listen

Share

How to protect your APIs from abuse, control traffic, reduce infrastructure costs, and build systems that remain reliable under pressure.

Imagine you have just launched a new API. Everything looks good during testing. Requests are fast, the database responds quickly, and your application handles normal traffic without any issues.

Then, one day, traffic suddenly increases.

It might be a marketing campaign that performs better than expected. It might be a client retrying failed requests in a loop. It might be a bot repeatedly hitting your login endpoint. Or it might simply be one customer integrating your API incorrectly.

Within minutes, your database connections are exhausted, response times increase, and legitimate users start experiencing errors.

The frustrating part? Your application might not have a bug at all. It might simply be receiving more requests than it can safely handle.

This is where rate limiting becomes one of the most important building blocks of a reliable backend system.

Rate limiting is not just about blocking users who send too many requests. It is about controlling how your system consumes resources, protecting shared infrastructure, and ensuring that one client’s behavior does not degrade the experience for everyone else.

Let’s explore how it works, how to implement it properly, and what you need to consider before using it in production.

Press enter or click to view image in full size

1. What exactly is rate limiting?

Rate limiting is a technique used to restrict how many requests a client can make within a defined period.

For example, an API might allow:

• 100 requests per minute per authenticated user.
• 10 login attempts per minute per account or client.
• 5 password-reset requests per hour per account.
• 1,000 requests per minute for a particular API key.

When a client exceeds the configured limit, the server can reject additional requests, delay them, or return a response explaining when the client can try again.

A typical HTTP response is:

429 Too Many Requests

Consider a simple example.

Suppose your application allows 100 requests per minute. A user sends 80 requests in the first 30 seconds, then sends another 40 requests before the minute ends.

Under a fixed-window policy, the first 100 requests may be accepted, while the remaining 20 are rejected. The exact result depends on the algorithm and the request timestamps.

The purpose is not necessarily to prevent high traffic. It is to prevent uncontrolled traffic from overwhelming resources or violating an agreed usage policy.

That distinction matters.

A system should still handle legitimate traffic spikes. Rate limiting is one part of a broader strategy that includes capacity planning, caching, queues, autoscaling, timeouts, and backpressure.

2. Why every serious backend eventually needs it

At first, rate limiting can feel like an optional feature. Your API works, your server has enough resources, and your user base is small.

But as usage grows, a few problems become increasingly common.

Server overload

Every request consumes resources: CPU, memory, database connections, network bandwidth, and sometimes expensive third-party API calls.

A single endpoint that performs several database queries can become a bottleneck when called thousands of times per second.

Rate limiting can cap incoming work before it reaches those expensive operations.

Increased infrastructure costs

Imagine an endpoint that generates a report by querying a large dataset. If one client repeatedly requests the same report, your application might process the same expensive operation hundreds of times.

A rate limit can reduce unnecessary computation and help control costs.

It should ideally be combined with caching and request deduplication, since blocking excess traffic alone does not eliminate redundant work within the allowed limit.

Brute-force attacks

Login, OTP verification, password reset, and account recovery endpoints are common targets for automated abuse.

An attacker may repeatedly try credentials, request verification codes, or attempt to enumerate accounts.

Rate limiting adds a layer of protection by restricting the frequency of these actions. However, it should be combined with appropriate authentication controls, monitoring, and other security measures.

Unfair resource consumption

Suppose your SaaS application has 1,000 customers. One customer runs a badly written integration that sends 10,000 requests per minute.

Without sensible limits, that single customer may consume a disproportionate share of your infrastructure.

A good rate-limiting policy protects the platform while giving legitimate customers predictable access.

Downstream service protection

Your API may depend on a payment gateway, an external API, or an internal microservice with limited capacity.

Even if your own server can handle the traffic, your dependencies might not.

Rate limiting can help protect those downstream services from receiving more work than they can safely process.

The key lesson is simple: your API’s capacity is determined not only by your application server, but also by the weakest dependency in the request path.

3. How rate limiting works internally

A typical rate-limiting flow looks like this:

1. A client sends an HTTP request.
2. The request reaches an API gateway or application server.
3. The rate limiter identifies the client or resource being protected.
4. It checks the relevant counter or token state.
5. It decides whether the request should be allowed.
6. If allowed, the counter or token state is updated and processing continues.
7. If rejected, the server returns a rate-limit response, usually HTTP 429.

The interesting part is step three: how do you identify the client?

Possible identifiers include:

• An authenticated user ID.
• An API key or tenant ID.
• An IP address.
• A combination of account and endpoint.
• A global service-level key.

IP-based limiting is useful for anonymous traffic, but it is not sufficient on its own. Many legitimate users can share a public IP address, while attackers can distribute requests across multiple addresses.

For authenticated APIs, user or tenant identity is often more meaningful.

In practice, you may need several limits operating simultaneously. For example, a login request could be subject to an IP limit, an account-specific limit, and a global protection limit.

A request should proceed only if it satisfies all applicable policies.

4. Choosing the right rate-limiting algorithm

There is no universally best algorithm. Each approach makes a different trade-off between simplicity, accuracy, burst handling, memory usage, and implementation complexity.

Fixed window

The fixed-window algorithm divides time into intervals and counts requests within each interval.

For example:

• Limit: 100 requests.
• Window: 60 seconds.
• Counter: reset at the next window boundary.

Once the counter reaches 100, additional requests are rejected until the next window begins.

Advantages

• Easy to understand and implement.
• Low memory overhead.
• Efficient counter operations.
• Good for straightforward API quotas.

Limitation

It can allow traffic bursts at window boundaries.

Imagine a client sends 100 requests at 12:00:59 and another 100 at 12:01:00. Both groups may be accepted even though 200 requests arrive within roughly two seconds.

This behavior is often called the boundary-burst problem.

Fixed windows are useful when simplicity matters more than perfectly smooth traffic.

Sliding-window log

A sliding-window log stores the timestamps of individual requests and removes entries outside the current time window.

To determine whether another request is allowed, the system counts the requests that occurred during the previous 60 seconds.

This provides accurate rolling-window behavior.

However, storing individual timestamps can consume substantial memory when request volumes are high. It also requires more complex state management than a single counter.

It is appropriate when precise rolling limits justify the additional storage and processing overhead.

Sliding-window counter

A sliding-window counter estimates recent traffic using the current window and the previous window.

Instead of storing every request timestamp, it combines counters with a weighting factor based on how far the current time has progressed into the window.

For example, assume:

• Previous-minute count: 80.
• Current-minute count: 30.
• Progress through the current window: 25%.

The estimated rolling count is:

30 + (80 × 0.75) = 90

If the limit is 100, the next request may be allowed.

This approach is more memory-efficient than a full timestamp log and reduces the boundary-burst problem. The result is an estimate rather than an exact rolling-window count.

It is a useful option for many high-traffic APIs.

Token bucket

The token-bucket algorithm models available capacity as a bucket containing tokens.

Each request consumes a token. Tokens are replenished at a configured rate, up to a maximum bucket capacity.

Suppose your configuration is:

• Refill rate: 10 tokens per second.
• Bucket capacity: 20 tokens.
• Request cost: 1 token.

A client can send 20 requests immediately if the bucket is full. After that, it can sustain approximately 10 requests per second as tokens are replenished.

This gives you two useful controls:

• Sustained rate: how quickly tokens are replenished.
• Burst capacity: how much temporary traffic the system can tolerate.

Token buckets are particularly useful for APIs where short bursts are acceptable but sustained excessive traffic must be controlled.

Leaky bucket

The leaky-bucket concept is commonly used to smooth traffic into a more consistent output rate.

Think of a queue that processes requests at a controlled pace. Incoming requests may accumulate temporarily, but if the queue fills up, additional requests must be rejected.

This can be useful for protecting services that cannot handle sudden bursts.

One important distinction: token buckets generally permit bursts up to their capacity, whereas a leaky-bucket traffic-shaping implementation aims to smooth the output rate. Some implementations use the term “leaky bucket” for related policing algorithms, so the exact behavior depends on the design.

Which algorithm should you choose?

For a simple public API, a fixed window might be sufficient. For smoother quotas, a sliding-window counter is worth considering. For burst-tolerant APIs, a token bucket is often a strong choice.

For expensive operations or systems with strict processing limits, a queue combined with backpressure may be more appropriate than simply rejecting traffic.

Do not choose an algorithm because it sounds sophisticated. Choose it because its behavior matches your actual traffic and business requirements.

5. Implementing rate limiting with Node.js, Express, and Redis

Let’s move from theory to code.

For a small Express application, a middleware-based limiter can be enough. But once you run multiple application instances, an in-memory counter becomes problematic.

Consider three Node.js instances behind a load balancer:

• Instance A receives 40 requests.
• Instance B receives 35 requests.
• Instance C receives 30 requests.

If every instance maintains its own independent counter, each might believe the client is still below a limit of 100. Together, however, the instances have processed 105 requests.

This is why distributed systems often use a shared counter store such as Redis.

Redis provides fast access to shared state and supports atomic operations that help coordinate decisions across application instances.

Install the dependencies

For a modern Express application, you can use the maintained rate-limiter-flexible package with its Redis adapter.

npm install express redis rate-limiter-flexible

The following example uses a Redis-backed fixed-window limiter.

import express from "express";
import { createClient } from "redis";
import { RateLimiterRedis } from "rate-limiter-flexible";

const app = express();const redis = createClient({
  url: process.env.REDIS_URL,
});redis.on("error", (error) => {
  console.error("Redis connection error:", error);
});await redis.connect();const apiLimiter = new RateLimiterRedis({
  storeClient: redis,
  keyPrefix: "api_limit",
  points: 100,
  duration: 60,
  blockDuration: 0,
});app.use("/api", async (req, res, next) => {
  // For authenticated APIs, prefer a verified user or tenant ID.
  const key = req.user?.id
    ? `user:${req.user.id}`
    : `ip:${req.ip}`;  try {
    const result = await apiLimiter.consume(key);    res.setHeader("RateLimit-Limit", "100");
    res.setHeader(
      "RateLimit-Remaining",
      String(result.remainingPoints)
    );    next();
  } catch (error) {
    if (error && typeof error.msBeforeNext === "number") {
      const retryAfterSeconds = Math.ceil(
        error.msBeforeNext / 1000
      );      res.setHeader("Retry-After", String(retryAfterSeconds));      return res.status(429).json({
        error: "Too many requests",
        message: "Please try again later.",
        retryAfterSeconds,
      });
    }    // Infrastructure errors need an explicit failure policy.
    console.error("Rate limiter error:", error);    return res.status(503).json({
      error: "Service temporarily unavailable",
    });
  }
});app.get("/api/products", (req, res) => {
  res.json({ products: [] });
});app.listen(3000, () => {
  console.log("API server listening on port 3000");
});

This is an illustrative starting point, not a complete production configuration. In particular, verify the installed package version and its Redis compatibility, and ensure your authentication middleware runs before the limiter if you rely on req.user.

Also configure your Express proxy settings carefully if the application sits behind a load balancer. Blindly trusting forwarded IP headers can allow clients to manipulate their apparent IP address.

What happens when a request arrives?

The limiter identifies the client, checks its current allowance in Redis, and consumes a point if the request is permitted.

When the client exceeds its allowance, the middleware returns HTTP 429 rather than continuing to the route handler.

Because the state is stored centrally, multiple application instances can share the same quota.

That solves one important problem: per-instance counters no longer allow each server to independently grant the full quota.

However, shared state alone does not automatically guarantee perfect global enforcement. You still need to consider the algorithm’s consistency guarantees, the number of Redis operations, network latency, and how limits are scoped.

A few implementation details worth getting right

Authenticate before using user-based keys. A user ID supplied directly in a request body or query parameter is not a trustworthy identity. Use the identity established by your authentication layer.

Set limits according to endpoint cost. Reading a cached product is not equivalent to generating a large report. Consider separate policies for cheap reads, expensive writes, exports, and authentication.

Keep error responses useful. A 429 response should ideally include a reasonable retry instruction. Clients should not have to guess when they can retry.

Handle Redis errors deliberately. The example returns 503 when the limiter cannot be checked. Some systems may choose to allow selected requests during a limiter outage, but that is a conscious availability-versus-protection decision, not a universal rule.

Avoid creating unbounded keys. If attackers can generate arbitrary identifiers, they may force your rate limiter to maintain excessive state. Normalize keys, validate identities, and configure appropriate expiration and memory policies.

6. Why Redis is so useful for distributed rate limiting

Redis is popular for rate limiting because it provides fast shared state, atomic operations, and expiration support.

But the real benefit is not simply speed. It is consistency across application instances.

Imagine your backend is deployed on Kubernetes and scales from two pods to ten pods during a traffic spike. If each pod maintains its own in-memory counter, the effective limit can change as traffic moves between pods or the number of replicas increases.

A shared Redis store allows those instances to coordinate around the same counters.

A simplified architecture looks like this:

Incoming Requests
                         |
                         v
                  Load Balancer
                         |
                         v
                   API Gateway
                         |
                         v
                 Rate Limit Check
                         |
                         v
                   Redis Store
                         |
                 +-------+-------+
                 |               |
              Allowed          Rejected
                 |               |
                 v               v
           Backend Service    HTTP 429
                 |
                 v
             Database

This diagram represents a logical flow. In a real deployment, the gateway and rate limiter might be the same component, and Redis checks may happen inside gateway infrastructure or application middleware.

Should every request go to Redis?

Not necessarily.

A centralized Redis check introduces a network operation into the request path. At high request volumes, that operation can become a meaningful source of latency and load.

Several approaches can help:

• Use atomic Redis scripts or suitable limiter libraries to combine related counter operations.
• Choose a limiter algorithm that does not require storing individual timestamps unnecessarily.
• Avoid applying expensive distributed checks to every endpoint if a cheaper gateway-level policy is sufficient.
• Use local protection to reject obviously excessive traffic before it reaches shared infrastructure.
• Consider approximate or locally allocated quotas when strict global precision is not required.

One advanced technique is to allocate small batches of request tokens to individual application instances. Each instance consumes its local allocation without contacting Redis for every request.

This can reduce Redis traffic substantially. However, it also changes the enforcement model: unused allocations may be stranded, and the system needs a clear policy for replenishment, expiry, and overshoot.

The general principle is that the stricter the global enforcement requirement, the more carefully you must account for coordination costs.

7. Where should rate limiting live: CDN, gateway, or application?

There is no single correct placement. Different layers protect different resources.

CDN and edge layer

An edge layer can reject excessive traffic before it reaches your infrastructure.

This is useful for protecting public endpoints from obvious abuse, reducing unwanted traffic, and enforcing broad request limits.

However, edge-level rules may not have access to your application’s authenticated user or tenant identity. They also cannot necessarily understand the cost of a particular business operation.

API gateway

An API gateway is a natural place for shared policies across many services.

For example, a gateway can enforce quotas per API key, apply route-specific limits, and reject excessive traffic before it reaches backend applications.

This is particularly useful when several microservices share a common entry point.

But a gateway may not know whether a request will trigger a simple database lookup or an expensive multi-stage report generation process. Application-level controls can handle that distinction.

Application middleware

Application middleware has access to authenticated identities, route information, and sometimes business context.

It can enforce rules such as:

• A maximum number of report exports per tenant.
• A limit on OTP requests for an account.
• Different quotas for free and paid subscription tiers.
• A maximum number of expensive operations running concurrently.

The trade-off is that requests have already consumed some application resources by the time the middleware runs.

A layered strategy

For a production system, a combination is often the most effective approach:

LayerPrimary purposeCDN/WAFFilter unwanted traffic and common attacksAPI gatewayEnforce shared API quotasApplicationApply identity-aware and business-specific limitsQueue/workerControl expensive asynchronous workloadsDatabase/dependencyApply connection, concurrency, and resource safeguards

Not every application needs all five layers. The right architecture depends on the traffic profile, cost of requests, threat model, and operational budget.

8. Rate limiting is not the same as throttling or concurrency limiting

These concepts are related, but they solve different problems.

Rate limiting controls how many operations can occur within a period. For example, 100 requests per minute.

Throttling is sometimes used as a broad term for restricting traffic. In some systems, it specifically means slowing requests down or shaping their delivery rather than immediately rejecting them.

Concurrency limiting controls how many operations can be active simultaneously. For example, allowing only 20 report-generation jobs to execute at once.

Why does this distinction matter?

Suppose an API receives 60 requests per minute, which is below its rate limit. However, every request takes 30 seconds to complete.

The API may still accumulate many concurrent operations and exhaust its worker pool or database connection pool.

A rate limit alone does not necessarily prevent that failure.

For expensive operations, consider combining a request-rate limit with:

• A concurrency limit.
• A bounded queue.
• A request timeout.
• A maximum payload size.
• A database connection-pool limit.

Each control protects against a different failure mode.

9. Designing rate limits around actual business requirements

One of the biggest mistakes is applying the same numerical limit to every endpoint.

Consider a SaaS application with these routes:

EndpointExample policyReasonGET /products100 requests/minute/userRelatively inexpensive readsPOST /login10 attempts/minute/IP, plus account-level controlsReduce automated guessingPOST /otp/sendAccount, destination, and IP limitsPrevent OTP abuse and cost spikesPOST /reports/export5 requests/minute/user, plus concurrency controlExpensive operationsPOST /paymentsBusiness-specific limit and idempotencyProtect dependencies and avoid duplicate effects

These are illustrative starting points, not universally safe production values. Real limits should be based on measured capacity, legitimate user behavior, business agreements, and security analysis.

For example, a limit of 10 login attempts per minute per IP may be reasonable for some applications but can inconvenience users behind shared corporate networks.

Likewise, a very strict account-based login limit can become an account-denial mechanism if an attacker deliberately submits failed attempts against another user’s account.

The goal is to control abuse without unnecessarily blocking legitimate users.

Per-user limits versus per-IP limits

IP-based limits are easy to apply to anonymous traffic, but public IP addresses are often shared by households, offices, universities, and mobile carrier networks.

User-based limits are more appropriate when requests can be associated with authenticated identities.

Tenant-based limits are useful in multi-tenant SaaS applications because they help control consumption at the customer-account level.

For robust protection, combine relevant dimensions rather than assuming one identifier solves every problem.

For example:

Request allowed only when:

IP quota is available
AND user quota is available
AND tenant quota is available
AND endpoint quota is available
AND global capacity is available

This is a conceptual policy. The actual implementation should avoid unnecessary serial network checks and define what happens when any one of those checks fails.

Rate limits for paid customers

Commercial APIs may offer different quotas depending on subscription plans.

A free customer might receive 60 requests per minute, while a paid enterprise customer might receive 1,000.

That can work well, but the policy should be transparent. Document the applicable limits, provide useful response headers, and give customers a way to monitor consumption.

Also consider whether the limit applies to each user, each API key, or the entire tenant. These choices produce very different results.

10. What happens when your rate limiter goes down?

This is one of the most important production questions, and it is often overlooked.

Your application depends on Redis to decide whether requests are allowed. Then Redis becomes unavailable.

Should the API continue serving requests?

There are two common strategies.

Fail open

If the limiter cannot be checked, the application allows the request to proceed.

The advantage is availability. A Redis outage does not automatically prevent customers from using the application.

The disadvantage is that rate limits may no longer protect your backend during the outage. An attack or traffic spike can reach expensive services without the usual controls.

This may be acceptable for selected low-risk endpoints, depending on the system.

Fail closed

If the limiter cannot be checked, the application rejects or temporarily defers the request.

The advantage is that the system does not silently bypass a critical protection mechanism.

The disadvantage is that an infrastructure problem can block legitimate traffic, even if the application and database are otherwise healthy.

This may be appropriate for particularly sensitive operations, but the user experience and operational implications must be understood.

A more practical production policy

Instead of making one decision for every endpoint, define the policy according to risk.

For example:

• Public product reads may continue under conservative local protection.
• Login and OTP endpoints may use stricter fallback controls.
• Expensive exports may be rejected or queued when quota enforcement is unavailable.
• Payment operations should retain their independent validation and idempotency protections regardless of rate-limiter availability.

Do not assume that fail-open or fail-closed is universally correct.

The important part is making the choice explicitly, testing it, and monitoring when the fallback is activated.

Also remember that a fallback policy does not remove the need for timeouts, bounded retries, connection management, and recovery procedures.

11. How to test rate limiting properly

A rate limiter can appear to work during manual testing and still fail under real traffic.

A meaningful test plan should cover more than sending 101 requests and checking whether the last one receives HTTP 429.

Test the basic limit

If the policy allows 100 requests per minute, verify that requests within the allowance succeed and excess requests are rejected according to the selected algorithm.

Use a controlled clock or carefully managed test environment when verifying time-window boundaries.

Test boundary bursts

For a fixed-window limiter, send requests immediately before and after a window reset.

Confirm that the behavior matches the documented policy. If the boundary burst is unacceptable, consider a sliding-window approach or token bucket.

Test multiple application instances

Send traffic through a load balancer to several replicas.

Verify that the effective quota is consistent with the intended shared policy rather than being multiplied by the number of instances.

Test identity spoofing

Try supplying a forged user ID or manipulating forwarded IP headers.

Confirm that the limiter relies on trusted authentication and correctly configured proxy information.

Test Redis failure

Simulate connection timeouts, Redis restarts, temporary network failures, and high latency.

Verify that the configured fallback policy behaves as expected and does not create an uncontrolled retry storm.

Test legitimate shared traffic

Simulate many users behind one public IP address.

This helps reveal whether the IP policy blocks legitimate customers who share a network.

Test latency and load

Measure request latency with rate limiting enabled and disabled.

Look at p50, p95, and p99 latency, Redis command volume, application CPU, rejected-request counts, and dependency load.

The goal is to understand the protection’s cost as well as its effectiveness.

12. Observability: know when your limits are helping or hurting

A rate limiter that rejects requests without generating useful telemetry is difficult to operate.

You need enough information to answer three questions:

1. Is the limiter protecting the system?
2. Is it blocking legitimate customers?
3. Is the limiter itself becoming a bottleneck?

Useful metrics include:

• Allowed requests.
• Rejected requests.
• Rejection rate by endpoint.
• Rejections by plan or tenant, where appropriate.
• Redis latency and error rate.
• Rate-limiter fallback activations.
• Backend latency and error rates.
• Database connection-pool saturation.
• Downstream dependency latency.

For example, if HTTP 429 responses increase at the same time that database latency decreases, the limiter may be successfully protecting the backend.

If 429 responses suddenly increase for one enterprise customer, however, you may have a configuration problem or an unexpected change in their workload.

Avoid using raw user IDs, API keys, or arbitrary client-supplied values as unrestricted metric labels. Excessive label cardinality can make monitoring expensive and unreliable. Keep sensitive identifiers out of logs and metrics unless there is a clear, protected operational need.

Alert on meaningful patterns, not simply on the existence of rejected requests. Rejections may be normal and expected.

13. Common mistakes developers make

Some rate-limiting failures come from the algorithm. Others come from how it is integrated into the application.

Mistake 1: Using in-memory counters across multiple replicas

An in-memory limiter can be useful for local development or single-process protection. But its state is not automatically shared between application instances.

If you need a shared quota, use a shared store or a gateway that provides the required coordination.

Mistake 2: Trusting the client-provided identity

Never let a user choose an arbitrary identifier that determines their quota.

Otherwise, a client may be able to bypass restrictions by changing a header, query parameter, or body field.

Use verified identities and carefully validated network metadata.

Mistake 3: Setting limits without measuring capacity

A limit of 1,000 requests per second sounds impressive, but it means very little without knowing the cost of each request.

A cached read and a multi-join analytical query have completely different resource profiles.

Measure representative workloads before choosing limits.

Mistake 4: Applying limits only to the public API

Internal services can also experience accidental request storms, retry loops, and runaway jobs.

Internal service-to-service calls may need separate quotas, concurrency controls, and deadlines.

Mistake 5: Ignoring client retry behavior

If clients immediately retry every 429 response, they may make the problem worse.

Return useful retry information and encourage clients to implement exponential backoff with jitter.

Mistake 6: Returning the wrong error

A request rejected because the client exceeded its quota is different from a request rejected because Redis is unavailable.

Use HTTP 429 for rate-limit rejection. Use an appropriate service-error response when the system cannot safely make the decision, and ensure the fallback policy is consistent with the endpoint’s risk.

Mistake 7: Treating rate limiting as complete security

Rate limiting does not replace authentication, authorization, input validation, idempotency, fraud detection, or a web application firewall.

It is one protective layer in a broader system.

14. Rate limiting in a real production architecture

Consider an e-commerce platform with a React frontend, Node.js APIs, Redis, PostgreSQL, and several external services.

A request to view a product might be inexpensive. A request to export a customer’s entire purchase history could be expensive. A payment request may trigger several downstream operations.

A sensible architecture might look like this:

Users / Clients
                           |
                           v
                        CDN / WAF
                           |
                           v
                      API Gateway
                           |
                  Shared API Quotas
                           |
                           v
                    Node.js Services
                           |
               +-----------+-----------+
               |           |           |
               v           v           v
            Redis       PostgreSQL   External APIs
               |
        Shared Rate-Limit State

Expensive asynchronous operations could use an additional queue and worker pool.

In this design:

• The CDN and WAF help filter unwanted traffic.
• The API gateway enforces broad quotas.
• The Node.js application applies business-specific limits.
• Redis coordinates shared quota state.
• The queue controls the rate at which expensive work is processed.
• The database and external services retain their own capacity safeguards.

This layered architecture is more resilient than expecting one rate limiter to protect every resource.

A note about Redis Cluster

If your application uses Redis Cluster, key placement and atomicity matter.

Operations that must run together may need to access keys in the same hash slot. Lua scripts and multi-key operations should be designed with the cluster’s constraints in mind.

For example, if a request must atomically check a user limit and a tenant limit, do not assume a multi-key script will work across arbitrary cluster slots. Use an appropriate key strategy, a supported limiter implementation, or a design that explicitly handles the coordination trade-off.

This is the kind of detail that often separates a local prototype from a reliable distributed implementation.

15. Is rate limiting worth the added complexity?

Not every endpoint needs a sophisticated distributed rate limiter.

A small internal tool with a handful of trusted users may be perfectly fine with basic gateway rules or lightweight middleware.

On the other hand, a public API with expensive operations, multiple application replicas, external dependencies, and strict customer quotas has a stronger case for shared rate limiting.

A useful decision framework is:

• How expensive is each request?
• What happens if traffic increases tenfold?
• Can one client affect other customers?
• Do multiple instances need to enforce the same quota?
• How precise must the limit be?
• What happens if the shared counter store becomes unavailable?
• How much additional latency and operational complexity can the system tolerate?

The answers will tell you whether you need a simple fixed window, a distributed token bucket, gateway-based enforcement, or a more comprehensive traffic-control system.

The right solution is the simplest design that meets your reliability, security, and business requirements.

Conclusion: Rate limiting is about protecting the whole system

Rate limiting is easy to underestimate because the basic implementation can fit into a few lines of code.

But production rate limiting involves much more than counting requests.

It requires choosing an algorithm, identifying clients correctly, coordinating state across instances, accounting for expensive dependencies, defining failure behavior, measuring the results, and adjusting policies as traffic changes.

Redis can make distributed enforcement practical, but it also introduces a dependency that needs its own capacity planning and failure strategy.

API gateways can provide broad protection, but application-level controls may still be necessary for business-specific operations. And even a well-designed rate limiter cannot replace concurrency limits, queues, timeouts, or good client retry behavior.

If you are building a backend today, start with the endpoints that can hurt you the most: authentication, expensive queries, report generation, third-party integrations, and any operation that consumes a scarce resource.

Define what healthy traffic looks like. Measure the cost of processing it. Choose a limit that reflects those measurements. Then test what happens when traffic exceeds the limit — and when the rate limiter itself fails.

The goal is not to reject as many requests as possible. The goal is to keep the system useful, predictable, and fair, even when traffic is no longer behaving as expected.

That is what makes rate limiting a reliability feature rather than just another piece of middleware.

What approach do you use for distributed rate limiting — API gateway policies, Redis-backed middleware, token buckets, or something else? The interesting engineering lessons often come from the edge cases, especially when multiple services and tenants share the same infrastructure.