Rate limiting is not simply “allow 100 requests per minute.” The algorithm controls fairness, burst behavior, storage cost, and what happens when several application instances see the same client. A good design also tells the client how long to wait instead of turning overload into confusion.

Fixed windows are simple but uneven

The fixed-window algorithm counts requests in a period such as 60 seconds. At the boundary, a client can send 100 requests at 12:00:59 and another 100 at 12:01:00. The configured limit is 100 per minute, but the client produced 200 requests in almost no time.

This approach is still useful for a basic public endpoint because it is easy to explain and inexpensive to store. Add a Retry-After response and choose the boundary carefully. Do not use it for sensitive operations where a boundary burst can cause real damage.

Sliding windows improve fairness

A sliding log records request timestamps and removes entries older than the active interval. It produces fairer decisions, but storing one timestamp per request can be expensive for a busy API. A sliding counter reduces memory by combining smaller buckets, with a small accuracy tradeoff.

If a route receives 16.9k requests per second, a naive timestamp list can become a serious memory problem. In a rate-limiter test, I measured 16.9k requests per second before comparing memory growth and Redis command cost. Measure the storage operations and key cardinality before choosing a precise algorithm.

Token buckets allow controlled bursts

A token bucket refills at a steady rate and holds a maximum number of tokens. Each request spends one token. A full bucket permits a short burst, while the refill rate controls long-term throughput.

This is often a useful model for APIs that should tolerate a short page-load burst but still protect a backend dependency. Store the token count and last-refill time atomically. In a distributed system, a read followed by a write can allow two instances to approve the same token.

Leaky buckets smooth work

A leaky bucket processes requests at a controlled rate. Extra requests wait in a queue or are rejected when the queue is full. This is useful when smoothing work matters more than immediate responses, such as sending notifications or calling a provider with a strict quota.

Do not put an unbounded queue behind a rate limiter. A queue that accepts everything may protect the upstream provider while exhausting your own memory. Give it a maximum size, a deadline, and a metric for rejected work.

Decide what the key means

The key might be an IP address, user ID, API key, route, tenant, or provider identity. IP-only limits can punish users behind one network. User-only limits are weak before authentication. A strong policy often combines a broad IP limit, an authenticated identity limit, and a stricter limit for expensive operations.

Return headers that explain the decision. At minimum, tell clients the limit, remaining allowance, and reset or retry time. Keep the response body stable so SDKs can handle it.

Test failure, not only success

Test two application instances competing for the final token. Test clock differences, Redis timeouts, retries, and a client that ignores Retry-After. A rate limiter should fail closed for a password reset or payment action, but a less sensitive read endpoint may use a bounded fallback.

Final checklist

  • Use fixed windows for simple, low-risk quotas.
  • Use token buckets when controlled bursts are legitimate.
  • Use queues only with hard size and time limits.
  • Make distributed updates atomic.
  • Choose keys that match abuse risk and user identity.
  • Measure allowed, rejected, delayed, and storage-failure requests.

The algorithm is only half the design. The other half is making the limit fair enough for real users and visible enough for operators to tune.

Distributed storage choices

An in-memory limiter is fine for a single process, but it becomes misleading as soon as traffic is spread across instances. Shared Redis state is a common choice because an atomic script can update the counter and return the decision together.

The storage key should have a bounded lifetime. Include the policy version, identity type, and time bucket where appropriate, then set an expiry so abandoned identities do not remain forever. Monitor key count, command latency, rejected requests, and storage errors. A limiter that protects the API but silently fails every Redis request needs an explicit fallback policy.

What clients should receive

Return 429 Too Many Requests when the request is rejected, with a stable error code and a useful retry hint. For sensitive writes, make retries safe with an idempotency key; otherwise a client that obeys Retry-After can still repeat a side effect.

Document which limit applies to which identity. A user-facing limit, an IP limit, and a provider quota are different controls. Naming them in logs and documentation helps support teams explain a rejection without exposing internal abuse-detection details.

Write down the policy next to the route that uses it. Include the window, burst allowance, identity key, failure behavior, and response headers. This prevents a later refactor from changing the algorithm while leaving an outdated limit in the API documentation. It also gives reviewers a concrete checklist when a new expensive endpoint is added.

Review those values with real traffic after launch. A limit that is harmless for a read endpoint may be far too generous for password resets, uploads, or payment attempts.