Rate Limiting Algorithms, Ranked

The Problem#

Rate limiting is not one algorithm. The right choice depends on whether you need burst tolerance, smooth traffic, strict fairness, or cheap distributed enforcement.

Why It Matters#

Rate limits protect shared systems: databases, APIs, queues, expensive reports, login endpoints, and third-party dependencies. They also create fairness between tenants so one customer or integration cannot consume the whole system.

Public APIs from companies such as GitHub, Stripe, Slack, and many cloud providers document rate limits because customers need predictable boundaries and retry behavior.

Project Example#

For a public API, use token bucket when customers may burst briefly but must stay within a sustained quota. Use sliding window counters for login attempts where fairness matters. Use concurrency limits for expensive operations such as report generation.

Implementation Example#

const allowed = await tokenBucket.consume({
  key: `tenant:${tenantId}:api`,
  capacity: 1_000,
  refillPerMinute: 100,
});

if (!allowed) {
  return new Response("rate limit exceeded", {
    status: 429,
    headers: { "Retry-After": "30" },
  });
}

For distributed systems, implement the counter atomically in Redis or another central store. Local in-memory limits are useful for per-instance protection but not for global tenant fairness.

Implementation Checklist#

• Define the resource being protected: CPU, database, third-party API, or abuse.
• Choose limits per user, API key, tenant, or IP.
• Return 429 with Retry-After.
• Make distributed counters atomic.
• Track allowed, rejected, and near-limit requests.
• Add separate limits for cheap and expensive endpoints.
• Expose remaining quota where API users need it.
• Give support teams a way to inspect current limiter state.

Algorithm Guide#

Token bucket is good for APIs that allow bursts. Leaky bucket smooths traffic. Sliding window is easier to reason about for abuse prevention. Concurrency limits are best when the protected resource is "work in flight", not request count.

Common Mistakes#

• Using IP-only limits for authenticated SaaS APIs.
• Applying one global limit to all endpoints.
• Forgetting internal clients and background jobs.
• Hiding rate-limit state from support teams.
• Rate-limiting after expensive work has already happened.
• Returning 500 instead of clear 429 behavior.

Summary#

Rate limiting is capacity and fairness policy. Pick the algorithm based on the failure mode you are preventing.