Rate Limiting, Throttling and Quota Management Questions

For an in-memory limiter, I’d choose token bucket first because it handles bursty security automation well: you can allow short spikes while enforcing a long-term average. Leaky bucket is better when you want a very smooth output rate, but it can reject useful bursts.

import time
import threading

class TokenBucket:
    def __init__(self, capacity, refill_per_sec):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_per_sec = refill_per_sec
        self.updated = time.monotonic()
        self.lock = threading.Lock()

    def allow(self, cost=1):
        with self.lock:
            now = time.monotonic()
            elapsed = now - self.updated
            self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_per_sec)
            self.updated = now

            if self.tokens >= cost:
                self.tokens -= cost
                return {"allow": True, "remaining": self.tokens}
            return {"allow": False, "remaining": self.tokens}

Concurrency: I’d protect state with a lock because multiple threads may check the same API key. If throughput becomes a bottleneck, I’d shard buckets by key or use per-key locks.

Why token bucket here

• Good for security automation jobs that run in bursts
• Easy to reason about remaining capacity
• Supports different weights per request if some actions are more expensive

When leaky bucket wins

• You need strict output smoothing
• You want to decouple request arrival from processing rate

For API keys in a defensive service, token bucket is usually the better operational fit.