Approach: Use a token-bucket that tracks current_tokens and last_refill timestamp. On each allow(n) call, compute tokens to add = min(capacity, current + elapsed * refill_rate), then if current_tokens >= n consume and return True else return False. Use monotonic clock and a lock for thread-safety.python
import time
import threading
class TokenBucket:
def __init__(self, capacity: float, refill_rate: float):
"""
capacity: max tokens
refill_rate: tokens per second
"""
self.capacity = float(capacity)
self.refill_rate = float(refill_rate)
self._tokens = float(capacity)
self._last = time.monotonic()
self._lock = threading.Lock()
def _refill(self):
now = time.monotonic()
elapsed = now - self._last
if elapsed <= 0:
return
added = elapsed * self.refill_rate
self._tokens = min(self.capacity, self._tokens + added)
self._last = now
def allow(self, n: float = 1.0) -> bool:
if n <= 0:
return True
with self._lock:
self._refill()
if self._tokens >= n:
self._tokens -= n
return True
return False
Key points:- Uses time.monotonic to avoid clock jumps.- Lock ensures correctness in multithreaded single-process.- Time & space: O(1) per request, O(1) memory.- Edge cases: fractional tokens supported, n > capacity should always return False, handle very high rates (float precision).Distributed deployment changes:- Single-process state isn't shared. Use a central, strongly consistent store (Redis) and perform refill+consume atomically (Lua script) to avoid race conditions. The Lua script would compute elapsed using server time, update tokens, and decrement if available. Consider clock skew if clients compute time—prefer server-side time. For high throughput, consider sharding keys across Redis cluster or use local token buckets as a fast first hop (local token fast-path) with periodic reconciliation to a central global limiter (approximate fairness). Trade-offs: central store gives correctness but adds latency and single-point cost; local caches improve latency but allow temporary bursts and require careful parameter tuning.