Approach: use a token-bucket per user with capacity=100, refill_rate=10 tokens/sec. For multi-process correctness and low memory, store per-user state in Redis and perform refill+consume atomically in a Lua script. Redis is memory-efficient for ~10k small keys and provides atomicity without extra locks.python
import redis
import time
# Redis client (configure connection pool in real deploy)
r = redis.Redis(host='localhost', port=6379, db=0)
# Parameters
CAPACITY = 100.0
RATE = 10.0 # tokens per second
# Lua script: KEYS[1]=user_key, ARGV[1]=now, ARGV[2]=cost, ARGV[3]=capacity, ARGV[4]=rate
TOKEN_BUCKET_LUA = """
local key = KEYS[1]
local now = tonumber(ARGV[1])
local cost = tonumber(ARGV[2])
local capacity = tonumber(ARGV[3])
local rate = tonumber(ARGV[4])
local v = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(v[1]) or capacity
local ts = tonumber(v[2]) or now
-- refill
local elapsed = math.max(0, now - ts)
tokens = math.min(capacity, tokens + elapsed * rate)
if tokens < cost then
-- not allowed; persist current state and return 0
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
-- optional TTL to free memory for inactive users
redis.call('EXPIRE', key, 3600)
return 0
else
tokens = tokens - cost
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', key, 3600)
return 1
end
"""
token_bucket_sha = r.script_load(TOKEN_BUCKET_LUA)
def allow_request(user_id: str, timestamp: float = None) -> bool:
now = timestamp if timestamp is not None else time.time()
key = f"bucket:{user_id}"
allowed = r.evalsha(token_bucket_sha, 1, key, str(now), "1", str(CAPACITY), str(RATE))
return bool(allowed)
Key points:- Atomic refill+consume via Lua prevents race conditions across processes.- Memory: one small Redis hash per active user (fields: tokens, ts). Use EXPIRE (e.g., 1h) to evict inactive users.- Performance: O(1) per request; Redis handles >10k users easily; use connection pooling and sharding if needed.- Edge cases: clock skew — use server time (Redis TIME) if untrusted client timestamps. Prevent negative elapsed. Handle bursts by allowing up to capacity.- Alternatives: in-process limiter (faster, not shared), Redis leaky-bucket or fixed window (simpler but less precise).