Use a doubly-linked list for recency and a dict for O(1) lookup. The DLL stores nodes (key, val); head = most-recent, tail = least-recent. get moves node to head; put inserts or updates and evicts tail when capacity exceeded.python
class Node:
def __init__(self, k, v):
self.k, self.v = k, v
self.prev = self.next = None
class LRUCache:
def __init__(self, capacity):
self.cap = capacity
self.map = {} # key -> Node
# dummy head/tail to simplify ops
self.head = Node(None, None)
self.tail = Node(None, None)
self.head.next = self.tail
self.tail.prev = self.head
def _remove(self, node):
p, n = node.prev, node.next
p.next, n.prev = n, p
def _add_to_head(self, node):
node.next = self.head.next
node.prev = self.head
self.head.next.prev = node
self.head.next = node
def get(self, key):
node = self.map.get(key)
if not node:
return -1
self._remove(node)
self._add_to_head(node)
return node.v
def put(self, key, value):
node = self.map.get(key)
if node:
node.v = value
self._remove(node)
self._add_to_head(node)
return
if len(self.map) >= self.cap:
# evict LRU (tail.prev)
lru = self.tail.prev
self._remove(lru)
del self.map[lru.k]
newn = Node(key, value)
self.map[key] = newn
self._add_to_head(newn)
Key points:- O(1) get/put: dict lookup + constant-time list ops.- Space O(capacity).Edge cases:- capacity <= 0 (should raise or treat as zero-capacity).- put existing key updates value and recency.- large keys/values (memory limits).Testing:- Unit tests: sequence of gets/puts verifying eviction order, updates, capacity boundary.- Property tests: randomized ops against a correct but slower model (OrderedDict or deque+dict).- Stress tests: high-throughput concurrent reads/writes (see below), memory/leak tests.Making thread-safe for a model-serving process:- For simple correctness: wrap get/put with a single reentrant lock (threading.RLock) to serialize ops—easy and safe.- For higher throughput: use reader-writer lock (many concurrent reads, exclusive writes) or shard the cache (multiple LRU partitions by key-hash) to reduce contention.- Be careful with eviction callbacks and interaction with model objects (avoid holding locks during expensive cleanup).- In async environments, use asyncio.Lock or an asyncio-friendly implementation.- Monitor contention (lock hold time) and add metrics; prefer sharding for heavy write workloads common in feature caches.This implementation is production-appropriate for an ML model-serving side cache; add TTL, size-based eviction, and metrics as needed.