Approach: Use lock striping (segmented cache) to allow high concurrency: split key space into N segments; each segment has its own ConcurrentHashMap and a doubly-linked LRU list protected by a ReentrantLock. This gives near-lock-free reads across segments and bounded lock contention during updates. I'll provide a concise Java implementation and then discuss trade-offs (coarse-grained lock, per-bucket LRU, alternatives) and SRE considerations.java
import java.util.concurrent.*;
import java.util.concurrent.locks.*;
public class SegmentedLRUCache<K,V> {
private final Segment<K,V>[] segments;
private final int segmentMask;
static final class Node<K,V>{
final K key; V value;
Node<K,V> prev, next;
Node(K k, V v){ key=k; value=v; }
}
static final class Segment<K,V>{
final ReentrantLock lock = new ReentrantLock();
final ConcurrentHashMap<K,Node<K,V>> map = new ConcurrentHashMap<>();
final int capacity;
Node<K,V> head, tail; // LRU: tail oldest
Segment(int capacity){ this.capacity = capacity; }
V get(K key){
Node<K,V> n = map.get(key);
if(n==null) return null;
lock.lock();
try { moveToHead(n); return n.value; }
finally { lock.unlock(); }
}
void put(K key, V value){
lock.lock();
try {
Node<K,V> n = map.get(key);
if(n!=null){ n.value=value; moveToHead(n); return; }
n = new Node<>(key,value);
addHead(n);
map.put(key,n);
if(map.size()>capacity) {
Node<K,V> ev = removeTail();
if(ev!=null) map.remove(ev.key);
}
} finally { lock.unlock(); }
}
private void moveToHead(Node<K,V> n){
if(n==head) return;
// unlink
if(n.prev!=null) n.prev.next = n.next;
if(n.next!=null) n.next.prev = n.prev;
if(n==tail) tail = n.prev;
addHead(n);
}
private void addHead(Node<K,V> n){
n.prev = null; n.next = head;
if(head!=null) head.prev = n;
head = n;
if(tail==null) tail = n;
}
private Node<K,V> removeTail(){
if(tail==null) return null;
Node<K,V> r = tail;
if(tail.prev!=null) tail.prev.next = null;
tail = tail.prev;
if(tail==null) head = null;
r.prev = r.next = null;
return r;
}
}
@SuppressWarnings("unchecked")
public SegmentedLRUCache(int totalCapacity, int concurrency){
int segs = 1;
while(segs < concurrency) segs <<=1;
segmentMask = segs -1;
segments = new Segment[segs];
int per = Math.max(1, totalCapacity / segs);
for(int i=0;i<segs;i++) segments[i] = new Segment<>(per);
}
private int idx(Object key){ return (key.hashCode() & Integer.MAX_VALUE) & segmentMask; }
public V get(K key){ return segments[idx(key)].get(key); }
public void put(K key, V value){ segments[idx(key)].put(key,value); }
}
Key points & trade-offs:- Coarse-grained lock: simple but serializes all ops → poor throughput under concurrency.- Lock striping/segmented (above): good balance — parallelism across segments, bounded eviction per-segment, predictable contention.- Per-bucket LRU (one LRU per hash bucket): minimizes lock scope further but increases complexity and memory; harder to maintain global capacity and causes uneven eviction.- Alternatives: pure ConcurrentHashMap + background eviction (approx-LRU) for very high throughput; use Caffeine/Guava for production-grade caches.SRE considerations:- Observability: expose metrics (hit/miss, eviction rate, segment contention, latency).- Capacity planning: segmented per-segment capacity can cause hot-key skew; consider adaptive rebalancing or global admission policy.- Failures & backpressure: avoid blocking callers on long lock waits; consider async writes or request coalescing.- Testing: load-test with realistic key distribution, measure contention and GC impact.- Safety: consider TTLs, write-behind persistence, and metrics-based autoscaling.Complexity:- get/put average O(1). Space O(N). Segment count trades latency vs concurrency.