Approach: implement a small client that uses the consensus store's compare-and-swap and TTL/lease primitives (etcd leases). Acquire creates a lease, writes a key with owner ID attached using a transaction that succeeds only if key absent or owned by expired lease. Release revokes the lease atomically. Renew keeps the lease alive.python
# pseudo-python
class DistLockClient:
def __init__(self, etcd, client_id):
self.etcd = etcd # consensus client with txn, lease, keepalive, revoke
self.client_id = client_id
self.leases = {} # key -> lease_id
def acquire(self, key, ttl_sec):
lease = self.etcd.lease(ttl_sec)
value = {"owner": self.client_id, "lease": lease.id}
# txn: only put if key missing or if existing lease is expired (etcd treats lease expiry by key removal)
success = self.etcd.txn(
compare=[("version", key, "=", 0)],
success=[("put", key, serialize(value), lease.id)],
failure=[]
)
if success:
self.leases[key] = lease
return True
else:
lease.revoke()
return False
def release(self, key):
lease = self.leases.pop(key, None)
if lease:
lease.revoke() # revoking removes attached key(s) atomically
return True
return False
def renew(self, key, ttl_sec):
lease = self.leases.get(key)
if not lease:
raise KeyError("not owner")
# create new lease and transfer via txn: verify current value owner==client_id then put with new lease
new_lease = self.etcd.lease(ttl_sec)
def cmp_owner():
val = self.etcd.get(key)
return val and deserialize(val).get("owner")==self.client_id
if cmp_owner():
self.etcd.put(key, serialize({"owner":self.client_id,"lease":new_lease.id}), lease_id=new_lease.id)
lease.revoke()
self.leases[key] = new_lease
return True
else:
new_lease.revoke()
raise Exception("lost ownership")
Key concepts:- Use consensus store leases so expiry is authoritative server-side.- All ownership checks must be server-side transactions (compare-and-swap); never rely on client clocks.- Keep local mapping of lease IDs to manage renew/revoke.Failure modes:- Client crash: lease will expire; other clients can acquire after TTL. Recommend short TTLs + automatic renewals for long-held locks.- Network partition: If client is partitioned from store, it may think it still holds lock—must stop using resource if keepalives to store fail. Client should treat lost connectivity as lost ownership.- Lease expiry: expiry is authoritative; treat missing key as available. Beware of operations after expiry—ensure operations include server-side owner check when acting.- Clock skew: avoid client-side time for correctness; use server leases/ttl. For logging/observability, record both server lease expiry and local timestamp.Safe usage patterns to document:- Use short TTLs (e.g., few seconds) with background keepalive threads; surface keepalive failures to callers.- On acquire success, perform critical section actions immediately and quickly; design idempotent operations or check ownership server-side before mutating shared state.- On network failure or keepalive loss, stop using the resource and attempt to reacquire.- Prefer leader-election patterns for single-writer workflows instead of long exclusive locks.- Instrument lease lifecycle (acquire, renew, revoke, expiry) and expose metrics/alerts for keepalive failures.Trade-offs:- Short TTLs reduce risk of stuck locks but increase churn/renew traffic.- Relying on server-side leases simplifies correctness at cost of needing consensus availability.