Distributed Locks Design
Coordinating exclusive access across many processes is one of the oldest problems in distributed systems. This recipe walks through a fencing-token lock pattern built on a strongly-consistent KV store, suitable for leader election, job dedup, and critical-section serialization at Meridian-grade throughput.
01.Acquire with a Fencing Token
Every successful acquisition returns a monotonically increasing token. Downstream resources reject any write whose token is older than the highest one they have seen. This prevents the classic split-brain problem where a paused holder wakes up and writes after its lease expired.
Tokens MUST be issued by the same authority that owns the lock state. Do not derive them from wall-clock time on the client.
02.Lease, Don't Hold
Every lock has a TTL. Holders renew via heartbeat; if a heartbeat is missed the lock becomes acquirable again. The TTL should be at least 3x the expected heartbeat interval to absorb GC pauses and network blips without spurious failover.
async function acquire(key, ownerId, ttlMs) {
const token = await kv.incr("lock:seq:" + key);
const ok = await kv.setNX(
"lock:" + key,
JSON.stringify({ ownerId, token }),
{ ex: ttlMs }
);
if (!ok) return null;
return { token, renew: () => kv.expire("lock:" + key, ttlMs) };
}03.Release Idempotently
Release MUST verify ownership before deleting. A naive DEL allows a holder whose lease expired to release another node's freshly-acquired lock. Use a Lua script or compare-and-delete transaction keyed by the ownerId you received at acquisition time.
Bonus: emit a structured log line on every acquire, renew, and release event. Locks are notoriously hard to debug without a complete audit trail of state transitions.