Data Consistency and Distributed Transactions Questions

Direct answer: Strong consistency means every read sees the most recent write, as if there were only one copy of the data; eventual consistency means replicas are allowed to temporarily disagree after a write, but will converge to the same value once updates stop arriving, with no guarantee about HOW LONG that convergence takes. Eventual consistency is acceptable wherever a brief window of staleness is harmless to the user or business; it's not acceptable wherever that staleness could cause a real, uncorrectable mistake.

Structured elaboration

Strong consistency. Every client, everywhere, sees the same, latest value immediately after a write commits, achieved by coordinating reads and writes (through a single leader, quorum reads/writes, or consensus), at the cost of added latency and reduced availability during network partitions (a replica that can't confirm it has the latest state must refuse to serve a strongly-consistent read rather than risk returning stale data).

Eventual consistency. A write is accepted quickly by one replica and propagates to others asynchronously; a read immediately after the write, served by a DIFFERENT replica, might return the OLD value until propagation catches up. The system guarantees convergence (given enough time with no new writes, all replicas agree), but makes no promise about how quickly, "eventual" is doing real work in that name, it's not a synonym for "soon."

Where eventual consistency is acceptable. Social media like/view counts (a brief delay in a count updating is invisible to the user experience), product catalog listings for browsing (a few seconds of staleness on a description or a non-critical attribute doesn't cause real harm), most analytics dashboards (a report a few minutes stale is still useful and expected to have some lag), and DNS (propagation delay is an accepted, well-understood part of how DNS already works).

Where it is not acceptable. Account balance displayed right before a withdrawal (showing a stale, too-high balance could let a user attempt to overdraw), inventory counts at the exact moment of checkout (stale "in stock" data causes oversold orders), and any operation with a real, hard-to-reverse consequence triggered directly off the read (an authorization check reading a stale "still has access" flag after access was just revoked).

Mitigation techniques for the anomalies eventual consistency introduces. Session guarantees (read-your-writes, monotonic reads) so at least the ACTING user's own experience doesn't feel inconsistent, even while the system is eventually consistent for other observers. Bounded staleness (an explicit SLA on how stale a read can be, e.g. "within 5 seconds," with monitoring to catch violations, converts "eventual, unbounded" into "eventual, but with a known worst case" the product can design around). Read-repair and anti-entropy (background processes that detect and correct divergent replicas even without a triggering read, shrinking the practical staleness window over time). UI-level staleness indicators (surfacing "last updated Xs ago" rather than presenting stale data as if it were current, letting the user calibrate their own trust in what they're seeing). Lost updates are a genuinely different anomaly from staleness (a concurrent write is silently overwritten and its information discarded, rather than just delayed) and need their own, separate mitigations: conditional/optimistic writes (compare-and-swap against an expected prior version, rejecting a write that would silently clobber a change it never saw, rather than blindly overwriting) turn a silent loss into a detectable, retryable conflict; version vectors let the system distinguish a genuine overwrite (one write causally after the other, safe to discard the older one) from a real concurrent conflict that needs resolving rather than one write silently discarding the other; and, where every concurrent write's information genuinely needs to survive, a CRDT or a Dynamo-style multi-value register (returning all conflicting "sibling" values to the application instead of picking one) avoids losing any of them silently in the first place.

Worked example. An e-commerce catalog shows "In Stock" for an item that was actually just sold out in another region's replica 2 seconds ago; a customer places an order that later needs to be cancelled and refunded, an annoying but recoverable outcome, acceptable trade for the throughput and availability eventual consistency buys at checkout-browsing scale. Contrast with the SAME system's actual checkout-confirmation step, where the business has decided inventory MUST be strongly consistent (a real reservation, checked synchronously) specifically because an oversell at that exact moment is the harder-to-recover-from failure the earlier browsing-page staleness never risked.

Trade-offs and pitfalls. The most common mistake is applying ONE consistency model uniformly across an entire product, rather than making this decision per-feature based on the actual cost of staleness for that specific read, as the worked example shows, the SAME system can reasonably use eventual consistency for browsing and strong consistency for the actual transaction, and conflating the two into one blanket policy either sacrifices throughput unnecessarily or accepts real risk where it shouldn't.

Direct answer: Choreography fits a small number of loosely-coupled services where each step's failure handling is simple, since there's no central process to build or operate; orchestration fits workflows with more than a handful of steps, complex or conditional failure handling, or a need for centralized visibility into where every saga instance currently stands.

Structured elaboration

Decision matrix

When to recommend choreography. A 2-3 service flow with straightforward, largely linear failure handling (if step 2 fails, always compensate step 1, no branching), where teams value not having a shared piece of orchestration infrastructure everyone depends on, and where the participating services are otherwise fully independent (no team already owns cross-service workflow logic).

When to recommend orchestration. More than about 4-5 services, conditional or branching compensation logic (different failure handling depending on WHICH step failed or WHY), a compliance or operational need to see "where is this specific transaction right now" without correlating distributed logs, or a workflow that changes often enough that touching every participating service's event contract on each change would be a real cost.

Worked example. A 2-service "reserve inventory then send confirmation email" flow is a reasonable choreography candidate: Inventory service reserves and publishes Reserved, Notification service (subscribed to Reserved) sends the email; if reservation fails, Inventory publishes ReservationFailed and nothing else needs to react. A 6-service order-fulfillment flow (Order, Inventory, Payment, Fraud-check, Shipping, Notification) with different compensation logic depending on which step failed (a fraud rejection needs different handling than a payment decline) is a stronger orchestration candidate: expressing that branching logic as a web of event subscriptions across 6 services becomes hard to reason about and debug.

Trade-offs and pitfalls. Teams sometimes start with choreography for its lower initial coupling and later regret it as the workflow grows past 3-4 services and failure handling gets more conditional, at which point migrating to orchestration means introducing a new piece of shared infrastructure AND unwinding event contracts multiple teams already depend on. It's usually cheaper to start orchestrated for anything expected to grow past a handful of steps, even though it means one more service to build up front.

Direct answer: Last-write-wins (LWW) resolves a conflict between two concurrent updates by simply keeping whichever one has the later timestamp and discarding the other, no merging, no user involvement, just pick one. It's acceptable when losing the discarded update silently is a genuinely low-stakes outcome; it's dangerous when the discarded update carried information that mattered and shouldn't have been thrown away.

Structured elaboration

Mechanism. Every write carries a timestamp (or a similar monotonically-comparable value). When two replicas' values for the same key disagree, whichever has the LATER timestamp is kept as the resolved value; the other is discarded entirely, with no trace and no attempt to combine the two.

Where LWW is acceptable. A user's "last active device" field, where the whole point IS to reflect the most recent update and there's no meaningful sense in which an older value should be preserved or merged. A cache of a frequently-refreshed external value (like a cached stock price snapshot), where an older concurrent write being discarded in favor of a newer one is exactly the desired behavior, not data loss. Any field where two concurrent writes represent genuinely REPLACING information rather than two pieces of information that both need to be kept.

Where LWW is dangerous. A shopping cart's item list: if LWW is applied naively to the whole cart object, two devices concurrently adding DIFFERENT items results in one device's additions being silently discarded entirely, real data loss a user would notice and be frustrated by. A counter or accumulator: LWW applied to "total items sold" would drop one of two concurrent increments rather than correctly summing them, an outright correctness bug, not just a UX nuisance. Any field relying on wall-clock timestamps for ordering when clock skew between replicas is a realistic risk, since LWW's correctness depends entirely on timestamps accurately reflecting real recency, a replica with a clock running even a few seconds fast can have ALL its writes incorrectly "win" against genuinely more recent writes from a correctly-clocked replica.

Worked example. A collaborative task list applies LWW at the WHOLE-LIST level (a common early design mistake): user A, offline, adds "buy milk"; user B, also offline and unaware of A's edit, adds "call dentist" to their own copy. On reconnect, LWW compares the two list versions' timestamps and keeps only ONE user's entire list, silently discarding the other user's addition entirely, "buy milk" or "call dentist" is gone, not because of a bug, but because that's exactly what LWW is designed to do: pick one and discard the other. The fix isn't a "better" LWW, it's recognizing this field needed a different merge strategy entirely (an OR-Set, in this case, which would have kept both additions).

Trade-offs and pitfalls. LWW's appeal is its simplicity, no merge logic to design or reason about, but that simplicity is exactly what makes it a trap when applied by default to a field that actually needed real merge semantics; the discipline worth internalizing is asking, for every field, "if two concurrent writes to this happen, is discarding one of them actually an acceptable outcome," before defaulting to LWW rather than after a customer reports lost data.

Direct answer: For user-profile objects replicated across regions, I'd use a G-Counter or PN-Counter for numeric fields (like a follower count), an LWW-Register (or a more careful conflict-preserving register) for single-valued fields like display name, and an OR-Set for multi-valued fields like a list of interests or tags, handling deletions with per-element tombstones and surfacing genuinely unresolvable semantic conflicts back to the application rather than silently picking a winner.

Structured elaboration

Per-field-type CRDT selection. A user profile isn't one homogeneous blob, different fields have different natural merge semantics, so the design applies a DIFFERENT CRDT per field type rather than forcing one structure onto the whole object. Counters (follower count, post count): PN-Counter, converges to the exact correct total regardless of which region incremented what. Single-valued text fields with no natural "combine" semantic (display name, bio): an LWW-Register (each write timestamped, last-write-wins per field), accepting that a genuine concurrent edit to the SAME field from two regions will silently keep only one, a reasonable trade for fields where a merge doesn't make sense anyway. Multi-valued fields (interests, tags, linked accounts): an OR-Set, so concurrent additions from different regions/devices all survive the merge, and removals only remove what was actually observed (not accidentally removing a concurrent, unrelated add of the same value).

Handling deletions and tombstones. An OR-Set's remove operation doesn't delete the underlying record, it marks the SPECIFIC observed instance(s) as removed (tracked via their unique add-IDs), while leaving room for a concurrent add of the "same" value (a different unique ID) to survive. Tombstones (records of what was removed) need periodic compaction (see metadata-growth discussion elsewhere in this topic) since they'd otherwise accumulate forever for a long-lived, frequently-edited profile.

Surfacing unresolved semantic conflicts. Some conflicts genuinely can't be resolved automatically by ANY generic merge rule, e.g. two regions concurrently setting a user's "primary email" to two DIFFERENT, both-valid-looking addresses, an LWW-Register would silently pick one, but that might not be what the user actually wants. For fields where a silent LWW resolution carries real risk of user confusion or harm, the design instead surfaces the conflict explicitly (both candidate values, with metadata about which region/device/time each came from) to the application layer, which can prompt the user to pick, rather than baking a silent, possibly-wrong resolution into the data layer.

Worked example. A user updates their bio from their phone while offline, and separately updates their location (a different field) from their laptop while also offline. On reconnect, these merge cleanly with no conflict at all (different fields, an LWW-Register per field means unrelated fields never interact). But if the SAME user, confused about which device has their latest edit, changes their bio on BOTH devices while both were offline with genuinely different text, the LWW-Register merge picks whichever write has the later timestamp, and the other bio edit is silently lost. If the product decides bio conflicts matter enough to protect against this, the field-level design would flag this specific case (same field, both devices, concurrent, per the register's own tracked metadata) and surface both candidate bios to the user on next sync rather than silently discarding one.

Trade-offs and pitfalls. Applying a single CRDT type uniformly across an entire heterogeneous object (treating the whole profile as one opaque LWW-Register, say) is a common shortcut that either loses information unnecessarily (for fields that could have merged cleanly, like the interests list) or, in the other direction, over-engineers simple fields with unnecessary OR-Set machinery, the per-field-type design costs more upfront modeling effort but avoids both failure modes.

Direct answer: Session guarantees are a set of promises a system can make to a SPECIFIC client about what it will see across its own sequence of reads and writes, weaker than full strong consistency (which promises something to every observer, not just one client's own session), but strong enough to avoid the most confusing user-facing anomalies. The four common ones: read-your-writes (a client always sees its own prior writes), monotonic reads (a client never sees data go backward in time on successive reads), monotonic writes (a client's writes are applied in the order it made them), and write-follows-reads (a client's write is guaranteed to be ordered after any write it has already observed).

Structured elaboration

Read-your-writes. After a client successfully writes something, every subsequent read BY THAT CLIENT reflects that write (or something newer), even if the underlying system is eventually consistent for OTHER clients. Without it: a user updates their profile, refreshes, and sees the old value, confusing even though the system is "eventually" correct.

Monotonic reads. Once a client has read a value at some point in time, it will never later read an OLDER value, even if it's routed to a different, lagging replica on a subsequent request. Without it: a user refreshes a page twice and the second refresh shows STALER data than the first, which reads as the system going backward.

Monotonic writes. A client's own writes are applied in the order the client issued them, even if they're processed by different replicas or arrive out of order over the network. Without it: two edits from the same user could be applied out of order, silently losing the later edit's effect if the earlier one "wins" by arriving second.

Write-follows-reads. If a client reads value V and then makes a write based on what it read, that write is guaranteed to be applied AFTER (causally ordered after) the write that produced V, even on a different replica. Without it: a user could see a comment, reply to it, and have their reply become visible to others before the original comment they're replying to is, a causally backward-looking result.

Implementation strategies for a client SDK

Sticky sessions (server-side). Route a given client's requests to the SAME replica for the duration of a session, so its own reads trivially see its own writes (since they're the same node). Simple, but limits load-balancing flexibility and doesn't survive the sticky replica failing over.

Client-side version tokens / causal tokens. The server returns a version token (a timestamp, a vector clock, or a simple monotonic counter) with every response; the client includes the LATEST token it has seen with every subsequent request; the server serving that request ensures the replica it reads from is at least as current as that token (waiting briefly, or routing to a sufficiently up-to-date replica) before responding. This works across server restarts and load-balancer changes, at the cost of a small amount of extra state the client (or an SDK on its behalf) needs to carry and the server needs to check against.

Metadata persistence and token expiry. For a client SDK, the causal token is typically stored alongside whatever session state the client already keeps (in-memory for a single session, or persisted, e.g. in a mobile app's local storage, if the guarantee needs to survive an app restart). A token needs no explicit expiry for correctness (an old, stale token just means the server might do a bit more waiting to catch up, never incorrect behavior), though in practice you might cap how large it can grow (e.g. compacting a vector-clock-style token) for efficiency.

Persisting across devices. Storing the causal token only in a single device's local storage (as described above) loses the guarantee the moment a user switches devices, a phone-issued token never reaches the laptop the user opens next, so the laptop's first read looks like a fresh, tokenless session and can show stale data relative to a write the user just made on the phone. To make the guarantee follow the USER rather than the device, the server should also persist the client's latest known token keyed by the authenticated user's account (not just return it to the device and forget it), and the SDK should fetch that stored token as part of session bootstrap on login, merging it with (or preferring it over) whatever token the local device already has, before issuing its first read. This costs a small amount of server-side storage per user and an extra lookup at login, but it's what keeps read-your-writes intact across a device switch instead of silently resetting.

Trade-offs and pitfalls. Confusing "the system is eventually consistent" with "session guarantees don't matter" is a common design mistake, an eventually-consistent system with NO session guarantees produces a genuinely confusing user experience (data appearing to go backward, a user not seeing their own actions reflected), while the same system WITH session guarantees can feel completely normal to users despite being eventually consistent under the hood for everyone else.