Database Scalability and High Availability Questions

Architectural approaches and operational practices for scaling and maintaining database availability. Topics include vertical versus horizontal scaling trade offs; replication topologies, leader and follower roles, read replicas and replica lag; read write splitting and connection pooling; sharding and partitioning strategies including range based, hash based, and consistent hashing approaches; handling hot partitions and data skew; federation and multi database federation patterns; cache layers and cache invalidation; rebalancing and resharding strategies; distributed concurrency control and transactional guarantees across shards; multi region deployment strategies, cross region failover and disaster recovery; monitoring, capacity planning, automation for failover and backups, and cost optimization at scale. Candidates should be able to pick scaling approaches based on read and write patterns and explain operational complexity and trade offs introduced by distributed data.

MediumSystem Design

36 practiced

Describe a practical resharding strategy to add a new shard to a distributed key-value store using consistent hashing. Outline client routing updates, data migration order, how to keep reads and writes correct during migration, validation steps, and how you'd rollback if migration shows issues.

HardTechnical

36 practiced

Evaluate trade-offs and create a recommendation for using a managed multi-region database service (e.g., Cloud Spanner, Cosmos DB, Aurora Global) versus running a self-managed sharded cluster on VMs/Kubernetes for a system handling ~10M QPS reads and 200k writes/sec. Cover latency, consistency, operational complexity, cost, scaling behavior, and DR capabilities.

HardTechnical

43 practiced

Explain how snapshot isolation (SI) can be implemented across shards and what is required to provide serializability across distributed partitions. Discuss MVCC, timestamp allocation (logical or physical), global ordering mechanisms (e.g., TrueTime or hybrid logical clocks), and the performance vs correctness trade-offs for each approach.

Sample Answer

**Implementing Snapshot Isolation (SI) across shards:**- Use MVCC: each shard keeps versioned rows; reads pick the latest version ≤ transaction's snapshot timestamp.- Timestamp allocation: transaction coordinator assigns a snapshot timestamp when transaction starts. Timestamps can be physical (wall-clock) or logical (monotonic counters).

**Timestamp mechanisms:**- Physical time (e.g., TrueTime):

text

if now ∈ [earliest, latest] then assign timestamp = now

Intuition: TrueTime gives bounded uncertainty, enabling external consistency with minimal coordination.- Hybrid Logical Clocks (HLC):

text

ts = max(local_clock, observed_ts) + logical_increment

Intuition: HLC combines monotonic physical time with logical counters to provide causality without strict clock sync.

**Providing serializability across partitions:**- SI alone allows write skew. To get serializability you need extra coordination: - Serializable SI via certification: at commit, run a global validation (or per-shard) to detect dangerous patterns and abort if needed. - Global ordering: use a global sequence (Paxos/Raft leader) to serialize conflicting commits. - Use dependency tracking (read/write sets) and a centralized certifier that ensures no conflicting concurrent commits commit.

**Trade-offs:**- TrueTime: strong correctness with low coordination for some workloads, but requires specialized infrastructure and tight clock sync; complexity in bounding uncertainty.- HLC/logical timestamps: simpler, works on commodity hardware, but needs more commit-time coordination to prevent anomalies; may incur higher aborts or latencies.- Certification vs global ordering: certification reduces synchronous cross-shard locking but can increase aborts and retry traffic; global ordering (single log) simplifies correctness but becomes a write-path bottleneck and increases latency.

**Practical guidance for a backend engineer:**- Use MVCC per shard and HLC for timestamps if you cannot rely on TrueTime. Implement a lightweight coordinator that performs per-shard prepare and a fast certification phase to enforce serializability for critical transactions; permit SI for read-mostly operations to maximize performance. Monitor abort rates and add stronger ordering for high-conflict workloads.

EasyTechnical

46 practiced

What are the essential database monitoring metrics a backend developer should track for capacity planning and high availability? Provide categories of metrics (resource, replication, performance) and at least two specific metrics per category with brief rationale for each.

MediumTechnical

34 practiced

Compare active-passive and active-active multi-region database deployment models for a globally distributed backend. For an application with strong write locality (users write in their home region) and low tolerance for write conflicts, recommend an approach and describe replication topology, failover, and latency trade-offs.

Unlock Full Question Bank

Get access to hundreds of Database Scalability and High Availability interview questions and detailed answers.

Join thousands of developers preparing for their dream job.