Advanced Caching and Data Pipeline Design Questions
Distributed caching, cache coherency, specialized stores (search engines, column stores, time-series databases). Data pipeline architecture: batch processing, stream processing, ETL design. Understanding Lambda and Kappa architectures.
EasyTechnical
36 practiced
A product team asks whether to use Elasticsearch or relational DB indexes for a new text-search feature with high update rates. Describe the engineering trade-offs, indexing latency, query expressiveness, and caching interactions you would evaluate to make a recommendation for fast search across millions of documents.
Sample Answer
Clarify requirements: confirm expected query types (full-text, phrase, fuzzy, facet/aggregation), update patterns (per-doc updates, deletes, bulk vs streaming), SLA for freshness (milliseconds, seconds, minutes), expected QPS, and read/write ratio.Trade-offs and evaluation criteria:- Indexing latency & freshness: Relational DB indexes (B-tree, trigram extensions) update synchronously with transactions → near-zero indexing latency and strong consistency. Elasticsearch uses near-real-time indexing (refresh interval, default 1s) and segments → typical refresh latency 0.5–2s; you can reduce refresh to improve freshness at the cost of indexing throughput and I/O.- Update rate & throughput: High update rates favor RDBMS for small updates if queries are simple; for large-scale text indexing, ES bulk APIs and tuned refresh/merge settings handle many writes but require careful resource tuning (refresh interval, refresh throttling, replica behavior). Frequent small updates in ES can cause segment churn and storage overhead.- Query expressiveness: ES offers rich full-text (analyzers, scoring, fuzzy, phrase, aggregations) and scale-out search; RDBMS full-text varies by engine — often less flexible and slower for complex relevance scoring or distributed search across shards.- Caching interactions: RDBMS benefit from buffer pool and query cache (if enabled) for repeated lookups; ES relies on OS page cache and query cache for frequent queries but caches become less effective with high-cardinality or highly dynamic data. Also consider application-layer caches (CDN, Redis) to hide freshness requirements.Recommendation (fast search across millions with high update rates):- If strict freshness and transactional consistency are primary and queries are simple (exact matches, basic full-text), use relational DB full-text or secondary trigram indexes.- If relevance, complex text queries, aggregations, and scale-out query performance matter more, use Elasticsearch but tune for updates: increase refresh interval, use bulk/batched writes, implement versioning/soft-deletes to reduce churn, and combine with a write-through cache or delta-index pattern (hot index for recent changes, periodic merge into main index) to balance freshness and throughput.Measure: prototype both approaches on realistic load to compare latency, CPU/I/O, and index growth; monitor merge/refresh metrics and cache hit rates before final decision.
EasyTechnical
48 practiced
You observe stale reads from a cache after updates to the primary database (e.g., product price updates). List practical approaches to avoid stale data in caches and discuss pros/cons: cache invalidation (synchronous/asynchronous), time-based TTLs, versioned keys, write-through/write-back, and read-your-writes guarantees.
Sample Answer
Situation: Stale reads from cache after DB updates (e.g., product price) are common; here are practical approaches and trade-offs.1) Cache invalidation- Synchronous (invalidate or update cache within same DB transaction or before commit) - Pros: Strong consistency, low window for staleness. - Cons: Adds latency to write path; risk of longer transactions; harder in distributed systems.- Asynchronous (publish invalidation events after commit) - Pros: Faster writes; decouples services. - Cons: Small window of staleness; needs reliable messaging and ordering; retries/duplicates handling.2) Time-based TTLs- Pros: Simple, avoids infrastructure changes, bounds staleness.- Cons: Either long TTL → stale data; short TTL → higher cache churn and backend load.3) Versioned keys (or cache key with version/epoch)- Pros: Easy invalidation by bumping version; atomic switch for many keys.- Cons: Requires version management and potential cache growth until old keys expire.4) Write-through / write-back- Write-through: writes update both DB and cache synchronously. - Pros: Read-after-write consistency for readers; simpler coherence. - Cons: Higher write latency.- Write-back: write to cache and lazily persist to DB. - Pros: Fast writes, reduced DB load. - Cons: Risk of data loss on cache failure; complex eviction/flushing logic.5) Read-your-writes guarantees (client/session affinity or causal reads)- Techniques: sticky sessions, read-after-write from primary, or use per-user read-after-write layer. - Pros: Users see their updates immediately. - Cons: Adds routing complexity; doesn't solve global consistency.Practical guidance: combine methods—use asynchronous invalidation + reasonable TTLs for general items, versioned keys for bulk updates (e.g., price list), and read-your-writes for critical user flows. Ensure reliable event delivery, idempotency, and monitoring to detect staleness.
MediumSystem Design
46 practiced
Design a streaming pipeline that uses Kafka for ingestion and Apache Flink (or equivalent) for sessionization and aggregations, then writes results to ClickHouse for analytics queries. Specify how to achieve fault tolerance, windowing/session semantics, and state management; discuss delivery semantics (at-least-once vs exactly-once) and their impact on downstream storage.
Sample Answer
Requirements:- Ingest high-throughput event stream (user events) into Kafka.- Perform sessionization (group events by user with inactivity gap) and aggregations (session length, counts).- Persist results to ClickHouse for analytics with low-latency reads.- Strong fault tolerance and clear delivery semantics.High-level architecture:- Producers -> Kafka (topic partitioned by user_id) -> Apache Flink job -> ClickHouse sink (via exactly-once-capable connector or idempotent/upsert writes).Core components & responsibilities:1. Kafka: durable, partitioned ingestion; choose partition key = user_id to ensure affinity.2. Flink: stateful stream processing for session windows using event time; keyed by user_id; use SessionWindow (gap e.g., 30m) or custom ProcessFunction for complex session logic.3. State backend: RocksDB + filesystem checkpointing (S3/HDFS) for large state and fast recovery.4. Checkpointing: enable periodic Flink checkpoints (e.g., 30s) and retain externalized checkpoints.5. Sink to ClickHouse: use transactional/atomic writes where possible (buffered batches), or write to Kafka topic for a separate insert worker to ensure control.State management & windowing:- Use event-time session windows so late events (watermarks) are handled; set allowed lateness and side outputs for late events.- Store per-key session state in RocksDB: current session start, last event time, aggregates. On session gap expiry, emit final session record and clear state.Fault tolerance:- Flink checkpoints capture state + Kafka consumer offsets atomically (via two-phase commit or Kafka's transactional source). Use Flink’s exactly-once integration with Kafka (FLIP-149 style) to ensure consistent offsets with state.- Store checkpoints in durable storage; enable high-availability JobManager.Delivery semantics and impact on ClickHouse:- Exactly-once: preferred. Use Flink two-phase commit sink or idempotent upserts into ClickHouse (e.g., replace into with unique session_id). Exactly-once prevents duplicates in ClickHouse and simplifies downstream analytics.- At-least-once: simpler but can cause duplicate session records. Mitigate with deduplication in ClickHouse (unique keys, TTL, dedupe materialized views) or use idempotent keys (session_id composed of user_id + session_start).- If ClickHouse connector doesn't support two-phase commit, write to an intermediate Kafka "sink" topic with exactly-once guarantees and have a separate idempotent consumer that writes to ClickHouse.Trade-offs:- Exactly-once adds complexity and potential latency (two-phase commits, larger checkpoints).- RocksDB + externalized checkpoints increases recovery time but supports large state.- Watermarks and allowed lateness balance completeness vs latency.Operational notes:- Monitor checkpoint durations, state size, Kafka consumer lag.- Test failure scenarios (task/worker/node failure) and validate end-to-end semantics with chaos testing.- Provide metrics for sessions emitted, duplicates, late events.
HardTechnical
48 practiced
Discuss cache coherence models for distributed caches: invalidation-based, update-based, lease-based, and version-vector approaches. For each model, analyze scaling behavior, network overhead, stale-read windows, and suitability for read-heavy vs write-heavy systems.
Sample Answer
I'll compare the four coherence models across scaling behavior, network overhead, stale-read window (how long a reader might see stale data), and read/write workload suitability. I'll give short examples and trade-offs.Invalidation-based- How it works: Writers send invalidation messages for keys/lines; readers must fetch fresh data on next access.- Scaling: Good for large systems if invalidations are multicast/filtered; however high fan-out writes harm scale.- Network overhead: Low for read-heavy workloads (only invalidations on writes). Overhead grows with number of cached replicas and write rate.- Stale-read window: Potentially small (staleness ends after next fetch) but readers can see stale values until they re-read; window = time between invalidation issuance and reader noticing.- Suitability: Best for read-heavy, low-update workloads or when cheap remote fetch is acceptable.Update-based (write-update)- How it works: Writers propagate new values to all replicas immediately.- Scaling: Poorer than invalidation as every write triggers payload distribution to all replicas; becomes expensive at high replica counts.- Network overhead: High for write-heavy systems (full data shipped), but low fetch latency for readers because caches are fresh.- Stale-read window: Very small (near-zero if propagation is synchronous); consistency tighter.- Suitability: Good when reads and writes both frequent but low-latency reads are critical and network/write rate is moderate (or value deltas are small).Lease-based- How it works: Writers grant time-bounded leases to holders; caches are valid until lease expiry; writers must revoke or wait for lease expiration.- Scaling: Scales well with controlled lease management; number of control messages depends on lease churn rather than replica count.- Network overhead: Moderate — lease grants and occasional renewals/revocations. Overhead proportional to lease renewal frequency and write contention.- Stale-read window: Bounded by lease duration (deterministic): readers may see stale values up to lease TTL unless explicit revocation is used.- Suitability: Good when you want bounded staleness and lower coordination cost than strict invalidation; fits geo-distributed read-heavy services with predictable read/write patterns.Version-vector / CRDT-style (per-key metadata)- How it works: Each replica tracks versions (vector clocks, timestamps); reads can detect divergence and either merge, fetch latest, or serve with metadata.- Scaling: Version vectors grow with number of writers/replicas or require compaction/approximation; careful design needed for many nodes.- Network overhead: Metadata overhead increases (vectors) and reconciliation traffic on conflicts. Write propagation can be selective; reconciliation may cost extra.- Stale-read window: Variable — can provide monotonic reads, causal guarantees, or detect staleness; resolution strategies determine effective staleness.- Suitability: Best when you need conflict detection/merge (multi-master), causal consistency, or offline updates. Good for distributed collaborative systems; less ideal if tiny metadata or strict global single-writer semantics required.Trade-offs summary- Read-heavy, low-write: invalidation or lease (longer leases) minimize network cost.- Read-heavy but require low-latency fresh reads: update-based or short-lease invalidation.- Write-heavy: update-based causes high network cost; leases can reduce coordination; version-vectors help if multi-master is needed but add metadata cost.- Bounded staleness: leases give deterministic bound; version-vectors+causal models give stronger ordering guarantees but more complexity.- Scalability levers: reduce replica count per key, use hierarchical/partitioned invalidation/update, diminishing vector size (dotted version vectors), and hybrid approaches (invalidate most replicas, update hot ones).Example hybrid: use leases for most reads, push updates to a small set of “primary” replicas and invalidate others—combines low write fan-out with bounded staleness.When designing, quantify write rate × replica fan-out, acceptable staleness TTL, network budget, and conflict model; choose the model (or hybrid) that fits those constraints.
HardSystem Design
58 practiced
Design a globally distributed cache that provides low read latency for users worldwide and eventual consistency. The system should support 100M keys, 100k writes/sec, and prefer read-local semantics. Describe partitioning, replication between regions, conflict resolution, cache warming, and acceptable consistency models.
Sample Answer
Requirements & constraints:- 100M keys, 100k writes/sec global, read-local preferred, eventual consistency acceptable, low read latency worldwide.High-level approach:- Sharded, multi-region, read-local cache with asynchronous geo-replication and conflict resolution via CRDTs or timestamped updates. Each region serves reads from local replicas; writes are accepted locally and propagated asynchronously.Partitioning:- Consistent hashing across a configurable number of virtual shards (e.g., 4096) to evenly distribute 100M keys and allow smooth re-sharding.- Each shard maps to a group of cache nodes (replica-set) in a region. Use shard ownership metadata in a lightweight placement service (ZooKeeper/Coordination).Replication between regions:- Multi-master (peer-to-peer) asynchronous replication per shard: each region holds one or more replicas for each shard. Local writes apply immediately to local replica-set, then are propagated via append-only change logs or Raft-wal style async streams to remote regions.- Use anti-entropy (gossip) + per-shard changelogs to ensure convergence. Support hinted handoff for temporarily unavailable regions.Conflict resolution:- Prefer CRDTs for commutative updates (counters, sets) to achieve automatic merge.- For arbitrary values, use LWW with Hybrid Logical Clocks (HLC) provided by clients/regions to order updates; on equal timestamps use deterministic tie-breaker (region ID).- Optionally track small vector clocks for hot keys where concurrent updates are frequent; expose resolved value or conflict metadata to application if needed.Cache warming & population:- Write-through mode: on writes to origin DB, push to caches in origin region and propagate to others.- Lazy-warm: on cache miss, read from origin and write locally plus asynchronous propagation.- Bulk-warm on deployment: background jobs prefetch hot key lists (from analytics) and populate regional caches in parallel.- Use TTLs and LFU eviction; provide prioritized warming (top-N hot keys first).Consistency models & guarantees:- Default: eventual consistency with read-local low latency.- Provide session guarantees per client: read-your-writes and monotonic reads via session token with last-applied HLC timestamp.- Offer bounded staleness option (e.g., "serve data no older than X ms") by tracking per-shard commit lag.- Allow stronger consistency per-key via synchronous cross-region quorum if needed (at cost of latency).Scalability & ops:- Scale horizontally by adding nodes and virtual shards; monitor per-shard throughput and re-balance automatically.- Metrics: replication lag, conflicts/sec, cache hit-rate, per-region write rate.- Failure handling: retry with exponential backoff, tombstones for deletes, compaction of changelogs.Trade-offs:- Asynchronous multi-master favors availability and low read latency at cost of staleness and occasional conflicts.- CRDTs reduce conflicts but require operation semantics; LWW/HLC simpler but may lose concurrent updates.This design gives global low-latency reads, handles 100M keys and 100k writes/sec by sharding and horizontal scaling, and uses pragmatic conflict-resolution & cache-warming strategies to meet business needs.
Unlock Full Question Bank
Get access to hundreds of Advanced Caching and Data Pipeline Design interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.