Stream Processing and Event Streaming Questions
Building on event-streaming platforms: Kafka and message queues, event sourcing, partitioning, consumer groups, exactly-once vs at-least-once delivery, and windowing. Covers handling late and out-of-order events, watermarks, and stateful stream operators. The core skill for real-time data engineering.
A stream of events from one source is producing duplicates, but only on a subset of partitions. Describe how you'd instrument and debug this down to a root cause.
Sample Answer
Direct answer
Duplicates confined to a subset of partitions point at something specific to those partitions, most commonly a producer retrying against just those partitions' leader (a transient broker issue on that leader) without idempotence enabled, or a consumer committing offsets inconsistently for just those partitions.
Structured elaboration
The instrumentation strategy is to correlate the affected partitions against three signals: producer-side retry counts and error logs scoped to those specific partitions' leader broker (a leader having transient issues would show elevated retries only for partitions it leads); consumer offset-commit history for those partitions specifically, checking for a pattern of commits that don't monotonically advance, or gaps consistent with a rebalance interrupting an in-flight commit; and whether producer idempotence is actually enabled and functioning (a duplicate purely from producer retry should be silently deduplicated by an idempotent producer, so duplicates surviving despite idempotence being enabled point elsewhere, most likely the consumer side).
Worked example
Concretely: pull the list of affected partitions, and check which broker leads each of them; if they share a common leader broker that had a recent restart or GC-pause episode, that's strong evidence of producer retries against an unstable leader. Cross-reference with whether enable.idempotence was set on the producer at the time; if it was, and duplicates still occurred, check the consumer's commit log for those partitions around the same time window for signs of a rebalance interrupting a commit (partitions revoked before an in-flight commit completed), which would cause reprocessing rather than a broker-side duplicate write.
Trade-offs and pitfalls
A common mistake is assuming duplicates always originate on the producer side and never checking the consumer's commit behavior; at-least-once consumption inherently allows for reprocessing-caused duplicates that have nothing to do with the producer at all. Fixing this at the source (enabling idempotence, or fixing the commit-timing bug) is preferable to only adding downstream deduplication, since downstream dedup is a safety net that should catch what slips through, not the primary correctness mechanism.
Design an idempotent sink that writes streaming results into an external database that does not support distributed transactions, ensuring no duplicate rows even when the streaming job restarts and reprocesses.
Sample Answer
Direct answer
Design the sink so writes are naturally idempotent, typically an upsert keyed by a deterministic identifier derived from the source event (its partition and offset, or a stable business key), so a replayed write after a restart produces the identical row rather than a second one.
Structured elaboration
Without distributed-transaction support at the sink, you can't get the streaming job's checkpoint and the sink's write to commit as a single atomic unit; the workaround is to make the write itself safe to repeat. Concretely: derive a deterministic key for every write (source offset, or a natural key already unique to the business event), and express the write as "set this row's value" (an upsert) rather than "append a new row." On a restart-and-replay after a crash, the same source records get reprocessed and re-written with the identical key and value, so the external database simply overwrites the same row with the same content, producing no visible duplicate.
Worked example
A sink writing aggregated hourly revenue per product to a relational table keyed by (product_id, hour): an upsert (INSERT ... ON CONFLICT (product_id, hour) DO UPDATE SET revenue = excluded.revenue) means that if the job crashes after writing hour 14's revenue but before its checkpoint commits, and recovery reprocesses and rewrites hour 14's revenue again, the second write simply overwrites the row with (in this case) the same recomputed value, leaving no duplicate row and no incorrect double-counted total.
Trade-offs and pitfalls
This only works cleanly when the write is naturally expressible as an idempotent upsert; an operation with an external side effect (sending a notification, charging a payment) can't be made idempotent this way without an additional mechanism, typically an idempotency key checked and recorded at the point of the side effect itself, not just at the database write. A common mistake is deriving the upsert key from something that isn't actually stable across a replay (a randomly generated ID rather than the source offset or a genuine business key), which silently reintroduces duplicates despite the upsert pattern looking correct.
Explain checkpoints and savepoints in a stateful stream-processing framework. What roles do they play in fault tolerance, and how does synchronous versus asynchronous checkpointing affect job throughput and recovery time?
Sample Answer
Direct answer
Checkpoints and savepoints are both durable snapshots of a stateful stream-processing job's internal state, used to recover after a failure without reprocessing from the very beginning; checkpointing frequency and whether it's synchronous or asynchronous directly trade off recovery time against how much steady-state throughput the job gives up while checkpointing.
Structured elaboration
A checkpoint captures, atomically, both the input offsets the job has consumed up to and the current value of all its internal state (window contents, aggregation accumulators, join buffers), so recovery resumes exactly where the job left off rather than replaying from the start of retention. Savepoints serve the same mechanism but are triggered manually rather than on a timer, typically before a deliberate action like a version upgrade or a planned rescaling, so the job can be stopped and later restarted from that exact point rather than an arbitrary automatic checkpoint. Synchronous checkpointing pauses processing while the snapshot is taken, which is simple and consistent but adds a real throughput dip on every checkpoint; asynchronous checkpointing lets processing continue while the snapshot is written in the background (typically via a copy-on-write mechanism), trading a bit of implementation complexity for much less throughput impact.
Worked example
A job checkpointing every 30 seconds, synchronously, on a state size that takes 2 seconds to snapshot, loses roughly 2 seconds of throughput out of every 30 (about a 6.7% steady-state throughput cost) purely to checkpointing pauses; switching to asynchronous checkpointing removes most of that pause at the cost of needing enough memory headroom to buffer in-flight state mutations during the snapshot. If that job crashes, recovery time is bounded by how long it takes to reload the last checkpoint (proportional to state size) plus reprocessing whatever was consumed since that last checkpoint, so a longer checkpoint interval means faster steady-state throughput but a longer worst-case recovery window.
Trade-offs and pitfalls
Checkpointing too frequently protects recovery time at a real, continuous throughput cost; checkpointing too infrequently risks a long recovery window (and a lot of reprocessing) after a rare failure, a trade-off that should be tuned against your actual recovery-time objective, not set to an arbitrary default. A common mistake is treating checkpoint interval as a purely technical knob rather than connecting it to a business-facing recovery-time expectation that stakeholders have actually agreed to.
A production broker cluster is showing frequent leader elections, shrinking in-sync-replica sets, and rising tail latency (or has just lost a large fraction of brokers with unclean leader election enabled). Walk through your triage checklist and how you'd stabilize the cluster and validate no data was silently lost.
Sample Answer
Direct answer
Frequent leader elections and a shrinking in-sync-replica set point to brokers falling behind or becoming unreachable; the triage priority is to stabilize replication first (find and fix whatever's causing followers to fall behind), and only then worry about performance tuning, because an unstable ISR is an active data-safety risk, not just a latency problem.
Structured elaboration
A broker showing repeated leader elections is either crashing/restarting, suffering long garbage-collection pauses that make it look unresponsive to the coordination layer, or is genuinely network-partitioned from the rest of the cluster. A shrinking ISR set means followers can't keep up with the leader's write rate, most commonly from disk I/O saturation, network bandwidth exhaustion, or the same GC-pause problem affecting the replication path. If a large fraction of brokers is lost and unclean leader election is enabled, any partition whose remaining replicas are all out-of-sync will still elect a leader from among them, silently accepting a replica that may be missing recently-acknowledged writes.
Worked example
A concrete triage checklist: (1) confirm the coordination layer's own health first, since a lagging controller can itself cause spurious leader-election churn; (2) check broker-level GC logs and disk/network I/O metrics for the affected brokers, correlated against the timestamps of the leader elections; (3) identify every partition whose ISR shrank and whether unclean leader election activated for any of them; (4) once the root cause (say, a disk nearing capacity causing write stalls) is fixed and brokers rejoin the ISR, run a targeted comparison (checksums or record counts) on any partition that underwent unclean leader election against the last known-good replica, to find out whether any acknowledged writes were actually lost.
Trade-offs and pitfalls
Restarting a struggling broker without first understanding why it's struggling can make things worse, briefly forcing every partition it led to fail over at once and adding load to the very brokers that must now serve as new leaders. Unclean leader election should generally be treated as an emergency-only escape hatch, not a standing configuration on any topic where losing recently-acknowledged data is unacceptable; the moment it fires, you should treat the affected data as needing a verification pass, not assume everything is fine because the cluster is technically serving traffic again.
Explain tumbling, sliding, and session windows in stream processing, and give a realistic use case for each.
Sample Answer
Direct answer
A tumbling window is a fixed-size, non-overlapping bucket (every event belongs to exactly one window); a sliding window is fixed-size but overlapping, advancing more frequently than its own length so events can belong to several windows at once; a session window has no fixed size at all, closing only after a period of inactivity.
Structured elaboration
Tumbling windows fit metrics you want reported once per fixed period with no overlap, like hourly active-user counts. Sliding windows fit metrics that should update more frequently than the window length itself, like a rolling 5-minute average recomputed every minute, useful when you want smoother, more responsive trend lines than a tumbling window's abrupt bucket boundaries would give. Session windows fit behavior that's naturally grouped by activity rather than by clock time, like a user's browsing session, where the right boundary is "this user went quiet for a while," not a fixed clock interval.
Worked example
For an e-commerce site: hourly revenue reporting is a tumbling window (each dollar counted in exactly one hour's bucket, no double-counting across hours). A "trending products in the last 10 minutes, updated every minute" feature is a sliding window (each product's count includes overlapping activity from the last 10 minutes, recomputed every 60 seconds so the trend feels responsive). A user's shopping session (used to compute cart-abandonment or session-length metrics) is a session window with, say, a 30-minute inactivity gap, since sessions genuinely vary in length and aren't meaningfully described by any fixed clock interval.
Trade-offs and pitfalls
Sliding windows multiply output volume relative to their overlap (a 10-minute window sliding every 1 minute emits roughly 10x more results than a 10-minute tumbling window), which has real storage and downstream-processing cost implications. Session windows need an explicit policy for what happens to a session when a late event arrives just after it closed, since "was the gap really over" is inherently a judgment call the engine has to make with an allowed-lateness parameter, unlike a tumbling window's fixed, unambiguous boundary.
Unlock Full Question Bank
Get access to all Stream Processing and Event Streaming interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.