Batch and Stream Processing Questions

Covers design and implementation of data processing using batch, stream, or hybrid approaches. Candidates should be able to explain when to choose batch versus streaming based on latency, throughput, cost, data volume, and business requirements, and compare architectural patterns such as lambda and kappa. Core stream concepts include event time versus processing time, windowing strategies such as tumbling sliding and session windows, watermarks and late arrivals, event ordering and out of order data handling, stateful versus stateless processing, state management and checkpointing, and delivery semantics including exactly once and at least once. Also includes knowledge of streaming and batch engines and runtimes, connector patterns for sources and sinks, partitioning and scaling strategies, backpressure and flow control, idempotency and deduplication techniques, testing and replayability, monitoring and alerting, and integration with storage layers such as data lakes and data warehouses. Interview focus is on reasoning about correctness latency cost and operational complexity and on concrete architecture and tooling choices.

MediumTechnical

88 practiced

Implement an efficient deduplication strategy in Python pseudocode using Apache Beam (or equivalent) that removes duplicate events based on event_id within a 24-hour sliding window while remaining memory efficient. Describe how you would use keyed state and timers, and how you would handle scale and restarts.

Sample Answer

Approach: use keyed, per-event_id state that records the last-seen timestamp and a processing timer to evict state after 24 hours. For scale, key by event_id (or a hash shard of it), use Beam’s per-key state + event-time timers so duplicates within a 24-hour sliding window are dropped, and state is cleaned automatically. Use durable state so restarts resume correctly.

python

# python (Beam-like pseudocode)
class Deduplicate(DoFn):
    # ValueState stores the timestamp of first-seen event
    SEEN_TS = ValueStateSpec('seen_ts', TimestampCoder())
    # Timer to clear state after window
    CLEAR_TIMER = TimerSpec('clear', TimeDomain.EVENT_TIME)

    def process(self, element, timestamp=DoFn.TimestampParam,
                seen_ts=DoFn.StateParam(SEEN_TS),
                clear_timer=DoFn.TimerParam(CLEAR_TIMER)):
        event = element  # assumes {'event_id':..., 'payload':...}
        key = event['event_id']
        first_seen = seen_ts.read()
        if first_seen is None:
            # first time -> emit and set state + clear timer 24h after event time
            yield event
            seen_ts.write(timestamp)
            clear_timer.set(timestamp + Duration(hours=24))
        else:
            # duplicate within 24h -> drop (no output)
            pass

    def on_timer(self, clear_timer=DoFn.TimerParam(CLEAR_TIMER),
                 seen_ts=DoFn.StateParam(SEEN_TS)):
        # timer fired -> clear state to free memory
        seen_ts.clear()

Key points:- Use event-time timers so lateness is respected; set watermark and allowed lateness.- Memory efficient: per-key single timestamp, removed after 24h.- Scaling: Beam shards keys; for very hot keys pre-shard (append hash suffix) to avoid single-key hotspot and reconcile downstream if needed.- Restarts: durable state and timers in runner (Dataflow/Flink) persist state; ensure checkpointing enabled and idempotent sinks.- Edge cases: out-of-order/late events (configure allowed lateness), clock skew (use event-time), very high cardinality (consider TTL eviction, approximate deduplication with Bloom filters for memory constraints).

HardTechnical

70 practiced

Design autoscaling and resource allocation strategies for streaming jobs and GPU inference clusters that process highly variable traffic (baseline 1M events/min, occasional 10x bursts). Ensure inference latency stays under 100ms and overall cost is minimized. Address strategies like predictive autoscaling, prewarming, spot instances, and warm pools.

Sample Answer

Requirements & constraints:- Baseline 1M events/min (~16.7k reqs/s), occasional 10x bursts (167k reqs/s)- P95 latency <100ms for inference- Minimize cloud/GPU cost, tolerate some non-critical degradation only during extreme bursts

High-level approach:- Combine predictive autoscaling (time-series + recent signals) with reactive autoscaling, warm pools/prewarmed instances, spot instances for non-critical capacity, and strict admission control/batching to meet latency SLA.

1) Capacity planning & model characteristics- Measure per-request GPU throughput and latency for your model (single-stream and batched): e.g., GPU can do 500 inferences/s at batch=8 with 30ms latency.- Derive required GPUs: baseline GPUs = ceil(16.7k / 500) = 34 (add 20% safety → ~41). Peak 10x → ~410 GPUs.

2) Predictive autoscaling- Train short-horizon predictor (ARIMA/Prophet/LSTM) on traffic patterns + calendar features; produce t+1-10min forecasts and confidence intervals.- Start scale-up actions proactively based on forecast + safety margin (e.g., forecast 95th perc + 10%). Use gradual ramp plans to allow instance boot times.

3) Warm pools & prewarming- Maintain a warm-pool of preprovisioned GPU instances (or GPU VMs with driver/runtime warm) sized to cover sudden multipliers (e.g., 2x baseline). Use image snapshots with drivers, model loaded into memory for fastest start (<30s).- For larger predicted spikes, prewarm additional nodes progressively.

4) Spot instances & heterogeneous fleet- Use mixed instance policies: on-demand for critical baseline capacity; spot/interruptible for overflow. Tag spot capacity for lower-priority work or as additional inference shards with fast fallback.- Implement graceful eviction handling: maintain model replicas on on-demand nodes to absorb spot loss; use node draining and re-routing.

5) Reactive autoscaling & fast paths- Short-window reactive autoscaler (1–2s metrics) to adjust request routing and worker concurrency. Scale pods/containers horizontally within node capacity; scale nodes with cooldowns tuned to boot times.- Maintain a small pool of ultra-fast instances (e.g., small number of high-end GPUs or CPU-based lightweight approximations) to handle sudden spikes for critical low-latency paths.

6) Inference optimizations & admission control- Dynamic batching with max batch size tuned to latency target; set preferred latency SLO per batch. Use adaptive batching: increase batch size under high load but cap expected per-request latency.- Prioritize requests (real-time vs background). For overload, apply shedding or best-effort queue fallback (degraded model or CPU-based cheaper model).- Model quantization, pruning and TensorRT/ONNX runtime to reduce per-request GPU time and increase throughput.

7) Queuing & SLA enforcement- Use front-door request queue with request deadlines. If queue wait + estimated inference > SLA, serve degraded path or reject with proper backpressure.- Track end-to-end latency budgets (network + queuing + inference) and feed this into scaling decisions.

8) Cost optimization & monitoring- Autoscale policy uses cost-aware objective: prefer spot for incremental capacity; scale-down gracefully after sustained low usage (observe 5–15 min cool-down).- Continuous monitoring/alerting: GPU utilization, queue lengths, P95/P99 latency, spot eviction rate, prediction error. Use RL/tuning to adjust safety margins.

Trade-offs & rationale:- Predictive autoscaling + prewarming reduces unmet demand and cold-start latency but risks overprovisioning if forecasts are wrong; mitigate with confidence intervals and cost-aware thresholds.- Spot instances cut cost heavily but require redundancy and fast fallback.- Warm pools provide fastest reaction (<30s) at moderate cost; size them to cover common short bursts.- Adaptive batching increases throughput but can hurt tail latency—use per-request deadlines to avoid violating 100ms P95.

Example policy (practical numbers):- Baseline: 40 on-demand GPUs running warmed model.- Warm pool: extra 20 prewarmed GPUs for instant 1.5x handling.- Predictive scale to add spot + on-demand to reach forecasted peak; keep an on-demand reserve = 10% of predicted peak.- Reactive scaler with 15s window to add nodes if queue length > threshold; cool-down 5–10 minutes to avoid thrash.

This design ensures sub-100ms latency via warm pools, adaptive batching, and priority admission, while minimizing cost with spot usage and predictive scale decisions. Continuous telemetry and iterative tuning (safety margins, warm-pool size, batch policies) are required.

EasyTechnical

70 practiced

Explain backpressure and flow control in streaming systems. Describe common operational techniques to handle backpressure from downstream sinks (such as rate limiting, buffering, throttling, and circuit breakers) and how these relate to model serving components like GPU-based inference clusters.

Sample Answer

Backpressure is the condition where downstream components cannot accept data as fast as upstream producers send it; flow control are the techniques to limit, buffer, or reshape the data stream so the system remains stable and meets latency/SLO goals. In streaming systems backpressure should be propagated so producers slow down or shed load instead of overwhelming buffers and causing failures.

Common operational techniques- Rate limiting: enforce max requests/sec at ingress or per-client (e.g., token bucket). Prevents bursts reaching downstream.- Buffering/queueing: absorb short spikes with bounded queues. Useful with retry/backoff and DLQs but requires sizing to avoid OOM.- Throttling/adaptive batching: temporarily reduce concurrency or increase batch processing size to improve GPU throughput while controlling latency. Adaptive batching groups requests to maximize GPU utilization without violating latency SLOs.- Circuit breakers & health checks: open circuits when error/latency rates exceed thresholds to fail fast, trigger fallback models, or return cached responses.- Backoff & retry policies: exponential backoff for retries to prevent retry storms.- Prioritization & shedding: drop low-priority traffic or return 429 when overloaded.

How this maps to GPU-based inference clusters- GPUs prefer high throughput via batching; adaptive batching + buffering raises throughput but increases tail latency—balance is key.- Monitor queue length, GPU utilization, latency percentiles, and OOM/rejection rates. Use these metrics to auto-scale replicas or adjust batch size.- Flow-control signals: use 429/HTTP flow-control headers or gRPC stream flow control to tell clients to slow down; internally, reactors propagate backpressure to upstream components.- Circuit breakers protect GPUs from sustained overloads by diverting traffic to cheaper CPU fallbacks or degraded models.- Practical pattern: admission control at API gateway (rate limit + priority), bounded request queues per model, adaptive batching on the GPU node, and autoscaler reacting to queue length and GPU utilization.

Key trade-offs: throughput vs latency, complexity vs robustness, and cost vs performance.

MediumTechnical

128 practiced

Using Flink SQL (or compatible stream SQL), write a query that computes per-user total purchase amount in 1-minute tumbling windows based on an event_time TIMESTAMP column. Allow 2 minutes of lateness and emit accumulating updates to a sink. Given schema: purchases(user_id STRING, amount DOUBLE, event_time TIMESTAMP). Explain any assumptions and how to ensure correctness.

Sample Answer

Approach: use event-time tumbling window with 1-minute size, specify watermark allowing 2 minutes lateness, aggregate SUM(amount) per user, and emit accumulating updates (UPDATE_BEFORE/UPDATE_AFTER) by using a changelog-producing sink (e.g., upsert-kafka or print with changelog). Use CREATE VIEW with WATERMARK, then window aggregate.

sql

-- Define source with watermark allowing 2 minutes lateness
CREATE TABLE purchases_src (
  user_id STRING,
  amount DOUBLE,
  event_time TIMESTAMP(3),
  WATERMARK FOR event_time AS event_time - INTERVAL '2' MINUTE
) WITH (
  -- connector and options...
);

-- 1-minute tumbling window aggregate
CREATE VIEW user_minute_totals AS
SELECT
  user_id,
  TUMBLE_START(event_time, INTERVAL '1' MINUTE) AS window_start,
  TUMBLE_END(event_time, INTERVAL '1' MINUTE) AS window_end,
  SUM(amount) AS total_amount
FROM purchases_src
GROUP BY
  user_id,
  TUMBLE(event_time, INTERVAL '1' MINUTE);
  
-- Sink that accepts upsert/changelog to emit accumulating updates
CREATE TABLE totals_sink (
  user_id STRING,
  window_start TIMESTAMP(3),
  window_end TIMESTAMP(3),
  total_amount DOUBLE,
  PRIMARY KEY (user_id, window_start) NOT ENFORCED
) WITH (
  'connector' = 'upsert-kafka',
  -- other sink options
);

INSERT INTO totals_sink
SELECT * FROM user_minute_totals;

Key points & correctness:- Watermark = event_time - 2 minutes allows late events up to 2 minutes; later events are considered late and may be dropped or routed to side output depending on connector.- Tumbling window on event_time ensures deterministic per-minute buckets.- Use an upsert/changelog-capable sink with a primary key (user_id + window_start) so Flink emits updates (accumulating SUM) as late events arrive.- Ensure event_time is properly assigned at ingestion (no processing-time timestamps); use Kafka/Source connector that supplies event_time column or extract from payload.- Test with out-of-order events and verify watermark progression; consider a larger watermark lag if clock skew is bigger.

EasyTechnical

76 practiced

Define event time and processing time in the context of stream processing. Explain why event-time semantics matter for correctness in AI feature computation and give an example where relying on processing time would produce incorrect model features. Describe 2 mitigation strategies to handle late or out-of-order events.

Unlock Full Question Bank

Get access to hundreds of Batch and Stream Processing interview questions and detailed answers.

Join thousands of developers preparing for their dream job.