Technical Foundation and Self Assessment Questions
Covers baseline technical knowledge and the candidate's ability to honestly assess and communicate their technical strengths and weaknesses. Topics include fundamental infrastructure and networking concepts, operating system and protocol basics, core development and platform concepts relevant to the role, and the candidate's candid self evaluation of their depth in specific technologies. Interviewers use this to calibrate how technical the candidate is expected to be, identify areas for growth, and ensure alignment of expectations between product and engineering for collaboration.
HardTechnical
49 practiced
You're paged: nightly ETL produced zero rows in the data warehouse, but upstream systems show normal traffic. Outline immediate triage steps (check scheduler, job logs, recent deployments, schema changes), how to determine scope and impact, how to run a safe failover or backfill, and how to prepare an incident report and postmortem to prevent recurrence.
Sample Answer
Situation: I’m paged that tonight’s ETL produced zero rows in the warehouse while upstream systems show normal traffic.Immediate triage (first 15–30 minutes)- Acknowledge the page and notify stakeholders (on-call lead, analytics consumers, SRE) with current known facts and expected cadence for updates.- Check scheduler/orchestration (Airflow, Dagster, Cloud Composer): verify DAG run status, task states, start/finish times, and any paused schedules.- Inspect job logs (driver/executor logs, stdout/stderr, application logs): look for errors, exceptions, retries, schema/permission errors, or silent early exits.- Verify recent deployments/config changes: check CI/CD pipeline, infra-as-code, config repo, and any config toggles (feature flags, environment variables) deployed in the last 24–48 hours.- Check schema changes upstream and downstream: did source column names/types change, or did target table DDL change (partitioning, constraints)?- Confirm resource issues: cluster health (YARN/Kubernetes), executor failures, disk/full, IAM/token expiry.- If safe, re-run only pipeline validation steps (schema validation, sample ingestion) in prod-read mode—don’t reprocess full data yet.Determine scope & impact- Identify which pipelines/tables are affected and time range (single job, entire schedule, incremental vs full).- Query upstream systems for last-good timestamps and compare row counts by source partitions.- Check downstream consumers/reports relying on these tables and mark critical SLAs and dashboards impacted.- Estimate data volume to backfill and time required per partition.Safe failover / backfill strategy- Prefer idempotent, partitioned backfill. If ETL supports partitions (date/hour), backfill missing partitions only.- Run backfill in a staging environment first; run with dry-run / validation mode to ensure deduplication and schema compatibility.- Use snapshotting or write-to-temp-table pattern: write to temp staging tables, validate counts/checksums, then swap/merge into production (atomic rename or transactional MERGE).- Ensure deduplication keys and watermark logic prevent double-processing.- Throttle backfill to protect clusters; coordinate with SRE for capacity and monitor cost.- If immediate availability required, run a safe fallback: restore last-known good snapshot for critical tables while full backfill runs.Incident report & postmortem- Triage notes (timeline, evidence, root cause hypothesis, actions taken), impact (who/what/when), and metrics (rows missing, time to detection, time to mitigation).- Root-cause analysis: use the 5 whys, link to commits/config changes/logs, and confirm with reproductions.- Actionable corrective items: rollbacks, hotfixes, validation checks, alerting improvements (row-count/heartbeat monitors, SLA alerts), automated canaries for schema drift, CI tests for migrations, runbook updates.- Assign owners, priorities, and deadlines for each remediation; schedule follow-up review.- Share postmortem with stakeholders and incorporate learnings into deployment and testing practices.Key principles: act quickly but conservatively, avoid blind reprocessing, validate before writing to production, automate detection/prevention, and document everything with clear owners.
HardSystem Design
49 practiced
Design a deployment strategy to run stateful streaming jobs (Flink/Kafka Streams) on Kubernetes with state persistence and zero data loss during rolling upgrades. Discuss persistent volumes, durable checkpoint storage, savepoints, leader election, pod restarts, job upgrades, and how to coordinate state migration and version compatibility.
Sample Answer
Requirements & constraints:- Zero data loss, low downtime during rolling upgrades- Durable state persisted across pod/node failures- Support Flink / Kafka Streams semantics (exactly-once when possible)- Scale to many jobs, multi-tenant cluster on KubernetesHigh-level architecture:- Kubernetes for orchestration- Stateful workloads run as Deployments/StatefulSets (Flink JobManager as Deployment, TaskManagers as StatefulSet) or for Kafka Streams use StatefulSet per instance with stable network/volume- Durable checkpoint/savepoint storage on an external object store (S3/GCS/Azure Blob) + RocksDB local state persisted to PVCs for fast I/O- Use Kafka (or KafkaMirror) for input with committed offsets stored in Kafka + job stateCore components & responsibilities:1. Persistent Volumes (PVCs) - TaskManagers / Kafka Streams pods mount a PVC for local RocksDB state to avoid losing large local RocksDB snapshots on restarts. - PVCs backed by network-attached storage (e.g., AWS EBS, GCE PD) or CSI volumes with ReadWriteOnce and pod-affinity via StatefulSet to keep pod->volume mapping stable. - For horizontal scaling and preemption resilience prefer ephemeral local SSD for performance + frequent durable checkpoints to object store.2. Durable Checkpoint Storage - Configure Flink checkpointing to write to object store (s3://...) with high durability. Checkpointing frequency tuned to RPO requirements. - For Kafka Streams, enable changelog topics (compact logs) and commit offsets to Kafka; consider backing up state stores to object storage if needed.3. Savepoints & Upgrades - For planned upgrades use savepoints: trigger a savepoint (stop-with-savepoint for Flink), store location in object store, deploy new job version pointing to savepoint for state restore. - Automate savepoint creation and verification before upgrade with CI/CD pipeline. Use job restart strategy that waits for savepoint completion.4. Leader election & high availability - Run Flink JobManager in HA mode with multiple JobManagers using Kubernetes leader election (e.g., Kubernetes ConfigMap or built-in ZK/raft metadata) and checkpoint metadata in object store. Use Flink HA with Kubernetes high-availability mode. - For Kafka Streams, rely on Kafka consumer group rebalancing; provide enough replicas and graceful shutdown handlers to migrate partitions with minimal churn.5. Pod restarts & fault handling - Ensure containers handle TERM signal: on SIGTERM, trigger a checkpoint/savepoint or gracefully stop so framework can checkpoint state and commit offsets. - Liveness/readiness probes: readiness false during restore; avoid killing pod during restore window by extending termination grace period. - Configure PodDisruptionBudgets to avoid simultaneous drain of all replicas for a job.6. Rolling upgrades & zero data loss - Use blue/green or rolling upgrade with savepoints and sticky PVCs: - Option A (Flink): Take a savepoint, deploy new JobManager/TaskManagers pointing to savepoint, then stop old job. Or use in-place rolling upgrade with compatible binary changes and state migration via Flink's state backend (enable state TTL and type serializers compatibility). - Option B (Kafka Streams): Use cooperative rebalancing (since Kafka 2.4) to reduce rebalance impact; deploy new version with staged rollout and use changelog topics + offset commits to ensure no loss. - Use health checks + operator (FlinkK8sOperator or Flink Kubernetes Operator) to orchestrate savepoint creation, job shutdown, and restore steps.7. State migration & version compatibility - Follow backward/forward-compatible serialization for state (Avro/Protobuf/Serde with schema registry). - Use Flink’s state migration: provide TypeSerializer snapshots or implement a migration step in code to read old state and write new shape; perform offline compatibility tests with savepoints. - For disruptive changes, do two-step migration: deploy a compatibility shim that writes state in new format, create savepoint, then deploy new logic.Data flow & coordination:- CI/CD job triggers: 1) create savepoint, 2) verify savepoint persisted, 3) deploy new image, 4) restore job from savepoint via operator, 5) validate metrics and lag, 6) drain old pods.- Monitor checkpoints success rate, restore time, RocksDB checkpoint sizes, Kafka consumer lag.Scalability & trade-offs:- Local SSD + PVC improves performance but complicates scaling and pod relocation. Frequent durable checkpoints mitigate that.- Relying solely on PVCs without object storage risks data loss if node replacement loses local disk; object store is required for durable state.- Savepoints provide controlled upgrades but add latency; use incremental checkpoints/savepoints if supported.Best practices:- Automate savepoint and restore with Kubernetes operator.- Keep serializers stable; use schema registry and versioned schemas.- Test upgrade paths in staging with realistic state volumes.- Set graceful termination (terminationGracePeriodSeconds) > restore/flush time.- Use PodDisruptionBudgets and anti-affinity for availability.This design balances durability (object store checkpoints), performance (local RocksDB + PVCs), and operational safety (savepoints, operator orchestration, graceful shutdown) to achieve rolling upgrades with zero data loss.
EasyTechnical
39 practiced
Explain Kafka core concepts: topic, partition, broker, consumer group, offset, replication factor. Describe how Kafka provides ordering guarantees (per-partition) and what ordering it cannot provide. If you need high ingestion throughput, how do you choose the number of partitions and what trade-offs does that choice create?
Sample Answer
Topic: a named feed of messages (log) in Kafka. Producers write to topics; consumers read from them.Partition: a topic is split into ordered, immutable partitions. Each partition is an append-only log with messages assigned sequential offsets (0,1,2...).Broker: a Kafka server that stores partitions and serves producers/consumers. Partitions are distributed across brokers.Consumer group: a set of consumers that jointly consume a topic; each partition is consumed by at most one consumer in the group, enabling parallelism and automatic failover.Offset: the integer position of a message within a partition; consumers commit offsets to track progress.Replication factor: number of copies of each partition stored on different brokers for fault tolerance (one leader, others followers).Ordering guarantees:- Kafka guarantees ordering only within a partition: messages with increasing offsets are read in the same order they were written.- Kafka does NOT guarantee ordering across partitions of the same topic. If a producer round-robins or keys map to different partitions, global ordering is lost.Choosing partitions for high ingestion throughput:- More partitions -> higher parallelism (more producers/consumers and I/O concurrency) and higher aggregate throughput because multiple brokers/cores can serve writes.- Trade-offs: - Increased partitions add overhead: higher metadata, more file handles, more network connections, increased GC/heap pressure on brokers. - Rebalance time and recovery (replica sync) grow with partition count. - Per-key ordering: if you need ordering by key, ensure same key hashes to same partition; more partitions can still preserve per-key ordering but changes to partitioning or scaling may complicate reassignments. - Operational complexity: monitoring, backup, and maintenance cost rise.Practical guidance: start with partitions sized to match expected parallel consumers and broker I/O (e.g., tens–hundreds per broker for heavy loads), monitor throughput/latency, and scale partitions conservatively (repartitioning is non-trivial).
EasyTechnical
38 practiced
Compare batch processing and streaming processing for data workloads. Define latency, throughput, windowing, stateful vs stateless processing, and explain processing semantics: at-most-once, at-least-once, and exactly-once. Give two real-world use cases where streaming is required and two where batch is sufficient.
Sample Answer
Batch processing vs streaming:- Batch: processes bounded datasets in jobs (e.g., nightly ETL). High throughput, higher latency (minutes–hours). Good for heavy aggregations, historical analytics.- Streaming: processes unbounded, continuous data with low latency (ms–s). Designed for real-time insights and event-driven use cases.Key terms:- Latency: time from data arrival to result availability. Streaming optimizes for low latency; batch accepts higher latency.- Throughput: amount of data processed per unit time. Batch often achieves higher aggregate throughput; streaming sustains continuous throughput.- Windowing: technique to group events over time in streams (tumbling, sliding, session windows) so you can compute aggregates on unbounded data.- Stateful vs stateless: stateless ops (map, filter) don’t keep history; stateful ops (joins, aggregations, sessionization) maintain state across events and need fault-tolerant state management.Processing semantics:- At-most-once: events processed zero or one time (possible data loss) — low overhead.- At-least-once: events processed one or more times (duplicates possible) — common default for performance.- Exactly-once: guarantees single logical effect per event across failures — strongest guarantee, higher complexity (idempotence, checkpointing, transactional sinks).When streaming is required:1. Real-time fraud detection (must react instantly to transactions).2. Live personalization/feature updates for recommendations (user experience depends on immediate signals).When batch is sufficient:1. End-of-day financial reporting, large-scale ETL into a warehouse.2. Machine-learning feature engineering that retrains models daily/weekly.As a data engineer, choose based on SLA (latency), consistency needs, cost, and operational complexity.
HardSystem Design
37 practiced
Design a dimensional model (star schema) for e-commerce orders to support cohort analysis and time-series metrics. Define fact tables (orders, order_items) and dimension tables (user, product, promotion). Show how to support SCD Type 2 for user attributes, handle returns/cancellations, and ensure queries for cohorts and lifetime value are efficient.
Sample Answer
Requirements:- Support cohort analysis (acquisition cohorts by signup/date, behavioral cohorts by first order)- Time-series metrics (daily/monthly orders, GMV, refunds)- Lifetime Value (LTV) and retention calculations- SCD Type 2 for user attributes (email, plan, region)- Track order lifecycle including returns/cancellations- Efficient at scale (partitioning, indexing, pre-aggregates)High-level star schemaFact: order_fact (one row per order)- surrogate_order_id (PK)- user_sk (FK → user_dim.surrogate_user_sk at order_time)- order_date (date), order_ts (timestamp)- channel, currency, order_status (Completed, Cancelled, Returned, Partially_Returned)- order_total_amount, order_total_cost, order_discount_amount, order_net_amount- refund_amount, cancellation_flag- first_order_flag, acquisition_cohort_key (derived)- load_date, valid_from_ts (for auditing)Fact: order_item_fact (one row per order_line)- surrogate_order_item_id (PK)- surrogate_order_id (FK)- product_sk (FK → product_dim)- sku, qty, unit_price, line_discount, line_net_amount- return_qty, return_amount- promotion_sk (nullable FK → promotion_dim)- load_dateDimension: user_dim (SCD Type 2)- surrogate_user_sk (PK)- user_id (natural key)- email, plan, region, segment- effective_from_ts, effective_to_ts, is_current (boolean)- signup_date, acquisition_sourceTo assign user_sk at order time: lookup user_dim where user_id = order.user_id AND effective_from_ts <= order_ts < effective_to_ts.Dimension: product_dim (SCD Type 1 or 2 depending on need)- product_sk, product_id, name, category, brand, price, is_active, effective_* if SCD2Dimension: promotion_dim- promotion_sk, promotion_id, type, discount_pct, start_date, end_dateSCD Type 2 handling- Maintain effective date ranges and is_current flag.- During ETL: when user attributes change, insert new user_dim row with new effective_from_ts and set previous effective_to_ts; backfill if necessary.- Use surrogate keys stored on facts so historical analysis uses snapshot of user/product attributes at order time.Handling returns & cancellations- Represent status on order_fact and order_item_fact (return_qty, refund_amount).- For partial returns create a new adjustment record: order_item_fact with negative qty or a separate order_adjustment_fact referencing original line to simplify auditing.- Keep original order rows immutable; record adjustments with timestamps and link to original order to compute metrics over time.Optimizations for cohorts & LTV- Partition order_fact and order_item_fact by order_date (day/month) and cluster by user_sk.- Create materialized views / aggregate tables: - daily_user_revenue (user_sk, date, revenue, refunds) for fast LTV rolls - cohort_retention_table (acquisition_cohort, day_since_acq, users_active, revenue)- Precompute first_order_date per user in a user_first_order table to speed cohort assignment.- Use columnar storage (Snowflake/BigQuery/Redshift Spectrum) with compression and predicate pushdown.Indexes & storage- Build secondary indexes on (user_sk, order_date), (acquisition_cohort_key, order_date)- Use surrogate integer keys to reduce storage- Use clustering keys on user_sk for LTV queriesSample cohort query pattern- Join order_fact → user_first_order (to get cohort_date) → aggregate revenue by date - cohort_date to produce days-since-cohort series; use pre-aggregates where possible.Trade-offs & notes- SCD2 increases storage but provides correct historical attribution—essential for cohort/LTV.- Materialized aggregates speed queries but need ETL refresh logic (daily incremental).- For heavy historical adjustments, maintain a reconciliation pipeline to re-derive aggregates if business rules change.This design balances correctness (SCD2 snapshots), auditability (immutable facts + adjustments), and performance (partitioning, aggregates) for cohort and LTV analytics at scale.
Unlock Full Question Bank
Get access to hundreds of Technical Foundation and Self Assessment interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.