Experimentation Platforms and Infrastructure

Infrastructure for A/B testing and experimentation: assignment/bucketing, metric pipelines for experiments, guardrail and variance-reduction plumbing, and experiment result storage. Covers building the platform that powers trustworthy online experiments at scale. Distinct from the statistics of experiment analysis.

77 questions

Data Reliability and Fault Tolerance

Designing pipelines that survive failures: retries, idempotency, checkpointing, exactly-once semantics, dead-letter handling, and recovery/replay. Covers reasoning about partial failures, poison messages, and consistency guarantees under faults. The resilience angle distinct from monitoring (detecting and alerting on a failure) and from Workflow Orchestration and Scheduling (the DAG/scheduler mechanics that decide whether and when a task runs again, including backfills and dependency management): this topic owns whether the data itself stays correct, not lost, not duplicated, not corrupted, when a process is retried or replayed.

50 questions

Data Warehousing and Data Lakes

Architecture of warehouses, data lakes, and lakehouses: storage-compute separation, medallion/zoned layouts, and when each is appropriate. Covers governance of a lake, table formats, and the trade-offs between warehouse-first and lake-first analytics stacks. A core infrastructure-design topic for analytics platforms.

22 questions

Customer Data Platforms and Marketing Data Integration

Unifying customer data across systems: CDP and CRM integration, identity resolution, segmentation and personalization at scale, and event-driven marketing data flows. Covers stitching first-party data into a single customer view and activating it downstream. The martech/customer-data slice of analytics infrastructure.

39 questions

ML Feature Pipelines and Feature Stores

Data infrastructure for machine learning: feature pipelines, feature stores, online/offline consistency, training-serving skew, and data preparation for models. Covers building reliable feature platforms and preventing leakage in the data path feeding models. The data-engineering-for-ML topic.

46 questions

Data Transformation and Processing Logic

Implementing transformation logic: joins, aggregations, deduplication, pivoting/reshaping, and business-rule application over datasets. Covers writing correct and maintainable transformation code, handling edge cases in the transform layer, and preparing data for downstream consumption. Focuses on the logic of turning raw data into analytics-ready outputs.

48 questions

Data Ingestion and Source System Integration

Getting data out of heterogeneous source systems and landing it reliably: APIs, operational databases, file drops, webhooks, message queues and third-party SaaS. Covers connector selection and design (managed platforms versus Debezium, DMS or Kafka Connect versus building your own), pull versus push and polling versus webhook patterns, incremental extraction and high-watermark strategy including what to do when a source offers no native change capture, authentication and credential rotation against third-party APIs, source-side rate limits and quotas, schema drift and contract breakage at the source boundary, backfill and replay of history, ingestion-time data-quality gates, reconciliation after a source outage, and negotiating with source-system owners. The scope stops at the boundary: once data has landed, transforming it, the architecture of the pipeline that carries it, stream-processing mechanics, and pipeline monitoring are all covered separately.

26 questions

Storage Formats, Partitioning, and Serialization

Physical storage layout for analytics data. Covers columnar and row format internals and their trade-offs (Parquet row groups and column chunks, ORC stripes, Avro), serialization choices, compression codec and column-encoding selection, partitioning and clustering or bucketing strategy including partition-key choice and pruning, the small-file problem and compaction, file-size and row-group tuning, and open table formats (Iceberg, Delta Lake, Hudi) at the layout and metadata level: ACID commits, snapshots and time travel, manifest and metadata-tree structure, schema and partition evolution, and the catalog that tracks them. The scope is how data is laid out on disk and what that layout makes cheap or expensive. Tuning the queries that read it, and choosing which managed platform to run it on, are covered separately.

74 questions

Cloud Data Platforms and Managed Services

Evaluating and choosing among managed cloud data platform PRODUCTS: cloud data warehouses (Snowflake, BigQuery, Redshift, Synapse) as vendor options, the storage-and-compute-separation model as a purchasing and operating decision, serverless versus provisioned compute models, warehouse and streaming-service sizing and capacity planning, concurrency and workload management as a platform operating concern, pricing-model comparison and platform-level cost trade-offs, vendor lock-in and portability, platform-to-platform migration, and the recurring managed-versus-self-managed decision applied to warehouses, databases, streaming, and ETL/orchestration services. Focuses on platform SELECTION and operation as a product, not designing the ingestion pipelines, ETL transform patterns, or streaming processing logic that run on top of a chosen platform, and not a single vendor's certification trivia.

36 questions

Dimensional Modeling and Schema Design

Designing analytical schemas: star and snowflake schemas, fact and dimension tables, grain selection, slowly changing dimensions, and normalization vs denormalization for analytics. Covers modeling business requirements into queryable structures and schema design for large datasets. The data-modeling backbone of BI and warehouse work.

76 questions

ETL and ELT Design Patterns

Trade-offs between extract-transform-load and extract-load-transform strategies, where transformation logic should live, and when to push compute into the warehouse. Covers incremental vs full loads, change-data-capture, slowly changing dimensions handling in the load path, and tooling (dbt-style transformation layers). Focuses on the processing-strategy decision rather than a specific vendor.

30 questions

Data Pipeline Monitoring and Observability

Observing pipeline health: freshness, volume, schema, and distribution monitoring; lineage; alerting; and data-downtime detection. Covers instrumenting pipelines, defining SLAs/SLOs for data, and observability tooling. The operational-visibility discipline for data platforms.

68 questions

Data Quality and Validation

Ensuring correctness and trust in data: validation rules, constraints, completeness/accuracy/timeliness checks, and quality frameworks. Covers designing validation into pipelines, quality gates before publishing, and handling edge cases and real-world dirty data. Central to any data engineering or analytics role.

63 questions

Data Governance, Contracts, and Classification

Governing data at scale: data contracts between producers and consumers, schema evolution/compatibility, data classification and sensitivity tagging, access control, and lineage/cataloging. Covers policy, ownership, and compliance-driven controls over data. The governance layer over the technical stack.

70 questions

Workflow Orchestration and Scheduling

Orchestrating multi-step data workflows with DAG schedulers (Airflow, Dagster, and similar tools): dependency management between tasks, scheduling strategies (cron-based, sensor and trigger patterns, event-driven runs), and backfills or catch-up runs for time-partitioned data. Covers task-level retries and idempotent task design, so a scheduler can safely re-run a failed step, plus SLA tracking and alerting when a run is late or missing. The core concern is coordination: given a set of dependent tasks that must run in some order on some schedule, how do you trigger, sequence, and re-run them reliably. This is distinct from whether the data itself stays correct across a failure (exactly-once processing, deduplication, checkpointing, and dead-letter handling for corrupted or poison messages, which is a data-consistency concern) and from how a specific compute engine executes a task internally (Spark or Hadoop mechanics). The operational glue of a data platform: getting the right task to run at the right time, in the right order, with visibility into failures.

84 questions

Stream Processing and Event Streaming

Building on event-streaming platforms: Kafka and message queues, event sourcing, partitioning, consumer groups, exactly-once vs at-least-once delivery, and windowing. Covers handling late and out-of-order events, watermarks, and stateful stream operators. The core skill for real-time data engineering.

79 questions

Batch, Streaming, and Real-Time Serving Trade-offs

Reasoning about when to use batch, micro-batch, or continuous streaming and how to serve low-latency analytics: latency, cost, complexity, and correctness trade-offs; lambda vs kappa architectures; and reprocessing semantics. Covers real-time aggregation, freshness vs consistency trade-offs, and reconciling streaming results with batch ground truth, including geospatial and high-throughput real-time workloads under eventual consistency. The data-systems judgment topic for choosing and reconciling batch versus real-time approaches, distinct from the hands-on streaming transport itself.

21 questions

Data Pipeline Architecture and Design

End-to-end design of data pipelines: source-to-sink flow, staging layers, idempotency, backfills, and reprocessing. Covers choosing between batch and streaming stages, decoupling ingestion from transformation, and designing for evolvability. The foundational systems-design skill for a data engineering interview.

46 questions

Business Intelligence, Reporting, and Dashboards

The reporting and presentation layer of analytics: semantic/metrics layers, report development and automation, self-service BI, and the architecture that feeds dashboards and reports. Covers dashboard and visualization design (tool selection across Tableau/Power BI/Looker-style platforms, drill-downs, information architecture, communicating metrics visually), refresh strategies, and query performance for interactive reporting workloads. Spans both the engineering behind the reporting layer and the design of the dashboards that consume it.

34 questions

Metric Definition and Implementation

Defining and computing business metrics correctly: single-source-of-truth metric definitions, handling edge cases (dedup, attribution windows, timezones), and reconciling real-time vs batch metric values. Covers metric governance and translating business questions into precise, reproducible calculations. A high-frequency analytics-interview topic.

89 questions

Data Platform Architecture and Technology Selection

System-level design of an end-to-end data platform: component selection, build-vs-buy, tool trade-offs, and aligning platform architecture with organizational and analytics needs. Covers reasoning about the whole stack (ingestion through serving) and technology-choice justification. The architect-altitude view above any single pipeline.

35 questions

Data Pipeline Scalability and Performance

Diagnosing and fixing throughput, cost, and capacity problems in data pipelines: batch ETL/ELT jobs and streaming ingestion that must handle growing data volume within cost and SLA constraints. Covers bottleneck diagnosis (I/O-bound versus CPU-bound stages, profiling a slow pipeline); partition and key design as a load-distribution DECISION for a processing job (choosing keys, avoiding and mitigating hot partitions); recognizing when a transformation forces an expensive shuffle or join and choosing a mitigation strategy at the decision level, not engine-internal tuning; capacity planning anchored in real data volume (cluster, storage, and network sizing from an event rate and payload profile); and throughput techniques including incremental processing, result caching and materialization, storage-format and compaction choices for throughput, and cost-efficient scaling (autoscaling policy, spot/reserved capacity trade-offs). Also covers multi-tenant resource governance for shared data-processing infrastructure (quotas, isolation, throughput enforcement). Distinct from distributed-data-processing-with-spark-and-hadoop (engine-internal execution mechanics of a specific framework), data-reliability-and-fault-tolerance (correctness and consistency guarantees under failure, e.g. exactly-once semantics and checkpointing), workflow-orchestration-and-scheduling (DAG/dependency coordination), and general application or service performance tuning (web request latency, generic distributed-systems architecture with no data-pipeline framing).

52 questions

Analytical Query Performance and Optimization

Diagnosing and tuning analytical queries on distributed and warehouse engines. Covers reading and reasoning about query plans, join strategy and ordering including broadcast versus shuffle, statistics and cardinality estimation, materialized views and result caching, skew and spill diagnosis, workload management and concurrency, and the cost side of query performance on engines billed by bytes scanned or compute time. The altitude is the deep, engine-specific operational layer: what a real plan on a real warehouse engine is telling you and what to change. Generic single-engine query-tuning fundamentals, the physical layout of the data being scanned, and the choice of which managed platform to run on are covered separately.

46 questions

Product Analytics Instrumentation and Event Tracking

Instrumenting products to collect behavioral data: event taxonomy/tracking plans, client and server-side collection, attribution implementation, and telemetry for web, mobile, and games (including crash reporting). Covers designing clean, analyzable event schemas and the collection infrastructure behind them. The data-collection foundation for product analytics.

63 questions

SQL-Based Data Cleaning and Anomaly Detection

Using SQL to profile, clean, and validate data: detecting duplicates, nulls, outliers, and referential-integrity breaks; writing production-safe diagnostic queries; and root-cause debugging of missing or incorrect data. Covers anomaly-detection queries and data cleaning for analytics. The hands-on SQL data-quality skill.

54 questions

Distributed Data Processing with Spark and Hadoop

Distributed compute engines for large-scale data processing, with Apache Spark as the primary focus: driver/executor architecture, the RDD/DataFrame/Dataset APIs, lazy evaluation and the Catalyst/Tungsten optimizers, shuffle and partitioning, data skew detection and mitigation (salting, broadcast joins, adaptive query execution), memory tuning, and caching/persistence strategies. Covers engine-internal execution and recovery within a single job: how the DAG scheduler breaks work into stages and tasks, task retries, speculative execution, stage recomputation from RDD lineage, and Structured Streaming's own checkpointing and write-ahead logs for exactly-once semantics. Also covers the Hadoop ecosystem as historical foundation (HDFS, MapReduce, YARN, Hive) and modern lakehouse table formats (Delta Lake, Iceberg) that have largely replaced it in current interview practice. The distributed-processing depth expected of data engineers at scale: writing and tuning jobs, diagnosing OOMs and skew, and choosing the right engine, storage format, and cluster configuration for a workload. Distinct from workflow-orchestration-and-scheduling, which owns the external, Airflow-style scheduler that invokes jobs like these as steps in a larger multi-system pipeline, and from data-reliability-and-fault-tolerance, which owns designing idempotent, cross-system pipeline behavior around job failures rather than the engine's own internal recovery mechanics.

112 questions

Data Engineering & Analytics Infrastructure Topics

Experimentation Platforms and Infrastructure

Data Reliability and Fault Tolerance

Data Warehousing and Data Lakes

Customer Data Platforms and Marketing Data Integration

ML Feature Pipelines and Feature Stores

Data Transformation and Processing Logic

Data Ingestion and Source System Integration

Storage Formats, Partitioning, and Serialization

Cloud Data Platforms and Managed Services

Dimensional Modeling and Schema Design

ETL and ELT Design Patterns

Data Pipeline Monitoring and Observability

Data Quality and Validation

Data Governance, Contracts, and Classification

Workflow Orchestration and Scheduling

Stream Processing and Event Streaming

Batch, Streaming, and Real-Time Serving Trade-offs

Data Pipeline Architecture and Design

Business Intelligence, Reporting, and Dashboards

Metric Definition and Implementation

Data Platform Architecture and Technology Selection

Data Pipeline Scalability and Performance

Analytical Query Performance and Optimization

Product Analytics Instrumentation and Event Tracking

SQL-Based Data Cleaning and Anomaly Detection

Distributed Data Processing with Spark and Hadoop