Data Engineering & Analytics Infrastructure Topics
Data pipeline design, ETL/ELT processes, streaming architectures, data warehousing infrastructure, analytics platform design, and real-time data processing. Covers event-driven systems, batch and streaming trade-offs, data quality and governance at scale, schema design for analytics, and infrastructure for big data processing. Distinct from Data Science & Analytics (which focuses on statistical analysis and insights) and from Cloud & Infrastructure (platform-focused rather than data-flow focused).
Experimentation Platforms and Infrastructure
Infrastructure for A/B testing and experimentation: assignment/bucketing, metric pipelines for experiments, guardrail and variance-reduction plumbing, and experiment result storage. Covers building the platform that powers trustworthy online experiments at scale. Distinct from the statistics of experiment analysis.
Data Warehousing and Data Lakes
Architecture of warehouses, data lakes, and lakehouses: storage-compute separation, medallion/zoned layouts, and when each is appropriate. Covers governance of a lake, table formats, and the trade-offs between warehouse-first and lake-first analytics stacks. A core infrastructure-design topic for analytics platforms.
Customer Data Platforms and Marketing Data Integration
Unifying customer data across systems: CDP and CRM integration, identity resolution, segmentation and personalization at scale, and event-driven marketing data flows. Covers stitching first-party data into a single customer view and activating it downstream. The martech/customer-data slice of analytics infrastructure.
ML Feature Pipelines and Feature Stores
Data infrastructure for machine learning: feature pipelines, feature stores, online/offline consistency, training-serving skew, and data preparation for models. Covers building reliable feature platforms and preventing leakage in the data path feeding models. The data-engineering-for-ML topic.
Data Ingestion and Source System Integration
Getting data out of heterogeneous source systems and landing it reliably: APIs, operational databases, file drops, webhooks, message queues and third-party SaaS. Covers connector selection and design (managed platforms versus Debezium, DMS or Kafka Connect versus building your own), pull versus push and polling versus webhook patterns, incremental extraction and high-watermark strategy including what to do when a source offers no native change capture, authentication and credential rotation against third-party APIs, source-side rate limits and quotas, schema drift and contract breakage at the source boundary, backfill and replay of history, ingestion-time data-quality gates, reconciliation after a source outage, and negotiating with source-system owners. The scope stops at the boundary: once data has landed, transforming it, the architecture of the pipeline that carries it, stream-processing mechanics, and pipeline monitoring are all covered separately.
Data Quality and Validation
Ensuring correctness and trust in data: validation rules, constraints, completeness/accuracy/timeliness checks, and quality frameworks. Covers designing validation into pipelines, quality gates before publishing, and handling edge cases and real-world dirty data. Central to any data engineering or analytics role.
Data Governance, Contracts, and Classification
Governing data at scale: data contracts between producers and consumers, schema evolution/compatibility, data classification and sensitivity tagging, access control, and lineage/cataloging. Covers policy, ownership, and compliance-driven controls over data. The governance layer over the technical stack.
Batch, Streaming, and Real-Time Serving Trade-offs
Reasoning about when to use batch, micro-batch, or continuous streaming and how to serve low-latency analytics: latency, cost, complexity, and correctness trade-offs; lambda vs kappa architectures; and reprocessing semantics. Covers real-time aggregation, freshness vs consistency trade-offs, and reconciling streaming results with batch ground truth, including geospatial and high-throughput real-time workloads under eventual consistency. The data-systems judgment topic for choosing and reconciling batch versus real-time approaches, distinct from the hands-on streaming transport itself.
Data Pipeline Architecture and Design
End-to-end design of data pipelines: source-to-sink flow, staging layers, idempotency, backfills, and reprocessing. Covers choosing between batch and streaming stages, decoupling ingestion from transformation, and designing for evolvability. The foundational systems-design skill for a data engineering interview.