Data Engineering & Analytics Infrastructure Topics
Data pipeline design, ETL/ELT processes, streaming architectures, data warehousing infrastructure, analytics platform design, and real-time data processing. Covers event-driven systems, batch and streaming trade-offs, data quality and governance at scale, schema design for analytics, and infrastructure for big data processing. Distinct from Data Science & Analytics (which focuses on statistical analysis and insights) and from Cloud & Infrastructure (platform-focused rather than data-flow focused).
Data Warehousing and Data Lakes
Architecture of warehouses, data lakes, and lakehouses: storage-compute separation, medallion/zoned layouts, and when each is appropriate. Covers governance of a lake, table formats, and the trade-offs between warehouse-first and lake-first analytics stacks. A core infrastructure-design topic for analytics platforms.
Customer Data Platforms and Marketing Data Integration
Unifying customer data across systems: CDP and CRM integration, identity resolution, segmentation and personalization at scale, and event-driven marketing data flows. Covers stitching first-party data into a single customer view and activating it downstream. The martech/customer-data slice of analytics infrastructure.
Data Ingestion and Source System Integration
Getting data out of heterogeneous source systems and landing it reliably: APIs, operational databases, file drops, webhooks, message queues and third-party SaaS. Covers connector selection and design (managed platforms versus Debezium, DMS or Kafka Connect versus building your own), pull versus push and polling versus webhook patterns, incremental extraction and high-watermark strategy including what to do when a source offers no native change capture, authentication and credential rotation against third-party APIs, source-side rate limits and quotas, schema drift and contract breakage at the source boundary, backfill and replay of history, ingestion-time data-quality gates, reconciliation after a source outage, and negotiating with source-system owners. The scope stops at the boundary: once data has landed, transforming it, the architecture of the pipeline that carries it, stream-processing mechanics, and pipeline monitoring are all covered separately.
Cloud Data Platforms and Managed Services
Evaluating and choosing among managed cloud data platform PRODUCTS: cloud data warehouses (Snowflake, BigQuery, Redshift, Synapse) as vendor options, the storage-and-compute-separation model as a purchasing and operating decision, serverless versus provisioned compute models, warehouse and streaming-service sizing and capacity planning, concurrency and workload management as a platform operating concern, pricing-model comparison and platform-level cost trade-offs, vendor lock-in and portability, platform-to-platform migration, and the recurring managed-versus-self-managed decision applied to warehouses, databases, streaming, and ETL/orchestration services. Focuses on platform SELECTION and operation as a product, not designing the ingestion pipelines, ETL transform patterns, or streaming processing logic that run on top of a chosen platform, and not a single vendor's certification trivia.
Dimensional Modeling and Schema Design
Designing analytical schemas: star and snowflake schemas, fact and dimension tables, grain selection, slowly changing dimensions, and normalization vs denormalization for analytics. Covers modeling business requirements into queryable structures and schema design for large datasets. The data-modeling backbone of BI and warehouse work.
Data Governance, Contracts, and Classification
Governing data at scale: data contracts between producers and consumers, schema evolution/compatibility, data classification and sensitivity tagging, access control, and lineage/cataloging. Covers policy, ownership, and compliance-driven controls over data. The governance layer over the technical stack.
Stream Processing and Event Streaming
Building on event-streaming platforms: Kafka and message queues, event sourcing, partitioning, consumer groups, exactly-once vs at-least-once delivery, and windowing. Covers handling late and out-of-order events, watermarks, and stateful stream operators. The core skill for real-time data engineering.
Batch, Streaming, and Real-Time Serving Trade-offs
Reasoning about when to use batch, micro-batch, or continuous streaming and how to serve low-latency analytics: latency, cost, complexity, and correctness trade-offs; lambda vs kappa architectures; and reprocessing semantics. Covers real-time aggregation, freshness vs consistency trade-offs, and reconciling streaming results with batch ground truth, including geospatial and high-throughput real-time workloads under eventual consistency. The data-systems judgment topic for choosing and reconciling batch versus real-time approaches, distinct from the hands-on streaming transport itself.
Data Pipeline Architecture and Design
End-to-end design of data pipelines: source-to-sink flow, staging layers, idempotency, backfills, and reprocessing. Covers choosing between batch and streaming stages, decoupling ingestion from transformation, and designing for evolvability. The foundational systems-design skill for a data engineering interview.