Data Engineering & Analytics Infrastructure Topics
Data pipeline design, ETL/ELT processes, streaming architectures, data warehousing infrastructure, analytics platform design, and real-time data processing. Covers event-driven systems, batch and streaming trade-offs, data quality and governance at scale, schema design for analytics, and infrastructure for big data processing. Distinct from Data Science & Analytics (which focuses on statistical analysis and insights) and from Cloud & Infrastructure (platform-focused rather than data-flow focused).
Data Architecture and Pipelines
Designing data storage, integration, and processing architectures. Topics include relational and NoSQL database design, indexing and query optimization, replication and sharding strategies, data warehousing and dimensional modeling, ETL and ELT patterns, batch and streaming ingestion, processing frameworks, feature stores, archival and retention strategies, and trade offs for scale and latency in large data systems.
Stream Processing and Event Streaming
Designing and operating systems that ingest, process, and serve continuous event streams with low latency and high throughput. Core areas include architecture patterns for stream native and event driven systems, trade offs between batch and streaming models, and event sourcing concepts. Candidates should demonstrate knowledge of messaging and ingestion layers, message brokers and commit log systems, partitioning and consumer group patterns, partition key selection, ordering guarantees, retention and compaction strategies, and deduplication techniques. Processing concerns include stream processing engines, state stores, stateful processing, checkpointing and fault recovery, processing guarantees such as at least once and exactly once semantics, idempotence, and time semantics including event time versus processing time, watermarks, windowing strategies, late and out of order event handling, and stream to stream and stream to table joins and aggregations over windows. Performance and operational topics cover partitioning and scaling strategies, backpressure and flow control, latency versus throughput trade offs, resource isolation, monitoring and alerting, testing strategies for streaming pipelines, schema evolution and compatibility, idempotent sinks, persistent storage choices for state and checkpoints, and operational metrics such as stream lag. Familiarity with concrete technologies and frameworks is expected when discussing designs and trade offs, for example Apache Kafka, Kafka Streams, Apache Flink, Spark Structured Streaming, Amazon Kinesis, and common serialization formats such as Avro, Protocol Buffers, and JSON.
Batch and Stream Processing
Covers design and implementation of data processing using batch, stream, or hybrid approaches. Candidates should be able to explain when to choose batch versus streaming based on latency, throughput, cost, data volume, and business requirements, and compare architectural patterns such as lambda and kappa. Core stream concepts include event time versus processing time, windowing strategies such as tumbling sliding and session windows, watermarks and late arrivals, event ordering and out of order data handling, stateful versus stateless processing, state management and checkpointing, and delivery semantics including exactly once and at least once. Also includes knowledge of streaming and batch engines and runtimes, connector patterns for sources and sinks, partitioning and scaling strategies, backpressure and flow control, idempotency and deduplication techniques, testing and replayability, monitoring and alerting, and integration with storage layers such as data lakes and data warehouses. Interview focus is on reasoning about correctness latency cost and operational complexity and on concrete architecture and tooling choices.
Data and Analytics Infrastructure
Designing building and operating end to end data and analytics platforms that collect transform store and serve event product and revenue data for reporting analysis and decision making. Core areas include event instrumentation and tag management to capture user journeys marketing attribution and experimental events; data ingestion strategies and connectors; extract transform load pipelines and streaming processing; orchestration and workflow management; and choices between batch and real time architectures. Candidates must be able to design storage and serving layers including data warehouses data lakes lakehouse patterns and managed analytical databases and to choose storage formats partitioning and indexing strategies driven by volume velocity variety and access patterns. Data modeling for analytics covers raw event layers curated semantic layers dimensional modeling and metric definitions that support business intelligence and product analytics. Governance and reliability topics include data quality validation freshness monitoring lineage metadata and cataloging schema evolution master data considerations and role based access control. Operational concerns include scaling storage processing and query concurrency fault tolerance and resiliency monitoring and observability alerting cost and performance trade offs and capacity planning. Finally candidates should be able to evaluate and select tools and frameworks for orchestration stream processing and business intelligence integrate analytics platforms with downstream consumers and explain how architecture and operational choices support marketing product and business decisions while balancing tooling investment and team skills.