Data Engineering & Analytics Infrastructure Topics
Data pipeline design, ETL/ELT processes, streaming architectures, data warehousing infrastructure, analytics platform design, and real-time data processing. Covers event-driven systems, batch and streaming trade-offs, data quality and governance at scale, schema design for analytics, and infrastructure for big data processing. Distinct from Data Science & Analytics (which focuses on statistical analysis and insights) and from Cloud & Infrastructure (platform-focused rather than data-flow focused).
Real Time and Batch Ingestion
Focuses on choosing between batch ingestion and real time streaming for moving data from sources to storage and downstream systems. Topics include latency and throughput requirements, cost and operational complexity, consistency and delivery semantics such as at least once and exactly once, idempotent and deduplication strategies, schema evolution, connector and source considerations, backpressure and buffering, checkpointing and state management, and tooling choices for streaming and batch. Candidates should be able to design hybrid architectures that combine streaming for low latency needs with batch pipelines for large backfills or heavy aggregations and explain operational trade offs such as monitoring, scaling, failure recovery, and debugging.
Data Architecture and Pipelines
Designing data storage, integration, and processing architectures. Topics include relational and NoSQL database design, indexing and query optimization, replication and sharding strategies, data warehousing and dimensional modeling, ETL and ELT patterns, batch and streaming ingestion, processing frameworks, feature stores, archival and retention strategies, and trade offs for scale and latency in large data systems.
Stream Processing and Event Streaming
Designing and operating systems that ingest, process, and serve continuous event streams with low latency and high throughput. Core areas include architecture patterns for stream native and event driven systems, trade offs between batch and streaming models, and event sourcing concepts. Candidates should demonstrate knowledge of messaging and ingestion layers, message brokers and commit log systems, partitioning and consumer group patterns, partition key selection, ordering guarantees, retention and compaction strategies, and deduplication techniques. Processing concerns include stream processing engines, state stores, stateful processing, checkpointing and fault recovery, processing guarantees such as at least once and exactly once semantics, idempotence, and time semantics including event time versus processing time, watermarks, windowing strategies, late and out of order event handling, and stream to stream and stream to table joins and aggregations over windows. Performance and operational topics cover partitioning and scaling strategies, backpressure and flow control, latency versus throughput trade offs, resource isolation, monitoring and alerting, testing strategies for streaming pipelines, schema evolution and compatibility, idempotent sinks, persistent storage choices for state and checkpoints, and operational metrics such as stream lag. Familiarity with concrete technologies and frameworks is expected when discussing designs and trade offs, for example Apache Kafka, Kafka Streams, Apache Flink, Spark Structured Streaming, Amazon Kinesis, and common serialization formats such as Avro, Protocol Buffers, and JSON.
Data Pipeline Architecture
Design end to end data pipeline solutions from problem statement through implementation and operations, integrating ingestion transformation storage serving and consumption layers. Topics include source selection and connectors, ingestion patterns including batch streaming and micro batch, transformation steps such as cleaning enrichment aggregation and filtering, and loading targets such as analytic databases data warehouses data lakes or operational stores. Cover architecture patterns and trade offs including lambda kappa and micro batch, delivery semantics and fault tolerance, partitioning and scaling strategies, schema evolution and data modeling for analytic and operational consumers, and choices driven by freshness latency throughput cost and operational complexity. Operational concerns include orchestration and scheduling, reliability considerations such as error handling retries idempotence and backpressure, monitoring and alerting, deployment and runbook planning, and how components work together as a coherent maintainable system. Interview focus is on turning requirements into concrete architectures, technology selection, and trade off reasoning.
Data Lake and Warehouse Architecture
Designing scalable data platforms for analytical and reporting workloads including data lakes, data warehouses, and lakehouse architectures. Key topics include storage formats and layout including columnar file formats such as Parquet and table formats such as Iceberg and Delta Lake, partitioning and compaction strategies, metadata management and cataloging, schema evolution and transactional guarantees for analytical data, and cost and performance trade offs. Cover ingestion patterns for batch and streaming data including change data capture, data transformation approaches and compute engines for analytical queries, partition pruning and predicate pushdown, query optimization and materialized views, data modeling for analytical workloads, retention and tiering, security and access control, data governance and lineage, and integration with business intelligence and real time analytics. Also discuss operational concerns such as monitoring, vacuuming and compaction jobs, metadata scaling, and strategies for minimizing query latency while controlling storage cost.
Data Processing and Transformation
Focuses on algorithmic and engineering approaches to transform and clean data at scale. Includes deduplication strategies, parsing and normalizing unstructured or semi structured data, handling missing or inconsistent values, incremental and chunked processing for large datasets, batch versus streaming trade offs, state management, efficient memory and compute usage, idempotency and error handling, and techniques for scaling and parallelizing transformation pipelines. Interviewers may assess problem solving, choice of algorithms and data structures, and pragmatic design for reliability and performance.
Data Quality and Governance
Covers the principles, frameworks, practices, and tooling used to ensure data is accurate, complete, timely, and trustworthy across systems and pipelines. Key areas include data quality checks and monitoring such as nullness and type checks, freshness and timeliness validation, referential integrity, deduplication, outlier detection, reconciliation, and automated alerting. Includes design of service level agreements for data freshness and accuracy, data lineage and impact analysis, metadata and catalog management, data classification, access controls, and compliance policies. Encompasses operational reliability of data systems including failure handling, recovery time objectives, backup and disaster recovery strategies, observability and incident response for data anomalies. Also covers domain and system specific considerations such as customer relationship management and sales systems: common causes of data problems, prevention strategies like input validation rules, canonicalization, deduplication and training, and business impact on forecasting and operations. Candidates may be evaluated on designing end to end data quality programs, selecting metrics and tooling, defining roles and stewardship, and implementing automated pipelines and governance controls.
Data and Analytics Infrastructure
Designing building and operating end to end data and analytics platforms that collect transform store and serve event product and revenue data for reporting analysis and decision making. Core areas include event instrumentation and tag management to capture user journeys marketing attribution and experimental events; data ingestion strategies and connectors; extract transform load pipelines and streaming processing; orchestration and workflow management; and choices between batch and real time architectures. Candidates must be able to design storage and serving layers including data warehouses data lakes lakehouse patterns and managed analytical databases and to choose storage formats partitioning and indexing strategies driven by volume velocity variety and access patterns. Data modeling for analytics covers raw event layers curated semantic layers dimensional modeling and metric definitions that support business intelligence and product analytics. Governance and reliability topics include data quality validation freshness monitoring lineage metadata and cataloging schema evolution master data considerations and role based access control. Operational concerns include scaling storage processing and query concurrency fault tolerance and resiliency monitoring and observability alerting cost and performance trade offs and capacity planning. Finally candidates should be able to evaluate and select tools and frameworks for orchestration stream processing and business intelligence integrate analytics platforms with downstream consumers and explain how architecture and operational choices support marketing product and business decisions while balancing tooling investment and team skills.
Azure Data Platforms (Synapse, Data Lake Storage, Data Factory)
Understanding Azure's data ecosystem: Synapse for data warehousing with both dedicated and serverless SQL pools, Data Lake Storage Gen2 for enterprise data lakes, Data Factory for orchestration. Understanding how components integrate and when to use each for different workloads.