Data Engineering & Analytics Infrastructure Topics
Data pipeline design, ETL/ELT processes, streaming architectures, data warehousing infrastructure, analytics platform design, and real-time data processing. Covers event-driven systems, batch and streaming trade-offs, data quality and governance at scale, schema design for analytics, and infrastructure for big data processing. Distinct from Data Science & Analytics (which focuses on statistical analysis and insights) and from Cloud & Infrastructure (platform-focused rather than data-flow focused).
Data Warehousing and Data Lakes
Architecture of warehouses, data lakes, and lakehouses: storage-compute separation, medallion/zoned layouts, and when each is appropriate. Covers governance of a lake, table formats, and the trade-offs between warehouse-first and lake-first analytics stacks. A core infrastructure-design topic for analytics platforms.
Customer Data Platforms and Marketing Data Integration
Unifying customer data across systems: CDP and CRM integration, identity resolution, segmentation and personalization at scale, and event-driven marketing data flows. Covers stitching first-party data into a single customer view and activating it downstream. The martech/customer-data slice of analytics infrastructure.
Data Transformation and Processing Logic
Implementing transformation logic: joins, aggregations, deduplication, pivoting/reshaping, and business-rule application over datasets. Covers writing correct and maintainable transformation code, handling edge cases in the transform layer, and preparing data for downstream consumption. Focuses on the logic of turning raw data into analytics-ready outputs.
Cloud Data Platforms and Managed Services
Evaluating and choosing among managed cloud data platform PRODUCTS: cloud data warehouses (Snowflake, BigQuery, Redshift, Synapse) as vendor options, the storage-and-compute-separation model as a purchasing and operating decision, serverless versus provisioned compute models, warehouse and streaming-service sizing and capacity planning, concurrency and workload management as a platform operating concern, pricing-model comparison and platform-level cost trade-offs, vendor lock-in and portability, platform-to-platform migration, and the recurring managed-versus-self-managed decision applied to warehouses, databases, streaming, and ETL/orchestration services. Focuses on platform SELECTION and operation as a product, not designing the ingestion pipelines, ETL transform patterns, or streaming processing logic that run on top of a chosen platform, and not a single vendor's certification trivia.
Dimensional Modeling and Schema Design
Designing analytical schemas: star and snowflake schemas, fact and dimension tables, grain selection, slowly changing dimensions, and normalization vs denormalization for analytics. Covers modeling business requirements into queryable structures and schema design for large datasets. The data-modeling backbone of BI and warehouse work.
ETL and ELT Design Patterns
Trade-offs between extract-transform-load and extract-load-transform strategies, where transformation logic should live, and when to push compute into the warehouse. Covers incremental vs full loads, change-data-capture, slowly changing dimensions handling in the load path, and tooling (dbt-style transformation layers). Focuses on the processing-strategy decision rather than a specific vendor.
Data Pipeline Monitoring and Observability
Observing pipeline health: freshness, volume, schema, and distribution monitoring; lineage; alerting; and data-downtime detection. Covers instrumenting pipelines, defining SLAs/SLOs for data, and observability tooling. The operational-visibility discipline for data platforms.
Data Quality and Validation
Ensuring correctness and trust in data: validation rules, constraints, completeness/accuracy/timeliness checks, and quality frameworks. Covers designing validation into pipelines, quality gates before publishing, and handling edge cases and real-world dirty data. Central to any data engineering or analytics role.
Data Governance, Contracts, and Classification
Governing data at scale: data contracts between producers and consumers, schema evolution/compatibility, data classification and sensitivity tagging, access control, and lineage/cataloging. Covers policy, ownership, and compliance-driven controls over data. The governance layer over the technical stack.