Evaluation of deep, domain specific technical knowledge relevant to the candidate's own team, product, or problem space, whatever that domain is. Candidates should demonstrate subject matter expertise in their problem space and be able to explain core concepts, architectures or system designs, domain specific algorithms or methods, and practical trade offs. The specific domain varies by role and industry: it might be recommendation systems and data platforms for a tech company, claims and underwriting systems for insurance, supply chain and logistics platforms, payment and settlement rails for fintech, clinical or health record systems for healthcare, or content and production pipelines for media. Expect questions on domain specific data flows and integration patterns, versioning and change management strategies, common customer or user workflows, typical pain points in that domain, and how domain constraints shape day to day priorities and decisions. For product facing roles, be ready to explain core product features, typical customer workflows, integration points, and how domain constraints influence product decisions. For engineering, platform, or delivery focused roles, describe how the domain shapes responsibilities and challenges, and outline an approach to initial discovery, diagnosis, and early improvements when picking up an unfamiliar part of that domain. This topic tests both conceptual depth in the candidate's actual domain and the ability to map that domain knowledge to concrete product and engineering decisions.
HardTechnical
64 practiced
Design an approach to handle PII across data pipelines: identify PII, decide between tokenization and pseudonymization, plan key management and encryption in transit and at rest, implement least-privilege access policy, and maintain audit logs. Discuss trade-offs between analytic utility and privacy risk.
Sample Answer
Situation/Goal: Build a repeatable, secure pattern so data pipelines can safely ingest, process and serve PII while balancing analytic utility and privacy risk.Approach (high-level):1. Identify & classify PII- Automated classifiers: regex + rule engines for SSNs/emails, ML/NLP for names/free text.- Integrate with a data catalog (e.g., AWS Macie, Google DLP, or an open-source tool) to tag columns/fields with sensitivity levels and retention policy.2. Decide transformation: tokenization vs pseudonymization- Tokenization (reversible, stored mapping in vault): use when downstream needs to re-identify or join records across systems (e.g., customers across services). Prefer format-preserving/token vaults (Vault, Thales, cloud tokenization) with HSM-backed keys.- Pseudonymization (irreversible or one-way hashing with salt): use when you need consistent identifiers for analytics but not reidentification (e.g., cohorting). Use keyed HMAC or salted hash for deterministic joins; use non-deterministic hashing when joins are not needed.- Consider differential privacy or noise-addition for aggregate outputs to reduce reidentification risk for analytics.3. Key management & encryption- Centralized KMS/HSM (AWS KMS + CloudHSM, Azure Key Vault, Google KMS) for master keys, with CMKs for envelope encryption.- Envelope encryption: data encrypted with DEKs; DEKs encrypted by KMS. Store DEKs with metadata, rotate CMKs periodically (rotate CMKs and re-encrypt DEKs).- Access to key material limited to tokenization service; never embed keys in code or configs. Use least-privilege IAM for KMS API calls and CloudHSM admin operations.4. Encryption in transit & at rest- TLS >=1.2 with certificate pinning between services.- At rest: SSE-KMS for object stores (S3/GCS), TDE for databases, column-level encryption for sensitive DB columns.- For caches/queues (Redis, Kafka), enable TLS and auth, and encrypt persisted logs.5. Least-privilege access policy- Principle of least privilege with role-based access: separate roles for pipeline operators, analysts, data scientists.- Implement column- and row-level access controls (e.g., Snowflake masking policies, BigQuery column-level ACLs) and attribute-based access policies for ad-hoc queries.- Use ephemeral credentials for compute (short-lived STS tokens), VPC endpoints, private networking for data stores.- Enforce approval workflows for key or token vault access (break-glass with multi-party approval).6. Audit logging & monitoring- Immutable audit logs for access to PII, key usage, tokenization/de-tokenization events (CloudTrail, Audit Logs).- Centralize logs into SIEM, alert on anomalous access patterns, retention per compliance.- Record provenance metadata in the data catalog for each dataset transformation and retention disposition.Concrete pipeline example:- Ingest raw events -> classification step tags PII -> route to transformation service: - If policy = tokenization: tokenization microservice (vault-backed) returns token; mapping stored in secure vault with access controls. - If policy = pseudonymize: apply HMAC-SHA256 with KMS-derived key for consistent pseudonym.- Store transformed data in data lake; original raw PII moved to secured archive with strict access and retention TTL.Trade-offs: analytic utility vs privacy- Deterministic pseudonymization (salted hashes) preserves joinability & ML feature utility but increases re-identification risk if adversary has auxiliary data. Tokenization is safer for reidentification control (mapping kept in vault) but complicates cross-system joins unless tokenization is centralized.- Irreversible hashes reduce risk but lose the ability to re-identify and can impede certain use cases (customer support, fraud investigations).- Stronger privacy (non-deterministic tokens, adding noise/differential privacy) reduces utility for record-level analysis; choose per-use-case: preserve identifiers for operational flows, pseudonymize or aggregate for analytics.- Cost/operational trade-offs: HSMs and vaults add complexity and cost but are necessary for high-risk PII.Key measures to mitigate risk while maximizing utility:- Apply least-privilege & separation of duties, central tokenization service, deterministic pseudonyms only when necessary, strong KMS + envelope encryption, monitoring and periodic privacy risk assessments (re-identification testing), and dataset-level policies (retention, minimization).This pattern lets data engineers provide analysts useful, joinable datasets while protecting PII through layered controls, auditable access, and policy-driven transformations.
HardTechnical
90 practiced
Design a governance process for feature lifecycle in a feature store: onboarding, versioning, deprecation, documentation requirements, ownership assignment, and metrics to identify unused or stale features. Include notification strategies to notify and protect downstream dependents when deprecating a feature.
Sample Answer
Goal: create a repeatable, low-friction governance process that ensures feature quality, traceability, safe evolution, and minimal downstream disruption.1) Principles & scope- Treat each feature as a product with metadata, owners, tests, and lifecycle states: Proposed → Active → Deprecated → ReadOnly → Removed.- Enforce via a Feature Registry (catalog) + CI checks + access controls.2) Onboarding- Automated checklist in PR/registry entry: schema, data type, semantic description, intended consumers, upstream sources, freshness SLA, compute/serving spec.- Required artifacts: unit/integration tests, data validation (statistical & null checks), example queries, privacy classification.- Gate: CI must pass tests and approver from owning team and platform before transitioning to Active.3) Versioning- Semantic versioning for feature transforms: MAJOR.MINOR.PATCH.- Immutable published versions; consumers bind to a specific version. Default alias (latest-stable) available but opt-in.- Registry tracks lineage across versions and stores change diffs; migrations require compatibility notes and a deprecation plan for prior version.4) Deprecation process- Criteria: zero/low usage over N weeks, replaced by superior feature, upstream changes, or privacy/regulatory needs.- Stages & timelines (example): - Warn: 30 days — notify owners/consumers, mark Deprecated in registry, set read-only flag optional. - Protected: 14 days — CI emits warnings on consumer builds; platform can create canary alerts for failures. - Removal: 30 days after Protected — remove from default alias; permanent removal after agreed freeze (e.g., 90 days).- Escalation path: if consumers object, schedule compatibility window or provide migration assistance.5) Documentation & ownership- Metadata required: owner (primary), backup owner, SLA, creation date, version, lineage, privacy tag, test coverage %, known caveats.- Owners responsible for monitoring, signing off on deprecations, and assisting migrations. Add on-call rotation for critical features.6) Metrics to detect unused/stale features- Consumer metrics: number of distinct downstream consumers, query frequency, last query timestamp.- Pipeline metrics: weekly compute runs, data freshness, error rate, cardinality drift.- Business/ML impact: feature importance in models (SHAP/coefficients), contribution to downstream KPIs.- Define thresholds: e.g., <1 consumer AND <1 query/week for 8 weeks → candidate for deprecation; priority increases if importance = 0.7) Automation & dashboards- Daily job computes usage/staleness signals and surfaces candidates in registry; owners get automated reports and SLA breaches.- Alerting integrates with Slack/email and ticketing (Jira) to create deprecation tasks with timelines.8) Notification & protection strategy when deprecating- Multi-channel notifications: registry banner, email to owners and listed consumers, Slack channel mention, and auto-generated ticket.- Dependency-aware notifications: use dependency graph to list impacted consumers and owners, include code pointers and suggested migration path.- Protections: - Soft warnings in CI and runtime logs for dependent teams. - Read-only mode toggles writes while allowing reads; prevents new consumers. - Feature flags/aliases: move consumers to alias pointing to new version while preserving old version for bounded time. - Blocking policy only with explicit approval; default is opt-out removal schedule to avoid indefinite zombie features. - Canary & rollback: deprecate for a subset of consumers first and monitor metrics before full removal.9) Governance cadence & reviews- Monthly triage meeting: platform + feature owners to review deprecation candidates, policy exceptions, and outstanding migrations.- Quarterly audit: validate metadata completeness, test coverage, and critical-feature SLAs.10) Tooling suggestions- Use a feature catalog (Amundsen/DataHub) extended with lifecycle states, CI integrations, and dependency graph.- Orchestrate checks via pipelines (Airflow/DBT/Spark) and embed tests in CI.- Logging/metric store (Prometheus/Datadog) for usage; automated tickets in Jira.Why this works- Combines automated detection with human judgment, clear timelines, and ownership accountability. It minimizes surprises for downstream teams, enforces quality, and provides safe migration paths while keeping the catalog healthy and maintainable.
MediumTechnical
56 practiced
Explain how partition pruning, statistics collection, and data skipping work in modern query engines like Spark and Presto. Given a slow query that appears to scan the entire table, list diagnostic steps, SQL changes, and storage layout changes you would make to improve performance.
Sample Answer
Partition pruning, statistics collection, and data skipping are orthogonal but complementary optimizations modern engines use to avoid full-table scans.- Partition pruning: The engine uses table partition metadata (e.g., date=2025-01-01) to restrict which files/directories to read based on query predicates. This happens before file listing and reading, so proper partition columns and predicate pushdown are critical.- Statistics collection: Engines collect table-level, partition-level, and column-level stats (row counts, NDV, min/max, histograms). The optimizer uses these to choose join orders, broadcast vs shuffle, and predicate selectivity estimates; accurate stats prevent bad plans.- Data skipping (file/column-level): Storage formats (Parquet/ORC) write min/max and bloom filters per row-group/stripe and column. Query engines consult these file-level indexes to skip reading row groups or entire files whose min-max don’t satisfy predicates.Diagnostic steps for a query that scans the whole table1. EXPLAIN (or EXPLAIN ANALYZE) to see planned vs actual stages, IO reads, and whether predicates were pushed down.2. Check partitions: SHOW PARTITIONS / list S3 paths; confirm predicate uses partition column(s) and types match.3. Inspect file metadata: parquet-tools meta / ORC file inspector to confirm row-group stats exist.4. Verify statistics: run ANALYZE TABLE ... COMPUTE STATISTICS or check metastore stats timestamps.5. Check connector/config: spark.sql.sources.partitionOverwriteMode, parquet.enable.summary-metadata, hive.metastore.stats.autogather, and predicate pushdown enabled.6. Sample query with LIMIT to reproduce; run with profiling to measure file/row-group reads.SQL changes to improve:- Filter on the partition column (e.g., WHERE dt BETWEEN '2025-01-01' AND '2025-01-31') or rewrite to use partition predicates.- Cast predicates to match column types to avoid function-wrapping (avoid where DATE(col) = …; instead use col >= … AND col < …).- Push down filters earlier: push predicates into subqueries, use WITH (materialized) or CTEs carefully (avoid non-inline CTEs that materialize).- Add column statistics: ANALYZE TABLE tbl COMPUTE STATISTICS FOR COLUMNS(col1, col2).- Force better join strategy if stats are stale: use /*+ BROADCAST(t) */ or join hints as temporary mitigation.Storage/layout changes:- Partition granularity: partition by high-cardinality? Choose meaningful partitions (date/hour) to limit files per query; avoid over-partitioning small files.- File size and layout: compact small files into larger Parquet/ORC files (e.g., 128–512 MB) and optimize row-group size; use compaction jobs (Spark repartition/coalesce + write).- Enable and preserve file-level statistics: ensure writers produce row-group min/max and bloom filters (Parquet/ORC defaults) and preserve summary metadata (e.g., _metadata/_common_metadata).- Sort within partitions by common filter columns (Z-ordering or sort-by) to increase locality and make min/max skipping more effective.- Use appropriate file format and encoding (Parquet/ORC), and consider creating data skipping indexes (Delta Lake data skipping, Hudi statistics, or third-party indexes like Hyperspace).Summary: run EXPLAIN to identify missing pruning or pushdown, ensure predicates align with partitioning and column types, compute/update statistics, compact and sort files so min/max and bloom filters are effective, and adjust partitioning/file sizes to balance pruning and file count. These steps usually convert full scans into targeted reads and large IO savings.
MediumTechnical
60 practiced
Design pipeline changes to support GDPR right-to-be-forgotten and data access requests. Discuss strategies like physical deletion, logical tombstones, pseudonymization, audit trails, downstream propagation, and how to balance regulatory compliance with analytics and ML requirements.
Sample Answer
Requirements & constraints:- Support Right-to-Be-Forgotten (RTBF) deletions and Data Subject Access Requests (DSARs) within legal SLA.- Ensure deletions propagate to all stores, backups, analytics extracts, and ML features while preserving ability to run analytics without personal identifiers.- Maintain auditable proof of actions.High-level approach:1. Orchestrator & workflow- Use a single request API that triggers a workflow (Airflow/Step Functions). Workflow runs discovery, propagation, verification, and audit logging.2. Identity-resolution & scope- Maintain a canonical identity table (PII index) mapping subject identifiers (email, account_id) to system-wide IDs. Use this to find all locations holding the subject’s data.3. Deletion strategies (per-store):- Physical deletion: For primary user stores (OLTP, CRM) do hard deletes when required and feasible.- Logical tombstones: For append-only stores (Kafka topic, audit logs, immutable data lake) write a tombstone message (Kafka null value / tombstone key) or add a tombstone flag so downstream consumers can filter; schedule compaction/GC where supported.- Pseudonymization/tokenization: Replace direct identifiers with tokenized IDs for analytics and ML pipelines; tokens stored in an encrypted vault keyed by KMS/HSM. For RTBF, either remove token mapping (irreversible) or rotate keys and delete mapping to render re-identification impossible.- Selective masking: For historical analytics where identifier not required, mask or aggregate fields.4. Downstream propagation:- Use CDC (Debezium/DB logs) + Kafka topics to publish deletion/tombstone events. Consumers (feature store, DW, ML pipelines) subscribe and apply deletes/filters. For batch ETL, include a "deletion window" filter step that joins against PII index or tombstone feeds.- For third-party exporters, maintain a registry and send deletion notifications; revoke access tokens and request confirmations.5. Audit & verification:- Write immutable audit records of request, scope, actions, timestamps, actor, and verification status to a WORM store (e.g., append-only S3 with object lock) and a searchable index for compliance.- Provide a verification job that re-scans downstream stores and reports outstanding items.6. Balancing analytics & ML:- Keep privacy-preserving feature engineering: use pseudonymized IDs for joins, store features without direct PII, apply differential privacy/aggregation for sensitive metrics.- For ML models requiring historical labeled data tied to users, use reversible tokenization with strict access controls and retention TTL; upon RTBF either (a) delete mapping and avoid retraining on that user, or (b) remove user's rows from feature store and re-train if required by policy.- Offer synthetic/aggregated datasets to replace deleted individuals in long-term analytics.Trade-offs & considerations:- Tombstones simplify compliance for immutable stores but require compaction to free space; physical deletion is stronger legally but harder in backups and caches.- Reversible tokenization eases analytics but increases re-identification risk; use strict KMS, audit, and TTLs.- Immediate deletion can impact model reproducibility; document provenance and consider marking models trained with now-deleted data.Operational checklist:- CI tests: simulate deletion flow and assert downstream removals.- Monitoring: metrics for outstanding deletion propagation, audit gaps.- Data catalog: map sensitive fields and downstream consumers.- Legal alignment: define retention policies and deletion SLAs with Legal.Example technical patterns:- Kafka tombstone: publish {key: user_id, value: null} → compacted topic cleans up consumer state.- Feature store purge: Spark job joining feature store to PII index to drop rows where subject marked deleted.- Tokenization: store token -> user_id in Vault; on RTBF delete mapping and publish tombstone so analytics use only token-less data.This design provides auditable, scalable deletion propagation while preserving analytics via pseudonymization, aggregation, or synthetic substitutes; trade-offs are controlled with key management, retention policies, and observable verification.
MediumTechnical
68 practiced
Outline a rollout plan for adopting a centralized schema registry and metadata catalog across multiple teams. Include pilot selection, migration steps, compatibility policies, CI integration, enforcement mechanisms, training, and measures to minimize breaking changes and developer friction.
Sample Answer
Situation: Our org needed consistent schemas and discoverable metadata to reduce pipeline failures and speed analytics.Plan (high-level goals):- Provide a single source of truth for schemas and dataset metadata- Minimize consumer breakages and developer friction- Integrate checks into CI/CD and observabilityPilot selection:- Pick 1 producer team (stable, high-usage dataset) and 1 consumer-heavy analytics team- Choose datasets with moderate churn and clear owners to iterate quicklyMigration steps:1. Inventory: catalog current topics/tables, owners, consumers, and schema formats.2. Register: add canonical schemas and metadata in registry/catalog with tags (env, owner, SLA).3. Adapter layer: add client libs/wrappers that read schemas from registry (backwards-compatible read).4. Dual writes (if needed): allow old+new paths for 2–4 weeks while monitoring.5. Cutover: route consumers to registry-backed endpoints, retire legacy definitions after validation.Compatibility policies:- Enforce semantic compatibility rules: backward compatible changes allowed; forward incompatible changes require major version bump and migration plan.- Use explicit versioning and deprecation metadata.CI integration & enforcement:- Schema lint and compatibility checks as pre-merge pipeline steps.- Gate PRs that change schemas with automated tests that run consumer contract tests (consumer-driven contracts or generated sample data).- Block merges if compatibility fails; provide automated suggestions to fix.Enforcement & monitoring:- Registry webhook emits events on breaking changes; alert owners and downstream test suites.- Add dashboards: schema change frequency, failed compatibility checks, number of consumers per schema.- Use automated canary consumers to detect runtime issues post-deploy.Training & onboarding:- Run workshops, recorded demos, and cookbooks showing library usage, versioning, and migration examples.- Create "schema champions" in each team as contacts.- Provide templates for deprecation notices and migration plans.Minimizing breaking changes & friction:- Default to additive changes only; require migration windows for removals.- Provide tooling to auto-generate migration scripts and sample payload transforms.- Offer a sandbox environment and a migration playbook for common scenarios.- Encourage incremental adoption via adapters and dual-write pattern.Measures of success:- Reduced pipeline schema-related failures by X% in pilot (target 80%).- Mean time to onboard a dataset under Y days (target <=5).- Number of teams actively registering metadata (adoption target).This approach balances governance with developer velocity through automation, clear policies, and hands-on support.
Unlock Full Question Bank
Get access to hundreds of Domain and Product Technical Knowledge interview questions and detailed answers.
