Assess a candidate's practical and conceptual understanding of technology stacks, including major programming languages, application frameworks, databases, infrastructure, and supporting tools. Candidates should be able to explain common use cases and trade offs for languages such as Python, Java, Go, Rust, C plus plus, and JavaScript, including differences between compiled and interpreted languages, static and dynamic type systems, and performance characteristics. They should discuss application frameworks and libraries for frontend and backend development, common web stacks, service architectures such as monoliths and microservices, and application programming interfaces. Evaluate understanding of data storage options and trade offs between relational and non relational databases and the role of structured query language. Candidates should be familiar with cloud platforms such as Amazon Web Services, Google Cloud Platform, and Microsoft Azure, infrastructure components including containerization and orchestration tools such as Docker and Kubernetes, and development workflows including version control, continuous integration and continuous delivery pipelines, testing frameworks, automation, and infrastructure as code. Assess operational concerns such as logging, monitoring and observability, deployment strategies, scalability, reliability, fault tolerance, security considerations, and common failure modes and mitigations. Interviewers may probe both awareness of specific tools and the candidate's depth of hands on experience, ability to justify technology choices by evaluating trade offs, constraints, and risk, and willingness and ability to learn and evaluate new technologies rather than claiming mastery of everything.
HardTechnical
69 practiced
Discuss strategies to optimize a transformer based model to meet a strict 1 millisecond p95 inference target on CPU only edge devices. Cover algorithmic approaches such as distillation and pruning, deployment techniques like operator fusion and model compilation with TVM or ONNX Runtime, and quantization trade offs including per channel quantization.
Sample Answer
Situation: Meeting a 1 ms p95 inference target on CPU-only edge requires end-to-end optimization—model, precision, and runtime. Below is a prioritized strategy and concrete techniques.1) Algorithmic compression- Distillation: Train a small student transformer using logits + intermediate-layer hints (TinyBERT-style). Target architectural changes: reduce layers (e.g., 12→4), narrower hidden dims, and fewer attention heads. Use knowledge distillation loss with temperature and feature-map matching to retain accuracy.- Structured pruning: Prefer head/channel/block pruning over unstructured sparsity for better CPU speedups. Use iterative magnitude or movement pruning with sparsity-aware retraining and optionally L0 regularization to remove entire attention heads or feed-forward blocks.- Architecture search: Consider hybrid architectures (e.g., Linformer, Performer) to reduce attention complexity if latency still high.2) Quantization and trade-offs- Start with int8 quantization; aim for post-training quantization with per-channel weight quantization and per-tensor activation quantization. Per-channel reduces accuracy degradation for heterogeneous weight ranges.- If int8 harms accuracy, use Quantization-Aware Training (QAT) with fake-quant ops and calibration.- Mixed precision: keep sensitive layers (first/last, layernorm, softmax) in fp16 or fp32 and quantize the rest.- Beware: per-channel offers better accuracy but needs runtime support; per-tensor is simpler but worse accuracy.3) Deployment/runtime optimizations- Operator fusion: fuse attention QKV matmuls + bias + transpose + softmax + context matmul where possible to eliminate memory traffic. Implement fused kernels for fused dense+gelu.- Model compilation: compile with TVM or ONNX Runtime with CPU backends (XNNPACK, MKL-DNN/oneDNN). Use ahead-of-time lowering, kernel autotuning, and graph-level optimizations (layout transforms, constant folding).- Runtime choices: ONNX Runtime with ORT-TRT equivalent for CPU (use DML or OpenVINO where available), or TVM runtime with tuned schedules produces small binary and minimal overhead.- Use thread pinning, set OMP_NUM_THREADS to 1 for strict latency, and use model warm-up to avoid page faults.4) Measurement and iteration- Profile end-to-end with realistic inputs, measure breakdown (compute vs memcpy). Target microseconds per op; if attention dominates, implement low-rank or block-sparse attention.- SLO strategy: tune model size until p95 <1 ms with acceptable accuracy loss (<1-3% absolute depending on product).- CI: include accuracy/latency regression tests on target hardware.Trade-offs summary:- Distillation + structured pruning + per-channel int8 (with QAT as needed) gives best accuracy/latency balance.- Operator fusion and AOT compilation (TVM/ONNX Runtime) convert model reductions into real wall-clock gains.- Accept complexity: mixed precision and fused kernels increase engineering work but are required to hit 1 ms p95 on CPU edge.
MediumTechnical
76 practiced
Design a GDPR compliant data and model lifecycle for a system where users can request deletion of their personal data. Explain how you would store raw data, engineered features, and models such that you can comply with deletion requests and still maintain audit logs, including trade offs between anonymization, pseudonymization, and retraining strategies.
Sample Answer
Requirements & constraints:- Users must be able to delete personal data (right to erasure) while preserving auditability and system integrity.- Minimize impact on model performance and operational complexity.- Keep provable lineage: which raw records contributed to features/models.High-level design:1. Storage separation- Raw data store: encrypted object store (S3/GCS) containing original records; each record has an internal RecordID (RID) — NOT user-facing. Access controlled via IAM + audit logs.- PID store (personal identifiers): small, highly protected DB (HSM/KMS backed) mapping UserID -> RID(s) via reversible cryptographic tokenization (pseudonymization). Tokenization keys rotated and access tightly logged.- Feature store: stores features keyed by FeatureID and RID-hash (a one-way hash of RID + salt). Do NOT store plaintext PII.- Models & training metadata: track training snapshots with immutable training-manifest that lists contributing RID hashes (or cryptographic Bloom filter) and training dataset version.2. Deletion workflow- Receive deletion request for UserID.- Lookup RIDs in PID store; delete entries from PID store (or mark revoked) and log deletion event.- Physically delete raw records from raw store by RID.- For feature store entries keyed by RID-hash: remove entries whose RID-hash matches deleted RIDs.- For models: consult training-manifest. Two options: a) Retrain: For critical models or when legal requirement demands, retrain model on dataset excluding deleted RIDs. Use automated retrain pipelines; version and swap. b) Patch/Prove: If retraining impractical, record that model contains deleted data, keep model but mark as "contains legacy data" and, where feasible, remove influence (e.g., data removal via SISA, Sharded Influence, or machine unlearning methods). Provide user-facing proof of deletion and audit logs.3. Auditability- Append-only audit log (WORM): record every data access, tokenization events, deletion requests, and deletion actions with hashes of deleted content (hash of RID + salt) so proof exists without storing PII.- Manifests & provenance: immutable training manifests, dataset versions, and pipeline run IDs stored in an audit DB. Store cryptographic hashes of datasets and model binaries to prove integrity.Trade-offs: anonymization vs pseudonymization vs retrain- Anonymization (irreversible): strong for privacy and avoids need to retrain — but destroys utility (cannot re-link for corrections or analytics) and may be hard to prove truly irreversible.- Pseudonymization (reversible tokenization): allows linkage for model updates and debugging; meets GDPR if controls + key management are strong. Deletion requires removing mapping and data; risk: key compromise re-identifies.- Retraining: strongest compliance and safest long-term; costly compute and latency. Incremental retraining or sharded training (SISA) reduces cost and enables efficient dataset removals.- Machine unlearning/approximate removal: lower cost but research frontier; may not be accepted by regulators as full deletion.Recommendations:- Use pseudonymization with strict key management + separate PID store; maintain hashed RID links in feature store for traceability.- Implement SISA-style sharded training so deletes require retraining only affected shards.- Keep immutable audit logs (hashes + events) so you can prove deletion actions without retaining PII.- Define policy: low-risk models -> patch & document; high-risk/regulated models -> full retrain within SLA.- Automate deletion pipelines, monitoring, and quarterly key rotations and audits.This design balances compliance, traceability, and operational cost while providing clear options and SLAs for full retraining vs pragmatic removal.
HardSystem Design
84 practiced
Design a multi region model serving system that provides low latency to global users, consistent model versions across regions, and graceful failover when a region becomes unavailable. Address model distribution, CI CD propagation, global traffic routing, consistency guarantees for registry and artifact storage, and data residency constraints.
Sample Answer
Requirements clarification:- Functional: global low-latency inference, consistent model versions across regions, automated CI/CD propagation, graceful failover.- Non‑functional: <50ms regional latency target, strong model-version consistency for correctness, cross‑region availability, data residency (some regions cannot store model artifacts or training data).- Constraints: budget, regulatory residency, network partitions possible.High-level architecture:- Global Traffic Layer: anycast + global LB (Cloud CDN/Global Accelerator) + geo-aware DNS for routing to nearest healthy region. Health checks drive routing weight.- Control Plane: central Model Registry (metadata) in a strongly consistent global DB (Google Spanner / CockroachDB / Cosmos DB with strong consistency) + CI/CD pipeline (e.g., Git + CI like Tekton + ArgoCD/Flux).- Artifact Store: object storage per-region (S3/GCS) with selective Cross-Region Replication (CRR). Global immutable artifact index referenced by registry.- Serving Plane (per-region): k8s cluster with autoscaling model servers (KFServing/KServe or custom microservices) using local cached artifacts and a sidecar for model lifecycle.- Observability: distributed tracing, metrics, and alerting (Prometheus + Grafana + centralized aggregator).Model distribution & consistency:- Model artifacts are built once in CI, content-addressed and signed (SHA256 + provenance). CI pushes artifact to a global immutable artifact storage or to a primary regional bucket with CRR to allowed regions. Registry stores model metadata (id, version, hash, allowed regions, residency flags).- Use strong consistency for registry so a single authoritative version exists (avoid split-brain). Registry change (promote version) triggers per-region CD controllers to pull artifact by hash.- Per-region deployment uses optimistic concurrency: deployment controller verifies artifact hash and applies canary rollout; once health checks pass, promote traffic.CI/CD propagation:- GitOps: commit → CI builds artifact → push to artifact store → update registry via transaction in global DB → emits event to pub/sub (Kafka or cloud pub/sub) → per-region CD agents subscribe and perform staged rollout (canary → 50% → 100%).- If a region has residency exclusion, CD agent skips pulling artifact; registry enforces deployment policy.Global traffic routing & failover:- Primary routing chooses nearest healthy region. Use health-aware global LB with weighted failover and TTL-based DNS fallback. On region failure, LB shifts traffic to next-closest region. Anycast/global accelerator reduces failover time.- For stateful inference needing user affinity, use sticky cookies + global session store with regional read replicas; on failover allow session migration with TTL.Artifact & registry consistency guarantees:- Registry: strong consistency (single source of truth) to prevent divergent model versions.- Artifact storage: eventual consistency acceptable for object stores, but guarantee read-after-write by: - Writing first to a primary region and only marking artifact available in registry after successful CRR/sync or after checksum validation in consumers. - Alternatively, use geo-replicated object stores that support strong read-after-write (where available) or maintain a global cache invalidation protocol.- Use content-addressed artifacts + signature to detect partial replication; CD agents validate hash before activation.Data residency:- Registry metadata includes allowed_regions flag. CD agents enforce policy: if region disallowed, they do not pull artifacts or deploy.- For inference requiring resident data, route those requests only to compliant regions (geo-DNS + request attribute routing). Use data access proxy that enforces region constraints.Graceful failover & safety:- Failover steps: detect region unhealthy → global LB reroutes → per-region deployment/stateful failover hooks run (drain, replicate minimal state) → circuit-breaker on clients to avoid overload downstream.- Staged rollbacks automated: if canary fails, controller reverts; registry remains authoritative.Scaling & performance:- Local caches for artifacts on nodes reduce cold-starts.- Autoscaling based on latency SLOs, queue depth.- Use model sharding or replicas per region for high throughput.Trade-offs:- Strong registry consistency increases latency for control operations but prevents split versions — acceptable.- Full synchronous artifact replication (strong) increases deploy time; chosen compromise: registry only marks artifact available after verification or uses async CRR + hash checks.- Cost vs latency: more regional replicas = lower latency but higher cost and residency complexity.Monitoring & testing:- Synthetic probes per-region verifying model hash, latency and output correctness.- End-to-end chaos testing (simulated region failures) and compliance audits for residency.This design ensures low-latency local serving, a single authoritative registry for version consistency, controlled CI/CD propagation with verification, and policy-driven residency enforcement while enabling fast, automated failover.
EasyTechnical
78 practiced
Describe the differences between batch processing and stream processing for ML data pipelines. Give examples of use cases for each, and describe typical platforms such as Apache Spark, Apache Flink, Kafka Streams, and Apache Beam. Include considerations such as latency, state, exactly once processing, and handling late arriving data.
Sample Answer
Batch vs Stream (short): Batch processing handles bounded datasets in chunks (e.g., daily ETL, model retraining). Stream processing handles unbounded, continuous events in near real-time (e.g., feature updates, online predictions, fraud detection).Use cases:- Batch: nightly feature aggregation, periodic model training, backfills, expensive global joins.- Stream: real-time feature engineering, online model scoring, anomaly detection, personalization.Typical platforms:- Apache Spark: strong batch (Spark SQL, MLlib) and micro-batch streaming (Structured Streaming). Easy scale and rich APIs; micro-batch gives near-real-time latency (secs).- Apache Flink: true low-latency, event-at-a-time streaming with powerful stateful processing and event-time semantics; good for complex windowing and low latency.- Kafka Streams: lightweight library for stream processing tightly integrated with Kafka; great for embed-in-app, low ops, stateful stream joins.- Apache Beam: unified programming model that runs on runners (Dataflow, Flink, Spark); good when you need portability and consistent event-time semantics.Key considerations:- Latency: batch = minutes–hours; stream = milliseconds–seconds.- State: streaming needs durable state stores (Flink state backend, RocksDB, Kafka Streams local state) for features/aggregation.- Exactly-once: achievable with Flink and Spark Structured Streaming (+ checkpointing), Kafka Streams with idempotent producers; requires careful sink semantics.- Late-arriving data: use event-time processing and watermarks (Flink/Beam) to handle lateness and update materialized features; decide allowed lateness vs correctness.Practical advice: combine both — use streaming for real-time features and monitoring, batch for heavy aggregates and retraining; ensure reproducibility, idempotent sinks, and robust monitoring.
HardSystem Design
63 practiced
Design an automatic hyperparameter tuning system that executes distributed experiments on spot instances while controlling total budget and ensuring reproducibility. Discuss scheduler choices such as population based methods, Bayesian optimization, or ASHA, checkpointing and resuming trials, metadata to store, and how to compare trials with confidence intervals.
Sample Answer
Requirements & constraints:- Run large distributed training on spot instances while enforcing a fixed monetary budget, keep experiments reproducible, support early stopping and resuming after preemption, and provide statistically sound comparisons between trials.High-level architecture:- Orchestrator (Kubernetes + custom controller) schedules trial pods onto spot node pools (multi-AZ). - Global Budget Manager tracks consumed cost in real time (cloud billing API + per-pod cost estimator) and enforces soft/hard caps. - Experiment Database (metadata store: Postgres/Elastic + object store S3/GCS for artifacts/checkpoints). - Scheduler layer pluggable: ASHA / Bayesian Optimizer / Population-Based Trainer (PBT). - Trial Runner container: standard image, provenance logging, checkpointing client that writes to object store. - Preemption handler: watch spot metadata, immediate checkpoint on SIGTERM, auto-requeue via controller.Scheduler choices & combination:- ASHA: excellent for large parallel search when training is expensive — quickly stops poor trials. Use as the low-level scheduler to save compute. - Bayesian Optimization (BO): sample-efficient global search when function evaluations are expensive; use BO as the high-level proposer of hyperparameter candidates. Combine BO + ASHA: BO suggests configs; ASHA allocates resources and early-stops. - Population Based Training (PBT): use when you want online adaptation of hyperparams and to exploit long-running training dynamics (e.g., learning-rate schedules). PBT is heavier; use selectively for model families that benefit.Checkpointing & resuming:- Mandatory periodic checkpointing frequency tuned to expected spot interruption rate (e.g., checkpoint every min(cost_of_checkpoint < expected_loss_on_interrupt)). On SIGTERM attempt quick checkpoint. - Store: model weights, optimizer state, RNG seeds, epoch/step, hyperparams, training/validation splits, environment/container image digest, git commit hash, data version id. Use efficient incremental checkpoints (deltas) for large models. - Resume: controller reads metadata, restores RNG/local seeds and optimizer state, continues with same trial id or clones for variance runs.Metadata to store:- Experiment id, trial id, hyperparams, config schema version, container image digest, git commit, dataset version & preprocessing hash, RNG seeds, start/end timestamps, cost consumed, node/zone, checkpoint URIs, training curves (time series), final metrics, success/failure reason (preempted, OOM, converged), scheduler decisions (promote/stop), and reproducibility hashes.Budget control strategies:- Real-time costing: meter per-node minute cost + data egress; Budget Manager enforces hard stop or reduces parallelism when approaching cap. - Cost-aware acquisition: for BO use acquisition function penalized by estimated cost (expected improvement per dollar). For ASHA, adapt bracket sizes / rung steps to reduce expensive brackets when budget tight. - Graceful degradation: reduce parallel workers, lower fidelity (smaller subset of data or fewer epochs) based on budget.Comparing trials with confidence intervals:- Run each promising config with multiple random seeds or use bootstrap over checkpoints to estimate variance. Store full training curves. - Compute CIs using bootstrap or t-based intervals on final metric across seeds: report mean ± 95% CI and use non-parametric tests (bootstrap percentile, Mann–Whitney U) when distributions non-normal. For early-stopped trials, compare best-interpolated metric at matched budget (e.g., performance at 10k steps) to avoid bias. - Use Bayesian ranking: compute posterior probability a config is best (via Gaussian process on metrics across seeds), and stop exploring configs with low probability of beating incumbent given cost.Failure & edge cases:- Corrupt checkpoint: keep multiple recent checkpoints + checksums. - Hotspot preemption: spread replicas, diversify AZs, stagger checkpoint times to avoid I/O spikes. - Reproducibility drift: pin base images, log exact environment, and provide reproduction script to spawn single-trial run from stored metadata.Trade-offs:- ASHA + BO balances sample efficiency and parallelism; PBT gives online adaptation but costs more. Frequent checkpointing increases overhead but reduces lost work on spot preemptions. Multi-seed increases CI reliability but multiplies cost—use sequential confirmation: run 1 seed for screening, then validate top-k with multiple seeds.This design yields a cost-aware, spot-resilient HPO system that is reproducible, statistically rigorous, and flexible across different optimization strategies.
Unlock Full Question Bank
Get access to hundreds of Technology Stack Knowledge interview questions and detailed answers.
