Conceptual understanding of important deep learning ideas: representation learning, why deep networks work, CNNs for feature extraction in images, RNNs for sequential data, attention mechanisms, and transformers. Understanding when to use deep learning vs simpler methods.
EasyTechnical
42 practiced
Explain representation learning and how it differs from manual feature engineering. As a machine learning engineer designing production systems, describe three practical advantages of learned representations (e.g., transfer learning, reduced preprocessing, robustness to raw inputs) and give one concrete situation where manual features might still be preferable. Illustrate with examples from image, text, or tabular pipelines and outline operational implications (storage, serving) for learned embeddings.
Sample Answer
Representation learning is the process where models (usually neural nets) automatically learn useful features (embeddings) from raw data, versus manual feature engineering where humans design features from domain knowledge. Learned representations capture complex patterns, are dense and task-adaptive, and reduce the need for hand-crafted pipelines.Three practical advantages:1) Transfer learning — pretrained image/text encoders (ResNet, BERT) let you fine-tune with less labeled data (example: fine-tune BERT for intent classification).2) Reduced preprocessing — end-to-end models accept raw pixels or tokenized text, removing bespoke normalization and feature-extraction code (example: CNN on images vs handcrafted edge/HOG features).3) Robustness/generalization — learned embeddings often handle noisy inputs and subtle correlations better (example: learned user/item embeddings improve recommendation under sparse signals).When manual features are preferable: small-tabular datasets with strong domain rules (credit scoring) — handcrafted, interpretable features and simple models (logistic regression) give better transparency, regulatory compliance, and often comparable performance.Operational implications:- Storage: embeddings are vectors (e.g., 128–1024 dims); store in vector DBs or feature stores with quantization/FAISS for space/time trade-offs.- Serving: low-latency retrieval via approximate nearest neighbor indices, cache hot embeddings, and version/lineage embeddings for reproducibility.- Monitoring: drift detection on embedding distributions, offline recomputation cost, and lifecycle for retraining/refresh schedules.
HardSystem Design
52 practiced
You're building a multi-tenant inference service for transformer models. Design the system to provide tenant isolation, enforce memory and throughput quotas, support per-tenant model versions, and meet latency SLAs. Discuss runtime choices (host per tenant vs containerized GPU sharing vs model multiplexing), cold-start mitigation, cost attribution, and operational monitoring.
Sample Answer
Requirements (clarify):- Per-tenant isolation (security, noisy-neighbor prevention)- Memory & throughput quotas per tenant- Per-tenant model versions/config- Latency SLA (p99 target), predictable tail latency- Cost efficiency, observability, autoscalingHigh-level architecture:- API gateway → Auth & tenant routing → Inference controller → Scheduler/orchestrator → GPU workers (hosts/containers) + CPU workers for preprocessing/postprocessing → Monitoring & billing pipeline → Model store (versioned artifacts)Runtime choices (trade-offs):1. Host-per-tenant (dedicated VM/GPU per tenant) - Pros: strongest isolation, simple quota enforcement, predictable latency - Cons: high cost for low-util tenants, cold starts when scaling - Use when strict compliance/latency required.2. Containerized GPU sharing (Kubernetes + device-plugin + cgroups) - Pros: better utilization, easier multi-tenant consolidation, per-container resource limits - Cons: partial isolation (driver/kernel shared), careful scheduling needed to avoid GPU OOM; overhead from container images - Good default for medium-sized tenants.3. Model multiplexing on same GPU (multiple models in one process using sharding/batching) - Pros: highest throughput, efficient batching and kernel reuse, lower latency for small requests via pinned models in memory - Cons: complex isolation & fair scheduling; harder to enforce strict memory quotas; risk of cross-tenant interference - Use for high-throughput tenants where strict isolation is relaxed.Recommendation: hybrid approach — small tenants share containerized GPU pools with strict soft quotas and throttling; enterprise/PCI tenants get dedicated hosts.Model versions & routing:- Maintain a versioned model registry with metadata (tenant, version, resource footprint).- Controller supports per-tenant routing rules: route to tenant-specific model instance (hot) or to shared model if allowed.- Blue-green or canary deploy via weighted routing; immutable artifacts + hash-based checksum.Quota enforcement & throughput:- Two-level quota system: - Admission control at API gateway: per-tenant QPS and token rate limiting. - Runtime throttling within orchestrator: GPU memory reservations, per-container/cgroup GPU time quotas (using NVIDIA MIG where available).- Use token-bucket for throughput, and adaptive backpressure (reject/queue) when overloaded.- Implement fair scheduler using weighted-leaky-bucket to guarantee throughput shares.Cold-start mitigation:- Warm pool of preloaded models (hot containers/processes) sized by predicted demand (use recent traffic + ML forecasting).- Model packing: keep multiple small models resident on one GPU (when memory allows) using model quantization (8-bit), operator fusion, and tensors offloading (Vulkan/MPS/host pinned memory) for less-active ones.- Lazy init + micro-batching to amortize kernel launch overhead.- Snap-start-like fast-paths: export models to optimized runtimes (ONNXRuntime, TensorRT/TF-TRT) and start from serialized engine caches.- For very cold tenants, return a “warming” response with ETA or route to lighter-weight fallback model for latency-critical flows.Autoscaling and SLA:- Multi-dimensional autoscaling: scale replicas based on GPU utilization, request latency p95/p99, queue length and per-tenant SLA.- Horizontal scale by adding replicas; vertical by loading lighter model variants (distillation, smaller size) for degraded mode.- SLO policing: prioritize traffic of higher-SLA tenants during contention (configurable weights).Cost attribution and billing:- Metering at request-level: - GPU-seconds per request (wall-clock on GPU) - Memory reservation time (GB-hours) - Token or input size processed - Additional CPU / egress costs- Emit events to billing pipeline: enrich with tenant id, model version, request latency, resource metrics.- Use sampling/profiling to estimate real GPU utilization per model and reconcile with measured wall-time for accuracy.- Provide per-tenant cost reports and alerts for overruns.Operational monitoring & alerting:- Metrics: per-tenant QPS, success/error rates, p50/p95/p99 latency, queue length, GPU utilization, memory usage, model load times, cold-start rates.- Traces: request-level distributed tracing to measure time spent in routing, preprocessing, model inference, postprocessing.- Logs: structured logs include tenant, model version, request id, failure reasons.- Alerts & SLOs: p99 latency breaches, high cold-start rate, GPU OOMs, skewed cost/noisy-neighbor signals.- Dashboards: heatmaps of tenant usage, top-cost tenants, trending cold starts.- Canary + chaos testing: verify isolation and throttling under failure and noisy neighbor scenarios.Security & isolation specifics:- Tenant data encryption in transit & at rest; model artifacts with ACLs.- Use hardware partitioning (NVIDIA MIG) where enforced isolation is required.- Kernel/driver-level hardening and runtime seccomp profiles in containers.Operational recommendations:- Start with containerized sharing + warm pools, implement strong admission control and per-tenant throttles.- Profile models to choose optimal runtimes (TensorRT for low-latency, ONNX for portability).- Provide tenant-facing SLAs and degrade gracefully (smaller model variants) when overloaded.- Automate billing reconciliation and expose cost dashboards to tenants to encourage right-sizing.This design balances isolation, cost, and SLA by combining orchestration, admission control, model optimization, and disciplined observability.
EasyTechnical
86 practiced
Compare commonly used activation functions (sigmoid, tanh, ReLU, leaky ReLU, GELU). For each, explain typical use-cases, gradient behaviour, saturation issues, and how the choice affects initialization and training stability in deep networks used in production systems. Mention any runtime trade-offs for inference on CPUs/GPUs.
Sample Answer
Sigmoid:- Use-cases: historically for binary outputs / final layer probability; rarely used in hidden layers for deep nets.- Gradient: small derivative for |x| large (σ' = σ(1−σ)), causes vanishing gradients.- Saturation: strong saturation at both tails.- Init/training: requires careful small-weight init; poor for deep nets—slower convergence, unstable for very deep models.- Runtime: cheap elementwise ops; numerically needs stable exp implementation.Tanh:- Use-cases: hidden layers in older RNNs; zero-centered output helps optimization vs sigmoid.- Gradient: similar saturation as sigmoid but steeper around 0 (better gradients near origin).- Saturation: both tails saturate -> vanishing gradients for large inputs.- Init/training: benefits from Xavier/Glorot init to keep variance; better than sigmoid but still problematic in deep stacks.- Runtime: similar cost to sigmoid.ReLU:- Use-cases: default for CNNs/MLPs in modern models.- Gradient: 1 for x>0, 0 for x<0 -> avoids vanishing for positive activations.- Saturation: “dying ReLU” where neurons stuck at 0 if large negative inputs/large negative updates.- Init/training: He/Kaiming init recommended (preserves variance for ReLU); stable and fast convergence for deep nets.- Runtime: very cheap (max(0,x)); highly efficient on CPU/GPU.Leaky ReLU (including PReLU):- Use-cases: when dying ReLU is a concern; small negative slope keeps gradient flow.- Gradient: small constant slope for x<0 (e.g., 0.01), avoids dead neurons.- Saturation: no hard saturation; better gradient propagation.- Init/training: still use He init; can improve stability and robustness to learning rate.- Runtime: slightly more expensive than ReLU (one multiply for negative), still cheap.GELU:- Use-cases: transformer architectures and state-of-the-art NLP/vision models (e.g., BERT).- Gradient: smooth, non-linear, stochastic-like gating (x * Φ(x)); better empirical performance.- Saturation: smooth tails reduce abrupt sparsity; not saturating like sigmoid/tanh.- Init/training: use He-ish init; can improve final performance but may require tuned learning rate and weight decay.- Runtime: more expensive (erf/approx), costlier on CPU; GPUs often have optimized kernels or approximate implementations (faster). For latency-sensitive inference, consider approximations (fast rational or tanh-based).Practical guidance for production:- Prefer ReLU/Leaky ReLU for most vision/MLP workloads (fast, stable); use He init.- Use GELU for transformers where accuracy gain justifies compute; use optimized library ops for inference.- Avoid sigmoid/tanh in deep hidden layers; reserve for outputs or shallow networks.- Monitor activation distributions (batch norm/LayerNorm can mitigate saturation), tune learning rates, and choose activations keeping inference latency and hardware (CPU vs GPU) in mind.
EasyTechnical
43 practiced
Describe the main components of a convolutional neural network: convolution, stride, padding, pooling, receptive field, and feature maps. Explain how these components contribute to translation equivariance and hierarchical feature extraction for images, and give an example sequence of layers suitable for 64x64 input images aimed at classification.
Sample Answer
Convolution: a local linear filter applied across the image with shared weights; extracts local patterns (edges, textures). Stride: step size the filter moves—larger stride reduces output spatial size and increases translational subsampling. Padding: adding border pixels (usually zeros) so outputs keep spatial size or control edge effects. Pooling: downsampling (max/avg) that reduces spatial resolution and increases invariance to small translations. Receptive field: the region of the input that influences a given unit; grows with more layers, larger kernels, strides or pooling. Feature maps: the per-filter activation maps produced by convolutions; each map detects a particular learned pattern across spatial locations.How they enable translation equivariance and hierarchical features:- Weight sharing in convolution makes responses equivariant to translation: shifting the input shifts feature maps similarly (same detector applied everywhere).- Pooling and strides introduce some translation invariance by reducing sensitivity to small shifts (outputs change less).- Stacking layers increases receptive field so early layers learn low-level features (edges, colors), mid layers combine them into motifs (corners, textures), and deep layers form high-level concepts (object parts), enabling hierarchical representation.Example 64×64 classification backbone:- Input 64×64×3- Conv 3×3, 64, stride=1, padding=1 → 64×64×64; ReLU- Conv 3×3, 64, stride=1, padding=1 → 64×64×64; ReLU- MaxPool 2×2, stride=2 → 32×32×64- Conv 3×3,128 ×2 (padding=1) → 32×32×128; ReLU; MaxPool → 16×16×128- Conv 3×3,256 ×2 → 16×16×256; MaxPool → 8×8×256- Conv 3×3,512 → 8×8×512; GlobalAvgPool → 1×1×512- FC (512 → num_classes) + softmaxThis sequence keeps small kernels (3×3) for efficient receptive field growth, uses pooling to reduce spatial size and increase invariance, and ends with global pooling to reduce parameters for classification—practical for production models.
EasyTechnical
40 practiced
You have a tabular dataset with 10,000 rows and 30 mixed numeric/categorical features for a binary classification task. As an ML engineer, decide whether to use deep learning or 'simpler' methods (e.g., XGBoost, logistic regression). Justify your choice considering sample size, feature types, interpretability, latency, and maintenance in production. Outline a pragmatic plan (model selection, validation strategy, and deployment) for your chosen approach.
Sample Answer
Decision and justification:For a 10,000‑row tabular dataset with 30 mixed features for binary classification I would choose a "simpler" tree‑based method (e.g., XGBoost or CatBoost) as the primary approach rather than a deep neural network. Reasoning:- Sample size: 10k is modest—trees generalize well on tabular data with limited samples; deep nets risk overfitting unless heavy regularization/augmentation is used.- Feature types: gradient-boosted trees handle mixed numeric/categorical (CatBoost native) with little preprocessing.- Interpretability: tree models support SHAP, feature importance — valuable for debugging and business stakeholders.- Latency & maintenance: smaller models are faster to serve, easier to version, cheaper to maintain and retrain in production.Pragmatic plan1. Baseline & preprocessing- Baseline: logistic regression with basic imputation + one‑hot/target encoding to set a performance floor.- Clean: impute, cap outliers, encode categoricals (use CatBoost/XGBoost native or target encoding with K‑fold smoothing), scale numerics if needed.2. Model selection & tuning- Primary: XGBoost and CatBoost; try LightGBM as alternative.- Hyperparameter tuning via randomized search + early stopping (20–50 trials), optimize AUC or business metric.- Try a small MLP only as a comparison (with dropout, early stopping).3. Validation strategy- Stratified k‑fold (k=5) to preserve class balance; use grouped CV if temporal or user groups exist.- Hold out a final test set (10–15%) for unbiased evaluation and calibration (Platt/isotonic).- Use metrics: ROC AUC, precision@k, F1 depending on business needs; monitor calibration and confusion matrix.4. Explainability & robustness- Produce SHAP explanations, partial dependence plots; run feature ablation.- Test for data drift, adversarial examples, and fairness checks if relevant.5. Deployment & maintenance- Package model with metadata into a container; register artifact in model registry (MLflow/Sagemaker).- Serve via REST/gRPC with latency budgets; add batching or async if needed.- Monitoring: track prediction distribution, input feature drift, performance on labeled feedback; set retrain triggers or scheduled retrains.- CI: unit tests for preprocessing, integration tests for inference, reproducible pipelines for training.Trade-offs: If later data grows to >100k or raw signals (images/text) are added, revisit deep learning. For now, gradient-boosted trees balance performance, interpretability, and production practicality.
Unlock Full Question Bank
Get access to hundreds of Deep Learning Concepts and Theory interview questions and detailed answers.
