Debugging and Testing ML Systems

Finding, diagnosing, and fixing problems in ML code, data, and models, and building tests that catch these problems before they reach users. Covers common ML pitfalls (data leakage, shape mismatches, silent training bugs, mis-specified loss or metrics), root-cause analysis of model regressions and production incidents (accuracy drops, calibration drift, intermittent or hard-to-reproduce failures), distributed-training-specific failures (multi-GPU divergence, intermittent OOM, precision-related instability), and the diagnostic tooling that supports it (reproducibility artifacts, structured logging, instrumentation). Also covers testing ML systems directly: unit tests for data and feature pipelines, validation checks for datasets and features, test oracles and acceptance criteria for probabilistic or non-deterministic model outputs, and integration and regression tests that catch model or pipeline regressions before deployment. Emphasizes the engineering rigor that keeps ML systems correct and maintainable.

86 questions

Responsible AI: Fairness, Bias, and Interpretability

Building ML and AI systems that are fair, explainable, and safe. Covers identifying and mitigating bias, fairness metrics and tradeoffs, model interpretability and explainability techniques, label-bias feedback loops, and responsible and safe development practices for production models. Emphasizes accountability and transparency as first-class design constraints.

142 questions

Generative AI and Large Language Models

The capabilities and behavior of modern generative and large language models. Covers how LLMs are pretrained, in-context learning and few-shot prompting, generative model families (autoregressive, diffusion), context windows, and tokenization and sampling. Emphasizes understanding what generative models can and cannot do and how they differ from discriminative ML.

21 questions

Model Evaluation and Validation

Measuring whether a model is good enough to trust and ship. Covers metric selection for classification, regression, and ranking (precision/recall, ROC-AUC, calibration, RMSE), offline validation design, evaluation-metric-to-business-objective alignment, and production safety guardrails. Emphasizes choosing metrics that reflect real objectives and avoiding misleading evaluations.

183 questions

Model Training Infrastructure and Distributed Training

Scaling model training across hardware and time. Covers GPU/accelerator considerations, data and model parallelism, distributed and large-scale training, experiment tracking and training infrastructure, and the training-versus-inference compute tradeoff. Focuses on the systems and resource decisions that make large-model training feasible.

22 questions

LLM Evaluation and Observability

Measuring, monitoring and releasing generative and LLM-powered systems. Covers evaluation of open-ended outputs (human review, calibrated model-graded judges, reference-based and embedding scores, perplexity), hallucination, toxicity, privacy-leak and red-team checks, offline benchmarks versus online monitoring, and production observability (logs, traces, metrics, cost and token attribution, privacy-safe telemetry, drift and quality alerts). Also release decisions: A/B tests, canary rollouts, SLOs and go or no-go gates. Building the grounding architecture itself, calibration of predictive models and general software testing are covered elsewhere.

6 questions

ML Research to Production

Bridging novel research and shipped systems. Covers the research-to-production pipeline, staying current with emerging techniques, prototyping and validating novel algorithms or system designs, and the tradeoffs of adopting cutting-edge methods in a production setting. Emphasizes translating advanced or experimental work into reliable, shippable ML.

6 questions

Machine Learning Fundamentals

Core concepts that underpin all machine learning work. Covers the difference between supervised, unsupervised, and reinforcement learning, the training/validation/test split, the learning objective, and how models generalize from data. Emphasizes conceptual clarity and knowing which learning paradigm fits a given problem rather than any single algorithm.

49 questions

Prompt Engineering

Designing prompts and instructions that steer large language models toward reliable, controllable outputs without changing model weights. Covers zero- and few-shot prompting and example selection, chain-of-thought and step-by-step reasoning prompts, system, developer and role instructions and instruction hierarchy, output formatting and structured output (JSON schemas, templates, constraints), task decomposition and prompt chaining, managing context and token budgets inside a prompt, prompt injection as a design constraint, and iterating on, versioning and regression-testing prompts against a test set. Also covers prompt-level ways to reduce hallucination (grounding in supplied text, quoting support, allowing abstention) and unsafe, toxic or off-policy output (scoped roles, refusal templates), and prompting a model to generate constrained, validated output such as SQL. Boundary: excludes how models are pretrained and behave internally, sampling and decoding internals, fine-tuning, retrieval pipelines, and general LLM evaluation and monitoring.

0 questions

Applied ML Problem Framing and Tradeoffs

Turning an ambiguous real-world problem into a well-posed ML solution. Covers problem definition and objective specification, mapping business goals to a modeling objective, stakeholder and objective-function tradeoffs, computational feasibility and resource constraints, and walking through past ML projects and their decisions. Emphasizes judgment about whether and how ML applies before any modeling begins.

69 questions

Deep Learning: Neural Networks and Architectures

How deep neural networks are built and trained. Covers network fundamentals, activation functions and non-linearity, loss function selection, backpropagation, optimization and learning-rate behavior, and diagnosing vanishing or exploding gradients, along with the major architecture families: convolutional networks for images and recurrent networks, LSTMs, and GRUs for sequences. Emphasizes both how to train deep networks stably and how to match an architecture family to the structure of the data.

129 questions

End-to-End ML System Design

Designing a complete machine learning system from problem to production. Covers the components and architecture of a production ML system, data flow from ingestion to serving, scalability, and integration of models into a larger product. Emphasizes the whole-system design tradeoffs that appear in ML system-design interviews.

52 questions

LLM Fine-Tuning and Alignment

Adapting foundation models to specific tasks and desired behavior. Covers transfer learning and using pretrained models, full and parameter-efficient fine-tuning, instruction tuning, and alignment methods such as RLHF and preference optimization. Focuses on when and how to customize a base model rather than prompt it, and the data and compute tradeoffs involved.

17 questions

Data Preparation and Class Imbalance for ML

Preparing training data and handling skewed or shifting distributions. Covers preprocessing and cleaning for model input, data augmentation, distribution shift, class imbalance techniques (resampling, reweighting, threshold tuning), and cold-start scenarios where labeled data is scarce. Focuses on the data-side decisions that determine whether a model can learn at all.

125 questions

On-Device and Edge ML

Running models on resource-constrained and privacy-sensitive devices. Covers model optimization for mobile and embedded hardware, on-device inference and privacy architecture, and dedicated neural accelerators. Focuses on the size, latency, power, and privacy tradeoffs of moving inference off the server and onto the edge.

9 questions

Anomaly and Fraud Detection

Detecting rare, abnormal, or adversarial events in data. Covers anomaly-detection techniques, fraud and risk modeling, handling extreme class imbalance, and the precision/recall and latency tradeoffs of real-time detection systems. Focuses on the modeling patterns unique to needle-in-a-haystack detection problems.

49 questions

Model Deployment and Inference Optimization

Serving trained models efficiently in production. Covers deployment and containerization, real-time and batch serving, latency budgets, throughput and cost optimization, quantization and model compression, and online/real-time learning constraints. Emphasizes meeting production performance targets without sacrificing model quality.

30 questions

Computer Vision

Modeling and pipelines for image data. Covers image classification, object detection (anchors, feature pyramids, non-maximum suppression, one-stage versus two-stage and anchor-free detectors), and semantic and instance segmentation; image preprocessing and augmentation; transfer learning, pretrained backbones and self-supervised pretraining; convolutional networks compared with vision transformers; annotation, labeling quality, dataset formats, active learning and leakage-safe splits; evaluation with intersection over union, average precision and mean average precision, and Grad-CAM; handling class imbalance, tiny objects and large images; and face detection and verification. Focuses on the fundamentals of applying learning to visual inputs and the practical constraints of vision systems. Model serving and compression, on-device deployment, distributed training, generic network mechanics, language and multimodal models, fairness and interpretability audits, and production monitoring and retraining pipelines are covered elsewhere.

5 questions

Classical Machine Learning Algorithms

Foundational non-deep-learning models and when to reach for each. Covers linear and logistic regression, decision trees and ensemble methods (random forests, gradient boosting), support vector machines, k-nearest neighbors, and clustering, including their assumptions, strengths, and failure modes. Focuses on algorithm selection and the numerical and implementation considerations behind these workhorse models.

86 questions

Recommendation, Ranking, and Personalization

Systems that select and order items for users. Covers candidate generation and ranking (two-stage retrieval, embedding-based retrieval with two-tower models and approximate nearest-neighbour candidate lookup, multi-stage scoring and cross-encoder re-ranking), collaborative and content-based filtering, matrix factorization (alternating least squares, Bayesian personalized ranking), sequence-aware and graph-based recommenders, cold start, learning-to-rank and negative sampling, turning noisy implicit feedback into labels, position, exposure and popularity bias and feedback loops, diversity, fairness-aware exposure and multi-objective re-ranking, counterfactual (off-policy) evaluation of recommenders (inverse propensity scoring, doubly robust estimators), and exploration with multi-armed and contextual bandits (epsilon-greedy, upper-confidence-bound and Thompson sampling, LinUCB, regret, non-stationarity, constraints, rollout, and when a bandit beats a fixed split or full reinforcement learning). Includes end-to-end recommender and personalized feed design. Focuses on the modeling and evaluation patterns specific to recommendation and ranking at scale. General A/B test design, generic ranking-metric definitions and ML serving infrastructure are covered elsewhere.

59 questions

Model Selection, Tuning, and Generalization

Choosing and tuning models so they generalize to unseen data rather than memorizing the training set. Covers the bias-variance tradeoff and its decomposition, diagnosing over- and under-fitting from learning curves, and regularization techniques such as L1/L2 penalties, dropout, and early stopping, alongside cross-validation strategies and grid, random, and Bayesian hyperparameter search. Emphasizes a principled, reproducible process for selecting model complexity and tuning against a real compute-versus-accuracy budget rather than ad-hoc trial and error.

84 questions

Natural Language Processing

Core natural language processing building blocks: text preprocessing and tokenization (word, character, subword BPE/WordPiece/unigram, multilingual, Unicode and noisy user-generated text), word and sentence embeddings (Word2Vec, GloVe, contextual embeddings, pooling and semantic similarity search), text classification, sentiment and intent detection (including limited labels, active learning and long documents), sequence labeling and named entity recognition (BIO tagging, CRFs, relation extraction, slot filling and semantic parsing), sequence-to-sequence models and machine translation (teacher forcing, exposure bias, beam search, BLEU, low-resource and cross-lingual transfer), topic modeling and theme extraction, language-model basics such as perplexity, and fine-tuning pretrained encoders like BERT for NLP tasks, including task-appropriate evaluation metrics, annotation and error analysis. Excludes transformer architecture internals, LLM pretraining and decoding behavior, retrieval-augmented generation and vector index engineering, parameter-efficient fine-tuning, and model serving or MLOps.

20 questions

Feature Engineering and Feature Stores

Transforming raw data into predictive model inputs and serving those features reliably. Covers feature creation and selection, encoding high-cardinality and categorical variables, representation learning, and the design of feature stores for training/serving consistency. Emphasizes features as a primary lever on model quality and the operational challenges of keeping them fresh and consistent.

62 questions

Retrieval-Augmented Generation (RAG)

Grounding language-model outputs in external knowledge at inference time. Covers document ingestion, chunking and embedding, sparse, dense and hybrid retrieval (BM25, score fusion, contrastive training of retrievers), vector search and approximate-nearest-neighbour index engineering (HNSW, IVF, product quantization, filtering, updates, sharding), re-ranking with cross-encoders, and context assembly with prompt templates and citations. Also covers reducing hallucination through verification and abstention, evaluating retrieval-augmented systems, and production concerns such as freshness, access control, prompt injection through retrieved documents, deletion and privacy, caching, cost, and weighing retrieval against fine-tuning. Generic LLM evaluation and release gates, the craft of prompts beyond retrieved-context templates, attention internals, parameter-efficient fine-tuning, model serving and distributed inference engineering, database indexes, and whole-product ML system design are covered elsewhere.

2 questions

MLOps: Monitoring, Retraining, and Lifecycle Management

Operating machine learning systems reliably over time. Covers model and data monitoring, drift and degradation detection, feedback loops, retraining and model-freshness strategy, versioning and model registries, and pipeline and workflow orchestration. Focuses on keeping deployed models healthy and reproducible across their lifecycle.

60 questions

Transformers and Attention

The transformer architecture behind modern language models. Covers queries, keys and values, scaled dot-product and multi-head attention (including why scores are scaled and worked numeric examples), masking, positional encoding (sinusoidal, learned, relative with bucketed biases, rotary, and 2D schemes for image patches), encoder-only, decoder-only and encoder-decoder variants, residual connections, layer normalization placement and the feed-forward block, parameter, memory and compute accounting, why transformers scale better than recurrent models, efficient and long-context attention variants (sparse, local window with global tokens, hashing-based, linear, low-rank, FlashAttention, multi-query), and what attention heads learn and what raw attention weights do and do not explain. Includes implementing attention and its components from scratch. Distributed training, serving and quantization, fine-tuning, pretraining objectives, and general optimizer practice are covered elsewhere.

1 questions

Machine Learning & AI Topics