Generative Models and Architectures Questions

Covers the fundamentals of how generative models are constructed and trained, including types such as variational autoencoders, generative adversarial networks, diffusion models, and large language models. Includes core concepts like attention and the transformer architecture, self supervised training objectives such as next token prediction, tokenization, scaling laws, and differences between generative and discriminative approaches. Also addresses practical techniques for adapting and improving models including fine tuning, transfer learning, prompt engineering, few shot and zero shot learning, inference trade offs, model compression, and deployment considerations such as latency, memory, and cost. Evaluation topics include likelihood based metrics and practical applied evaluation methods for generation quality.

HardSystem Design

32 practiced

Design a globally-distributed, low-latency service for a multimodal generative assistant (text + image) supporting privacy constraints where some user data must remain in-region. Describe data locality, model placement (edge vs region), cross-region coordination for models and embeddings, and how you would handle model updates and rollback with minimal downtime.

Sample Answer

Requirements & constraints:- Low-latency multimodal assistant (text+image) globally.- Some user data must remain in-region (data residency).- Strong privacy (at-rest/in-transit encryption, optional DP/MPC).- Minimal downtime for model updates/rollbacks.

High-level architecture:- Edge PoPs in major geo regions + regional data centers (region = legal boundary).- Lightweight edge inference (client-side or PoP) for tokenization, image preprocessing, and small distilled models for ultra-low-latency replies.- Heavy generation and multimodal fusion run in-region on regional clusters that hold resident user data.- Global control plane (no user data) handles orchestration, model catalog, config, and cross-region coordination.

Data locality:- User personal data and any model inputs tagged as "residency-required" are stored & processed only inside the user's region. Use region-scoped namespaces, encrypted volumes, and IAM policies enforced by the regional cluster.- Non-residency metadata (telemetry, anonymized stats) can flow to global control plane using aggregation with differential privacy.

Model placement & serving:- Edge: small distilled encoders/vision backbones and caching for common prompts; runs inference on PoP to precompute embeddings and validation checks.- Region: full multimodal models (fusion/generation) colocated with in-region data stores. Use autoscaled GPU/TPU pools.- Cold-model archive in a global object store; warm images pulled into regional caches on demand.

Cross-region coordination for models & embeddings:- Model catalog with immutable, versioned artifacts and checksums in global registry.- Embeddings: primary index kept per-region for resident user vectors. A global index stores only encrypted, privacy-safe summaries and cross-region pointers. Retrieval uses federated k-NN: query local index first; if allowed, query other regions via encrypted proxy and return scored candidates without moving raw user vectors.- Secure syncing: models and embedding indices transferred via TLS, signed artifacts, and at-rest encryption. Optionally use homomorphic encryption/MPC for cross-region similarity when privacy requires.

Model updates & rollback with minimal downtime:- Immutable model versions + descriptors. Deploy using staged rollout: 1. Canary in a small subset of PoPs and non-residency test traffic inside each region. 2. Blue-green within region—keep previous model serving pool healthy while warming new model. 3. Gradually shift traffic via weight-based routing in service mesh (Envoy/Istio) and observe latencies/quality metrics.- Fast rollback: traffic shift back to previous model (instant) + keep previous model loaded until new version validated.- Automated safety gates: metric thresholds, A/B quality tests, anomaly detectors; if tripped, automatic rollback.- For embeddings schema changes, support dual-indexing: serve both old and new indexes in parallel, migrate vectors asynchronously with backfill jobs; fall back to old index if retrieval errors increase.

Trade-offs and rationale:- Edge distillation reduces latency and cost but may lower fidelity—use for drafts, with regional model completing final output.- Keeping data and heavy models regionally ensures compliance but increases footprint and model sync complexity—mitigated by versioned artifacts and federated indices.- Canary + blue-green minimizes downtime and risk; dual-indexing avoids large migration windows.

Operational concerns:- Audit logs, signed artifacts, key management per-region (KMS per legal boundary).- Observability: per-region SLOs, embedding drift alerts, privacy-violation detectors.- Cost: balance global replication of models vs. cache-on-demand; use spot GPU for background backfills.

This design preserves in-region data residency, provides low-latency experiences via edge distillation and regional generation, coordinates embeddings/federated retrieval with privacy protections, and supports safe, low-downtime model rollout and rollback.

MediumBehavioral

39 practiced

Behavioral: Describe a time when you had to persuade product and infra teams to change the model-serving approach (for example, moving from CPU-based microservices to GPU inference). Use the STAR format: situation, task, action, result. Focus on metrics you used to make the case and how you handled stakeholder objections.

Sample Answer

Situation: At my previous company we served a vision model via CPU-based microservices. As usage grew, we saw rising latency and SLA breaches during peak traffic, impacting user engagement for our core image-tagging feature.

Task: I needed to persuade product and infra teams to adopt GPU inference for the model — not just for performance but in a cost-justified, low-risk way that aligned with product KPIs.

Action:- Collected metrics: 95th-percentile latency (p95) = 820ms, average throughput = 18 QPS per replica, SLA violation rate = 12% during peak, and cost per 1k inferences = $2.10 on CPU.- Built a prototype: containerized the model with TensorRT on a single GPU node and ran a 48-hour A/B test against the CPU service under realistic traffic patterns.- Measured prototype results: p95 latency dropped to 160ms, throughput rose to 120 QPS per GPU, and end-to-end user conversion for the feature increased by 6% in A/B traffic.- Created a cost model: though GPU hourly cost was higher, cost per 1k inferences fell to $0.85 due to consolidation and higher utilization. I included sensitivity analysis for utilization and traffic variance.- Addressed objections: for product (concerned about risk to user-facing metrics) I proposed gradual rollout with feature-flagged traffic and rollback thresholds tied to error-rate, latency, and conversion. For infra (concerned about operational complexity and vendor lock-in) I proposed using Kubernetes GPU autoscaling, single-vendor-agnostic runtime (ONNX/TensorRT), runbooks, and a 2-week pilot with full runbook validation.- Obtained sign-off by presenting a one-page ROI summary: performance gains, reduced SLA breaches, improved conversion, and breakeven timeline (6 weeks).

Result:- After phased rollout, p95 latency stabilized at ~170ms, SLA violations dropped from 12% to 1.5%, and feature-driven conversions improved 5.8% company-wide. Cost per 1k inferences decreased by ~60% versus CPU baseline at production utilization. The infra team adopted the GPU autoscaling runbook and we documented learnings for future models. Stakeholders remained confident because decisions were metric-driven, incremental, and reversible.

EasyTechnical

36 practiced

Explain the difference between generative and discriminative models in the context of machine learning. Provide concrete examples of each (e.g., autoregressive language models, VAEs, GANs vs. logistic regression, classifiers), discuss what objectives they optimize, and describe three practical implications of choosing a generative vs discriminative approach for a production AI system.

MediumTechnical

31 practiced

List and explain quantitative and qualitative metrics for evaluating generative model outputs in text and images. For text include perplexity, BLEU/ROUGE, BERTScore, and human evaluation; for images include FID, IS, and human preference. For each metric, state what it measures, known limitations, and when it can be misleading in production evaluation.

Sample Answer

Text metrics

- Perplexity - Measures: average surprisal of tokens under the model (lower = better predictability). - Limitations: correlates with next-token likelihood, not semantic quality or factuality. - Misleading when: comparing models with different tokenizers, temperature, or decoding strategies; low perplexity can coexist with hallucinations or dull repetitive text.

- BLEU / ROUGE - Measures: n-gram overlap (BLEU for precision, ROUGE for recall) with reference(s). - Limitations: penalizes valid paraphrases, brittle for creative tasks, sensitive to reference quality and number. - Misleading when: single-reference evaluations, diverse-valid outputs (dialogue, creative writing), or when higher scores reflect safe, generic language.

- BERTScore - Measures: semantic similarity via contextual embeddings (precision/recall/F1) between candidate and reference. - Limitations: depends on underlying embedding model and domain; can miss factual errors or fine-grained structure. - Misleading when: references are imperfect or when embeddings capture topical similarity but not factual correctness or coherence.

- Human evaluation (text) - Measures: subjective judgments (fluency, relevance, factuality, preference). - Limitations: costly, slow, annotator variance and bias, scale and rubric design affect results. - Misleading when: poorly defined instructions, non-representative annotators, or artificial prompts that differ from production usage.

Image metrics

- FID (Fréchet Inception Distance) - Measures: distance between real and generated image feature distributions (lower = closer). - Limitations: depends on the feature extractor (Inception), sensitive to dataset size and preprocessing. - Misleading when: domain mismatch (medical images), mode collapse vs. perceptual quality trade-offs, or small sample sizes.

- IS (Inception Score) - Measures: generated images’ classifiability and diversity via a pretrained classifier (higher = better). - Limitations: assumes classifier labels are meaningful, ignores closeness to real data and intra-class quality. - Misleading when: images are out-of-domain for the classifier, or high IS arises from adversarial or unfaithful artifacts that are confidently classified.

- Human preference (images) - Measures: subjective choices for realism, preference, fidelity to prompt. - Limitations: same as text human eval—cost, bias, and variability. - Misleading when: small, non-representative panels, unclear criteria, or interface effects (rendering quality affects judgment).

Practical guidance: combine automated metrics with targeted human evaluation against production prompts; track multiple complementary metrics, control for preprocessing/tokenization, and use calibration datasets to avoid deceptive improvements.

HardTechnical

28 practiced

Describe practical model compression techniques to reduce a 100B-parameter model footprint for inference: post-training quantization (8-bit, 4-bit), pruning (structured/unstructured), knowledge distillation, and parameter-efficient fine-tuning (LoRA). For each, explain expected gains, risks to downstream performance, and rollout strategies in production.

Sample Answer

Brief overview: For a 100B-parameter model you can reduce memory, latency, and cost using quantization, pruning, distillation, and parameter-efficient fine-tuning (LoRA). Below I list practical expectations, risks, and safe rollout patterns for each.

1) Post‑training quantization (8-bit / 4-bit)- Expected gains: 8-bit (INT8) cuts activation+weight memory ≈2× and inference throughput often 1.5–2× on supporting HW; 4-bit (INT4, quant-aware formats) can reach ~2.5–4× memory reduction and further throughput gains on specialized accelerators.- Risks: Out-of-distribution degradation, numeric instability for attention/LayerNorm, poorer calibration and generation quality if naive.- Rollout: Start with mixed-precision (keep sensitive layers FP16/FP32), use calibration on representative data, run automated NLU/NLG QA suites, A/B test; implement fallback to higher precision for failure cases.

2) Pruning (structured / unstructured)- Expected gains: Unstructured pruning (sparse weights) can reduce parameter storage nominally 2–4× but needs sparse kernels/hw to speed up; structured pruning (heads, MLP dims, layers) yields real latency/memory reduction ~1.2–3× depending on aggressiveness.- Risks: Loss of generalization, brittle behavior on rare inputs, reduced few-shot capabilities.- Rollout: Prefer gradual magnitude/pruning schedule with retraining or fine-tuning; validate across many tasks; deploy as canary with traffic shaping; maintain a “degraded quality” monitor.

3) Knowledge distillation- Expected gains: Student models can be 5–50× smaller (e.g., 10–20B student) with modest quality loss when distilled well; large latency and cost savings.- Risks: Distillation may compress spurious behaviors, lose emergent capabilities, or overfit teacher biases.- Rollout: Distill with mixed objectives (logits + supervised + RLHF if relevant), validate end-to-end metrics, roll out staged (internal users → small external cohort → full), keep teacher available for failing queries.

4) Parameter‑efficient fine‑tuning (LoRA)- Expected gains: Retains full model capacity while only storing small adapters (e.g., <1% of params). For multi-task/ensemble use, storage and deployment cost fall drastically without modifying base weights.- Risks: Slight task-specific performance regression versus full fine-tune; interaction when stacking many adapters.- Rollout: Serve base model with adapter loading on demand; test adapter composition; use canary and SLOs; maintain ability to revert to base.

Cross‑cutting rollout best practices- Comprehensive evaluation: task suites, safety checks, calibration, latency/p95, memory, cost.- Progressive deployment: internal → canary → gradual ramp with rollback triggers.- Monitoring & observability: accuracy, hallucination rates, latency, per-client degradation, and fallback paths.- Tooling: use libraries (e.g., ONNX/ORT, NVIDIA TensorRT, Intel OpenVINO, Fairseq/transformers + bitsandbytes/nn_pruning) and hardware-aware tuning.- Hybrid strategies: combine methods (INT8 + structured pruning + LoRA) for the best trade-offs; always preserve a higher-precision fallback or teacher service for high-risk requests.

Unlock Full Question Bank

Get access to hundreds of Generative Models and Architectures interview questions and detailed answers.

Join thousands of developers preparing for their dream job.