Pre training and Fine tuning Questions

Covers principles and practical strategies for pre training large models and adapting them to downstream tasks. Includes pre training objectives such as causal language modeling, masked language modeling, and next sentence prediction; reasons why pre training enables transfer such as representation learning and emergent capabilities; and modern self supervised and curriculum pre training approaches. Details fine tuning and adaptation methods including full model fine tuning, supervised fine tuning, instruction tuning, domain adaptation, few shot and in context learning, and parameter efficient methods such as low rank adaptation, prefix tuning, and prompt tuning. Addresses trade offs between pre training versus fine tuning versus using in context methods, cost and compute considerations, catastrophic forgetting and regularization, evaluation and benchmarking for downstream tasks, and when to retrain or continuously adapt models. Applicable to text and vision foundation models and other pretrained architectures.

HardTechnical

55 practiced

You must fine-tune a multilingual model to improve performance on low-resource languages with limited labeled data. Propose a combined approach using pretraining continuation, cross-lingual transfer, synthetic data generation, and parameter-efficient fine-tuning. Include evaluation and fairness checks.

Sample Answer

Approach overview: combine language-adaptive continuation pretraining, targeted cross-lingual transfer, high-quality synthetic-data augmentation, and parameter-efficient fine-tuning to maximize signal from scarce labels while controlling compute and preserving fairness.

1) Clarify scope & data: list target low-resource languages, downstream tasks, available unlabeled corpora, and label budgets. Define success metrics (task F1/EM, calibration, fairness).

2) Pretraining continuation (Language-Adaptive Pretraining, LAP):- Continue MLM/denoising on unlabeled in-language + related-language corpora for a few epochs to adapt token/grammar priors. Use domain and script-specific tokenizers if needed (or sentencepiece update + embedding remapping).- Regularize to avoid catastrophic drift (mix with original multilingual data at small ratio).

3) Cross-lingual transfer:- Select high-resource anchor languages with typological or lexical proximity.- Use multitask joint fine-tuning on combined labeled data (weighted sampling) and incorporate language ID tokens.- Add language-specific adapters (lightweight) so shared backbone transfers general knowledge while adapters capture language-specific nuances.

4) Synthetic data generation:- Back-translation: translate high-resource labeled examples into target languages with strong NMT; round-trip and quality-filter via language model scoring.- LLM augmentation: prompt a calibrated multilingual LLM to generate labeled examples; constrain diversity and include uncertainty estimation.- Filter with ensemble scorers (perplexity, bilingual lexicon coverage, semantic similarity) and human spot-checks for sensitive attributes.

5) Parameter-efficient fine-tuning (PEFT):- Use adapters or LoRA on top of the continued-pretrained backbone to fine-tune with limited labels. Freeze base weights to preserve shared knowledge.- Optionally use prefix-tuning for generation tasks.- Curriculum: warm-start with pseudo-labeled synthetic data, then fine-tune on true labels with higher weight.

6) Evaluation & robustness:- Hold out in-language test sets; measure task metrics, confidence calibration (ECE), and cross-lingual transfer gap (delta vs anchor).- Perform ablations: with/without LAP, synthetic data, adapters, to quantify contribution.- Test OOD/robustness with noise, orthography variants, code-switching.

7) Fairness, bias & coverage checks:- Demographic parity and subgroup metrics where labels/demographics exist.- Linguistic fairness: evaluate performance across dialects, scripts, formality levels.- Bias probes: template-based tests for stereotypes and toxic generation; toxicity classifiers in target languages (or translated probes).- Human evaluation focusing on harms, mistranslation, and cultural appropriateness; preferentially involve native speakers and community stakeholders.- Mitigation: reweight training, debiasing on representations, selective data curation, and refusal or fallback strategies for high-risk outputs.

8) Deployment & monitoring:- Monitor per-language metrics in production, feedback loops for mislabeled examples, and automated drift detection.- Budget/compute: prioritize LAP on smaller models or use mixed-precision; favor PEFT to reduce GPU/time.

Why this works: LAP improves language priors; cross-lingual transfer leverages abundant annotations; synthetic data expands label space with quality controls; PEFT enables efficient adaptation without overfitting. Combined with rigorous evaluation and fairness audits, this yields strong, responsible performance on low-resource languages.

HardTechnical

56 practiced

Discuss feasibility and trade-offs of combining LoRA (or adapters) with aggressive quantization (4-bit or lower) for inference on CPU-based serving. Include accuracy impacts, calibration/quantization-aware fine-tuning steps, and strategies to keep adapter updates working with quantized base models.

Sample Answer

Feasibility summary: Combining adapters/LoRA with aggressive (4-bit or lower) quantization for CPU inference is feasible and commonly used (e.g., QLoRA-style pipelines), but requires careful engineering to preserve accuracy and keep adapter updates operational. Main trade-offs are accuracy vs memory/latency and complexity of deployment (dequantization overhead, special kernels).

Accuracy impacts and causes:- Aggressive quantization (4-bit, INT4/NF4) can hurt activation/distribution tails and attention numeric stability; this primarily affects base model representational fidelity.- LoRA/adapters introduce small instructive deltas; if applied naïvely on top of severely quantized base weights, their corrective effect can be diminished or mis-scaled.- Errors concentrate in outlier channels and higher-precision-sensitive layers (attention, layernorm, first/last linear layers).

Calibration and quantization-aware fine-tuning steps:- Start with sensible PTQ: per-channel weight quantization, asymmetric ranges, and NF4 or custom codebook (for transformers).- Calibrate with representative corpus (100–200 batches) to capture activation ranges; collect min/max, histogram or MSE objectives.- Apply GPTQ/GPTQ-like second-order methods (Hessian approximations) or AWQ/GPTQ variants to reduce PTQ error on transformer blocks.- For best accuracy, run quantization-aware fine-tuning (QAT) or QLoRA: freeze quantized base, keep quantized weights in memory but simulate quantization noise during adapter training. Use straight-through estimators for integer ops.- Bias correction and layerwise re-scaling (e.g., update output biases after quantization) help recover final logits.

Strategies to keep adapters effective with quantized bases:- Store and train adapters/LoRA in higher precision (FP16/FP32). At inference, either: - Apply adapters as separate low-rank updates in FP16: dequantize only the layers needed to apply the adapter (hybrid execution). This keeps adapter efficacy while minimizing dequantization scope. - Fold LoRA into base weights after dequantizing that layer, then requantize the updated weights using per-channel quantization plus local GPTQ to reduce added error—best for static deployments.- Use hybrid precision: keep sensitive layers (query/key/value projections, output projection, first/last MLP) in int8/FP16 while others are int4.- Use multiplier scaling: learn small layerwise scaling factors for adapter outputs to compensate quantization scale mismatch.- During training, simulate target inference quantization (activation & weight quantization) so adapters learn to correct quantized dynamics (quantization-aware adapter fine-tune).- Employ inference kernels optimized for CPU (vectorized int4/int8, fbgemm, oneDNN) and fuse dequantize+gemm where possible to reduce latency.

Operational considerations:- Latency vs memory: dequantization and hybrid FP ops increase latency; folding adapters reduces runtime cost but needs retraining or safe requantization.- Validation: evaluate on downstream tasks and measure perplexity, exact-match/F1, and generation quality; focus on edge cases sensitive to numeric error.- Rollout: A/B test quantized+adapter stack; keep rollback path to FP16 baseline.

Bottom line: Aggressive quantization plus adapters is practical for CPU serving if you (1) use careful PTQ/GPTQ and calibration, (2) train adapters with quantization-in-the-loop or keep them higher-precision, and (3) adopt hybrid/folding strategies to balance accuracy, memory, and latency.

HardTechnical

55 practiced

Technical-domain-specific: For a retrieval-augmented generation system, discuss how you would fine-tune the base model so that it respects external knowledge (from a corpus) and abstains or defers when the retrieval is insufficient. Include loss design, calibration, and runtime checks.

Sample Answer

Approach summary: fine-tune the generator to ground outputs in retrieved passages and to abstain when retrieved evidence is insufficient by combining grounded generation objectives, an explicit abstention verifier, calibrated confidence, and runtime safety checks.

Training / loss design:- Base LM loss: standard cross-entropy L_CE on tokens conditioned on (query + top-K retrieved documents).- Grounding consistency loss: encourage generated spans to match retrieved evidence. Use contrastive/document-grounding loss L_ground = -log softmax(sim(h_gen, h_doc_i)) where h_gen are representations of factual tokens and h_doc_i are retrieved passage embeddings. This penalizes hallucinations.- Abstention/verifier loss: train an auxiliary classifier p(abstain|context) with binary cross-entropy L_abs where positive = cases where no retrieved doc supports the gold answer. Create training data by masking or corrupting retrievals.- Combined objective: L = L_CE + λ1 L_ground + λ2 L_abs. Tune λs to trade recall vs abstention.

Calibration:- After training, calibrate the verifier and generator confidences with temperature scaling or isotonic regression on a held-out dev set labeled for "supported / unsupported". Learn separate temps for generator token logits and abstain logit. Evaluate ECE and Brier score; optimize thresholds to meet target precision for non-abstain answers.

Uncertainty techniques:- Use deep ensembles or MC dropout at inference to estimate epistemic uncertainty; aggregate to boost abstention reliability.- Use retrieval-aware uncertainty: combine verifier p(abstain) with retrieval scores (BM25 / dense sim) via logistic fusion.

Runtime checks / orchestration:1. Retrieve top-K passages and compute retrieval coverage (fraction with high semantic overlap).2. Generate answer + generator confidence.3. Run verifier (and ensemble uncertainty) to get p_supported = 1 - p_abstain.4. Decision rule: - If p_supported > τ_high and generator_confidence high → return answer. - If p_supported < τ_low or high epistemic uncertainty → abstain/ask for clarification or return "I don't know" with reason ("insufficient evidence"). - If borderline → invoke secondary retrieval (larger K, different index) or a verifier model that extracts direct supporting spans.5. Always attach provenance: list top supporting passages with passage-level scores.

Evaluation & metrics:- Measure supported-precision, abstention-recall, F1 on supported answers, ECE, and utility-weighted metrics (penalize incorrect confident answers heavily).- Perform adversarial retrieval tests and human evaluation for faithfulness.

Practical notes:- Synthesize unsupported examples during fine-tuning to teach abstention.- Monitor trade-offs: higher λ_abs reduces hallucinations but can increase unnecessary abstention.- Use lightweight verifier to keep latency acceptable; escalate to expensive checks only when needed.

HardTechnical

56 practiced

For designing a multimodal (text+vision) foundation model, argue which pretraining objectives you would pick (contrastive, multimodal MLM, causal LM conditioning on image tokens) and why. Discuss dataset composition, aligned vs. non-aligned data trade-offs, and evaluation tasks to validate alignment between modalities.

Sample Answer

I would combine complementary objectives: contrastive pretraining + multimodal masked language modeling (M-MLM) + a conditional causal LM head on image tokens. Rationale:- Contrastive (e.g., CLIP-style) yields strong global alignment and robust retrieval/zero-shot classification; it’s sample-efficient for learning modality-agnostic embedding spaces.- M-MLM forces fine-grained cross-modal grounding (predict masked text conditioned on image and vice versa), improving phrase-level and region-level semantics.- Conditional causal LM (autoregressive text generation conditioned on image representations or discrete image tokens) trains the model for generative, instruction-following behaviors and captures multimodal coherence in long-form outputs.

Dataset composition and trade-offs:- Use a large corpus mixing: high-quality aligned image-caption pairs (10–30% of data) for strong grounding; noisy web crawls with weak/no alignment (50–70%) for breadth; synthetic pairs (caption augmentation, multi-caption paraphrases) for diversity.- Aligned data => better fine-grained grounding and supervised evaluation performance; non-aligned/weakly aligned => scale, robustness to domain shift, and coverage of rare concepts. Balance by curriculum: start contrastive + M-MLM on aligned data, then scale contrastive/self-supervised objectives on weak/noisy data, and finally fine-tune conditional LM on aligned + synthetic instruction pairs.

Evaluation tasks to validate alignment:- Retrieval (image→text, text→image) and recall@K for global alignment.- Visual question answering / grounded QA and referring expression comprehension for fine-grained grounding.- Image captioning and multimodal generation (BLEU/METEOR + human eval for factuality/coherence).- Cross-modal probing: masked-token prediction accuracy conditioned on image regions.- Robustness checks: compositionality tests, compositional perturbations, and factuality benchmarks (e.g., VCR, GQA, POPE).- Calibration and hallucination metrics (e.g., truthfulness score) during generative evaluation.

Trade-offs and final note:- Contrastive optimizes global similarity but can ignore fine details; M-MLM and conditional LM fill that gap. Combining objectives with staged curriculum and careful dataset mixing yields a foundation model that is both aligned and generative.

HardSystem Design

50 practiced

Design monitoring and automated rollback rules for a fine-tuned model serving in production. Include what metrics to track (performance, latency, calibration, safety signals), when to trigger rollback or retraining, and how to perform canary rollouts safely.

Sample Answer

Requirements & constraints:- Low latency (<200ms P95), high availability, safe outputs (no toxic content), calibrated confidences, ML drift detection, automated rollback with human-in-loop for high-risk failures, canary rollout support.

High-level architecture:- Inference cluster behind API gateway → traffic router (weights for canary) → observability pipeline (metrics collector, logging, metadata) → monitoring & decision service (rules + ML detectors) → orchestration (k8s + model registry) for rollout/rollback.

Key metrics to track (with example thresholds):- Performance: accuracy/F1 or task-specific metric on labeled evaluation stream (daily). Trigger if delta > -3% absolute or relative drop >10%.- Latency: P50/P95/P99 and tail errors. Trigger if P95 > 200ms or error rate >1%.- Calibration: ECE (expected calibration error). Trigger if ECE increases >0.05 or predicted-confidence vs. observed accuracy diverges.- Safety signals: toxicity rate, hallucination rate, policy violations from safety classifier. Trigger if >0.1% absolute increase or exceeds policy thresholds.- Traffic & business: conversion/drop-off, user satisfaction (NPS), manual flags.- Data drift: feature distribution shift (KL >0.2) or embedding drift.

When to rollback vs retrain:- Immediate automated rollback: - Critical safety breach (policy violation score high) OR latency/availability SLO breach OR error spike (>5%).- Automated rollback + alert: - Significant metric degradations (model quality drop > threshold, calibration loss) that persist for X minutes/requests (e.g., 30m or 10k requests).- Retrain trigger: - Persistent concept drift, rising label mismatch on evaluation set, or scheduled periodic retraining when labeled buffer > N samples (e.g., 50k).- Human-in-loop: Any false-positive sensitive cases or ambiguous safety signals require review before retrain.

Canary rollout procedure (safe steps):1. Create new model artifact with metadata, tests (unit, integration, safety unit tests).2. Internal canary: 1% traffic to new model + mirrored logging; run shadow mode for 24h to collect metrics.3. Automated checks: compare canary vs baseline across all metrics; require pass on gating rules (no safety increase, latency within 10%, quality >= baseline - small delta).4. Gradual increase: 1% → 5% → 20% → 50% → 100% with 30–60 min observation windows; each step gated by automated rules.5. Rapid rollback: orchestration supports instant weight revert; pre-warmed baseline instances to avoid cold-start.6. Post-deploy monitoring: 24–72h intensified monitoring; human review for flagged anomalies.

Implementation details:- Observability: Prometheus/Grafana for infra/latency; Kafka + feature-store + labeling pipeline for evaluation stream; ML-specific monitors (Evidently, WhyLabs) for drift and calibration.- Decision engine: rule-based + anomaly detection model with cooldown and hysteresis to avoid flapping (require sustained violation over window).- Audit & explainability: persist inputs/outputs, embeddings, safety scores for sampled requests; provide dry-run diffs for debugging.- Security & compliance: encrypt logs, PII redaction, retention policies.

Trade-offs:- Strict thresholds reduce risk but increase rollbacks; use canary and shadowing to minimize false alarms.- Automated rollback accelerates safety responses but must be conservative for business-critical metrics — combine automated rollback for safety/availability and human approval for marginal quality regressions.

This design balances fast mitigation (automated rollback for safety/availability), measured evaluation (canary + shadowing), and longer-term correctness (retraining triggers and human review).

Unlock Full Question Bank

Get access to hundreds of Pre training and Fine tuning interview questions and detailed answers.

Join thousands of developers preparing for their dream job.