Problem Decomposition Questions
Break complex problems into smaller, manageable subproblems and solution components. Demonstrate how to identify the root problem, extract core patterns, choose appropriate approaches for each subproblem, sequence work, and integrate partial solutions into a coherent whole. For technical roles this includes recognizing algorithmic patterns, scaling considerations, edge cases, and trade offs. For non technical transformation work it includes logical framing, hypothesis driven decomposition, and measurable success criteria for each subcomponent.
EasyTechnical
118 practiced
Write a Python function that, given a dataset represented as a list of dictionaries with keys 'features' (dict) and 'label' (int), computes precision, recall and F1 for each label and returns the top 3 labels by lowest F1 (i.e., worst-performing slices). Include clear docstring and handle cases with zero predictions for a label safely.
Sample Answer
Approach: iterate samples, accumulate per-label true positives (TP), false positives (FP), false negatives (FN), compute precision/recall/F1 with safe handling when denominators are zero (precision or recall = 0), then return top 3 labels with lowest F1.Key points:- Time O(n) single pass, space O(L) for number of labels.- Edge cases handled: no predictions for a label (precision=0), no true instances (recall=0), ties broken by label order.- For unit tests: test balanced labels, labels with zero predictions, labels with zero true instances, and empty inputs.
python
from collections import defaultdict
from typing import List, Dict, Tuple
def compute_worst_f1_labels(dataset: List[Dict], predictions: List[int], top_k: int = 3) -> List[Tuple[int, float, float, float]]:
"""
Compute precision, recall, F1 for each label and return top_k labels with lowest F1 (worst-performing).
Args:
dataset: list of dicts, each must contain key 'label' (int) for ground truth.
predictions: list of predicted labels (ints), same length as dataset.
top_k: number of worst labels to return (default 3).
Returns:
List of tuples (label, precision, recall, f1) sorted ascending by f1 (worst first).
Notes:
- If there are zero predicted positives for a label, precision is set to 0.0 safely.
- If there are zero true positives + false negatives, recall is set to 0.0 safely.
"""
if len(dataset) != len(predictions):
raise ValueError("dataset and predictions must have same length")
tp = defaultdict(int)
fp = defaultdict(int)
fn = defaultdict(int)
labels_set = set()
for item, pred in zip(dataset, predictions):
true = item['label']
labels_set.add(true)
labels_set.add(pred)
if pred == true:
tp[true] += 1
else:
fp[pred] += 1
fn[true] += 1
results = []
for lbl in labels_set:
tp_v = tp[lbl]
fp_v = fp[lbl]
fn_v = fn[lbl]
prec = tp_v / (tp_v + fp_v) if (tp_v + fp_v) > 0 else 0.0
rec = tp_v / (tp_v + fn_v) if (tp_v + fn_v) > 0 else 0.0
f1 = (2 * prec * rec) / (prec + rec) if (prec + rec) > 0 else 0.0
results.append((lbl, round(prec, 4), round(rec, 4), round(f1, 4)))
# sort by f1 ascending and return top_k worst
results.sort(key=lambda x: x[3])
return results[:top_k]EasyTechnical
61 practiced
Explain the trade-offs when decomposing an ML inference system as a set of microservices versus a monolithic service. Discuss operational complexity, latency, model versioning, testing, and deployment speed, and give scenarios where one approach is preferable over the other.
Sample Answer
High-level trade-off summary:- Microservices split inference into small, focused services (preprocessing, model A, model B, postprocessing). Monolith bundles everything into one process.Operational complexity:- Microservices: higher ops overhead — more services to monitor, secure, scale, and manage networking. Requires service discovery, observability, and CI/CD per service.- Monolith: simpler ops, fewer moving parts and deployments.Latency:- Microservices: network hops add tail latency and serialization costs; better if services can be colocated or use fast RPC.- Monolith: lower in-process latency, fewer serialization costs — preferable for low-latency SLAs.Model versioning:- Microservices: per-model versioning and can A/B different models independently; easier to rollback a single model.- Monolith: versioning entangles components; rolling back requires full-service deployment.Testing:- Microservices: easier unit/integration testing per component but harder end-to-end testing and reproducibility.- Monolith: simpler end-to-end tests and local replication, but harder to isolate model regressions.Deployment speed:- Microservices: faster, targeted releases for single model/component; but CI/CD pipelines multiply.- Monolith: slower releases (bigger artifact), but simpler pipeline.When to prefer which:- Microservices: heterogeneous models, independent scaling needs (large vision model + small text filter), frequent model experimentation/A-B, large teams owning different components.- Monolith: strict low-latency needs, small team, simple pipeline, or early-stage prototypes where simplicity and fast iteration matter.Best practices:- Start monolithic for MVPs; migrate to microservices when scale, independent scaling, or team boundaries justify the ops cost. Use feature flags, model registry, end-to-end tests, and service meshes or fast RPC (gRPC) to mitigate microservice latency and complexity.
HardTechnical
78 practiced
You must evaluate trade-offs between increasing model size, reducing latency, and lowering energy consumption for a deployed vision model. Decompose a quantitative evaluation plan: what experiments you'd run, cost models to build, metrics to track, and how you would present the Pareto frontier to leadership to justify a choice.
Sample Answer
Requirements & framing:- Objective: choose a deployed vision model variant that balances accuracy, latency (SLA), and energy (cost/carbon) under target traffic.- Constraints: target hardware (GPU/CPU/edge), throughput requirement, budget, accuracy floor.Quantitative evaluation plan — experiments1. Model variants: scale width/depth, checkpoints of pretraining, sparsity levels (pruning), quantization (8/4-bit), knowledge distillation, and architecture alternatives (MobileNet/ViT/ConvNeXt).2. Microbenchmarks: measure single-inference latency, P50/P95/P99, energy-per-inference (J) using power sensors (RAPL/nvml) and wall/host measurements; memory usage and peak bandwidth.3. Throughput stress tests: measure latency vs QPS curves for target batch sizes and concurrency; find knee where latency degrades.4. End-to-end A/B traffic test: deploy candidate models to small % of traffic to measure real-world latency, accuracy (CTR/TPR/F1), user-impact metrics.5. Robustness tests: dataset shift, adversarial/noise, temperature effects on energy/latency.6. Repeatability: run each experiment N times to estimate variance and confidence intervals.Cost models to build- Per-inference model: - EnergyCost = Energy_per_inference (J) * Energy_price_per_J ($/J) → $/inference - LatencyCost = penalty function if P95 > SLA (e.g., SLA breach fines or user drop-off estimated revenue loss) - InfrastructureCost = (GPU_hours * $/hour + amortized HW) / total_inferences - TotalCost = EnergyCost + InfrastructureCost + LatencyCost- Throughput scaling model: maps replicas to QPS and incremental cost; includes autoscaling inefficiencies.- Carbon model: CO2e = Energy_per_inference * grid_emissions_factor (kgCO2e/J)Metrics to track (per variant)- Accuracy: top-1/top-5, precision/recall, calibration metrics, AUC- Latency: mean, P50, P95, P99, tail behavior under load- Energy: Joules/inference (avg and distribution), watts under load- Resource: GPU memory, CPU usage, bandwidth- Cost: $/1M inferences, $/month at expected traffic- Reliability: error rates, OOMs, retries- Business: revenue impact, user engagement delta during A/BAnalysis & presentation of Pareto frontier1. Construct 2D Pareto frontiers for pairwise trade-offs (Accuracy vs Latency, Accuracy vs Energy, Latency vs Energy). Use scatter of variants; compute nondominated set (convex hull not required—use Pareto dominance).2. For 3-objective view, present: - 3D scatter (Accuracy, Latency, Energy) plus interactive filtering; or - Fix business constraints (e.g., max P95=100ms) and show 2D frontier for remaining objectives.3. Annotate "knee points" — variants where small accuracy loss yields large gains in latency/energy.4. Overlay cost-isolines ($/inference) or ROI contours to translate technical trade-offs into business impact.5. Uncertainty: show error bars/confidence intervals, and sensitivity analyses (how frontier shifts with traffic, energy price, SLA strictness).Decision rubric for leadership- Present recommended candidate(s) with explicit trade-offs: - If SLA & user experience critical: choose variant on frontier with lowest P95 under SLA and acceptable accuracy drop (<X%). - If cost reduction prioritized: choose knee point maximizing energy savings per % accuracy lost, showing annualized $ savings. - If carbon reduction prioritized: choose variant minimizing CO2e with acceptable latency.- Provide numerical scenarios: expected monthly cost/energy/accuracy at projected traffic, worst-case tail, and migration effort (engineering time, testing).- Include rollback plan and A/B rollout plan with metrics to monitor.Why this approach- Empirical experiments quantify real deployment behavior (not just FLOPs).- Cost models translate technical metrics into business outcomes.- Pareto presentation makes trade-offs explicit, enabling leadership to pick the point matching strategic priorities while seeing the marginal gains/costs.
MediumTechnical
56 practiced
Design how you would decompose a Retrieval-Augmented Generation (RAG) system into components (document store, retriever index, reranker, context builder, generator, and monitoring). For each component list one evaluation metric, one operational challenge, and one approach to validate it independently from the others.
Sample Answer
I'll decompose the RAG pipeline into the requested components and for each give: one evaluation metric, one operational challenge, and one independent validation approach.1) Document store (source corpus + metadata)- Metric: freshness latency (time between source update and index availability) or coverage (%) of expected documents.- Challenge: keeping store consistent and up-to-date with heterogeneous sources (sync/duplication).- Independent validation: run a controlled ingest test — add N known documents, then query by ID/metadata to verify they appear and metadata fields are correct; measure ingest-to-availability time.2) Retriever index (ANN / lexical index)- Metric: recall@k (fraction of gold passages retrieved in top-k).- Challenge: index staleness and distribution shift (new vocab, domain drift) degrading recall.- Independent validation: offline benchmark with held-out Q->relevant-passage pairs; compute recall@k and latency on representative hardware.3) Reranker (cross-encoder / re-scoring)- Metric: NDCG@k or MRR (ranking quality).- Challenge: high compute cost / latency for cross-encoders under production QPS.- Independent validation: run reranker on cached candidate lists from test set and compute NDCG/MRR; measure throughput/latency and GPU/CPU utilization separately.4) Context builder (prompt/context assembly + prompt templates)- Metric: context relevance ratio (percentage of context tokens that are from top-n relevant passages) or prompt length vs. token efficiency.- Challenge: prompt token budget and hallucination risk from irrelevant context.- Independent validation: synthetic assembly test — feed controlled candidate sets with known relevance labels and verify built prompts include top items, respect length limits, and maintain order; run ablation to measure effect on generator when replaced.5) Generator (LM that produces final answer)- Metric: answer factuality/precision (e.g., exact-match on factoid tests) and ROUGE/BLEU for expected outputs; or human-evaluated correctness rate.- Challenge: attribution/hallucination and latency/cost at scale.- Independent validation: use a held-out QA dataset where gold answers exist; supply generator with fixed contextual prompts (from context builder test) and measure factuality, fluency, and latency.6) Monitoring & observability- Metric: alerting coverage (proportion of critical failure modes with active alerts) and key SLA metrics (error rate, latency P95).- Challenge: noisy signals vs. real issues; correlating downstream quality drops to upstream component failures.- Independent validation: inject synthetic anomalies (e.g., drop retriever recall, corrupt metadata) and confirm monitors detect and correlate anomalies; run end-to-end golden queries to validate alerts trigger.Notes on integration testing:- Beyond independent validation, keep a small continuously-run end-to-end suite of golden queries that exercises the full pipeline to catch integration regressions and measure overall answer quality (human review or automated factuality checks).
HardTechnical
82 practiced
Case: Your product needs to integrate a third-party vision API to extract labels. Decompose integration risks (SLA, latency, accuracy mismatch, versioning, privacy), a testing plan, fallback strategies, and a rollout plan that minimizes user impact while enabling quick rollback. Provide one concrete fallback sketch.
Sample Answer
Decomposition of integration risks- SLA / availability: Third-party uptime, scheduled maintenance, regional outages. Risk: blocking label extraction pipeline.- Latency: Variable response times add tail latency to user flows or batch jobs.- Accuracy mismatch: Labels taxonomy, confidence calibration, domain shift (train/test mismatch) — leads to false positives/negatives.- Versioning: API model updates or schema changes breaking parsing or semantics.- Privacy / compliance: PII in images, transmission/storage policies, data residency, and retention rules.- Cost & rate limits: Unexpected bills or throttling under load.- Security: auth token compromise, man-in-the-middle.Testing plan- Contract tests: validate request/response schema, field types, enums; run on API stubs and real sandbox.- Accuracy evaluation: run a labeled holdout (1–5k images) through provider and baseline in-house model; compute precision/recall, confusion matrix, and calibration by confidence buckets.- Latency/load tests: synthetic concurrent requests to measure p95/p99 and error behavior under rate limits.- Chaos fault-injection: simulate timeouts, 429/5xx, and degraded responses.- Privacy compliance tests: verify redaction, encryption-in-transit, and no persistent logging of raw images.- End-to-end QA: UI/UX flows with degraded labels, human review for edge cases.Fallback strategies (hierarchy)1. Local lightweight model (edge or service) for basic labels — deterministic, lower accuracy but predictable latency.2. Cached recent/known labels from previous runs.3. Queue-and-process: accept request, return provisional state, process async and notify when labels ready.4. Human-in-the-loop: route low-confidence or critical cases for manual review.5. Feature-flagged degradation: hide labels or show “labels unavailable” UI.Concrete fallback sketch (implementation)- Deploy a small ResNet/efficientnet-based classifier container as “label-fallback” service trained on top N frequent labels (top 200). Expose same API contract.- Routing logic: - Primary: call third-party with timeout 800ms. If success and confidence >= threshold, return. - If timeout/5xx/low-confidence -> call fallback model (<=150ms) synchronously. - If fallback returns label with confidence >= lower threshold -> return with metadata: source=fallback, confidence. - Else enqueue image to async worker + human-review queue and return 202 with provisional UI state.- Metrics: track source distribution, delta-accuracy vs ground truth, cost per request.Rollout plan (minimize user impact, quick rollback)- Feature flag + staged rollout: 1. Internal canary (10% internal users): enable calls to third-party and fallbacks, heavy monitoring. 2. Small external canary (1% of traffic) with auto-rollback on SLO breach (availability, p99 latency, accuracy delta). 3. Gradually increase to 10% → 50% → 100% with health gates.- Monitoring & SLOs: - SLOs: end-to-end p95 latency < X ms, label accuracy drop < Y% vs baseline, error rate < 0.1%. - Set automated alerts and an ops runbook with steps: flip feature flag, throttle traffic, switch to fallback-only mode, revoke keys.- Quick rollback mechanisms: - Single feature-flag toggle to route all requests to fallback/local model. - DNS/ingress rule to stop sending traffic to third-party proxies. - Automated scripted revoke of API keys and roll-forward to cached responses.- Post-launch: periodic A/B evaluation, drift detection, retrain fallback model on a rolling dataset, negotiate version/stability SLAs with vendor.Why this minimizes risk- Multi-layered fallbacks preserve UX while limiting exposure to vendor outages or accuracy shifts.- Contract and accuracy testing catch semantic/version changes early.- Feature flags + canary + automated rollback allow rapid safe rollback with minimal user impact.
Unlock Full Question Bank
Get access to hundreds of Problem Decomposition interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.