Technical Communication and Explanation Questions

The ability to explain technical concepts, architectures, designs, and implementation details clearly and accurately while preserving necessary technical correctness. Key skills include choosing and defining precise terminology, selecting the appropriate level of detail for the audience, structuring explanations into sequential steps, using concrete examples, analogies, diagrams, and demonstrations, and producing high quality documentation or tutorials. Candidates should demonstrate how they simplify complexity without introducing incorrect statements, scaffold learning with progressive disclosure, document application programming interface behavior and workflows, walk through code or system designs, and defend technical choices with clear rationale and concise language.

MediumTechnical

38 practiced

Design an internal model card template that documents training data provenance, intended uses, performance across demographic subgroups, limitations, ethical considerations, and a maintenance schedule. For each field include a short description and provide an example of how to summarize subgroup performance in plain language for non-technical stakeholders.

Sample Answer

Model Card: Internal Template

1. Model Overview- Description: Brief purpose, model version, architecture, input/output.- Example: "Text classification v2.1 — BERT-based encoder; input: user query; output: intent label + confidence."

2. Training Data Provenance- Description: Sources, collection dates, sampling methods, preprocessing, consent/usage rights.- Example: "Sources: internal support tickets (2018–2024), public forums (CC BY 4.0). 60% sampled from high-frequency intents; PII removed."

3. Intended Uses & Out-of-Scope- Description: Primary use-cases, users, and prohibited uses.- Example: "Intended: routing customer support; Not for legal/medical advice."

4. Performance Metrics- Description: Overall accuracy, precision/recall, calibration, uncertainty measures; evaluation datasets and test procedures.- Example: "Overall accuracy: 91% on holdout set (n=50k)."

5. Subgroup Performance (format + plain-language summary)- Description: Per-demographic metrics (e.g., age, gender, language, region), statistical tests, sample sizes, confidence intervals.- Example (table): Age 18–25: acc 89% (n=8k); 26–45: 92% (n=20k); 46+: 85% (n=5k).- Plain-language summary for stakeholders: "The model works well for most age groups, correctly classifying about 9 out of 10 queries overall. Performance is slightly lower for users over 46 (about 85% correct), which may reflect fewer training examples from that group. We recommend monitoring and adding more older-user data."

6. Limitations & Failure Modes- Description: Known weaknesses, high-risk inputs, adversarial sensitivity.- Example: "Struggles with multi-intent queries and domain-specific jargon."

7. Ethical Considerations & Risk Assessment- Description: Potential harms, fairness risks, privacy concerns, mitigation strategies.- Example: "Risk of poorer service to underrepresented languages; plan: human-in-loop for low-confidence predictions."

8. Compliance & Data Governance- Description: Legal/regulatory constraints, data retention, anonymization, access controls.- Example: "GDPR considerations met; training logs retained 2 years."

9. Maintenance & Monitoring Schedule- Description: Retraining cadence, drift detection, ownership, rollback plan, alerting thresholds.- Example: "Monthly performance checks; retrain quarterly or when accuracy drops >2% for any critical subgroup."

10. Action Items & Contact- Description: Responsible owners, next steps, escalation path.- Example: "Owner: AI Platform Team; Contact: ai-platform@example.com."

Notes:- Always include sample sizes and CIs when reporting subgroup metrics.- For non-technical summaries, use frequencies (e.g., "9 out of 10") and explain causes and mitigation next steps.

MediumTechnical

33 practiced

Compose a short, persuasive email to your infrastructure manager requesting additional GPU resources to train a new model. Include a one-paragraph justification linking the model to business outcomes, an estimated cost and training time, potential risks, and a clear ask with proposed timeline. Keep the tone concise and actionable.

HardTechnical

33 practiced

Write a comprehensive FAQ and troubleshooting guide targeted at DevOps and ML engineers for intermittent GPU out-of-memory errors observed during training. Include probable root cause hypotheses, diagnostic commands, temporary mitigations like gradient accumulation, permanent fixes such as model architecture changes, and how to capture reproducible OOM traces for debugging.

Sample Answer

Situation: Intermittent GPU OOMs during training can be caused by transient allocation spikes, fragmentation, non-deterministic batching, race conditions in distributed runs, or memory leaks from retained tensors. Below is an FAQ + troubleshooting guide targeted to DevOps and ML engineers.

FAQ — probable root causes- Batch-size variance: last batch larger or augmentation increases memory.- Non-deterministic DataLoader yielding different shapes.- Accumulated gradients or optimizer states growing (e.g., fp16 scaler).- Memory fragmentation from many short-lived allocations.- Leaked references (keeping tensors in lists/logging).- Multi-process/device contention (shared GPU between jobs).- GPU driver/CUDA non-determinism or bug.- Distributed training race where rank occasionally holds extra tensors.

Quick diagnostics (commands)- Check basic GPU usage:

nvidia-smi --query-gpu=index,name,utilization.gpu,utilization.memory,memory.total,memory.used -f csv -l 2

- List processes on GPU:

nvidia-smi pmon -c 1
ps -ef | grep python

- System-level metrics (sampling):

nvidia-smi dmon -s u -d 2

- Container cgroup memory limits:

cat /sys/fs/cgroup/memory/memory.usage_in_bytes

In-process diagnostics (PyTorch examples)- Snapshot memory stats:

import torch
print(torch.cuda.memory_summary(device=0, abbreviated=False))
print("allocated:", torch.cuda.memory_allocated())
print("max_allocated:", torch.cuda.max_memory_allocated())

- Reset/track peaks:

torch.cuda.reset_peak_memory_stats()
# run step
print(torch.cuda.max_memory_allocated())

- Synchronous stack trace on OOM:set environment before running:

export CUDA_LAUNCH_BLOCKING=1

- Capture profiler trace:

from torch.profiler import profile, ProfilerActivity
with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], record_shapes=True) as p:
    model(input)
print(p.key_averages().table(sort_by="cuda_time_total"))
p.export_chrome_trace("trace.json")

How to capture reproducible OOM traces1. Create a minimal script that reproduces the failure deterministically: fix random seeds, use a fixed batch from disk, and run the single problematic step in a loop.2. Run with CUDA_LAUNCH_BLOCKING=1 to get accurate stack traces.3. Use torch.cuda.reset_peak_memory_stats() before the step and record torch.cuda.max_memory_allocated() after.4. Use PyTorch profiler to capture allocation shapes and kernels (export trace.json to open in Chrome tracing or Nsight).5. If intermittent only in multi-GPU, reproduce with the same number of processes and use logs per-rank.6. Collect nsys trace:

nsys profile -o trace -t cuda,osrt,drm -- python train.py ...

Temporary mitigations (fast to apply)- Reduce batch size by 1–2 steps.- Enable mixed precision (AMP) to cut activation/optimizer memory:

from torch.cuda.amp import GradScaler, autocast
with autocast():
    out = model(x)

- Gradient accumulation: keep effective batch size without increasing memory:

accumulate_steps = 4
loss = ...
loss = loss / accumulate_steps
loss.backward()
if (step+1) % accumulate_steps == 0:
    optimizer.step(); optimizer.zero_grad()

- Free caches between iterations:

del tensors_you_no_longer_need
torch.cuda.empty_cache()

- Disable pin_memory=True in DataLoader if CPU->GPU pinned memory causing pressure.- Reduce num_workers or prefetch_factor to avoid queued allocations.

Permanent fixes / robust changes- Model architecture: reduce hidden dims, remove redundant layers, use depthwise separable convs.- Activation checkpointing (recompute activations to trade compute for memory).- Use optimizer state sharding (ZeRO from DeepSpeed/Fairscale) to reduce per-GPU memory.- Use model parallelism or pipeline parallelism when model exceeds single GPU limits.- Convert parameters/optimizer states to fp16 with careful loss-scaling or use bfloat16 if supported.- Fix code-level leaks: avoid storing tensors in global lists, ensure detach().cpu() before logging.- Upgrade drivers/CUDA/cuDNN and framework to versions with known memory fixes.- Use static shapes for inputs (pad/truncate) to avoid dynamic allocation spikes.

Debugging checklist- Reproduce with minimal script and seed.- Capture GPU allocation trace (nsys/torch.profiler).- Verify process count and shared GPU usage.- Inspect logs for gradually increasing RAM/GPU memory → memory leak.- Run with reduced batch size to confirm threshold.- Try torch.backends.cudnn.benchmark=False to reduce nondeterministic workspace allocations.

Examples of subtle bugs- Logging a tensor each iteration without .item() or .cpu() retains GPU memory.- Using torch.no_grad() incorrectly so gradients retained.- Mixed precision plus in-place ops causing unexpected growth.

When to escalate to infra/driver- If nvidia-smi shows unexpected non-Python processes using memory.- If OOM occurs only under container orchestration — check cgroup limits and GPU sharing settings (MPS on multi-tenant).- If profiling shows CUDA runtime error or driver crashes — collect driver logs (dmesg, /var/log/nvidia-installer.log) and open an issue with framework and driver versions and captured traces.

Example minimal reproducible pattern (PyTorch)

import torch, random
torch.manual_seed(0)
x = torch.randn(8,3,224,224, device='cuda')
model = MyModel().cuda()
torch.cuda.reset_peak_memory_stats()
with torch.no_grad():
    for i in range(100):
        out = model(x)  # log shapes, probe memory
        print(torch.cuda.max_memory_allocated())

Final notes- Start with simplest checks (batch size, other processes). Use CUDA_LAUNCH_BLOCKING and profilers to obtain deterministic traces. Apply temporary mitigations to continue training while implementing longer-term fixes like checkpointing, ZeRO, or architecture changes. Collect and attach profiler traces and nsys outputs when filing issues.

MediumTechnical

40 practiced

Write a full API specification for a text generation endpoint named /v1/generate that accepts a JSON body with fields prompt, max_tokens, temperature, top_p, and stop. Define field types, default values, valid ranges, and error responses for 400, 429, and 500. Include a curl example showing a typical request and the expected JSON response structure. Target audience is backend engineers integrating the API.

Sample Answer

Overview/v1/generate — POST — synchronous text-generation endpoint. Accepts JSON body; returns generated text and metadata. Auth: Bearer token via Authorization header. Content-Type: application/json.

Request body schema (application/json)- prompt (string) — required - description: input text or instruction for the model - max length: 50,000 chars- max_tokens (integer) — optional, default: 256 - valid range: 1–4096- temperature (number) — optional, default: 1.0 - valid range: 0.0–2.0 (float) - semantics: higher = more random- top_p (number) — optional, default: 1.0 - valid range: 0.0–1.0 (float) - semantics: nucleus sampling- stop (array[string] or string) — optional, default: null - semantics: token(s) where generation stops; max 5 items, each ≤ 100 chars

Headers- Authorization: Bearer <API_KEY> — required- Content-Type: application/json — required- Accept: application/json — recommended

Successful response (200)Body (application/json):{ "id": "gen_12345", "model": "gpt-xyz", "prompt": "<echoed prompt or id>", "generated_text": "<string>", "tokens_used": { "prompt_tokens": 12, "completion_tokens": 45, "total_tokens": 57 }, "finish_reason": "stop" | "length" | "eos", "created_at": "2025-11-22T12:34:56Z"}

Error responses400 Bad Request — invalid input- Cause examples: missing prompt, invalid types, values outside ranges, too-long prompt, malformed JSON.- Body:{ "error": { "code": "invalid_request", "message": "prompt is required and must be a string", "details": {"field":"prompt"} }}

429 Too Many Requests — rate limit / quota exceeded- Headers: Retry-After: seconds- Body:{ "error": { "code": "rate_limit_exceeded", "message": "Rate limit exceeded. Retry after 30 seconds." }}

500 Internal Server Error — server-side- Body:{ "error": { "code": "server_error", "message": "An unexpected error occurred. Try again later.", "trace_id": "abc123" // for support/debugging }}

Validation and behavior notes- If both temperature and top_p provided, they are combined per sampling rules; use either or both.- If stop matched mid-token, model returns finish_reason "stop".- max_tokens caps generated tokens; may return fewer tokens.- Rate-limiting and per-account quotas apply; prefer exponential backoff on 429.

Curl example

bash

curl -X POST "https://api.example.com/v1/generate" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Write a concise summary of transformers in NLP.",
    "max_tokens": 120,
    "temperature": 0.7,
    "top_p": 0.9,
    "stop": ["\n\n"]
  }'

Expected example response (200)

json

{
  "id": "gen_01F...",
  "model": "gpt-xyz",
  "prompt": "Write a concise summary of transformers in NLP.",
  "generated_text": "Transformers use self-attention to model long-range dependencies, enabling parallel training and state-of-the-art performance across NLP tasks...",
  "tokens_used": {
    "prompt_tokens": 8,
    "completion_tokens": 42,
    "total_tokens": 50
  },
  "finish_reason": "stop",
  "created_at": "2025-11-22T12:34:56Z"
}

MediumTechnical

40 practiced

A deployed model shows biased outcomes for a protected group. Explain how you would communicate the issue, its potential impact, and your proposed remediation steps to a mixed audience of legal counsel, product managers, and senior engineers. Provide the structure of the message, sample language to avoid, and a prioritization of short-term and long-term actions.

Sample Answer

Structure of the message (opening → impact → evidence → remediation → ask/next steps):1. Opening (one-line): state the issue, audience-specific framing. - Example: “We’ve detected statistically significant adverse outcomes for [protected group] in Model X’s decisions; I want to explain scope, legal/regulatory risks, product impact, and a remediation plan.”2. Facts & evidence (concise): metrics, data slices, period, reproducibility. - Example: “Over the past 30 days, false positive rate for Group A = 18% vs others = 6% (p<0.01). This holds across two datasets and our A/B shadow traffic.”3. Impact by stakeholder: - Legal: compliance, litigation, regulatory reporting obligations. - PM: user trust, retention, feature rollout risks, KPI drift. - Engineers: technical root causes, reproducibility, rollback/feature flags.4. Proposed remediation (short-term vs long-term) with owners & timeline.5. Ask & next steps: decisions needed (pause rollout, data collection, budget), meeting cadence.

Sample short-term actions (priority):- Immediately enable safety guardrails (threshold tuning or feature-flag rollback) — owner: Eng — 24–72 hours.- Stop/limit affected decisions in production and enable monitoring dashboard — owner: Eng/Product — 48 hours.- Notify legal and prepare incident log for regulators if required — owner: Legal/PM — 48 hours.- Run root-cause analysis on training data and model behavior (bias attribution tests) — owner: AI Eng — 1 week.

Sample long-term actions:- Re-balance or augment training data, add counterfactual/data-synthesis — 2–8 weeks.- Retrain with fairness-aware objectives or post-processing (e.g., equalized odds) and cross-validate across slices — 4–12 weeks.- Add continuous bias monitoring, dashboarding, and escalation playbook — 4–6 weeks.- Incorporate fairness into model governance: review gates, audit trail, stakeholder sign-off — ongoing.

Sample language to avoid (and why):- “The model is racist/biased” — inflammatory; use precise, evidence-based phrasing.- “It’s just a data problem” — dismissive of systemic issues and stakeholder concerns.- “We can fix it quickly” — overpromising; give realistic timelines.- “Not our problem / low probability” — minimizes impact and damages trust.

Why this works: concise facts build credibility; stakeholder-tailored impacts align incentives; clear short/long-term actions with owners enable prompt decisions and compliance.

Unlock Full Question Bank

Get access to hundreds of Technical Communication and Explanation interview questions and detailed answers.

Join thousands of developers preparing for their dream job.