Natural Language Processing Fundamentals Questions

Foundational knowledge of Natural Language Processing including how text is represented and processed, common tasks, model classes, and practical tooling. Core concepts include tokenization and subword segmentation, embedding representations and word vector methods such as Word2Vec and GloVe, attention mechanisms, sequence to sequence models, recurrent neural networks, and transformer based architectures. Common tasks to understand include text classification, sentiment analysis, named entity recognition, and machine translation. Candidates should be able to explain how pretrained transformer models are used conceptually, trade offs between model types, basic training and evaluation approaches for language tasks, and practical experience or familiarity with common libraries and toolkits used in the field.

HardSystem Design

16 practiced

Design a scalable approximate nearest neighbor (ANN) pipeline for semantic search over 500 million sentence embeddings. Specify indexing approach, sharding and replication, recall vs latency trade-offs, memory/disk layout, choice of ANN library (FAISS/HNSW/ScaNN), and operational considerations for incremental updates and monitoring.

Sample Answer

Requirements & constraints:- 500M sentence embeddings (e.g., 768-d float32), target 10s–100s QPS, 50–200 ms p95 latency, recall@10 ≥ 0.9 desirable, budget & infra limits.

High-level architecture:- Embedding service (model) → Query router → ANN clusters (sharded) → Reranker/score normalization → API.

Indexing approach:- Use hybrid: coarse quantization + HNSW on PQ-compressed vectors. Build IVF (inverted file) or OPQ + Product Quantization (PQ) to reduce memory/disk, and per-cell HNSW for graph-based fast reranking inside candidates. This balances compression (low memory) and high recall.

ANN library choice:- FAISS (GPU for build/search with IVF+PQ+HNSW support) for bulk throughput and mature tooling. Optionally HNSWlib for CPU-only low-latency, ScaNN if on TPU-friendly stacks.

Sharding & replication:- Vertical shards by contiguous ranges of IVF cells (static) or by hashing embeddings into N shards (e.g., 128–512 shards). Each shard hosts its index (IVF+PQ+local HNSW). Replicate each shard 2–3× across nodes for HA and to smooth tail latency. Use routing layer to fan-out queries to subset of replicas; for multi-shard queries use async parallel calls + priority fusion.

Memory/disk layout:- Store compressed PQ codes and coarse centroids on SSD. Keep HNSW graph and frequently-accessed PQ dequant tables in RAM. For GPUs, keep hot shard(s) resident in GPU memory; cold shards on NVMe with mmap + async prefetch. Use memory-mapped FAISS indexes for fast loading.

Recall vs latency trade-offs:- Tuning knobs: nprobe (IVF cells searched), PQ bytes, HNSW efSearch/efConstruction. Increasing nprobe/efSearch → higher recall, higher latency. Set target latency budget and tune parameters per shard automatically using calibration dataset; maintain a latency vs recall curve and choose operating point per SLA.

Incremental updates:- Use hybrid index: base immutable snapshot + writeable delta shard for recent inserts. Periodically rebuild merges (offline, rolling) to compact PQ and update graphs. For deletes/updates, maintain tombstone layer and background compaction. Use append-only segment files and index versioning with atomic switch.

Operational considerations & monitoring:- Metrics: QPS, p50/p95/p99 latency, recall/quality (sampled queries with golden set), index size, memory pressure, GC, replica health, queue depths. Alerts for recall degradation, latency spikes, high CPU/IO.- CI/CD: automated index-build pipeline, deterministic seeds, canary rollout of new indexes and parameter sets.- Autoscaling: scale replicas by QPS; pre-warm indexes. Use circuit breakers and degrade to approximate-only or fallback to sparse retrieval when overloaded.- Testing: synthetic stress tests, adversarial queries, A/B tests for recall+latency.

This design provides scalable, tunable ANN search for 500M embeddings with clear trade-offs and operational practices.

EasyTechnical

22 practiced

What problem does the attention mechanism solve compared to vanilla encoder-decoder seq2seq models with a fixed-size context vector? Provide an intuitive example (e.g., translating a long sentence) and describe how attention changes gradient flow and learning for long-range dependencies.

HardTechnical

20 practiced

Design an active learning loop for an intent classification system to minimize human labeling cost. Describe sampling strategies (uncertainty, diversity, committee), stopping criteria, annotation UI needs for fast labeling, and how to incorporate labels into retraining with minimal disruption to serving models.

Sample Answer

Requirements & goals:- Minimize human labeling cost while maintaining or improving intent-classifier accuracy and calibration in production.- Fast human throughput, low model-serving disruption, and clear metrics for when to stop.

Active learning loop design:

1) Data pipeline & orchestration- Continuously ingest user utterances into a staging queue. Keep metadata (timestamp, user locale, session).- Maintain an “unlabeled pool” with model scores, embeddings, and provenance.

2) Sampling strategies (hybrid)- Uncertainty: pick examples near decision boundary (e.g., max entropy, margin sampling for top two logits, or lowest softmax confidence). Good for direct error reduction.- Diversity: use semantic embeddings (sentence-transformers) with clustering (k-means/soft clustering) or core-set selection to avoid redundant examples and cover tail intents.- Committee (model disagreement): maintain small ensemble (different seeds/architectures/data augmentations) or dropout-MC; sample utterances with high disagreement to capture model epistemic uncertainty.- Hybrid scheduler: allocate batch budget (e.g., 50% uncertainty, 30% diversity, 20% committee), adapt proportions by validation gain per strategy (bandit-style).- Prioritize by production impact: weight samples by traffic volume or business-critical intents.

3) Stopping criteria- Labeling budget exhausted OR marginal gain threshold: track validation AUC/accuracy or calibration improvement per labeled batch; stop when rolling improvement < δ for N batches.- Confidence plateau: population-level confidence distribution stabilizes (e.g., % of high-confidence predictions converges).- Business metric constraint: downstream KPIs (NLU routing error, deflection rate) meet SLA.

4) Annotation UI for speed & quality- Minimal friction: single-screen, keyboard shortcuts, and hotkeys for common intents.- Provide context: previous utterances, system prompt, confidence & top-k candidates pre-filled (reduce cognitive load).- Batch-mode and review-mode: allow annotators to label similar clustered examples together.- Quick negative sampling: “none of the above” and multi-intent tagging support.- Quality controls: gold checks, inter-annotator agreement, live feedback, annotation time monitoring.- Capture metadata: uncertainty reason, ambiguous flag, suggested canonical phrasing.

5) Incorporating labels into retraining with minimal serving disruption- Use incremental / continual learning pipeline: - Maintain production model and a shadow model. Periodically (daily/weekly) train shadow model on union of existing labeled data + recent active labels. - Validate on holdout and A/B test shadow vs prod; start canary rollouts for low-risk traffic.- Training strategies: - Warm-start from production weights (fine-tune) to reduce training time and retain learned features. - Class-imbalance handling: upsample new rare-intent examples or use focal loss / label smoothing. - Use calibration layers or temperature scaling post-training to stabilize probabilities.- Deployment pattern: - Blue-green or rolling deploy with traffic splitting (5%→25%→100%) and automated rollback on metric degradation. - Zero-downtime inference: keep model signature stable, versioned vocab/label-mappings; if label set changes, use mapping layer to translate.- Fast-turnaround updates: - For high-impact small updates, use lightweight adapters (e.g., intent-specific last-layer retraining) to push changes quickly, then retrain full model asynchronously.- Monitoring: - Post-deploy monitoring for drift, new unseen intents, confidence shifts, and annotator-verified errors to feed next loop.

6) Metrics & feedback- Track label efficiency: model gain per labeled example.- Annotation throughput & accuracy, time per label.- Production KPIs: intent accuracy, misroute rate, user satisfaction.- Use these to tune sampling mix, batch sizes, and stopping rules.

Trade-offs and rationale:- Uncertainty is sample-efficient but can cluster on redundant utterances—hence diversity/committee mitigation.- Shadow + canary avoids service disruption; adapters allow rapid fixes.- Human-in-loop UI reduces per-label time and improves quality, maximizing ROI on labeling.

This loop balances sample efficiency, annotation speed, and safe production updates while enabling continuous improvement.

MediumTechnical

19 practiced

Given token-level BIO-labelled sequences for NER, provide pseudocode for the Viterbi decoding algorithm for a linear-chain CRF. Explain time and space complexity, and how you would batch decode many sequences efficiently in production.

Sample Answer

Approach: use the standard dynamic-programming Viterbi for a linear-chain CRF in log-space to avoid underflow. At each time step t compute best score for each tag y_t as max_{y_{t-1}} (score_{t-1}[y_{t-1}] + transition[y_{t-1},y_t] + emission[t,y_t]). Store backpointers to reconstruct the best path.

python

# pseudocode (python-like)
def viterbi_decode(emissions, transitions, start_trans, end_trans, mask):
    """
    emissions: (L, T) log-potentials for each position and tag
    transitions: (T, T) matrix, from_tag -> to_tag (log-space)
    start_trans: (T,) log score to start with tag
    end_trans: (T,) log score to end from tag
    mask: (L,) boolean mask for valid positions (if variable length)
    Returns: best_path (L,) tag indices, best_score
    """
    L, T = emissions.shape
    dp = [-inf] * T            # dp for t-1
    backpointers = [[0]*T for _ in range(L)]
    # init
    for y in range(T):
        dp[y] = start_trans[y] + emissions[0, y]
    # forward pass
    for t in range(1, L):
        if not mask[t]:
            break
        next_dp = [-inf] * T
        for y in range(T):                        # current tag
            # compute best previous tag
            best_prev = argmax_y_prev(dp[y_prev] + transitions[y_prev, y])
            next_dp[y] = dp[best_prev] + transitions[best_prev, y] + emissions[t, y]
            backpointers[t][y] = best_prev
        dp = next_dp
    # termination
    for y in range(T):
        dp[y] += end_trans[y]
    best_last = argmax_y(dp)
    best_score = dp[best_last]
    # backtrack
    best_path = [0]*L
    idx = best_last
    for t in reversed(range(L)):
        best_path[t] = idx
        idx = backpointers[t][idx]
    return best_path, best_score

Key points:- Use log-space additions and argmax; store backpointers to reconstruct path.- Enforce BIO constraints by masking illegal transitions (set transition log-prob = -inf).

Time & space complexity:- Time: O(L * T^2) per sequence (L = sequence length, T = num tags) because for each position and current tag we consider all previous tags.- Space: O(L * T) to store backpointers and O(T) per time-step for dp.

Batch decoding in production:- Vectorize with tensors (PyTorch/TF): represent emissions as (B, L, T), transitions as (T, T). Compute dp across batch using broadcasting and torch.max over prev-tag axis to reduce the inner loop; this yields GPU-accelerated O(B * L * T^2) but high throughput.- Handle variable lengths with masks and pack/unpack or by sorting+bucketing by length to reduce wasted compute.- Optimize: reduce T via tag constraints (BIO pruning), use beam search (beam size K) to reduce complexity to O(L * T * K), or use 1-best greedy if acceptable.- Memory: keep backpointers as int tensors (B, L, T) and reconstruct per batch element; consider checkpointing or streaming backpointers for very long sequences.- Production tips: precompile transition constraints, run on GPU for large B, use mixed precision for throughput, and test correctness with masked positions and BIO-validity checks.

EasyTechnical

23 practiced

Explain masking in transformer models: describe padding masks vs. causal (autoregressive) masks, how they are applied in attention computations, and give concrete examples of where each is necessary (e.g., classification vs. autoregressive generation).

Sample Answer

Masking in transformer models prevents attention from attending to undesired positions. Two common masks:

1) Padding mask (key padding mask)- Purpose: prevent attention to padded tokens (not real data) in variable-length batches.- How applied: mask out keys/values for padded positions so their attention scores become -inf before softmax.- Example use: classification, sequence labeling, encoder stacks.- PyTorch example (additive mask):

python

# q,k: [B, T, D]; pad_mask: [B, T] bool True for padding
scores = torch.matmul(q, k.transpose(-2,-1)) / math.sqrt(D)
scores = scores.masked_fill(pad_mask.unsqueeze(1), float('-inf'))  # broadcast over query positions
attn = torch.softmax(scores, dim=-1)

2) Causal / autoregressive mask (subsequent mask)- Purpose: prevent positions from attending to future tokens, enforcing autoregressive factorization.- How applied: mask upper-triangular (j > i) entries in attention score matrix to -inf so token i only sees ≤ i.- Example use: language generation, decoder self-attention during training and inference.- PyTorch example:

python

T = seq_len
causal = torch.triu(torch.ones(T, T), diagonal=1).bool()  # True where future
scores = scores.masked_fill(causal.unsqueeze(0), float('-inf'))

Notes:- Combine masks by OR-ing boolean masks or summing additive masks.- In PyTorch's MultiheadAttention there are separate arguments: key_padding_mask (padding) and attn_mask (causal/subsequent).- Padding masks are batch-dependent; causal masks usually sequence-shared. Proper masking avoids leakage of information and ensures correct gradients/metrics.

Unlock Full Question Bank

Get access to hundreds of Natural Language Processing Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.