EasyTechnical

76 practiced

Describe batch normalization and how it stabilizes training. Compare batch normalization to layer normalization and group normalization, and explain which you would prefer in training Transformers or RNNs for production stability.

EasyTechnical

75 practiced

List a few widely used pre-trained models for computer vision and NLP (e.g., ResNet, EfficientNet, BERT, RoBERTa, GPT) and describe one realistic example use-case for each where using the pre-trained checkpoint is a clear advantage in production.

HardTechnical

68 practiced

Implement a Transformer encoder block in PyTorch-like pseudocode that supports configurable pre-norm or post-norm, multi-head attention, dropout, residual connections, and a two-layer feed-forward network. Ensure the block accepts a mask and batched inputs, and note shape assumptions.

Sample Answer

Approach: implement a single Transformer encoder block with configurable pre-norm or post-norm, multi-head self-attention, dropout, residuals, and a 2-layer feed-forward (with GELU). Accepts batched inputs and attention mask.

python

import torch
import torch.nn as nn
import torch.nn.functional as F
from math import sqrt

class EncoderBlock(nn.Module):
    def __init__(self, d_model, n_heads, d_ff, dropout=0.1, prenorm=True):
        super().__init__()
        self.prenorm = prenorm
        self.d_model = d_model
        self.n_heads = n_heads
        assert d_model % n_heads == 0
        self.head_dim = d_model // n_heads

        # Multi-head projection layers
        self.q_proj = nn.Linear(d_model, d_model)
        self.k_proj = nn.Linear(d_model, d_model)
        self.v_proj = nn.Linear(d_model, d_model)
        self.out_proj = nn.Linear(d_model, d_model)

        # Feed-forward
        self.ffn = nn.Sequential(
            nn.Linear(d_model, d_ff),
            nn.GELU(),
            nn.Dropout(dropout),
            nn.Linear(d_ff, d_model),
        )

        # Norms and dropout
        self.norm1 = nn.LayerNorm(d_model)
        self.norm2 = nn.LayerNorm(d_model)
        self.dropout = nn.Dropout(dropout)

    def _multi_head_attn(self, x, mask=None):
        # x: (B, T, d_model); mask: (B, 1, T, T) or (B, T, T) with True for allowed positions
        B, T, _ = x.size()
        q = self.q_proj(x).view(B, T, self.n_heads, self.head_dim).transpose(1,2)  # (B, H, T, Hd)
        k = self.k_proj(x).view(B, T, self.n_heads, self.head_dim).transpose(1,2)
        v = self.v_proj(x).view(B, T, self.n_heads, self.head_dim).transpose(1,2)

        scores = torch.matmul(q, k.transpose(-2,-1)) / sqrt(self.head_dim)  # (B, H, T, T)
        if mask is not None:
            # mask: broadcastable to (B, 1, T, T); masked positions should be -inf in scores
            scores = scores.masked_fill(~mask, float('-inf'))
        attn = torch.softmax(scores, dim=-1)
        out = torch.matmul(attn, v)  # (B, H, T, Hd)
        out = out.transpose(1,2).contiguous().view(B, T, self.d_model)
        return self.out_proj(out)

    def forward(self, x, mask=None):
        # x: (B, T, d_model)
        # mask: boolean tensor where True=allowed (shape (B, T, T) or (B,1,T,T))
        residual = x
        if self.prenorm:
            x = self.norm1(x)
        attn_out = self._multi_head_attn(x, mask)
        attn_out = self.dropout(attn_out)
        x = residual + attn_out  # residual

        if not self.prenorm:
            x = self.norm1(x)

        # FFN block
        residual = x
        if self.prenorm:
            x = self.norm2(x)
        ffn_out = self.ffn(x)
        ffn_out = self.dropout(ffn_out)
        x = residual + ffn_out
        if not self.prenorm:
            x = self.norm2(x)
        return x

Key points:- Shapes: input (B, T, d_model); mask broadcastable to (B, 1, T, T) or (B, T, T).- Complexity: O(B * T^2 * d_model) due to attention; FFN O(B * T * d_model * d_ff).- Edge cases: ensure mask dtype/shape, handle all-equal sequences, n_heads divides d_model.- Alternatives: use efficient attention (sparse/linear) for long sequences; fused QKV for speed; add scaling, attention dropout, or gating.

HardSystem Design

91 practiced

Design an explainability plan for a production Transformer-based recommendation model to satisfy regulatory requirements: which explainability techniques you would implement (feature attribution, counterfactuals, example-based explanations), how to log explanations, maintain audit trails, and integrate human-in-the-loop review for critical decisions.

Sample Answer

Requirements & constraints:- Explainability for a Transformer-based recommender in production to satisfy regulators: per-decision explanations, reproducible audit trails, human-review for high-risk decisions, privacy-preserving logs, bounded latency (soft real-time), scalable at N requests/s.

High-level architecture:Client → Inference API (serving Transformer) → Explainability Service → Logging/Audit Store + Human Review UI → Feedback loop to Model Ops

Explainability techniques (why and how):- Feature attribution: use Integrated Gradients (IG) for token-level attributions on content/user features; complement with TreeSHAP or KernelSHAP for tabular/context features. IG is gradient-based (efficient with backprop), SHAP gives consistent global/local feature importance.- Counterfactual explanations: generate sparse counterfactuals constrained to realistic changes using a learned generator (VAE over user/item features) + optimization (minimize change cost + flip predicted label). Useful for “what minimal change would alter recommendation?”- Example-based: provide k-nearest-neighbor exemplars from embedding space (approximate with FAISS) and influence-function-like scores to show which training examples most affected prediction (bounded approximation using TracIn for scale).- Global summaries: periodic surrogate models (interpretable decision trees) and aggregated SHAP/IG distributions to satisfy macro-level transparency.

Logging & audit trail:- Log schema per decision: request_id, user_id (hashed), model_version, input_snapshot (hashed + encrypted raw if allowed), prediction, explanation bundle {IG vector, SHAP values, counterfactuals summary, exemplar ids}, timestamps, latency, confidence, explanation_model_versions.- Store immutable, append-only audit logs in secure object store and index metadata in audit DB (WORM storage). Keep explainability artifacts TTL-aligned with retention policy; encrypt at rest, role-based access, and maintain hash signatures for tamper-detection.- Capture training metadata: training dataset hashes, random seeds, dependency versions, model weights checksum. Link decision logs to model training run id.

Human-in-the-loop:- Risk scoring policy: define thresholds (e.g., high-impact changes, personalization affecting finance/health). For high-risk decisions, route to Human Review UI showing: input snapshot, attribution heatmap, counterfactuals, top exemplars, recommended action.- Reviewer workflow: allow accept/override with reasons; record reviewer id, decision, time; feed overrides to monitoring and label store.- Retraining triggers: automated alerts when reviewer overrides exceed threshold or explanation drift detected (distributional shift of attributions).

Operational considerations:- Latency & scaling: compute lightweight explanations synchronously (IG with limited steps, nearest neighbors via precomputed indexes); compute expensive artifacts (full SHAP, counterfactual search, influence scores) asynchronously and attach later; mark initial response “explainability_pending” when needed.- Privacy: pseudonymize PII; differential privacy for exemplar release (limit similarity, add noise); only surface aggregated/global explanations for sensitive cohorts.- Monitoring & validation: monitor explanation stability, explanation-model drift, and correlation between attributions and business metrics. Periodic audits: run synthetic tests to verify explanation fidelity and counterfactual plausibility.

Trade-offs:- Fidelity vs latency: choose hybrid sync/async approach; store higher-fidelity explanations for audits.- Transparency vs privacy: restrict raw examples; use hashed/excerpted exemplars and consent controls.

This plan provides per-decision, reproducible, and auditable explanations combining attributions, counterfactuals, and example-based evidence, with secure logging and an integrated human-review pipeline to meet regulatory compliance.

MediumTechnical

95 practiced

Explain gradient accumulation: why it's used, how it mimics larger batch sizes, and trade-offs compared to increasing physical batch size or using multi-GPU data-parallel training. Include pseudo-implementation details for PyTorch optimizer steps with accumulation.

Sample Answer

Gradient accumulation lets you simulate a larger effective batch size by accumulating gradients across several forward/backward passes before applying an optimizer step. It's used when a single-device GPU cannot hold a large batch due to memory limits, but you want benefits of larger batches (smoother gradient estimates, different learning dynamics, better utilization).

Why it mimics larger batches- Compute: doing N micro-batches and summing their gradients equals computing gradients on one big batch (assuming no stateful ops like BatchNorm with different behavior).- Effective batch size = micro_batch_size * accumulation_steps. The optimizer update is performed once per effective batch.

Trade-offs vs increasing physical batch or multi-GPU data-parallel- Pros: Lower memory requirement, simple to implement, reproducible on a single device.- Cons: More kernel launches and overhead; slower wall-clock if accumulation_steps adds CPU/GPU synchronization cost. BatchNorm behaves differently unless you use SyncBatchNorm or keep stats fixed. Larger effective batch from accumulation may change training dynamics—may need to retune learning rate or use learning-rate scaling rules.- Compared to multi-GPU DP: Multi-GPU reduces wall time by parallelizing micro-batches across devices and often synchronizes grads with allreduce. Accumulation is sequential and can't reduce wall-clock as effectively. Multi-GPU requires more infra but preserves per-step wall-clock update frequency.

PyTorch pseudo-implementation

python

# assume model, optimizer, loss_fn, dataloader provided
accum_steps = 4
optimizer.zero_grad()
for step, (x, y) in enumerate(dataloader):
    out = model(x)
    loss = loss_fn(out, y) / accum_steps  # scale loss to average gradients
    loss.backward()  # gradients accumulate in .grad

    if (step + 1) % accum_steps == 0:
        # optional: clip gradients
        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
        optimizer.step()
        optimizer.zero_grad()

Practical tips- Scale learning rate carefully (linear scaling rule often used).- If using BatchNorm, prefer SyncBatchNorm in multi-GPU or replace with GroupNorm/LayerNorm.- Monitor instability and adjust accumulation or LR schedule.

Common Deep Learning Architectures Questions

Sample Answer

Sample Answer

Sample Answer

Sample Answer

Sample Answer

Unlock Full Question Bank