MediumTechnical

42 practiced

Write a Python/NumPy function that performs a 2D convolution forward pass for a single-channel image with a single filter, supporting padding and stride. Specify input shape (H, W), kernel shape (kH, kW), stride, and padding. Describe how you would extend it to multi-channel and batched inputs and how im2col could accelerate the computation.

Sample Answer

To implement a single-channel 2D convolution with stride and padding in NumPy, we slide the kernel over a padded input and compute elementwise multiplications and sums.

python

import numpy as np

def conv2d_single_channel(image, kernel, stride=(1,1), padding=(0,0)):
    """
    image: 2D array shape (H, W)
    kernel: 2D array shape (kH, kW)
    stride: (sH, sW)
    padding: (pH, pW) -- symmetric padding on all sides
    returns: 2D convolved output
    """
    H, W = image.shape
    kH, kW = kernel.shape
    sH, sW = stride
    pH, pW = padding

    # pad input
    img_padded = np.pad(image, ((pH, pH), (pW, pW)), mode='constant', constant_values=0)
    out_H = (H + 2*pH - kH) // sH + 1
    out_W = (W + 2*pW - kW) // sW + 1
    out = np.zeros((out_H, out_W), dtype=image.dtype)

    for i in range(out_H):
        for j in range(out_W):
            r0 = i * sH
            c0 = j * sW
            patch = img_padded[r0:r0+kH, c0:c0+kW]
            out[i, j] = np.sum(patch * kernel)
    return out

Key points:- Time complexity O(out_H * out_W * kH * kW).- Handle invalid sizes: ensure (H+2pH - kH) % sH == 0 (or document floor behavior).- Padding can be asymmetric if needed by adjusting np.pad.

Extensions:- Multi-channel: for input shape (C, H, W) and kernel (C, kH, kW), sum per-channel convs: out = sum_c conv(image[c], kernel[c]).- Batched: loop or vectorize over batch dimension (N, C, H, W) producing (N, out_H, out_W) or (N, F, out_H, out_W) for multiple filters.

im2col acceleration:- Rearrange all kH*kW*(C) patches into columns (matrix of shape (kH*kW*C, out_H*out_W)), flatten kernels to rows, then compute a single matrix multiply (GEMM). This turns the inner loops into a highly optimized BLAS operation, drastically improving speed on large inputs and multiple filters.

HardSystem Design

47 practiced

Design an end-to-end platform for training and serving large transformer models (10B–100B parameters) in production that supports multi-tenant fine-tuning, on-demand inference with low latency (<200 ms for 1k tokens), model versioning and rollback, cost controls, and compliance. Address compute architecture, model parallelism choices, storage, orchestration, autoscaling, monitoring, safety filters, and tenant isolation.

Sample Answer

Requirements (clarify):- Support training & fine-tuning of 10B–100B models for multiple tenants- On-demand low-latency inference: <200 ms for 1k tokens- Model versioning & rollback, cost controls, compliance, tenant isolation, safety filters- Throughput and cost trade-offs

High-level architecture:- Control plane (API gateway, auth, tenant management, billing)- Orchestration plane (Kubernetes + custom operator)- Compute plane: separate clusters for training and inference- Storage plane: object store (S3-compatible), model registry, feature/metadata DB- Monitoring & compliance plane (metrics, logging, audit, policy engine)

Core components and responsibilities:1. Model Registry & Versioning- Stores model artifacts (sharded checkpoints), metadata, signatures, provenance.- Allows promotion, rollback, immutable tags, GPG signing for compliance.

2. Training / Fine-tuning Cluster- Managed on GPU/accelerator fleet (A100/H100 or next-gen): node pools by SKU.- Use model parallelism library (e.g., combination of tensor-slicing (Megatron-LM) + pipeline parallelism + ZeRO-offload (DeepSpeed)) to scale to 100B.- Auto-provision ephemeral clusters per job with cost limits and quotas.- Checkpointing to object store with incremental/delta checkpoints to reduce storage.

3. Inference Cluster (low-latency)- Two-tier inference: a) Low-latency serving: model shards loaded in GPU memory across nodes using tensor- & pipeline-parallel optimized serving (e.g., NVIDIA Triton with custom plugins or vLLM with distributed GPU serving). Keep hot models warm in GPU/RAM. b) Cost-efficient batch/async: CPU or cheaper GPU for non-latency-critical requests.- Use kernel-level optimizations (FlashAttention, quantization—INT8/4) and model distillation to meet <200 ms for 1k tokens.- Autoscaler based on latency SLOs and queue depth. Use request-aware routing: prefer warmed model instances.

4. Orchestration & Autoscaling- Kubernetes for control, with custom operators managing distributed jobs (DeepSpeed/Megatron jobs).- Horizontal Pod Autoscaler + custom metrics (latency, GPU memory utilization).- Pre-warming and predictive scaling using usage forecasting, tenant priority queues, and spot fleet integration for cost savings.

5. Storage & Dataflow- S3 for checkpoints + object lifecycle policies (cold storage).- Sharded checkpoint layout compatible with model-parallel loading to avoid expensive all-to-one transfers.- Metadata DB (Postgres) for model versions, tenancy, audit logs.- Encryption-at-rest and in-transit; tenant-specific KMS keys for compliance.

6. Tenant Isolation & Multi-tenancy- Namespace-based isolation in control plane; network policies and IAM for compute access.- Per-tenant resource quotas, encryption keys, and dedicated model-serving sandboxes for high-risk tenants.- Sidecar-based data access proxies to enforce data governance and policy checks.

7. Monitoring, Compliance & Safety- Observability: Prometheus/Grafana for infra metrics; application traces for latency; model telemetry (token-level latencies, prompt jitter).- Compliance: immutable audit logs, model lineage, data consent metadata, PII detectors during fine-tuning.- Safety filters: pre- and post-generation filters (safety classifier, regex/pattern, rate limits). Deploy a runtime policy engine (OPA) to enforce allowed behaviors per tenant.- Canary rollout + automated canary metrics (latency, error rate, toxicity) before promotion.

Scalability & Trade-offs:- Model-parallel approach (tensor + pipeline + optimizer sharding) minimizes GPU memory; increases cross-node bandwidth—requires high-speed interconnect (NVLink/InfiniBand).- Latency SLOs push for persistent warm GPU pools — higher cost vs. on-demand cold starts.- Quantization and sparse attention improve latency/cost but may reduce fidelity—expose per-tenant options.

Operational practices:- Cost controls: per-tenant budgets, preflight cost estimates for fine-tuning, spot instances with fallback.- Rollback: immutable model versions, automated rollback triggers on SLO breaches.- Security: RBAC, tenant encryption keys, vulnerability scans, and SIEM integration.

Example flow:- Tenant requests fine-tune -> authenticate -> allocate ephemeral training cluster (DeepSpeed) -> checkpoints stored incremental to S3 -> register new model version -> run canary deployments to small inference pool -> monitor safety/latency -> promote to production with autoscaling policy.

This design balances performance, isolation, and cost with practical engineering choices for 10B–100B models.

EasyTechnical

53 practiced

Compare RNN, LSTM, and GRU at a high level. Explain the gating mechanisms and how they help with vanishing gradients. For a sequence classification task with moderate-length sequences, how would you decide which cell type to use and why?

MediumTechnical

57 practiced

Compare SGD with momentum, RMSProp, Adam, and AdaBelief in terms of convergence speed, generalization behavior, and hyperparameter sensitivity for deep networks. For training a large transformer with many layers and long sequences at scale, which optimizer and schedule would you choose and why?

Sample Answer

SGD with momentum, RMSProp, Adam, and AdaBelief differ across three axes:

Convergence speed- SGD + momentum: Slower to reach low training loss but stable, especially with large batch smoothing; benefits from careful LR schedule (warmup + decay).- RMSProp: Faster than plain SGD on noisy/nonstationary objectives early, because per-parameter adaptive scaling.- Adam: Fastest to reduce training loss and robust out-of-the-box; handles sparse / varying gradients well.- AdaBelief: Similar speed to Adam initially but often converges to slightly better minima by adapting to “surprise” (variance from predicted gradient).

Generalization behavior- SGD + momentum: Tends to generalize best in many large-scale vision/NLP settings when properly tuned — finds flatter minima.- RMSProp/Adam: Often worse generalization (can find sharper minima), though not always; careful weight decay and schedule mitigate this.- AdaBelief: Reports show improved generalization vs Adam by regularizing updates, closer to SGD in final test performance.

Hyperparameter sensitivity- SGD + momentum: More sensitive to learning rate and batch size; needs LR warmup/decay and tuned momentum.- RMSProp: Moderate sensitivity (decay, epsilon).- Adam: Less sensitive initially; still needs betas, eps and well-chosen weight decay; known issues with large-batch scaling if naive.- AdaBelief: Comparable to Adam; slightly more robust to eps and lr in practice.

Recommendation for a large transformer (many layers, long sequences, training at scale)- Optimizer: AdamW or AdaBelief with AdamW-style decoupled weight decay. AdamW is the safe, battle-tested default for transformers; AdaBelief is an option if you observe Adam’s generalization gaps.- Schedule: Linear warmup (e.g., 1–10% of total steps) from 0 to base LR, followed by either cosine decay or polynomial/linear decay to zero. Use layer-wise LR decay (smaller LR for lower layers) and gradient clipping (norm) for stability. Scale LR with batch size (LARS/linear scaling rule) and tune weight decay (often 0.01) plus Adam betas (0.9, 0.999) and eps (1e-8 or 1e-6).Rationale: AdamW + warmup stabilizes early training across deep stacks and long sequences, while decoupled weight decay and a decay schedule help align generalization toward SGD-like minima; layer-wise decay and careful clipping mitigate catastrophic instabilities in very deep transformers.

EasyTechnical

51 practiced

Explain the concept of attention in neural networks. Contrast additive (Bahdanau) attention with dot-product (Luong / scaled dot-product) attention and give an intuitive example (e.g., machine translation) showing why attention improves sequence-to-sequence models.

Deep Learning Concepts and Theory Questions

Sample Answer

Sample Answer

Sample Answer

Sample Answer

Sample Answer

Unlock Full Question Bank