Technical Fundamentals Check Questions

Checklist for core technical fundamentals expected of technical roles: algorithms and data structures, time and space complexity analysis (Big-O/Theta/Omega), basic applied mathematics and probability (e.g. Bayes' theorem), cryptography basics (symmetric vs asymmetric, common use cases), and core systems concepts. Used to evaluate whether a candidate can reason about fundamental technical problems and apply foundational techniques, calibrated to the depth appropriate for the candidate's role (e.g. hands-on implementation for engineering roles, conceptual fluency for technical non-coding roles).

MediumTechnical

94 practiced

Compare LRU, LFU, and TTL-based caching strategies. For an ML model serving scenario where embedding lookups have skewed access patterns with some hot keys and many cold keys, which strategy would you choose and why? Discuss implementation complexity and memory overhead.

EasyTechnical

56 practiced

Basic probability calculation: Given events A and B with P(A)=0.3, P(B)=0.4 and P(A∩B)=0.12, compute P(A|B) and P(A∪B). Explain the difference between independence and conditional probability and why these concepts matter for Naive Bayes classifiers.

MediumTechnical

64 practiced

Given a stream of integers too large to fit into memory, design an algorithm to return the k-th smallest element approximately in a single pass. Discuss trade-offs between accuracy, memory, and time and implement a sketch using heaps or randomized sampling in Python pseudo-code.

Sample Answer

Approach: For a single-pass approximate k-th smallest on a massive stream, two practical sketches are: (1) a max-heap of size m (memory budget) that keeps the m smallest seen values — if m >= k the heap's root approximates k-th smallest; (2) randomized reservoir sampling + exact selection on the sample. Trade-offs: larger m improves accuracy but uses more memory; sampling reduces memory/time but loses rare small values (bias). Heaps give monotonic guarantees (never miss values smaller than kept), sampling gives unbiased estimation of distribution.

Heap-based sketch (preferred when you can store m elements, m >= k recommended):

python

import heapq

def kth_smallest_stream_heap(stream, k, m):
    """
    Keep a max-heap (store negatives) of up to m smallest elements.
    Return heap_root as approximate k-th smallest (if m>=k, closer to true).
    """
    if m < 1: raise ValueError("m must be >=1")
    max_heap = []  # will store negatives to simulate max-heap
    for x in stream:  # stream yields integers
        if len(max_heap) < m:
            heapq.heappush(max_heap, -x)
        elif x < -max_heap[0]:
            heapq.heapreplace(max_heap, -x)
    if len(max_heap) < k:
        return None  # sample too small to estimate k-th
    # approximate k-th: sort kept elements and pick k-th
    kept = sorted([-v for v in max_heap])
    return kept[k-1]

Sampling-based sketch:

python

import random

def kth_smallest_stream_sample(stream, k, m):
    """
    Reservoir sample m items, then compute exact k-th on the sample.
    Good when memory m << stream size; variance decreases with m.
    """
    sample = []
    for i, x in enumerate(stream, start=1):
        if i <= m:
            sample.append(x)
        else:
            j = random.randint(1, i)
            if j <= m:
                sample[j-1] = x
    if len(sample) < k: return None
    sample.sort()
    return sample[k-1]

Complexity:- Heap: O(n log m) time, O(m) memory. Accuracy increases with m; if m = k exact if no smaller items were evicted.- Sampling: O(n) time to sample (O(m log m) final sort), O(m) memory. Accuracy depends on sample representativeness (variance ~1/m).

Edge cases:- m < k (cannot guarantee); highly skewed distributions where rare very small values may be missed.- Stream with duplicates—both handle duplicates naturally.

Alternatives: quantile sketches (t-digest, GK) provide probabilistic error bounds for rank queries and are better when you need guaranteed rank error bounds. Use them for stricter accuracy-memory trade-offs.

MediumTechnical

54 practiced

Compare forward-mode and reverse-mode automatic differentiation. For training deep neural networks with many parameters and a scalar loss, which mode is more efficient and why? Provide cost reasoning in terms of number of inputs and outputs.

MediumTechnical

63 practiced

Explain how learning rate, momentum, and batch size affect convergence behavior of gradient descent. When would you use SGD with momentum versus Adam? Discuss trade-offs in stability, convergence speed, and generalization for training large neural networks.

Sample Answer

Learning rate, momentum, and batch size each shape the optimization trajectory and trade off speed, stability, and generalization.

- Learning rate (LR): Controls step size. Too large → oscillation/divergence; too small → very slow convergence / gets stuck in sharp minima. Use LR schedules (decay, cosine, step) and warmup for large-batch training to stabilize early steps. Rule-of-thumb: scale LR roughly linearly with batch size (with warmup).

- Momentum: Adds velocity that accumulates past gradients (typically 0.9). It accelerates convergence along consistent directions and damps oscillations across steep, narrow directions (helpful in ravines). Momentum helps escape shallow local minima and reduces effective noise from stochastic gradients.

- Batch size: Smaller batches give noisy gradient estimates which can help exploration and often improve generalization; they have higher variance but more frequent updates. Large batches reduce gradient noise and improve hardware throughput but can get stuck in sharp minima and harm generalization unless LR scheduling and regularization are adjusted.

When to use SGD with momentum vs Adam:- Adam (adaptive moments) is great for fast prototyping, sparse or noisy gradients, and problems needing quick convergence (transformers pretraining, finetuning with limited compute). It is robust to LR choices early in training.- SGD with momentum often yields better final generalization for large-scale vision and many large networks when trained long enough with carefully tuned LR schedules and weight decay. Use SGD+momentum for final training/production runs when you want the best generalization.

Trade-offs summary:- Stability: Adam is more stable and forgiving initially; SGD+momentum may require careful LR tuning and warmup.- Convergence speed: Adam → faster initial loss decrease; SGD+momentum → competitive/asymptotic convergence and often better minima if given time.- Generalization: SGD+momentum typically generalizes better; Adam can converge to sharper minima unless combined with tweaks (decoupled weight decay, AMSGrad variants, or switching to SGD later).

Practical recipe: start with Adam for experiments; for large-scale training, use SGD+momentum with cosine decay/warmup and strong regularization (weight decay, data augmentation). Consider hybrid: AdamW warmup then switch to SGD for final epochs.

Unlock Full Question Bank

Get access to hundreds of Technical Fundamentals Check interview questions and detailed answers.

Join thousands of developers preparing for their dream job.