Amazon AI Engineer Interview Preparation Guide - Staff Level

Direct answer

Flag a row as an outlier when its value falls below $Q1 - k\cdot IQR$ or above $Q3 + k\cdot IQR$, computing Q1, Q3, and IQR from the column while gracefully skipping any missing values rather than letting them break the calculation.

Structured elaboration

import numpy as np
import pandas as pd

def detect_outliers_iqr(df, column, k=1.5):
    """Return the index labels of rows considered outliers via the IQR method."""
    values = pd.to_numeric(df[column], errors="coerce")   # tolerate numeric-looking strings, NaN otherwise
    valid = values.dropna()
    q1, q3 = valid.quantile(0.25), valid.quantile(0.75)
    iqr = q3 - q1
    lower, upper = q1 - k * iqr, q3 + k * iqr
    is_outlier = (values < lower) | (values > upper)   # NaN comparisons are False, so NaNs are never flagged
    return df.index[is_outlier.fillna(False)]

df = pd.DataFrame({"amount": [10, 12, 11, 13, 9, 500, None, 14]})
outlier_idx = detect_outliers_iqr(df, "amount", k=1.5)
print(outlier_idx.tolist())

Handling NaNs gracefully: computing the quantiles on valid = values.dropna() means missing values never distort Q1/Q3/IQR, and the final boolean mask's comparisons against NaN naturally evaluate to False (never flagged as an outlier), with an explicit .fillna(False) as a defensive safeguard in case a comparison chain produces something other than a clean boolean.

Worked example

Running the code above on [10, 12, 11, 13, 9, 500, None, 14], Q1 and Q3 are computed from the 7 valid values, giving a fence that easily flags 500 while leaving None untouched (not flagged, correctly excluded rather than erroring); the function returns the index position of the 500 row only.

Trade-offs and pitfalls

The IQR method's known limitation on a skewed distribution: because Q1 and Q3 are computed from the RAW (unlogged, untransformed) values, a heavily right-skewed feature can have its upper fence pulled disproportionately high by the skew itself, causing the method to under-flag genuinely unusual values on the long tail; for such features, applying a skew-reducing transform (log, Box-Cox) before running the IQR check, or using a method less sensitive to skew, often catches outliers this straightforward version misses.

To build a performant PyTorch pipeline for millions of images, focus on efficient storage access, batched/parallel decoding, minimal CPU-GPU transfers, reproducible shuffling, and keeping GPU fed. Approach: store images in an efficient format (LMDB / WebDataset / TFRecords) or use fast randomized shuffles of file lists, implement a lightweight Dataset that decodes on-the-fly (or uses preprocessed bytes), use DataLoader with multiple workers, persistent_workers, pin_memory, prefetch_factor, and a collate_fn that does minimal work. Seed worker RNGs for reproducibility.

import os, random
from PIL import Image
import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms

class ImageLMDBDataset(Dataset):
    def __init__(self, lmdb_env, keys, transform=None):
        self.env = lmdb_env  # opened readonly elsewhere to avoid re-opening per worker
        self.keys = keys
        self.transform = transform

    def __len__(self):
        return len(self.keys)

    def __getitem__(self, idx):
        with self.env.begin() as txn:
            img_bytes = txn.get(self.keys[idx])
        img = Image.open(io.BytesIO(img_bytes)).convert('RGB')
        if self.transform:
            img = self.transform(img)
        return img

def worker_init_fn(worker_id):
    # Ensure reproducible, independent RNGs per worker
    seed = torch.initial_seed() % 2**32
    random.seed(seed + worker_id)
    np.random.seed(seed + worker_id)

transform = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]),
])

# Example DataLoader
loader = DataLoader(
    dataset, 
    batch_size=256,
    shuffle=True,                # rely on DataLoader shuffle + worker_init_fn for reproducible seeds
    num_workers=16,              # tune to machine (CPU cores, disk bandwidth)
    pin_memory=True,             # faster host->GPU transfers
    persistent_workers=True,     # avoid worker spawn cost each epoch
    prefetch_factor=4,           # number of batches per worker to prefetch
    worker_init_fn=worker_init_fn,
    drop_last=True,
)

Key points:

• Storage: LMDB or WebDataset bundles avoid many small-file overhead; alternatively use SSDs with file lists and sharded tar files.
• Decode: Use Pillow / Intel SVT / libjpeg-turbo or GPU decode if available; consider pre-encoded tensors/mmap for fastest loads.
• Workers & prefetch: num_workers tuned to CPU I/O; prefetch_factor (default 2) increased to reduce stalls; persistent_workers avoids repeated spawning.
• pin_memory: reduces copy time to GPU.
• Reproducible shuffling: set DataLoader seed via torch.manual_seed before each epoch and use worker_init_fn to derive worker seeds.
• Collate: keep collate_fn lightweight; perform complex augmentations in Dataset transform or use NVIDIA DALI for heavy augmentation pipelines offloading to CPU/GPU.
• Monitoring: measure dataloader throughput and GPU utilization; instrument with perf counters to find bottlenecks.

Trade-offs/Alternatives:

• If CPU becomes bottleneck for augmentations, move augmentations to GPU (DALI, Kornia) or precompute augmentations.
• If storage I/O limited, increase sharding, use NVMe, or use distributed shuffling with each node reading local shard.

Cold-start: when a model-serving instance (container, function, or VM) is started from a cold state and must load the runtime, model weights, and any JIT/Warm caches before it can serve requests. Warm-up: bringing an instance to a ready state ahead of traffic—loading model, warming caches (e.g., GPU kernels, TF graph optimizations), and optionally running dummy inferences so latency on first real request is low.

Why it matters for capacity planning:

• Cold-starts produce high tail latency and can violate SLOs if unaccounted for.
• You must provision enough pre-warmed capacity (or tolerate extra latency) to meet peak SLA; poor planning causes either wasted resources or bad user experience.
• Predictable traffic patterns, startup times, and model memory/GPU load influence how many warm instances to keep.

Operational steps to reduce cold-start latency (serverless and containerized):

• Pre-warming / keep-alive: keep a small pool of warm instances running during expected peaks.
• Lazy vs eager loading: load only required parts on demand, but eagerly load critical layers to avoid first-request spikes.
• Lightweight bootstrap: reduce container image size, minimize framework overhead (use TF-TRT, ONNX Runtime, or custom C++ runtimes).
• Model optimizations: quantization, pruning, and using serialized runtime-optimized formats to speed load time.
• Snapshot/fast restore: persist warmed process state (e.g., cached memory maps, model sharding) and restore quickly if supported.
• GPU warm-up: run a short dummy inference to initialize GPU kernels and memory allocations.
• Traffic shaping: route a small percentage of traffic to new instances to warm them before full load.
• Autoscaler tuning: configure scale-up cooldowns, target warm concurrency, and provisioned concurrency (e.g., AWS Lambda Provisioned Concurrency).
• Monitoring & testing: instrument start-up latency, cold-start rate, and tail latency; run chaos tests and load tests to validate warm strategies.

These measures let you balance cost vs SLOs by quantifying warm instance counts needed and minimizing unpredictable latency from cold starts.

Direct answer

A p-value answers one narrow question: if there were truly no effect, how surprising would data like ours be? A small p-value, commonly under 0.05, means the pattern we saw would be unusual if nothing were really going on, so it counts as evidence worth acting on rather than random noise. Caution in one sentence: a p-value does not tell you the probability the effect is real, or how big or important it is, only how surprising the data would look under a "nothing is happening" assumption.

Structured elaboration

Explaining a statistics term to a non-technical stakeholder is less about simplifying the definition and more about choosing what to omit and what to guard against:

1. Pick one mental model and don't mix in a second. "How surprising the data would be if nothing were going on" is enough; don't also try to explain sampling distributions or the formal null hypothesis by name in the same breath.
2. State the caution as its own sentence, not a footnote. The most common misread is treating the p-value as "the probability we're right." Say plainly what it is not.
3. Separate statistical significance from size. A p-value can be tiny on an effect too small to matter, or the reverse: with a small sample, a real and sizeable effect can fail to look significant. That is where a confidence interval (the range the true effect probably falls in, not just whether it beat a cutoff) and margin of error earn their keep: they answer "how big, and how sure," a different question from "was this surprising."
4. Check understanding by asking the stakeholder to restate it in their own words. If they say "so it proves we're right," the caution hasn't landed yet.

The same "how surprising is this" frame extends past hypothesis tests. If a stakeholder asks how confident a model's predicted probabilities are, model calibration answers a related but different question: not "was this one result surprising" but "when the model says 70%, does that outcome actually happen about 70% of the time." And when a CFO worries a result is a fluke of a small sample, the honest reassurance isn't the p-value at all, it's showing the sample size and how much the estimate would tighten with more data, since a small or biased sample can produce a small p-value that still doesn't generalize.

Worked example

An A/B test on a new checkout flow comes back with a borderline p-value of 0.04. In the room:
"We ran the new checkout against the old one with about 20,000 customers on each side. The lift we saw would only happen by chance about 4 times out of 100 if the new flow actually made no difference, so this is likely real, but it's close to our usual cutoff, not a landslide. Before we roll it out everywhere, I'd want to see the confidence interval, the range of lift the data actually supports, because at this sample size a real-but-small effect and a borderline-noise effect can look similar on the p-value alone. If that range includes 'basically no change,' I'd want one more week of data before we call it, rather than lock in a decision off a single borderline number."
Caution restated: a p-value close to the cutoff isn't something you can round away in either direction, it's a signal to look at the interval and the trend, not settle the question on the spot.

Trade-offs and pitfalls

Dropping the caution sentence to keep the pitch upbeat is the most common mistake, and it's exactly what produces "the data proves it" overconfidence later. Overcorrecting the other way, hedging so hard that a genuinely strong result reads as shaky, costs credibility too. Borderline p-values, roughly 0.03 to 0.07, deserve more nuance than either "significant, ship it" or "not significant, ignore it": show the interval and the trend, not just the single number against the 0.05 line.

Direct answer. Feature toggles let you deploy code and ENABLE its behavior as two separate, independently-controllable steps, so a large refactor can ship dormant, get exercised safely (canaried, tested in production), and be switched on incrementally -- with an instant rollback (flip the toggle) if something's wrong, instead of a code revert.

Toggle types and when each applies

• Release toggles: hide in-progress work behind a flag so it can merge to the main branch continuously (avoiding long-lived feature branches) without being user-visible until it's ready. Short-lived by design -- removed once the feature ships fully.
• Ops toggles: let operators disable a specific behavior under load or incident conditions (e.g., 'disable the new caching layer') without a deploy. Longer-lived, sometimes permanent for critical kill-switches.
• Experiment toggles: control which users see variant A vs. B for measurement purposes; tied to an experiment's lifecycle, removed once the experiment concludes.

For a large refactor, you typically want a RELEASE toggle around the new code path, letting old and new implementations coexist and be flipped independently of any deploy.

How this enables safe incremental refactor

1. Ship the new implementation behind a toggle, defaulting OFF, so the deploy itself carries zero behavior-change risk.
2. Turn it on for internal users/a small percentage of traffic; compare outputs/error rates against the old path.
3. Ramp the percentage up gradually, watching the same metrics at each step, with the toggle giving you an immediate off-switch if a regression appears -- no need to revert a deploy under pressure.
4. Once fully ramped and stable, remove the OLD code path and the toggle itself.

The part people skip: toggle lifecycle management

A toggle that isn't tracked with an OWNER and a planned REMOVAL DATE tends to become permanent clutter -- both code paths stay alive indefinitely, doubling the maintenance surface and creating combinatorial testing problems as more toggles accumulate (does path A work correctly with toggle X on AND toggle Y off?). Treat every toggle as having a birth and an expected death: log it in an inventory, review stale ones on a cadence, and make removing a fully-ramped toggle part of the DEFINITION OF DONE for the feature it gated, not an optional follow-up.

Trade-offs and pitfalls

• Toggled code paths add real complexity to the codebase WHILE both exist (more branches, more test combinations) -- the toggle's value comes from how quickly you retire it, not from how cleverly you implement the branching.
• Toggling deep, stateful behavior (a data format change, a schema migration) is much riskier than toggling a stateless code path, since flipping back 'off' after the new path has already written data in the new format may not be a clean rollback -- plan the data-compatibility story separately from the toggle mechanism itself.

Goal: test whether adding personalization increases users' perceived fairness of system outputs.

Experiment setup:

• Randomized controlled trial (A/B): assign users randomly to Control (non-personalized baseline) or Treatment (personalized model). Ensure randomization at user level; run for sufficient duration to collect stable behavior (power calculation below).
• Primary outcome: perceived fairness measured via a validated survey question (e.g., Likert scale “I think the results were fair to me” 1–7) plus behavioral proxies (appeal rates, retention, complaint rate).
• Pre-register hypothesis: treatment increases mean fairness score.

Power & sampling:

• Conduct power analysis using expected effect size (e.g., d=0.2–0.3), target 80–90% power, account for multiple comparisons.

Statistical tests:

• Compare mean fairness scores with two-sample t-test (or Wilcoxon if non-normal). Use linear regression adjusting for covariates (age, prior satisfaction) to estimate treatment effect and CIs. For binary outcomes (complaints), use logistic regression.
• Correct for multiple subgroup tests (Benjamini–Hochberg or Bonferroni).

Subgroups to examine:

• Demographics (age, gender, region)
• Usage intensity / power users vs. new users
• Prior bias-vulnerable groups (protected attributes)
• Cold-start vs. long-history users
  Report heterogenous treatment effects with interaction terms; treat subgroup analyses as exploratory unless pre-registered.

Potential confounders and controls:

• Ensure randomization balance; if imbalance, adjust via regression or propensity scores.
• Temporal effects: run arms concurrently to avoid time trends.
• Exposure bias: ensure equal visibility of items; control for UI differences.
• Feedback loop: personalization may change behavior that further trains the model—use static model during experiment or frozen features.
• Survey response bias: differential response rates—weight responses or impute missingness and report response rates by arm.
• Spillover: avoid cross-user contamination (e.g., social sharing) or randomize by cluster.

Interpretation:

• Report effect sizes, CIs, p-values, and practical significance. Combine self-reported fairness with behavioral metrics and qualitative feedback to triangulate conclusions.

Direct answer

SageMaker, Vertex AI, and Azure ML all cover the same broad capability set: managed training, serving, and a registry/pipeline layer, but differ in ecosystem integration, portability, and how opinionated their pipeline abstractions are, which is usually the deciding factor over raw feature parity.

Structured elaboration

1. Managed training and hyperparameter tuning: all three offer managed training jobs with built-in hyperparameter-tuning services (SageMaker's Automatic Model Tuning, Vertex AI's Hyperparameter Tuning, Azure ML's HyperDrive): broadly comparable capability, differing mainly in configuration syntax and integration depth with each platform's other services.
2. Inference serving options: each supports serverless/on-demand inference, persistently-hosted real-time endpoints, and batch inference: again broadly comparable at a feature-checklist level, with real differences showing up in cold-start latency characteristics and autoscaling behavior under load, which matter more in practice than the checklist suggests.
3. Model registry and pipeline offerings: each has a native registry and a pipeline-orchestration capability (SageMaker Pipelines, Vertex AI Pipelines built on Kubeflow, Azure ML Pipelines): Vertex AI's being Kubeflow-based gives it a genuine portability edge if you might want to run similar pipeline definitions outside GCP later; SageMaker and Azure ML's pipeline systems are more tightly coupled to their respective ecosystems.
4. Key limitations pushing toward self-hosted: portability (all three managed offerings create real platform lock-in at the pipeline and tooling level, even though the underlying models themselves are usually portable); custom networking (highly specific VPC/network requirements can be easier to satisfy with self-hosted infrastructure you fully control); very custom GPU configurations (a managed platform's supported instance types may lag behind the newest hardware, or not offer the exact multi-GPU topology a specialized training job needs); and compliance (certain regulated environments have specific infrastructure requirements a managed platform's shared responsibility model may not cleanly satisfy without significant additional configuration).

Worked example

A concrete decision case: a team already deeply invested in GCP's data ecosystem (BigQuery, Dataflow) gets outsized integration value from Vertex AI specifically because of how tightly it connects to those existing services, an advantage that wouldn't transfer if the team evaluated SageMaker or Azure ML in isolation on feature checklists alone: the existing ecosystem investment is often the deciding factor in practice, more than any single platform capability being objectively superior.

Trade-offs & pitfalls

The trap in this kind of comparison is evaluating platforms on a feature checklist alone, since all three genuinely do cover the same broad capabilities at a surface level: the real differentiators are ecosystem fit (what else does your organization already run on this cloud), portability needs (how much does vendor lock-in actually matter for your situation), and operational specifics (autoscaling behavior, cold-start latency) that only show up under real production load, not in a documentation comparison.

Direct answer
Build the rubric around whether the writing actually works for its reader: does it state its point clearly, fit the audience it's for, give the reader something to do with it, and use a tone appropriate to its purpose, then justify each criterion by what a failure on it costs the reader.

Structured elaboration
Proposed criteria, with the reasoning for each:

1. Clear main point: can a reader state the document's core message in one sentence after reading it? Justification: this is the single biggest failure mode in unclear writing, so it anchors the rubric.
2. Appropriate structure: does the important information come early, with supporting detail after, rather than requiring the reader to read to the end to find the point?
3. Audience fit: is the level of jargon and assumed background knowledge appropriate for who's actually going to read this, rather than written for the author's own level of familiarity?
4. Concision: is there padding, hedging, or restatement that could be cut without losing meaning?
5. Actionable next step: if the document implies an action or a decision, is that action stated explicitly, rather than left for the reader to infer?
6. Precision: are claims specific and checkable, or do vague quantifiers stand in for actual numbers where numbers were available?
7. Tone fit: is the tone appropriate to the stakes and relationship, neither over-casual for a high-stakes audience nor needlessly formal for a quick internal note?
8. Honesty about caveats: does the document surface real limitations or risks, rather than smoothing them over to look cleaner?

Worked example
Applying this to a short vendor-status email: main point ("vendor is delayed two weeks") is clear in the first sentence; structure is fine; audience fit is appropriate (no unnecessary jargon for a business reader); concision is good at three sentences; the next step (approve a revised deadline) is explicitly stated; precision holds (a specific date is given, not "soon"); tone is appropriately direct without being alarmist; and the caveat (a small risk of a further one-week slip) is honestly included rather than hidden. That's 8 for 8, which is a genuinely well-written status update by this rubric.

Trade-offs and pitfalls

• A rubric with too many criteria becomes tedious to apply consistently; six to eight, as here, is usually enough to catch the failure modes that matter most without turning review into a lengthy checklist exercise.
• Some criteria trade off against each other (concision versus caveats, for instance); the rubric should make clear that cutting a genuine caveat to satisfy concision is a failure, not a win, on this rubric.
• A rubric like this works best as a discussion tool during review, not as a rigid pass/fail gate; a document can reasonably fail one criterion (say, tone) for a good reason specific to its context.

Direct answer

Developing independent judgment, not just correct outputs, requires repeated exposure to the same class of decision with you gradually receding from it, and requires the person to narrate their reasoning, not just report their choice. You know it's internalized, not just imitated, when their reasoning transfers to a situation you never coached them on directly, ideally one you weren't even present for.

How judgment gets built and verified

Coach the decision class, not the individual decision. A one-off answer to "should we do X" teaches them what to do this time. Judgment comes from recognizing the same underlying trade-off recurring in different clothes, which means you have to name the pattern explicitly rather than just resolving each instance.

Recede deliberately in stages. Start by explaining your own reasoning out loud when a decision comes up. Then ask them to predict what you'd decide, and why, before you weigh in. Then let them make the call and explain their reasoning to you after the fact. Then stop reviewing it at all. Each stage removes a layer of your safety net.

Make them narrate the criteria, not just the outcome. If someone can only say "I did X because I figured that's what you'd want," they've pattern-matched to you specifically, not internalized the underlying principle. You're listening for whether their stated reasoning would still hold up in a case where the "obvious" answer is actually wrong.

Verify with a novel or unobserved case. The strongest signal is watching them apply the same reasoning to a situation they haven't seen before, particularly one where you weren't in the loop and only heard about the decision afterward.

Worked example

Someone you're mentoring kept bringing you a specific recurring trade-off as if it were a one-off question each time: whether to fix a flaky, intermittently-failing test or ship a feature that was ready and waiting on it. Each time, you could have just answered the immediate question. Instead you treated it as a judgment gap and built a repeatable heuristic with them: is the flake masking a real intermittent bug or is it environment noise, what's the actual blast radius of shipping with it unresolved, and is there a way to quarantine the test that unblocks delivery without hiding the underlying risk.

Weeks later, a similar trade-off came up and they handled it without asking you first, only mentioning the decision afterward along with their reasoning. Their stated criteria matched the heuristic you'd built together, but in their own words, applied to a case with a different shape than the original one. That, not their confidence in the moment, was the signal it had actually internalized rather than just been remembered.

Trade-offs and pitfalls

Asking someone "do you understand?" tells you almost nothing; people say yes regardless of whether it's true. The only real test is watching the reasoning survive a situation you didn't script.

A subtle failure mode: rewarding a decision because it matches what you personally would have done, rather than evaluating whether the reasoning behind it was sound. If the original case was genuinely a coin toss, insisting they land on your exact answer trains obedience, not judgment.

The deeper trade-off is time and tolerance for being wrong. Actually receding means letting them face real stakes without a safety net, which means tolerating some decisions that turn out wrong in hindsight. That's not a bug in the process; it's the cost of judgment actually being tested rather than simulated.

A mentor who never truly recedes, who keeps reviewing every instance of the decision "just to be safe," never actually finds out whether the judgment transferred, because it's never been tested without the net.

Direct answer: Full-batch gradient descent computes the gradient over the ENTIRE dataset of size N before each update - O(N) work per step, but very few steps needed since each step's gradient is exact. Pure SGD (batch size 1) computes a gradient from a single example - O(1) work per step, but needs many more steps and each step's gradient is a noisy, high-variance estimate. Mini-batch SGD (batch size b) is the practical middle ground, O(b) work per step with gradient variance that decreases as $1/\sqrt{b}$, and - critically for modern hardware - the per-step work is highly PARALLELIZABLE across the batch dimension on a GPU, which is why throughput doesn't simply plateau immediately as batch size grows from 1.

Structured elaboration

• Per-step cost: proportional to batch size b (O(b) forward+backward work), true for all three variants (full-batch is simply b=N, pure SGD is b=1).
• Gradient variance: averaging over b independent samples reduces the VARIANCE of the gradient estimate by a factor of b (standard error shrinks as $1/\sqrt{b}$) relative to a single-sample gradient - this is why larger batches give a more accurate (lower-variance) estimate of the true full-dataset gradient direction, at the cost of more compute per step.
• GPU throughput and batch size: a GPU has a large but FIXED amount of parallel compute capacity; for small batch sizes, the GPU is under-utilized (idle compute units, since there isn't enough parallel work to fill them), so increasing batch size increases THROUGHPUT (examples processed per second) roughly linearly at first. Past some batch size, the GPU's compute units are fully saturated, and further batch-size increases no longer improve throughput per unit time (you're now compute-bound, not parallelism-starved) - this is why throughput plateaus, not because the underlying O(b) per-step cost changes, but because the WALL-CLOCK cost of processing b examples stops decreasing per-example once hardware parallelism is maxed out.

Worked example

Consider a GPU with enough parallel compute to fully utilize itself at batch size 256 for a given model. At batch size 32, the GPU processes each step in roughly the SAME wall-clock time it would take at batch size 256 (both are "cheap enough" to fit within one wave of parallel execution, dominated by fixed per-step overhead like kernel launch latency, not by the actual per-example compute) - so throughput (examples/second) at batch 32 is roughly 8x lower than at batch 256, purely from underutilizing available parallelism. Increasing batch size from 256 to 1024 (4x), once the GPU is already saturated at 256, roughly QUADRUPLES the wall-clock time per step (now truly compute-bound, scaling with actual work) while processing 4x the examples - so throughput stays roughly flat past this point, confirming the plateau.

Trade-offs & pitfalls

• Very large batch sizes, beyond the GPU-saturation throughput benefit, introduce their own OPTIMIZATION-QUALITY trade-off separate from the raw throughput question: overly large batches can generalize worse (a well-documented empirical phenomenon, sometimes called the "generalization gap"), requiring learning-rate scaling and other adjustments to compensate.
• Memory, not just compute, constrains batch size in practice - larger batches need proportionally more memory for activations (as discussed in the dense-layer-complexity survivor), and can hit a hard memory ceiling well before hitting a compute-bound throughput plateau.
• The variance-reduction benefit of larger batches has DIMINISHING returns ($1/\sqrt{b}$, not $1/b$) - doubling batch size from 1024 to 2048 reduces gradient noise by only about 29% (since $1/\sqrt{2} \approx 0.71$), a much smaller relative improvement than the first doubling from 1 to 2, which is part of why very large batch sizes give diminishing optimization benefit even before hitting hardware throughput limits.