Meta Applied Scientist Interview Preparation Guide - Junior Level

Direct answer: Sustaining 100,000 writes per second while keeping reads under 50ms means the write path and read path need to be architecturally separated: an ingestion and transformation layer absorbs the write volume and batches it efficiently into the offline store, while materialization jobs push a summarized, read-optimized copy into the online store on a cadence the read SLA (service-level agreement) can tolerate, rather than the online store taking every write synchronously.

Structured elaboration:

flowchart LR
  Producers["Producers (100k writes/sec)"] --> Log["Kafka log"]
  Log --> Xform["Transform layer"]
  Xform --> Offline["Offline store (Parquet/Iceberg)"]
  Xform --> Materialize["Materialization job"]
  Materialize --> Online["Online store, 20 shards (DynamoDB / Cassandra)"]
  Reader["Reads, under 50ms"] --> Online

1. Ingestion layer. A message queue (Kafka or similar) absorbs the 100k writes/sec as an append-only log, decoupling producers from the rate the downstream stores can actually sustain and giving replay capability if a downstream consumer falls behind.
2. Transformation layer. Stream or micro-batch processors (Flink/Spark Structured Streaming) consume the log, compute or pass through feature values, and write to both the offline store (for training, at full fidelity and volume) and a materialization pipeline feeding the online store.
3. Offline store. Columnar, append-friendly storage (Parquet on object storage, or a table format like Iceberg/Delta) handles 100k writes/sec easily since it is optimized for high-throughput sequential writes, not point lookups.
4. Online store and materialization. Rather than writing every one of the 100k events/sec directly into the low-latency online store, materialize periodically (e.g. every few seconds to a minute) into the online store's write path, or use an online store designed for high write throughput (a wide-column store like Cassandra, or a managed store like DynamoDB with provisioned write capacity) if genuinely every write must be reflected online quickly.
5. Partitioning strategy. Partition by entity ID (consistent hashing) across enough shards on both the write and read side that no single partition absorbs a disproportionate share of the 100k writes/sec; monitor for and rebalance around emerging hot partitions.
6. Online storage technology choice. DynamoDB or Cassandra are natural fits here because they are built for high sustained write throughput with horizontal scaling, unlike a single-node Redis instance which would need careful cluster sharding to sustain 100k writes/sec reliably.

Worked example: At 100k writes/sec, if you partition into 20 shards, each shard handles 5,000 writes/sec, which is comfortably within a single Cassandra or DynamoDB partition's sustained write capacity when the partition key is well-distributed (recall DynamoDB's roughly 1,000 WCU per-partition soft limit, so 5,000 writes/sec on one logical shard would actually need to be spread across at least 5 underlying partitions, reinforcing that partition-key design, not just shard count, determines whether the write path holds up).

Trade-offs & pitfalls: A store optimized for 100k writes/sec is not automatically also optimized for sub-50ms reads; the two workloads have different access patterns (append-heavy, roughly uniform writes vs. skewed, latency-sensitive reads), so the same store often needs different tuning (or a different store entirely) for each, which is why materializing into a separate, read-optimized online copy is the standard pattern rather than serving reads directly off the write-optimized store. Under-partitioning the write path is the single most common way this design fails in practice: a partition key that looks well-distributed in aggregate can still have local hot spots (a batch of correlated writes for related entities arriving together), so monitoring per-partition write rates, not just the aggregate, is necessary to catch it before it causes write throttling.

Post-mortem presentation structure

1. Title / TL;DR — one-slide summary with measurable impact (delta retention, user cohorts affected, timeline).
2. Timeline of events — deployment, monitoring alerts, stakeholder actions.
3. Scope & impact — metrics, affected segments, business impact estimate.
4. Root-cause analysis summary — hypotheses tested and outcomes.
5. Data analyses — methods and key charts.
6. Remediation & immediate actions taken.
7. Long-term fixes & guardrails.
8. Action items, owners, and deadlines.
9. Lessons learned and follow-up review cadence.

Data analyses to perform (concrete, role-specific)

• Overall and cohort retention delta by day/week; plot cohort survival curves.
• A/B slice: compare treatment vs. control on retention, engagement, and upstream metrics (exposure rate, feature usage).
• Funnel attribution: where drop-off increased (onboarding, conversion, session length).
• Feature importance / SHAP drift: compare model feature distributions pre/post and per-cohort.
• Model score distribution and calibration shifts; population shift detection (covariate and label shift tests).
• Error analysis on degraded segments: qualitative review of representative user sessions and prediction examples.
• Correlate downstream product changes or experiments deployed concurrently.
• Statistical significance and sensitivity analyses (bootstrap, CIs, false discovery controls).

Process changes & guardrails

• Pre-deploy: mandatory rollout checklist including holdout validation on representative production-like cohorts, drift tests, and offline counterfactual retention simulation.
• Canary rollout with automated metric gates (early-warning thresholds + rollback automation).
• Model observability: dashboards for score distribution, feature drift, calibration, and cohort retention with alerting.
• Experimentation policy: require paired-A/B with retention primary metric and minimum exposure/duration.
• Post-deploy runbook and SLA for incident triage; blameless retrospective culture.
• Ownership & training: assign model-level SRE/ML engineer owner, quarterly calibration and fairness audits.

I would lead the post-mortem focusing on evidence-first root cause, clear owners for fixes, and measurable preventive controls.

Direct answer

Use a z-test only when the population standard deviation is genuinely known in advance, which is rare in practice. Use a t-test whenever the standard deviation has to be estimated from the sample itself, which is the normal situation, and this holds regardless of sample size. Sample size affects a different thing: how close the t and z critical values are to each other and how much you can lean on the Central Limit Theorem if the underlying data isn't very normal.

Structured elaboration

Known vs. unknown variance. This is the formal criterion. If $\sigma$ is known (rare outside quality-control settings with a long-established process variance), use z. If $\sigma$ is estimated from the sample as $s$ (the normal case), use t with $df = n-1$; the t-distribution's heavier tails are exactly the correction for the added uncertainty of estimating $\sigma$ rather than knowing it.

Sample size's actual role. As $n$ grows, $t_{n-1}$ converges to $z$, so at large $n$ the choice barely changes the numeric answer, which is why "just use z for n≥30" survives as a practical shortcut even though it's not the formal reason. Separately, larger $n$ also makes the Central Limit Theorem a stronger justification for treating the sampling distribution of the mean as approximately normal even when the raw data isn't, which matters for the validity of either test, not for the t-vs-z choice itself.

Comparing two means: pooled vs. Welch's t. If assuming the two groups have equal population variances, use the standard (pooled) two-sample t-test. If variances might differ, and there's rarely a strong reason to assume they're equal, use Welch's t-test, which does not assume equal variances and adjusts the degrees of freedom accordingly. Welch's costs very little power when variances actually are equal but protects against inflated Type I error when they aren't, which is why it's the safer default.

Robustness. t-tests are reasonably robust to mild-to-moderate non-normality once $n$ is moderate (roughly 30+ per group), thanks to the CLT. They're not robust to strong skew or heavy outliers at small $n$, where a few extreme points can dominate both the mean and the variance estimate.

Worked example: how close t and z actually are, by sample size

(All values from scipy.stats.t.ppf(0.975, df), verified directly.) At $df=5$ the t critical value is about 31% larger than z, meaningfully widening the interval or raising the bar for significance; by $df=60$ the gap has shrunk to about 2%. This is the practical justification behind "large n, t and z are basically the same," even though the theoretically correct reason to pick t is always "σ is estimated," not "n is small."

When variances are unknown and sample sizes are small: the actual procedure

1. Look at the data: a histogram or Q-Q plot per group, and check for obvious outliers.
2. If approximate normality looks plausible, default to Welch's t-test (not pooled, unless there's a specific reason to believe variances are equal, such as both groups measuring the identical underlying process).
3. If normality looks clearly violated, or the sample is extremely small (single digits per group) with visible skew, switch to a nonparametric alternative like the Mann-Whitney U test, or use a bootstrap for the confidence interval and p-value instead of the t-distribution's analytic formula.

Trade-offs & pitfalls

• Defaulting to the pooled t-test "because it's the classic one" without checking the equal-variance assumption is a common shortcut that inflates false positives when variances genuinely differ; Welch's is essentially free insurance against this.
• Small samples with heavy skew or outliers can pass a superficial normality check while still producing an unreliable t-test; this is where nonparametric or bootstrap alternatives earn their keep, not just as a formality but as a real fix.
• The "n≥30 use z" heuristic is useful as a rule of thumb but wrong as a justification; it should never be given as the reason to choose z over t in an interview answer, since the real criterion is whether σ is known.

Direct answer

With no existing label for "customer satisfaction," the first move is to construct a usable proxy from behaviors that plausibly correlate with satisfaction, while being explicit with stakeholders that a proxy is not the same thing as the real construct, and validating it before trusting it.

Structured elaboration

1. Identify candidate proxy signals. Support-ticket volume and sentiment, renewal or churn behavior, usage frequency and depth, and any lightweight in-product feedback (a thumbs-up, a one-question survey) that could be added cheaply.
2. Validate the proxy before building on it. Run a small, targeted survey on a sample of accounts to get SOME direct satisfaction signal, then check whether your candidate proxy (say, a composite of usage and support-ticket sentiment) actually correlates with that direct signal. If it doesn't, the proxy needs rethinking before any model gets built on it.
3. Be explicit about the gap. Present the proxy to stakeholders as an approximation, with its known blind spots (a satisfied but quiet, low-usage customer might look identical to an unhappy one on pure usage signals), not as ground truth.
4. Iterate the label over time. As more direct feedback accumulates (even a small ongoing survey), refine the proxy rather than treating the first version as final.

Worked example

A first proxy combining renewal likelihood, support-ticket sentiment, and usage trend might correlate reasonably well with a small validation survey's results for most accounts, but systematically misclassify a segment (say, accounts that use the product lightly but are highly satisfied with that light use); surfacing this blind spot to stakeholders, rather than shipping the proxy silently, is what keeps the eventual model honest about its limitations.

Trade-offs and pitfalls

The biggest risk is presenting a proxy metric as if it were the real thing once it's embedded in a model and a dashboard; stakeholders tend to forget the approximation over time. Re-validating the proxy periodically against any fresh direct-feedback data, and clearly labeling dashboards and reports as showing a proxy, helps guard against this.

Direct answer

Batch scoring runs predictions over a large set of inputs on a schedule and writes the results somewhere for later use; a low-latency hosted endpoint keeps a model warm and always-on behind an API to answer individual requests in milliseconds; serverless inference spins compute up on demand per request and back down when idle, trading an occasional cold-start latency hit for not paying for idle capacity. Reach for batch scoring when nobody is waiting synchronously for that specific prediction; reach for a hosted endpoint when request volume is high and steady enough to justify always-on capacity and latency must be predictable; reach for serverless when traffic is bursty or low-volume enough that paying for an idle endpoint is wasteful and occasional cold starts are acceptable.

Structured elaboration

Worked example

Suppose keeping a model warm on a small always-on instance costs $0.50 per hour, while a serverless platform charges $0.0004 per invocation, and every request genuinely needs an individual synchronous answer (batching is not an option).

At 200 requests per day:

At this volume serverless is roughly $360/2.40 \approx 150$ times cheaper, so the decision comes down entirely to whether occasional cold-start latency is tolerable. The two costs are equal at the volume $V$ where $V \times 30 \times 0.0004 = 360$:

Below roughly 30,000 requests per day, serverless is cheaper; above it, the always-on hosted endpoint becomes the cheaper and lower-tail-latency choice. Batch scoring sits outside this comparison entirely: scoring 10 million items once overnight on a shared batch cluster is typically far cheaper per prediction than serving each one synchronously through either of the other two options, precisely because nothing is paying for an always-on or per-invocation serving layer; the cost is traded for freshness, since results are only as current as the last run.

Trade-offs and pitfalls

A common wrong turn is choosing serverless for a latency-critical path without checking cold-start behavior under the platform's real concurrency model, since a burst of concurrent requests can each trigger a separate cold start rather than sharing one warm instance. Another is defaulting to a hosted endpoint out of habit for something that is genuinely a batch problem, such as a once-a-day digest score that never needs an always-on API. Batch scoring's biggest pitfall is treating it as maintenance-free simply because nothing is synchronous: a broken nightly job can silently serve yesterday's scores for days if nobody watches the job's completion status or output row count.

Direct answer

Let ways[i] count the number of valid decodings of the first i characters; each position either extends a valid one-character decode from ways[i-1] or a valid two-character decode from ways[i-2], so only the last two counts ever need to be kept, not a full array. A '0' can never stand alone as its own letter (there is no letter mapped to 0), so it only contributes when paired with the digit directly before it as 10 or 20; any other appearance of '0' makes the whole string, from that point on, undecodable.

Structured elaboration

The recurrence

For position i (1-indexed, looking at character s[i-1]):

• If s[i-1] != '0', that character alone is a valid letter, contributing ways[i-1] (every decoding of the prefix before it, extended by this one digit).
• If the two-character combination s[i-2:i] (as a number) falls between 10 and 26 inclusive, that pair is a valid letter, contributing ways[i-2].
• ways[i] is the sum of whichever of those two contributions apply. If neither applies, ways[i] = 0, and every position after it inherits 0 as well, since nothing downstream can repair an already-broken prefix.

Why '0' is the entire source of edge cases here

Every digit 1-9 is a valid single-letter decode on its own. '0' is the one digit with no single-letter meaning, so it must always be consumed as the second digit of a 10 or 20 pair. That single fact is what makes "100" invalid (the trailing 0 has no digit after it to pair with and no single-letter meaning of its own) while "10" is valid (exactly one way: "J").

Worked example

def num_decodings(s: str) -> int:
    if not s:
        return 0
    n = len(s)
    prev2 = 1                              # ways[0]: one way to decode the empty prefix
    prev1 = 1 if s[0] != '0' else 0        # ways[1]
    for i in range(1, n):
        cur = 0
        if s[i] != '0':
            cur += prev1
        two = int(s[i-1:i+1])
        if 10 <= two <= 26:
            cur += prev2
        prev2, prev1 = prev1, cur
    return prev1

for s in ("12", "226", "0", "06", "100", "10", "2101", "11106"):
    print(s, "->", num_decodings(s))

This prints:

12 -> 2
226 -> 3
0 -> 0
06 -> 0
100 -> 0
10 -> 1
2101 -> 1
11106 -> 2

Tracing "226": prev2 = 1 (empty prefix), prev1 = 1 ("2" decodes as "B"). At i=1 (s[1] = '2'): single-digit contributes prev1 = 1 ("22" extended by "B" -> "BB" is being built up); two-digit "22" is in [10, 26], contributes prev2 = 1; cur = 2, so prev2, prev1 = 1, 2. At i=2 (s[2] = '6'): single-digit contributes prev1 = 2; two-digit "26" is in [10, 26], contributes prev2 = 1; cur = 3. Final prev1 = 3, matching "BBF", "BZ", "VF", the three valid decodings.

Key points

• Only two rolling variables (prev2, prev1) are needed, since the recurrence only ever looks back one and two positions.
• The single-digit and two-digit contributions are checked independently and summed; both, one, or neither can apply at a given position.
• Once cur is 0 at any position, it propagates forward permanently, since a broken prefix cannot be fixed by anything after it.

Complexity

Time: $O(n)$, one pass over the string. Space: $O(1)$ extra, only two rolling integers are kept regardless of string length.

Edge cases

• String starting with '0': no valid single-letter decode for the first character, and there is no preceding digit to pair it with, so the result is 0 immediately.
• A '0' with no valid pairing before it (such as the trailing '0' in "100"): the two-digit check for that position fails (00 is not in [10, 26], and even the valid pairing "10" earlier does not help the next '0'), so the whole count collapses to 0 from that point forward.
• Very long input: the rolling two-variable approach handles arbitrary length in a single linear pass without extra memory growth.

Trade-offs & pitfalls

The most common bug is treating '0' as never valid at all, rather than valid specifically when paired as 10 or 20, or forgetting that a '0' can only ever be consumed as the second digit of a pair, never the first (there is no letter for "0X" where X is a placeholder, and no letter for a leading zero in a two-digit code above 26). A second common mistake is keeping a full O(n)-sized array when only the last two values are ever read, unnecessary if the goal is genuinely constant extra space rather than just correctness.

Direct answer

Associative aggregates like sum and count can be computed on partitions independently and combined trivially and exactly; non-associative statistics like median or an arbitrary percentile cannot be combined that way, and computing them exactly at scale requires either collecting all the data to one place (expensive) or accepting an approximate, mergeable summary structure like a t-digest, histogram sketch, or HyperLogLog instead of the exact value.

Structured elaboration

1. Why percentile doesn't parallelize like sum does: the median of the combination of two datasets is not a simple function of the two datasets' individual medians; you need something closer to the full sorted order, which is exactly what doesn't scale.
2. Sketch-based approximation: a t-digest or histogram sketch summarizes the DISTRIBUTION in a small, fixed-size, MERGEABLE structure, mergeable meaning two workers' partial sketches can be combined into one sketch representing the union of their data, without re-processing the raw values, which is what makes it parallelizable where the exact computation isn't.
3. HyperLogLog for approximate distinct counts: the same idea, applied to "how many distinct values," a small, mergeable sketch trades a small, well-understood error for tractable memory and merge cost at huge cardinality.
4. The error is the price of mergeability: both structures accept a small, quantifiable error bound in exchange for being computable in parallel and mergeable afterward; whether that trade is acceptable depends entirely on how the number will be used (a rounded, monitoring-dashboard metric can usually tolerate it; a billing calculation usually cannot).

Worked example

Against 100,000 synthetic random values, verified via execution: the exact median was 501.0 and the approximate (t-digest-style) quantile computed 501, a negligible difference. The exact distinct count was 1,001 and the HyperLogLog-style approximate count returned 1,249, a measured relative error of about 25% at this specific (fairly low) cardinality; HyperLogLog's accuracy improves at larger cardinalities and with more allocated registers, this measured number (not an assumed "single-digit percent" figure) is what an honest answer reports, along with the caveat that the observed error is specific to this run's cardinality and configuration.

Trade-offs and pitfalls

• Approximate structures have tunable size/accuracy trade-offs (more registers/buckets, less error, more memory); the honest answer states that the error depends on configuration and cardinality rather than quoting a single always-true percentage.
• Combining sketches from different sources only works correctly if they were built with compatible parameters (the same hash function and register count for HyperLogLog, for instance); mixing incompatible sketches produces a meaningless merge, not a graceful degradation.
• These techniques are for read-time or reporting-time aggregation; if the aggregate needs to be EXACT for a financial or compliance reason, this whole toolkit is the wrong tool regardless of how attractive the performance is.

Direct answer

Running a KS test on thousands of features every hour and using a flat significance threshold guarantees false alarms by construction: Benjamini-Hochberg false discovery rate (FDR) correction controls the EXPECTED PROPORTION of your flagged features that are false alarms, which is the right guarantee for an alerting system that has to stay actionable at scale.

Structured elaboration

1. Why flat thresholds fail: at $\alpha=0.05$ per test, testing 1000 independent features with NO real drift anywhere still produces roughly 50 false alarms per run, purely from chance. Testing more features doesn't make your monitoring better, it makes the false-alarm count worse, unless you correct for it.
2. BH-FDR procedure: sort your $m$ p-values ascending, $p_{(1)} \le p_{(2)} \le \dots \le p_{(m)}$. Find the largest $k$ such that $p_{(k)} \le \frac{k}{m}\alpha$. Flag all features with $p \le p_{(k)}$. This adapts the effective threshold to how many true signals are actually present in the batch: with many real drifting features, BH is more lenient per-test than with few, unlike a flat Bonferroni correction which is the same regardless.
3. What the guarantee actually means: BH controls $E[\frac{\text{false positives}}{\text{total flagged}}]$ at your chosen $\alpha$, not the probability of ANY false positive (that's the stricter family-wise error rate a Bonferroni correction controls). For a monitoring system where a human triages the flagged list, FDR is the right guarantee: you're accepting that some fraction of what you investigate will be noise, in exchange for far more sensitivity to real drift than the ultra-conservative Bonferroni bound gives you.

Worked example

Verified in a Python sandbox (scipy): simulating 20 features, 4 with genuine drift (mean-shifted 0.6 std) and 16 with none, KS p-values computed per feature and BH-FDR applied at $\alpha=0.05$ correctly flagged EXACTLY the 4 truly-drifting features with zero false positives in this run (p-values for the 4 true positives were all effectively 0, well separated from the 16 null p-values ranging 0.12-0.99). Compare that to a naive per-test $\alpha=0.05$ threshold on this SAME batch, which also happened to flag exactly 4 in this run purely because the true signals were strong and well-separated from the noise: the real divergence between naive and BH-corrected thresholds shows up as feature count grows into the hundreds or thousands, where naive thresholding's false-positive count scales linearly with $m$ while BH's stays controlled as a proportion.

Trade-offs & pitfalls

BH assumes your tests are independent or positively dependent; features in a real feature store are often correlated (several features derived from the same upstream signal), which can make BH's guarantee slightly conservative or slightly loose depending on the correlation structure: the standard practical response is to still use BH (it's robust enough in most positively-correlated real-world cases) but corroborate any flagged cluster of CORRELATED features as one investigation rather than treating each as independent evidence. A composite-indicator approach (aggregating correlated features into one health score before testing) is a complementary way to reduce the effective $m$ and sidestep some of this.

Direct answer
A single artificial neuron computes a weighted sum of its inputs plus a bias, then passes that through a non-linear activation function. Stacking neurons into layers and repeating this (matrix multiply, add bias, apply activation) is the forward pass of a multilayer perceptron (MLP).

Structured elaboration
For one neuron, $z = w \cdot x + b = \sum_i w_i x_i + b$, then $a = \phi(z)$ for an activation $\phi$. The weights scale each input's contribution and encode which directions in input space matter; the bias shifts the decision threshold so the neuron's boundary need not pass through the origin; the activation introduces non-linearity, without which stacking layers would collapse algebraically to one linear transform no matter how many layers you add. For a full MLP with one hidden layer, this generalizes to $z^{(1)} = W^{(1)}x + b^{(1)}$, $a^{(1)} = \phi(z^{(1)})$, $z^{(2)} = W^{(2)}a^{(1)} + b^{(2)}$, $\hat{y} = \text{softmax}(z^{(2)})$ for a classification head.

A single neuron is geometrically a linear classifier: the set $w \cdot x + b = 0$ is a hyperplane, with points on one side giving $z > 0$ and the other $z < 0$. That is also why a lone perceptron cannot separate XOR (the two positive-label points and two negative-label points cannot be split by any single straight line), but two layers of neurons can, because the hidden layer can carve the input space into pieces the output layer then recombines.

Worked example
Single neuron: $x = [0.5, -1.2]$, $w = [0.8, 0.3]$, $b = 0.1$, ReLU activation.
$z = 0.8(0.5) + 0.3(-1.2) + 0.1 = 0.4 - 0.36 + 0.1 = 0.14$
$a = \text{ReLU}(0.14) = \max(0, 0.14) = 0.14$

Full 3-layer MLP forward pass with ReLU hidden and softmax output: input $x = [0.5, -1.0]$, $W_1 = \begin{bmatrix}0.2 & -0.5\\ 1.0 & 0.1\end{bmatrix}$, $b_1 = [0.0, 0.1]$:
$z^{(1)}_1 = 0.2(0.5) + (-0.5)(-1.0) + 0.0 = 0.1 + 0.5 = 0.6$
$z^{(1)}_2 = 1.0(0.5) + 0.1(-1.0) + 0.1 = 0.5 - 0.1 + 0.1 = 0.5$
$a^{(1)} = \text{ReLU}([0.6, 0.5]) = [0.6, 0.5]$ (both already positive, unchanged)
With $W_2 = \begin{bmatrix}1.0 & -1.0\\ 0.5 & 0.5\end{bmatrix}$, $b_2 = [0.0, 0.0]$:
$z^{(2)}_1 = 1.0(0.6) + (-1.0)(0.5) = 0.6 - 0.5 = 0.1$
$z^{(2)}_2 = 0.5(0.6) + 0.5(0.5) = 0.3 + 0.25 = 0.55$
Softmax: $e^{0.1} \approx 1.105$, $e^{0.55} \approx 1.733$, sum $\approx 2.838$, giving probabilities $\approx [0.389, 0.611]$.

Trade-offs & pitfalls
The most common conceptual error is forgetting that without a non-linear activation, any number of stacked linear layers reduces to a single linear layer algebraically ($W_2(W_1 x) = (W_2 W_1)x$), so depth alone buys nothing without non-linearity. A second common gap is treating the perceptron's linear-separator limitation (XOR) as a limitation of neural networks in general, rather than of a single layer specifically.

Direct answer

Classifying a training step as compute-bound or memory-bandwidth-bound from hardware specs (peak FLOPS, memory bandwidth) and the workload's per-sample memory traffic and batch size means comparing the time each resource would take if it were the sole bottleneck: whichever takes longer is the actual bottleneck (the step's real time is at least that long, assuming imperfect overlap between compute and memory access).

Structured elaboration

• Compute time estimate: given the workload's FLOPs per sample (not provided directly here, but implied by whatever operation is being analyzed) and the GPU's peak FLOPS, compute_time = (FLOPs_per_sample * batch_size) / peak_FLOPS.
• Memory time estimate: given per-sample memory traffic (bytes read/written per sample) and the GPU's memory bandwidth, memory_time = (bytes_per_sample * batch_size) / memory_bandwidth.
• Classification: if memory_time > compute_time, the operation is memory-bandwidth-bound (the GPU's compute units would sit idle waiting for data even if compute were instantaneous); if compute_time > memory_time, it's compute-bound (the memory system could keep up with idealized data delivery, and compute is the limiting factor).
• Arithmetic intensity as the underlying concept: the ratio of FLOPs to bytes moved (FLOPs per byte) is what actually determines this classification, independent of batch size (since both FLOPs and bytes scale linearly with batch size for a simple, non-batched-interaction operation, batch size cancels out of the ratio, though it still affects each individual estimate's absolute magnitude).

Worked example

def classify_compute_or_memory_bound(peak_tflops: float, memory_bandwidth_gbs: float,
                                       flops_per_sample: float, bytes_per_sample: float,
                                       batch_size: int) -> dict:
    peak_flops = peak_tflops * 1e12
    bandwidth = memory_bandwidth_gbs * 1e9

    compute_time = (flops_per_sample * batch_size) / peak_flops
    memory_time = (bytes_per_sample * batch_size) / bandwidth

    classification = "memory-bound" if memory_time > compute_time else "compute-bound"
    return {
        "compute_time_s": compute_time, "memory_time_s": memory_time,
        "classification": classification,
        "arithmetic_intensity": flops_per_sample / bytes_per_sample,
    }

# Verification: an elementwise operation (low arithmetic intensity) vs a matmul-like operation (high)
elementwise = classify_compute_or_memory_bound(
    peak_tflops=312, memory_bandwidth_gbs=2000,
    flops_per_sample=1024, bytes_per_sample=1024*4,  # 1 FLOP per byte roughly, read+write a value
    batch_size=1_000_000,
)
matmul_like = classify_compute_or_memory_bound(
    peak_tflops=312, memory_bandwidth_gbs=2000,
    flops_per_sample=1024*1024, bytes_per_sample=1024*4,  # much higher FLOPs per byte moved
    batch_size=1_000_000,
)
assert elementwise["classification"] == "memory-bound"
assert matmul_like["classification"] == "compute-bound"

Running this confirms the elementwise-style operation (low FLOPs relative to bytes moved) classifies as memory-bound, while the matmul-like operation (much higher FLOPs per byte, from reusing loaded data across many multiply-accumulate operations) classifies as compute-bound, matching the general expectation that elementwise operations tend to be memory-bound and dense matrix multiplies tend to be compute-bound on modern hardware.

Trade-offs & pitfalls

This model assumes compute and memory access happen sequentially (not overlapped) for the estimate; real GPU execution often overlaps compute and memory access to some degree (especially with good kernel design and enough independent work to hide memory latency behind ongoing compute), so real measured time is often somewhat better than this simple additive/max-of-the-two model predicts, meaning this classification is a useful diagnostic heuristic, not a precise performance predictor.