Netflix AI Engineer (Junior Level) Interview Preparation Guide

Direct answer

Acceptance criteria for a new model version should cover three tiers: does it perform at least as well as what's currently live, measured against the CURRENT production model on the same window rather than an absolute number in isolation; is it operationally safe to run, meaning it's been load-tested and has a rollback path that's actually been exercised; and can degradation be detected after it ships, meaning monitoring and alerting exist and are tested before the version receives real traffic.

Structured elaboration

1. Offline evaluation gate. Evaluate on a holdout that matches the production data distribution, and compare against the CURRENT production model as the baseline, not an absolute threshold set once and left static. Just as important: check slice-level performance, the new version must not regress badly on any important subgroup even if the aggregate metric improves, since an aggregate gain can hide a real regression concentrated in one slice.

2. Operational readiness gate. A versioned, reproducible artifact; integration tests passing in the actual serving stack; load testing to the expected peak traffic with a defined latency SLO (service-level objective) that's actually met; and a rollback path that has been exercised in practice, not just written down as a runbook step.

3. Observability gate. Dashboards and alerting for the new version's key signals (latency, error rate, prediction distribution) need to exist and be tested, for example by triggering a deliberate fire-drill alert, BEFORE the version takes real traffic. Wiring this up after go-live defeats the point: a regression in the first hour would otherwise be invisible.

4. Staged exposure. Ship behind a canary or a small percentage of traffic first, with a pre-agreed hold time and a pre-agreed rollback trigger (for example, a key metric moving outside an agreed tolerance band relative to the current production baseline).

Worked example

Version B is compared head-to-head against the currently-live version A on the same week of traffic. Suppose version A has 90% aggregate accuracy on that holdout and version B has 91%, an aggregate improvement. But on the "new users" slice specifically, version A scores 85% and version B scores 79%, a 6-point regression on that named slice even though the aggregate number went up. Under a slice-level acceptance gate, this fails: the aggregate improvement does not clear version B for release, because it's masking a real regression on a specific, identifiable population. This is exactly why acceptance criteria that only check the aggregate metric are insufficient on their own.

Trade-offs & pitfalls

Setting the acceptance bar as a fixed absolute threshold, rather than relative to the model currently running, can let a genuinely worse model in during an easy traffic period, or block a genuinely better model during a harder one; comparing head-to-head against the current production model on the same window controls for that. Skipping the observability gate under launch-timeline pressure is the most common shortcut, and it's precisely what turns a bad ship into a slow, invisible regression instead of a fast, visible, and quickly-rolled-back one. A rollback path that has never actually been exercised is not a proven rollback path; "documented" and "tested" are different guarantees, and only the second one should count toward sign-off.

Direct answer

Build trust before you need anything, by being reliable on small things, transparent about your constraints and capacity, and by giving the other person visibility into your world so they aren't surprised later. Waiting to invest in the relationship until you need a favor makes the ask feel transactional.

Framework

Lead with reliability on small things. Deliver on small, early commitments, answer a question promptly, show up to their planning session, so your word has a track record before there's a high-stakes ask on either side.

Be transparent about constraints. Proactively share capacity, risk, and known limitations rather than letting the other person find out the hard way, mid-project.

Give visibility into your world. Invite them into a review or share a roadmap or dashboard, so they understand your constraints without needing you to explain from scratch every time.

Make it reciprocal early. Ask what they need and what's on their plate too. Trust runs both directions, not just from you demonstrating value to them.

Worked example

Situation: a new product manager joins and will depend on your team, for example a platform or infrastructure team, for their roadmap.
Action: in the first couple of weeks, gave the PM read access to the team's capacity and roadmap view along with a short walkthrough, rather than waiting for them to ask. Proactively flagged one known constraint, a piece of infrastructure that was close to capacity, before it affected their planning. Followed through quickly and visibly on a small early request, answering a scoping question the same day, to establish reliability before anything high-stakes came up.
Result: by the time the PM had a genuinely high-stakes ask, an accelerated timeline, there was already a working relationship and a shared understanding of constraints. The conversation started from what's actually possible given what you already know, instead of starting from zero.

Trade-offs and pitfalls

• Trust-building gestures can look like busywork if they aren't tied to something concrete. Keep them small and genuinely useful, not performative.
• Over-sharing every constraint upfront can read as excuse-making before there's even a request. Calibrate to what's actually relevant to their planning.
• The senior differentiator is doing this proactively, before there's a need, rather than scrambling to build rapport only once you need something from the other person, which reads as transactional.

Direct answer
One Adam step is four small computations in sequence: update the two moving averages, bias-correct them, then apply the rescaled update to the parameter.

Structured elaboration

import numpy as np

def adam_step(param, grad, m, v, t, lr=1e-3, beta1=0.9, beta2=0.999, eps=1e-8):
    """One Adam update. param, grad, m, v: same-shape arrays. t: 1-based step count.
    Returns (param_new, m_new, v_new)."""
    m_new = beta1 * m + (1 - beta1) * grad
    v_new = beta2 * v + (1 - beta2) * (grad * grad)
    m_hat = m_new / (1 - beta1 ** t)
    v_hat = v_new / (1 - beta2 ** t)
    param_new = param - lr * m_hat / (np.sqrt(v_hat) + eps)
    return param_new, m_new, v_new

Worked example
As a genuine correctness test (rather than just running one isolated step), I ran this function for 2,000 steps against the simple convex quadratic $f(x)=(x-3)^2$ with gradient $2(x-3)$, starting from $x=0$ with $lr=0.01$: the parameter converged to $x=3.0$ (the exact analytic minimum) to within floating-point precision, confirming the implementation genuinely descends toward the correct minimum over many steps, not just that it runs without error on a single call.

np.random.seed(0)
x, m, v = 0.0, 0.0, 0.0
for t in range(1, 2001):
    grad = 2 * (x - 3)
    x, m, v = adam_step(x, grad, m, v, t, lr=0.01)
print(f"final x = {x:.10f}")
print(f"error from true minimum (3.0) = {abs(x - 3.0):.2e}")

Executed output:

final x = 3.0000000000
error from true minimum (3.0) = 2.43e-13

The printed error, 2.43e-13, is exactly what "to within floating-point precision" means concretely here: not literally zero (each Adam step still takes a nonzero-sized update even arbitrarily close to the minimum, so it converges toward 3.0 rather than landing on it exactly), but many orders of magnitude below anything that would matter for an optimizer implementation.

Trade-offs & pitfalls
A common bug is starting the step counter $t$ at 0 instead of 1; the bias-correction denominators $1-\beta_1^t$ and $1-\beta_2^t$ are exactly zero at $t=0$, causing an immediate division by zero. Complexity is $O(n)$ per step for $n$ parameters, with $O(n)$ additional memory for the $m$ and $v$ state (Adam's memory footprint is 2x a plain SGD optimizer's, since it must persist both moment estimates between steps, not just the parameters themselves). A natural extension is AdamW, which adds one more term, $-\eta\lambda\,\text{param}$, applied directly to the parameter update, decoupled from the $m/\sqrt{v}$ computation entirely.

Direct answer

A/B testing, canary, shadow, and phased rollout all reduce risk when releasing a new model, but they route traffic and reveal information differently: shadow reveals nothing to users but validates behavior; canary and phased rollout expose a small, growing slice of real users; A/B testing runs two versions concurrently at a fixed split specifically to measure a statistically valid comparison.

Structured elaboration

Worked example

For a model that affects revenue, start with shadow testing first regardless of urgency: it's the only strategy with zero user-facing risk, and it catches the most basic classes of failure (crashes, wildly implausible predictions, latency blowups) before any real user is exposed at all. Once shadow testing clears, move to a canary with a small initial percentage, watching BOTH technical metrics (latency, error rate) and business metrics (the actual revenue-relevant signal) before progressing: since a model can be technically healthy (fast, error-free) while still being a business regression (worse recommendations, lower conversion), and only real traffic exposure reveals that. Feature flags matter throughout this progression as the mechanism that makes each traffic-percentage step instantly reversible without a code redeploy; a database or schema migration that the feature computation depends on needs to be BACKWARD-compatible for the duration of the rollout, since old and new model versions may be serving simultaneously mid-rollout and both need the schema to work.

Trade-offs & pitfalls

Shadow testing's blind spot is real: because nothing is actually served, it can't catch problems that only manifest from genuine user REACTION to a different prediction (a subtly-different ranking that changes user behavior in a way shadow testing, which never shows the prediction to a real user, structurally cannot observe): this is exactly why shadow testing is a necessary first gate, not a sufficient one on its own, and always needs to be followed by a real-traffic stage (canary or A/B) before full rollout.

Direct answer

Normalization (min-max scaling) rescales a feature into a fixed range, usually $[0,1]$; standardization (z-score) centers a feature at zero mean with unit variance. Standardization is the safer default for most models because it isn't anchored to the exact min and max seen in training, while normalization is preferred when you need a strictly bounded range, such as feeding pixel intensities into a neural network.

Structured elaboration

Outliers affect the two very differently: a single extreme value inflates $x_{\max}$ (or deflates $x_{\min}$) and compresses every other normalized value toward one end of the range, while standardization's mean and standard deviation are also outlier-sensitive but the effect is comparatively milder since it doesn't hard-clip against two data points.

Model sensitivity: distance-based and gradient-based models (k-NN, SVM, neural networks, anything using Euclidean distance or gradient descent) are sensitive to feature scale because a feature with a larger numeric range dominates the distance calculation or the gradient magnitude purely by virtue of its units. Tree-based methods (Random Forest, gradient boosting) are scale-invariant because they split on threshold comparisons within a single feature at a time, and a monotonic rescaling doesn't change which side of a threshold a value falls on.

Rule of thumb: standardization for neural networks and most classical models by default; normalization specifically when you need a bounded range (image pixel values, or a model like a min-max-based similarity metric that explicitly expects $[0,1]$ inputs); tree models need neither.

Worked example

For a feature with values $[10, 20, 30, 40, 1000]$ (one clear outlier): min-max scaling maps 10→0.0 and 1000→1.0, but now 20, 30, and 40 all crowd into the range $[0.01, 0.03]$, nearly indistinguishable from each other. Standardization on the same data ($\mu = 220$, $\sigma \approx 390.1$) gives z-scores around $-0.54, -0.51, -0.49, -0.46$ for the four normal values and about $2.00$ for the outlier, less visually crushed together though still outlier-influenced through $\mu$ and $\sigma$.

Trade-offs and pitfalls

To recover original units for interpretability, invert the transform: $x = x_{\text{std}} \cdot \sigma + \mu$ (or the corresponding inverse for min-max), using the exact $\mu, \sigma$ (or min, max) computed on the TRAINING data, never recomputed on new data, or you'll silently apply a different transform than the model was trained on.

Direct answer

When a model improves engagement but hurts a related revenue metric, the first step is confirming both effects are real and not artifacts of measurement, then deciding based on the actual net effect on the metric the business ultimately cares about, not on which single metric looks better in isolation.

Structured elaboration

1. Verify both effects with proper experimental rigor. Confirm the engagement gain and the click-through-rate drop are both measured from the same, valid experiment (not one from an A/B test and the other from a noisier before-after comparison), so you're comparing effects you actually trust.
2. Trace the revenue drop to its mechanism. Is the CTR decline because the new model is surfacing more organic content and pushing ads further down, or some other specific behavior change? Understanding the mechanism tells you whether it's fixable without giving up the engagement gain.
3. Compute the net business effect. Translate both the engagement gain and the revenue drop into the same unit (typically dollars, over a comparable time horizon), including a reasonable estimate of engagement's longer-term value (retention), not just its immediate effect.
4. Consider a modify option before a binary keep-or-rollback decision. If the mechanism is traceable (ads pushed down), a modification (reserve specific slots for ads regardless of the new ranking) might recover most of the revenue while keeping most of the engagement gain.
5. Communicate the recommendation with the mechanism and the net number, not just a verdict. Stakeholders need to understand WHY you're recommending keep, modify, or rollback, not just which option won.

Worked example

If the mechanism turns out to be that better organic ranking pushed sponsored placements further down the page, reserving a fixed number of above-the-fold slots for ads regardless of the ranking model's organic scores might recover most of the CTR loss while preserving the bulk of the engagement improvement, turning what looked like a binary trade-off into a modify-and-keep-both outcome.

Trade-offs and pitfalls

The common mistake is treating this as purely a math problem (whichever metric moved by a bigger relative percentage wins) without tracing WHY the trade-off is happening, which is often what reveals a modify option that a pure keep-or-rollback framing would miss. The other risk is under-valuing engagement's longer-term effect on retention because it's harder to quantify than an immediate revenue number, systematically biasing the decision toward the metric that's easier to measure.

A Docker image is a read-only, versioned filesystem snapshot that contains everything needed to run an application: OS libraries, binaries, runtime, and your application files. It is built as a stack of immutable layers (each Dockerfile instruction typically creates a new layer). Layers are content-addressed and cached: unchanged layers are reused, which speeds builds and reduces storage duplication.

A running container is an instance of an image with a writable thin layer on top. The image stays unchanged; the container holds runtime state, ephemeral filesystem changes, and process isolation.

Why images improve reproducibility for ML deployments

• Immutable, versioned artifacts pin OS, library versions, and code, so models run the same across dev, CI, and prod.
• Layered caching enables reproducible, fast rebuilds and deterministic dependency snapshots.
• Combined with image registries and tags (or digests), you can roll back exactly to a tested image.

Common tools

• Build: docker build / BuildKit, docker buildx, kaniko (cluster builds), Podman, Google Cloud Build.
• Inspect: docker images, docker history, dive, nerdctl, skopeo.
• Registries: Docker Hub, AWS ECR, GCR/Artifact Registry, GitHub Container Registry, private registries.

Trade-offs: baking large model artifacts into images vs pulling at runtime

• Bake into image (pros): single artifact with pinned code+model, faster startup (no runtime download), works offline, simpler deployment atomicity.
• Bake (cons): very large images → slower CI pushes/pulls, more storage, harder incremental updates, registry quota issues.
• Pull at runtime (pros): smaller images, flexible model rollout (can swap models without rebuilding), use of CDN/object storage, streaming large files, better for frequent model updates.
• Pull (cons): adds startup latency, needs robust retry/validation, requires secure credentials and bandwidth, can complicate reproducibility unless you pin checksums/versions.

Practical suggestions

• For small/rarely-updated models, bake into image. For large or frequently updated models, store artifacts in model registries/object stores and download on startup (or use init containers), and always pin checksums or use image digests to retain reproducibility. Use multi-stage builds and minimal base images to keep images small.

Also covers (folded from merged near-duplicates): Also folds model-artifact-vs-container (the artifact-vs-image packaging distinction) into the same foundational Docker/packaging staple.

Pooling and strided convolution both reduce spatial resolution but differ in behavior and inductive bias.

Definitions:

• Pooling (max/average): fixed, non-learned operation summarizing a local patch.
• Strided convolution: learned filters applied with stride >1 that both extract features and downsample.

Comparison:

• Translation invariance:
  • Pooling (especially max) increases local translation invariance because it discards precise location within the pool window.
  • Strided conv offers less built-in invariance; invariance must be learned via filters and architecture.
• Learnable parameters:
  • Pooling has no parameters.
  • Strided conv has learnable weights, allowing more flexible, task-specific spatial aggregation.
• Information loss:
  • Pooling is a coarse, fixed reduction — can discard useful info (average smooths, max keeps strongest activation).
  • Strided conv can learn to preserve or transform important signals before decimation, potentially reducing harmful information loss.
• Computational & representational trade-offs:
  • Strided conv increases params and compute but can improve feature richness.
  • Pooling is cheaper and regularizes by limiting capacity.

When modern architectures prefer one:

• Many modern CNNs (ResNets, EfficientNets) favor strided convolutions for downsampling because learned filters integrate feature extraction and spatial reduction, improving accuracy.
• Pooling still appears in lightweight or classical designs, or where strong built-in invariance/regularization is desired (e.g., global average pooling before classification).
• Some architectures combine both (conv then pooling) or use anti-aliasing filters (blur + downsample) to reduce aliasing while keeping invariance.

Bottom line: use pooling for parameter-free invariance and simplicity; use strided conv when you want learned, task-specific downsampling and better representational power.

Direct answer: Fine-tuning a model to respect retrieved evidence and abstain when retrieval is insufficient (optionally combined with RLHF-style reward signals) requires a grounding-aware loss or reward, a verifier component that checks generated claims against retrieved passages, a calibrated confidence signal that the abstention threshold can actually trust, and evaluation specifically designed to measure hallucination reduction, not just general answer quality.

flowchart LR
    Q[Query] --> Retriever
    Retriever --> Reranker
    Reranker --> Generator[Generator: query + top-N passages]
    Generator --> Verifier[Claim Verifier: per-claim support score]
    Verifier -->|unsupported claims below threshold| Answer[Answer with citations]
    Verifier -->|too many unsupported claims| Regen[Constrained regeneration or refusal]
    Regen --> Generator

Structured elaboration:

• Grounding and verification: after generation, extract the individual claims from the model's answer and check each one against the retrieved passages using an entailment or QA-style verification model, producing a per-claim support score and a link back to the supporting passage; when the fraction of unsupported claims exceeds a threshold, trigger either a constrained regeneration (forcing citations) or an explicit refusal-to-answer rather than shipping an unverified claim.
• Calibration: since the whole design gates behavior on a per-claim support score and an overall unsupported-claim-rate threshold, that score needs to be genuinely calibrated, not merely monotonic, measure expected calibration error and build reliability diagrams comparing the verifier's support score (and any confidence the generator itself expresses) against actual ground-truth claim correctness on a held-out labeled set, and apply temperature scaling or isotonic regression when miscalibrated; recalibrate periodically as the retrieval corpus, generator, or verifier model changes, since a regeneration or refusal threshold tuned against a stale calibration curve silently drifts out of alignment with the model's actual, current reliability.
• Reward design for the RLHF-style component: combine several signals rather than a single blended score, a positive reward for claims that align with retrieved evidence, a heavy penalty specifically for unsupported claims (weighted large relative to other terms, since factuality failures are the most costly failure mode this design targets), a positive signal for citing high-trust sources and a penalty for citing low-trust ones, and a small penalty for unnecessary verbosity or speculation; train the reward model on a mixture of real preference data and synthetic perturbations (deliberately corrupted or unsupported variants) so it has seen concrete examples of the failure mode it needs to penalize.
• Training procedure: apply a policy-optimization method (PPO-style) using the verifier's signal as part of the reward, with a KL penalty against the base model to prevent the fine-tuning from drifting into degenerate behavior purely to satisfy the grounding reward (for example refusing everything to trivially avoid unsupported claims).
• Evaluation specifically for hallucination reduction: measure an unsupported-claim rate on datasets with known ground truth, run a retrieval-ablation test (comparing behavior with correct versus deliberately corrupted or "poisoned" retrieved passages, which directly tests whether the model's grounding behavior is real rather than just correlated with typically-correct retrieval), measure citation accuracy (what fraction of citations actually contain the claimed supporting text), and run blind human evaluation specifically probing for overconfidence, not just general helpfulness.

Worked example: The retrieval-ablation test is the sharpest evidence of whether grounding-aware fine-tuning actually worked: feeding the fine-tuned model a deliberately WRONG or irrelevant retrieved passage and checking whether it still asserts claims as if they were supported (a failure) versus correctly abstaining or flagging low confidence (the intended behavior) directly measures whether the model learned to actually depend on the retrieved evidence, rather than merely learning to produce fluent, citation-shaped text that happens to usually be correct because retrieval usually IS correct in normal operation. This same evaluation set is exactly what the calibration check above needs, the ablation's known ground truth (supported versus unsupported) is what lets you plot the verifier's stated support score against actual correctness and compute a genuine calibration error, not just an accuracy number.

Trade-offs and pitfalls: A reward that penalizes unsupported claims too heavily without a countervailing helpfulness signal can push the model toward excessive refusal or hedging, exactly the over-refusal failure mode discussed elsewhere in this pipeline, so the reward design needs both the factuality penalty and a genuine helpfulness or conservatism-in-moderation term to avoid trading one failure mode for another. A second pitfall is validating grounding only under normal retrieval conditions, without the retrieval-ablation test, a model can appear well-grounded purely because retrieval is usually accurate in testing, while never having been genuinely stress-tested against a case where retrieval fails. A third pitfall is deploying the unsupported-claim threshold without ever checking whether the underlying support score is calibrated, an uncalibrated score can still rank claims in roughly the right order (fooling a quick spot-check) while being systematically over- or under-confident in a way that makes the deployed threshold trigger far too often or far too rarely in practice.

Direct answer

NVIDIA Tensor Cores are specialized hardware units, distinct from a GPU's general-purpose CUDA cores, built specifically to accelerate matrix multiply-accumulate operations (the dominant computation in deep learning) at much higher throughput than general-purpose cores achieve, generally requiring reduced-precision numeric formats (fp16, bf16, tf32, or int8) to reach their full speed advantage.

Structured elaboration

• What Tensor Cores do: they perform small matrix multiply-and-accumulate operations (e.g. multiplying two small matrix tiles and adding the result to an accumulator) as a single hardware operation, far more efficiently than performing the equivalent sequence of individual multiply-and-add instructions on general-purpose CUDA cores.
• Why they need specific precisions: Tensor Cores' hardware design is built around specific input/accumulation precision combinations (e.g. fp16 or bf16 inputs, accumulating in fp32 for numerical stability); using standard fp32 for both inputs and accumulation generally doesn't engage the fastest Tensor Core paths (some GPU generations support a reduced-precision fp32 variant, tf32, specifically to get partial Tensor Core benefit without fully switching to fp16/bf16), which is a large part of why mixed-precision training (discussed extensively elsewhere in this topic) is so tightly linked to actually realizing a modern GPU's advertised peak performance.
• What changes in model implementation and numerics: using Tensor Cores effectively generally means adopting mixed-precision training (autocast-style automatic casting, or explicit dtype management) so that the matrix-multiplication-heavy operations (linear layers, convolutions, attention) actually run in a Tensor-Core-compatible precision, while numerically sensitive operations may still need to stay in fp32; this is not automatic just from having Tensor-Core-capable hardware, it requires the software (and specifically the numeric-precision choices in the code) to actually engage them.
• Practical magnitude: Tensor Cores can offer several times (often cited around 4-8x, generation- and precision-dependent) the throughput of the same GPU's general-purpose CUDA cores for the specific matrix-multiply-accumulate operations they accelerate, which is why nearly all large-scale deep learning training today deliberately uses mixed precision specifically to capture this hardware advantage, not merely for the memory savings mixed precision also happens to provide.
• Tile shapes: Tensor Cores compute fixed-size small matrix-multiply tiles per instruction (e.g. a 16x16x16 fp16 multiply-accumulate as the classic WMMA fragment shape, with newer generations supporting additional shapes and precisions including sparsity-aware tiles); matrix dimensions that aren't already multiples of the relevant tile size get padded up to the next multiple internally (or by the framework/library), so choosing model dimensions (hidden size, head dimension, vocabulary size) as multiples of 8 (fp16/bf16) or 16 (int8) avoids wasted padding compute and keeps the kernel on the fastest Tensor Core path.
• Preferred data layouts: Tensor Core kernels generally prefer a channels-last-style memory layout (e.g. NHWC for convolutions, or contiguous inner dimensions matching the tile's fastest-varying axis for matmuls) over the classic channels-first NCHW layout, since it lets the hardware read a tile's worth of contiguous, tile-aligned memory in fewer, more efficient transactions; using a suboptimal layout can silently fall back to a slower, non-Tensor-Core or poorly-tiled path even when the precision is otherwise correct.
• Fused matmul+epilogue ops benefit most: the operations that gain the most from Tensor Cores are large matrix multiplies immediately followed by a small "epilogue" op (bias add, activation function, or residual accumulation) fused into the same kernel, since fusing avoids writing the raw matmul output to HBM and reading it back just to apply the epilogue in a second kernel; libraries like cuBLASLt and CUTLASS expose these fused matmul+epilogue kernels directly, and it's this combination (large tiled matmul plus fused elementwise epilogue), more than an unfused matmul alone, that captures the largest share of Tensor Cores' theoretical throughput advantage in practice.

Worked example

A transformer's linear layers (the majority of its FLOPs) run as large matrix multiplications; training the same model in full fp32 (not engaging Tensor Cores' fastest paths on many GPU generations) versus in fp16/bf16 with autocast (engaging Tensor Cores fully) can show a measured multi-fold difference in training throughput on the same hardware, a difference attributable specifically to Tensor Core engagement, distinct from (though it compounds with) the separate memory-bandwidth benefit of moving smaller, lower-precision tensors.

Trade-offs & pitfalls

A common misconception is assuming a GPU with Tensor Cores automatically gets their speed benefit regardless of the precision used in the code; without deliberately using a Tensor-Core-compatible precision (via mixed precision training or explicit dtype choices), a model trained entirely in fp32 on Tensor-Core-capable hardware may leave a substantial fraction of that hardware's actual throughput potential unused.