Spotify Staff-Level Machine Learning Engineer Interview Preparation Guide

Decision framework (stepwise):

1. Clarify requirements & constraints
   • Scale (requests/day), model types (batch, real-time, streaming), latency/availability, compliance (GDPR, HIPAA), global regions, expected growth (3–5 years).
2. Evaluate dimensions (score 1–5) and weight by business priority
   • Long-term cost (TCO): infra + ops + engineering + data egress/transfer
   • Vendor lock-in: migration cost, proprietary APIs
   • Speed to market: deployment time, CI/CD, MLOps maturity
   • Talent availability: SRE/ML infra vs cloud-managed expertise
   • Security & compliance: data residency, auditability, certifications
   • Customizability & performance: custom schedulers, bespoke runtimes, HW acceleration
3. Gather data: pilot with cloud-managed POC and cost model; estimate build effort via internal metrics.
4. Score & decide: choose option with higher weighted score; set guardrails (SLA, exit plan).

Evaluation summary (for a midsize enterprise with global customers):

• Speed to market: cloud-managed wins (rapid POCs, managed endpoints, global regions).
• Talent: cloud-managed lowers need for specialized infra engineers; in-house demands senior platform engineers (scarce/costly).
• TCO: short-to-medium term cloud-managed likely cheaper; long-term at high scale in-house can be cheaper but requires sizable upfront investment and operating overhead.
• Vendor lock-in: in-house wins; mitigate by using portable artifacts (ONNX, Docker, Terraform) and avoiding proprietary services.
• Security/compliance: both can meet requirements; cloud-managed often provides certifications and built-in IAM/monitoring, but in-house gives maximum control for strict regimes.
• Customizability/performance: in-house wins for highly specialized runtimes or custom schedulers.

Recommendation for this midsize, global company:
Adopt a pragmatic hybrid strategy: start with cloud-managed ML (e.g., SageMaker) to accelerate delivery, run global POCs, and implement best practices for portability (containerized models, infra-as-code, data export policies). Parallelly, roadmap an internal ML-platform only where justified (high steady-state scale, unique security needs, or heavy custom infra). Define clear KPIs (cost per inference, time-to-deploy, MTTR, compliance audit time) and a 12–24 month review to decide on partial migration. This balances speed, cost control, and reduces long-term lock-in risk.

Clarify goal & constraints: reduce ~10s cold-start when pods scale up for a 20GB-parameter model on K8s while controlling cost and GPU resource usage. My strategy combines autoscaling policy changes, warm pools, memory-sharing techniques (mmap/snapshots), and cost controls.

1. Autoscaling adjustments

• Use predictive/autoscaling (KEDA or cluster-autoscaler with metrics from request rate, queue length, and GPU utilization). Add horizontal pod autoscaler with custom metric: in-flight requests per ready-model.
• Raise minReplicas above zero for critical windows (business hours) and use slower cooldowns to avoid thrash.
• Use burst capacity: allow short-lived overprovisioning when sudden spikes predicted.

2. Pre-warming / keep-alive pools

• Maintain a small pool of warm replicas (on CPU or cheap GPU) that hold the model loaded and ready. For GPU-heavy models, keep 1–2 GPU-backed warm pods; for lower-cost, keep CPU-only lightweight replicas for quick routing fallback with degraded latency.
• Implement a warm-worker queue: when scale decisions occur, create pods into warm state before routing traffic; use readiness probe only set once model init complete.

3. Model-sharing patterns

• Use multi-tenant model servers (e.g., Triton, TorchServe) to host multiple models/versions in one process to amortize memory.
• Consider model sharding across GPUs for very large models, or use parameter-server style sharing for memory-heavy layers.
• If serving many identical models, use a shared memory region (see mmap below) so multiple containers can map the same file-backed weights without extra RAM copies.

4. Memory snapshotting / mmap

• Prepare a memory-mapped model artifact: serialize weights to a file in a contiguous layout (e.g., PyTorch torch.save(...) to write a checkpoint, then load it with torch.load(f, mmap=True) (PyTorch's boolean mmap flag, added in PyTorch 2.1+ - not NumPy's string-valued mmap_mode argument, which torch.load does not accept)) or use a fused shared object (.pt with mmap). Containers then mmap the same file to avoid duplicate physical memory (kernel COW benefits). For GPU, use NVidia’s CUDA IPC (inter-process communication) or unified memory where possible to reduce copy time.
• Use process checkpointing (CRIU) or container snapshot (where supported) to capture a warmed process and restore fast - useful on node pools that support it.
• Use lazy loading of layers (load-on-first-use) to reduce initial warm cost if acceptable.

5. Routing & fallbacks

• Use a staging tier that routes to warm pools first; if busy, route to cold-starting pods but return preference to warm ones.
• Implement async request queues with informative 202 responses or streaming to mask occasional longer startups.

6. Cost controls

• Use scheduled scale-to-min windows to lower minReplicas at night; combine with predictive scaling to avoid cold starts at known peaks.
• Use spot/preemptible instances for non-critical warm pools with quick failover, and reserve small guaranteed nodes for latency-critical warm pods.
• Autoscale GPU nodes separately from CPU nodes; right-size GPU types (use smaller multi-GPU vs large expensive GPUs where memory fits).
• Monitor idle GPU time and set TTL for warm replicas based on request patterns; use incremental backoff to scale down gradually.

Trade-offs & monitoring

• Warm pools increase cost; tune minReplicas and TTL with traffic patterns and SLOs. Mmap/shared-memory reduces RAM but requires careful artifact preparation and may not eliminate GPU memory duplication.
• Measure: cold-start latency distribution, cost per hour, p95 latency, queue length. Iterate thresholds and pool size.

This combined approach reduces cold starts by keeping a small, efficiently shared warm capacity, using mmap/restore tricks to speed load, and applying predictive autoscaling and cost controls to balance SLOs and spend.

Also covers (folded from merged near-duplicates): b7bcb01d folds a concise 3-strategy list as a quick-reference companion.

Direct answer

Build trust before you need anything, by being reliable on small things, transparent about your constraints and capacity, and by giving the other person visibility into your world so they aren't surprised later. Waiting to invest in the relationship until you need a favor makes the ask feel transactional.

Framework

Lead with reliability on small things. Deliver on small, early commitments, answer a question promptly, show up to their planning session, so your word has a track record before there's a high-stakes ask on either side.

Be transparent about constraints. Proactively share capacity, risk, and known limitations rather than letting the other person find out the hard way, mid-project.

Give visibility into your world. Invite them into a review or share a roadmap or dashboard, so they understand your constraints without needing you to explain from scratch every time.

Make it reciprocal early. Ask what they need and what's on their plate too. Trust runs both directions, not just from you demonstrating value to them.

Worked example

Situation: a new product manager joins and will depend on your team, for example a platform or infrastructure team, for their roadmap.
Action: in the first couple of weeks, gave the PM read access to the team's capacity and roadmap view along with a short walkthrough, rather than waiting for them to ask. Proactively flagged one known constraint, a piece of infrastructure that was close to capacity, before it affected their planning. Followed through quickly and visibly on a small early request, answering a scoping question the same day, to establish reliability before anything high-stakes came up.
Result: by the time the PM had a genuinely high-stakes ask, an accelerated timeline, there was already a working relationship and a shared understanding of constraints. The conversation started from what's actually possible given what you already know, instead of starting from zero.

Trade-offs and pitfalls

• Trust-building gestures can look like busywork if they aren't tied to something concrete. Keep them small and genuinely useful, not performative.
• Over-sharing every constraint upfront can read as excuse-making before there's even a request. Calibrate to what's actually relevant to their planning.
• The senior differentiator is doing this proactively, before there's a need, rather than scrambling to build rapport only once you need something from the other person, which reads as transactional.

Direct answer

The intercept is the expected outcome for an observation at the reference (baseline) level of every categorical variable and zero on all continuous predictors; each dummy coefficient is the average shift relative to that baseline, holding everything else fixed. You avoid the dummy-variable trap by dropping one level per categorical variable, encoding a K-level category with K-1 dummy columns so the design matrix stays full rank.

Structured elaboration

Encoding: for a K-level categorical variable, create K-1 dummy columns and drop one level (the reference).

Intercept: if continuous predictors aren't centered, the intercept is technically the prediction at continuous = 0, which may be outside the realistic data range. Centering continuous predictors at their sample mean makes the intercept read as "typical prediction for the baseline category" instead of a mathematical extrapolation.

Dummy coefficients: each one is interpreted as "holding the continuous predictors fixed, this category's prediction is [coefficient] higher or lower than the baseline category's prediction."

The dummy-variable trap: including all K dummy columns plus an intercept makes the K dummy columns sum to the constant intercept column, a perfect linear dependency. This makes X^T X singular, so OLS either has no unique solution or the solver silently drops a column / produces unstable coefficients depending on the library. Fixes: drop one level (reference coding, the standard choice), or drop the intercept and keep all K dummies (each coefficient then directly estimates that group's mean), or apply regularization if you specifically want all levels represented (ridge tolerates the redundancy but the individual coefficients lose the clean "versus baseline" reading).

Worked example

Predictors: sqft (continuous) and a 3-level region variable (North = baseline, South, West). Suppose the fitted model is (illustrative coefficients, in $000s):

$\widehat{\text{price}} = 50 + 0.20\,\text{sqft} + 20\,\text{South} + 35\,\text{West}$

The intercept, 50, is the predicted price for a North home at sqft = 0 (a mathematical anchor, not a realistic home). For a South home at sqft = 1500: 50 + 0.20(1500) + 20 = 50 + 300 + 20 = 370 ($000). For a West home at the same sqft: 50 + 300 + 35 = 385 ($000). So, holding square footage fixed, a West home is predicted $15,000 more than an equivalent South home (385 - 370) and $35,000 more than an equivalent North home, which is exactly what the West coefficient of 35 says on its own.

Trade-offs & pitfalls

Forgetting to drop a level triggers the trap; whether that fails loudly or silently depends on the tool (pandas get_dummies keeps all levels unless drop_first=True is passed explicitly; scikit-learn's OneHotEncoder needs drop='first'). The choice of reference level is statistically arbitrary but affects readability, pick a meaningful baseline (most common category, or a natural "control" group) for stakeholder-facing reports. Rare categories can have dummy coefficients with very large standard errors since few observations inform them; grouping rare levels into an "other" bucket is a common practical fix.

Direct answer

Run exploratory analysis to propose a concrete, measurable definition rather than accepting the vague request as-is: inspect what data sources plausibly relate to "satisfaction" (support tickets, survey responses, churn, product usage), look at their distributions and how they relate to each other, and sanity-check any candidate definition against cases where you already have strong intuition about whether a customer was satisfied or not.

Building toward a defensible definition

Start by asking what decisions this score is meant to inform, since that shapes what "satisfaction" should even mean operationally. Then explore candidate inputs (survey scores if they exist, support-ticket sentiment or volume, usage trends, churn or renewal behavior) individually: their distributions, how much missing data each has, and whether they move together or tell conflicting stories. Before publishing, sanity-check the proposed score against a handful of accounts where you already have strong outside knowledge of whether the customer is happy or not, since a definition that contradicts obvious cases needs revisiting before it goes on a dashboard.

Worked example

Exploring three candidate signals: an NPS-style survey (Net Promoter Score, only 15% response rate, but where it exists it's directly interpretable), support-ticket volume normalized by account size, and month-over-month usage trend. The survey alone is too sparse to use on its own; ticket volume alone conflates a heavy but happy power-user with a genuinely frustrated one. Combining usage trend with ticket volume (weighted more heavily since it covers 100% of accounts) and validating the resulting score against ten accounts the account management team already has strong opinions about shows reasonable agreement in eight of ten cases; the two disagreements turn out to be accounts with unusually low usage for legitimate seasonal reasons, prompting a seasonal adjustment before the definition ships.

Trade-offs and pitfalls

A metric built this way should ship with its definition and known limitations attached, not just a number, since "customer satisfaction score" invites people to treat it as an objective fact rather than the constructed proxy it actually is.

Overview: defend training pipelines with layered detection (pre-train, during-train, post-train) plus an incident response that isolates suspected poison, quantifies impact, and remediates.

1. Pre-ingest / provenance checks

• Enforce dataset provenance: cryptographic hashes, signed sources, dataset manifests and ingestion logs.
• Automated metadata checks: unexpected author/source, sudden metadata spikes, duplicate samples.
• Simple filters: remove exact duplicates, detect anomalous label distributions or unnatural class imbalance.

2. Influence analysis (post-train auditing)

• Use influence functions (or TracIn: a method that estimates how much removing one training example would have changed a given prediction, tracked by replaying the gradient updates that example contributed during training, without actually retraining) to compute per-example influence on validation loss/predictions. Rank training points by positive influence on target misclassification or high loss-change when removed.
• Flag top-K influential samples for manual review or automated quarantine.
• Quantify impact: estimate change in target metric when removing top N influences.

3. Holdout & canary validation

• Maintain clean holdout/validation sets isolated from the main training pool. Regularly evaluate model on holdout and on targeted "security" tests (adversarial/backdoor testsets).
• Canary examples: inject small number of labeled canaries under controlled conditions to detect selective poisoning (if canaries get misclassified, alarm).

4. During-training anomaly detection

• Monitor per-batch gradients: compute distribution stats (norms, cosine similarities) per-example or per-batch. Sudden outliers in gradients or samples with persistently large gradient contributions may be poisoned.
• Activation-space clustering: track embeddings for each class; detect small dense clusters far from class centroid (potential trigger/backdoor cluster).
• Loss dynamics: track training/validation loss per-class; label-flip attacks often cause small but persistent validation loss increases for specific classes.

5. Backdoor/trigger scanning

• Use trigger synthesis (neural cleanse: for each class, searches for the smallest input pattern, a patch of pixels or a mask, that would flip many different inputs to that one target class if added; a backdoor trigger tends to be much smaller than would normally be needed to change a prediction) to find minimal perturbations that force target label; low L1/L2 size suggests backdoor.
• Test by appending candidate triggers to clean samples and measuring targeted misclassification rate.
• Perform neuron interpretability: identify neurons that highly correlate with triggered behavior.

6. Metrics & thresholds

• Influence score threshold (e.g., top 0.1% influence → investigate)
• Gradient z-score thresholds (e.g., >4σ)
• Backdoor attack success rate threshold (e.g., >1% targeted misclassification)
• Maintain baselines and use statistical tests to avoid false positives.

7. Response plan

• Quarantine: isolate flagged training examples and mark them in dataset manifests; stop current training if high-confidence.
• Quantify: retrain model without quarantined examples on a canary/replicate to measure delta in targeted behavior and overall performance.
• Remediate: if poisoning confirmed, remove poisoned samples, retrain from last clean checkpoint, or fine-tune with robust methods: data sanitization (filtering out training points whose influence score or activation-cluster membership marks them as statistical outliers before retraining), robust loss functions (loss terms that automatically down-weight high-loss, likely-mislabeled examples instead of fitting them fully), label-noise tolerant training, differential privacy, certified defenses.
• Post-incident provenance & controls: tighten ingestion rules, require signed datasets, audit logs, restrict contributor access.
• Communication & forensics: produce incident report with influence analysis, gradient traces, affected models/versions, and mitigation steps.

8. Operationalization & tooling

• Automate checks in CI for data and training (data-quality pipelines, MLOps hooks).
• Canary training runs and shadow models for continuous validation.
• Integrate alerts into monitoring (PagerDuty) with prioritized playbooks.

Trade-offs & limitations

• Influence methods scale poorly for very large datasets: approximate/incremental methods (TracIn, sampling) help.
• False positives are possible; pair automated detection with human review.
• Attacker adaptive strategies may evade single-signal detectors: use multi-signal fusion for higher confidence.

This layered approach detects both targeted backdoors and broader poisoning, quantifies the attack via influence/ablation, and prescribes containment and robust retraining plus stronger provenance to prevent recurrence.

Hypothesis:

• Null (H0): Current feature set does not predict user click propensity better than a baseline (e.g., global click rate or simple logistic model on small feature subset).
• Alternative (H1): A model trained on current features predicts click propensity with higher AUC (or log-loss reduction) than baseline.

Experimental population:

• Random sample of active users over the next 4 weeks, stratified by device type and geography to preserve heterogeneity. Exclude bots and test accounts. Unit of analysis: impression (or user-session) depending on model granularity.

Data collection plan:

• Instrument impressions with full feature payload used by the model (features, timestamps, context), ground-truth label (click/no-click), and metadata for strata. Ensure consistent logging schema and backfill validation for sampling bias.
• Collect at least one holdout dataset (temporal split) and an online randomized holdout where model scores are not shown to downstream systems to avoid feedback.

Measurement plan:

• Primary metric: AUC-ROC for ranking propensity; secondary: calibrated probability Brier score or log-loss and calibration plots (reliability curves), and business metric (CTR lift on top decile).
• Compare model vs baseline on held-out offline test set and separately on randomized online holdout.
• Report uncertainty: 95% confidence intervals via bootstrap (for AUC) and delta method (for log-loss).

Statistical considerations:

• Power analysis: determine sample size to detect a minimal practically significant AUC improvement (e.g., ΔAUC = 0.02) at α=0.05 and power=0.8. For binary outcomes with class imbalance, compute required number of positive events; use pilot CTR to estimate.
• Control for multiple comparisons if testing several feature subsets (Bonferroni or Benjamini-Hochberg).
• Check assumptions: independence (cluster by user if multiple impressions per user), stationarity (test for drift across time windows), and confounding (use stratified analyses).

Stopping rules:

• Pre-register stopping criteria: collect minimum sample size from power calc; only assess results after full sample. Allow interim checks only for data quality (not efficacy). If performing interim efficacy looks, use alpha-spending (O’Brien-Fleming) to control Type I error.
• Early stop for futility if interim conditional power < 20% or for safety if model induces adverse business metric drop > predefined threshold.

Post-experiment checks:

• Validate calibration and fairness across strata, analyze feature importance and stability, inspect errors and edge cases, and run an online A/B test for business impact if offline validation passes.

Direct answer

Look for observable, checkable signals rather than vibes: contradictions between what different interviewers say about the same thing, evasive answers to concrete structural questions, and a mismatch between how the process itself is run and how the company describes itself. A single hedge from one person is noise; the same gap showing up across multiple sources is signal.

The framework

1. Cross-check across interviewers: ask the same structural question (how is priority X actually decided, how is success measured) to two or three different people in the loop; contradictions between their answers are more reliable than any one person's polish.
2. Watch for evasiveness on concrete questions: vague answers to "how is this measured," "who owns this decision," or "what happened the last time priorities conflicted" are more telling than a vague answer to an open-ended culture question.
3. Treat the process itself as a data point: how the company runs the interview loop, respect for your time, quality of feedback, whether commitments made during the process are kept, tends to correlate with how it treats people after you join.
4. The after-the-fact version of this same skill: if you missed these signals and the mismatch shows up post-hire, the team's actual priorities differ from what was pitched, or the day-to-day work drifts from what motivated you to apply, the fix is the same habit applied late: go back to specifics, ask direct structural questions of your manager, and give it a defined, honest window before concluding it's a pattern rather than a rough patch.

Worked example

In one interview I asked how the team decides between a reliability fix and a new feature when both are urgent. The hiring manager described a clear, named process. Two rounds later I asked a peer on the team the same question and got a different, vaguer answer with no reference to that process. That gap, not either answer alone, was the signal worth investigating further, so I asked a third person, informally, the same question. (If you don't have a story where the process itself surfaced a red flag, use the after-the-fact version instead: describe a moment post-hire where the day-to-day didn't match the pitch, and how you handled confirming the pattern before deciding what to do.)

Trade-offs and pitfalls

A single inconsistent answer from one interviewer is not enough on its own to conclude a culture mismatch; people vary in how well they represent the org, and being too quick to read one hedge as a red flag can talk you out of a good role. Look for a pattern across sources, not a single data point.

Direct answer

Nightly latency spikes at constant request volume mean the cause is TIME-based, not LOAD-based, which immediately rules out anything that scales with traffic and points toward scheduled work (background jobs, cache expiry/warming cycles, scheduled maintenance) or periodic garbage-collection/compaction cycles that happen to cluster around the same time each night, independent of how many requests are actually arriving.

Structured elaboration

1. Confirm the pattern is genuinely time-correlated, not coincidentally volume-correlated. Check request-volume metrics for the exact spike windows across several nights; if volume is truly flat while latency spikes, that rules out ordinary capacity-driven causes and strongly implicates something scheduled or periodic. Alongside the metrics, pull application-level logs for the exact spike windows specifically (not just the surrounding hours), looking for warning-level entries, retry counts, or error codes clustered in that window that a metrics dashboard alone would not surface.
2. Check for scheduled background jobs running on the same host/cluster around the spike time: a cache-warming job, a log-rotation or compaction task, a batch analytics job sharing infrastructure, a scheduled backup; cross-reference the exact spike timestamps against the cron/scheduler's OWN execution logs (start time, end time, and any error or retry entries for each scheduled run), not just a summary job-history view, since a summary view can round timestamps in a way that obscures a precise correlation.
3. Check caching policies specifically for a periodic expiry pattern. If a cache (in-process or shared) has a TTL (time-to-live: how long a cached entry is kept before it expires) that causes many entries to expire around the same time each night (a fixed-time cache-refresh schedule, or a TTL set relative to a fixed daily reset rather than per-entry insertion time), the resulting simultaneous cache-miss storm can produce exactly this nightly-latency-spike-at-flat-volume signature.
4. Check for batch windows in any request-batching logic; if requests are batched with a time-based flush (rather than purely size-based), and something periodic changes the batch-fill rate at that hour (fewer concurrent requests overnight meaning batches take longer to fill and therefore wait longer before flushing, even at flat OVERALL system load), that's a plausible, easy-to-overlook mechanism, especially ironic since it's a case where LOWER traffic at night could paradoxically increase per-request latency via the batching wait.
5. Check garbage-collection patterns for a scheduled or load-triggered full GC that happens to land at a consistent time nightly (some GC strategies trigger based on heap-growth patterns that, combined with a consistent daily traffic/memory-allocation shape, can cluster around the same hour even without being explicitly scheduled); the runtime's own GC logs (pause duration and frequency per collection cycle, not just an aggregate GC-time metric) show directly whether pause TIMING specifically clusters at the same nightly window, rather than requiring that to be inferred indirectly from latency alone.
6. Check for multi-tenant interference if the infrastructure is shared: another tenant's own scheduled nightly job competing for the same underlying resources (CPU, disk IO, network) can produce a latency spike for YOUR service with zero change to your own traffic or code, visible only by checking host-level (not just your-service-level) resource metrics during the spike window.
7. Check for scheduled maintenance: automated OS patching, container/host restarts, or infrastructure-level maintenance windows that a platform team runs on a schedule independent of any application team's knowledge.

Worked example

Cross-referencing spike timestamps against the platform's job scheduler shows a company-wide log-aggregation/rotation job running nightly on shared infrastructure, and host-level (not service-level) CPU and disk-IO metrics show a clear spike during exactly the same window, confirming multi-tenant interference from that job rather than anything in the service's own code or configuration. Immediate mitigation: request the shared job be rescheduled to a lower-traffic window for this service, or move this service to isolated infrastructure if the interference is severe enough to warrant it; a code-level fix wouldn't have helped at all here, since nothing about the service's own logic was the actual cause.

Trade-offs and pitfalls

The temptation to profile the SERVICE's own code first (since that's what's directly controllable) can waste significant time when the actual cause is external, shared-infrastructure interference invisible to service-level metrics alone; checking host-level, not just service-level, resource metrics during the spike window is often the fastest way to distinguish "something in my own code" from "something external happening to my host," and should be checked early given the flat-volume/time-correlated signature already points away from the service's own request-handling logic.

Direct answer

Treat the first ninety days as three distinct feedback phases rather than one long ramp: the first thirty days is mostly listening and asking calibrated questions of a wide set of people, the next thirty is testing that understanding through visible small contributions and targeted follow-up questions, and the last thirty is asking for a harder, more evaluative read now that there's real work to point to. At each phase, write down what you heard and what you changed because of it, so the loop is visible, not just felt.

Structured elaboration

• Days one to thirty: who and what. Talk to your manager (what does success look like at thirty, sixty, and ninety days, what's the biggest risk if this goes wrong), two or three peers doing similar work (what do you wish someone had told you when you started, what's the thing that trips people up here), and, if relevant, a couple of people upstream or downstream of your work (what do you actually need from this role that isn't written down anywhere). Questions here are deliberately open and low-stakes: "what should I be paying attention to that I don't know to ask about yet?"
• Days thirty-one to sixty: who and what. After producing something real, a first change, a first analysis, a first design, a first proposal, ask more targeted questions of whoever reviewed it: "was this the right level of detail," "did I miss context I should have had," "is there a pattern in what you're correcting that I should watch for?" This is also when to bring a specific check-in back to your manager: "here's what I've done, here's what I'm still unsure about."
• Days sixty-one to ninety: who and what. Ask for a more evaluative read, since there's now enough of a track record for the answer to be specific rather than generic: "if you were coaching me for the next quarter, what's the one thing I should focus on?" Ask this of your manager and at least one peer whose judgment you trust, since a manager's view and a peer's view often surface different things.
• Tracking that you're acting on it. Keep a simple running log, one line per piece of feedback: what was said, who said it, and what you changed or decided not to change and why. Bring this log into one-on-one check-ins with your manager (a regular short meeting between you and your manager), especially around the day-thirty and day-sixty marks, so your manager sees the pattern, not just individual points, and so you have to be honest with yourself about whether you actually followed through.

Worked example

The questions above stay constant, but what "producing something real" means in days thirty-one to sixty varies by the kind of work. For someone in a data-facing role, the first real deliverable is often getting the data model and who-needs-what-from-it right, so the targeted day-forty-five question becomes, "does my understanding of how this data actually gets used match reality," checked against a specific report or query. For someone in a design role, early feedback is often more about building credibility through a couple of small, well-executed pieces of work before asking for a harder critique, since a design opinion carries more weight once colleagues have seen competent delivery. For someone building technical proposals, such as an architecture document, a natural day-forty-five checkpoint is asking for feedback specifically on the early proposal itself and on communication style, since how something is proposed matters as much as what's proposed when you're new to a team. In every case, the log entry looks the same: what was said, what changed.

Trade-offs and pitfalls

Asking only your manager and skipping peers misses the day-to-day texture a manager doesn't see. Asking the same broad question the whole ninety days, instead of narrowing it as you get more context and more real work to point to, wastes the growing specificity available to you. Collecting feedback but never visibly acting on it reads as performative rather than genuinely coachable. And waiting until day ninety to ask for anything evaluative wastes the early window when small corrections are cheapest to make.