Applied Scientist (Staff Level) Interview Preparation Guide for Lyft

Direct answer

"Increase engagement" isn't an ML problem yet; it's a direction. My first move is to make it measurable and falsifiable: pick a concrete, observable proxy for engagement, define what "increase" means numerically and over what horizon, and only then ask what data and model would move that number.

Structured elaboration

1. Stakeholders to involve up front: product (what does "engagement" mean to them, retention or session frequency or depth of use), analytics (what's already being tracked and what's the current baseline), and legal or trust and safety if the mechanism involves anything sensitive (notifications, personalization of content).
2. Turning the goal into a metric. Engagement is usually a bundle of several things pulling in different directions (session count, session length, feature breadth), so pick ONE primary metric the model will be judged on, and one or two guardrails (unsubscribe rate, complaint rate, time spent outside the intended feature) that must not regress even if the primary metric improves.
3. Data and instrumentation. What behavioral data already exists, and is it enough to define a training label (e.g., "sessions per week over the next 30 days")? If the tracking needed to compute the metric doesn't exist yet, that instrumentation gap has to be closed before the ML work starts, not after.
4. The actual ML formulation. Only at this point do you pick a target: is this a ranking problem (surface better content), a propensity problem (predict who is at risk of disengaging and intervene), or something else? The formulation follows from the metric, not the other way around.

Worked example

Concretely: "increase engagement" for a content feed becomes "increase 7-day return-session rate for users in their first month, without decreasing average session length." That's now falsifiable. A parallel example for a subscription product: "reduce monthly churn by 10% over the next quarter" is the same kind of translation problem, but the natural target becomes a churn-propensity model with a defined observation window, and the guardrail becomes something like support-ticket volume (a retention tactic that just annoys people isn't a win).

Trade-offs and pitfalls

The biggest failure mode is picking a metric that's easy to move but doesn't represent the actual goal (a model can trivially increase "sessions" by sending more notifications, at the cost of long-term satisfaction). The fix is to always pair the primary metric with at least one guardrail that would catch that kind of gaming. The second failure mode is skipping the instrumentation check: teams frequently discover, mid-project, that the metric they picked can't actually be computed from existing logs.

Direct answer
A residual block lets its convolutional layers learn a correction $F(x)$ on top of the input, rather than the full desired mapping from scratch: the block output is $y = F(x) + x$, and this identity path is what makes very deep networks trainable at all.

Structured elaboration
For a block input $x$ (projected via a learned $1\times1$ convolution when channel/spatial dimensions change, giving $y=F(x)+W_s x$), backpropagating through the addition gives $\partial y/\partial x = I + \partial F/\partial x$: the identity term $I$ provides a direct, unattenuated path for the gradient back to earlier layers, regardless of how small $\partial F/\partial x$ might be. This is the core reason residuals mitigate vanishing gradients in very deep stacks: without the identity shortcut, the gradient would have to pass through every block's full Jacobian in a repeated product, which (as with any deep stack) can shrink toward zero; with the shortcut, at minimum the identity term keeps the gradient flowing, even if $F$'s own contribution is small.

Post-activation (the original ResNet design) applies the block's normalization and activation, THEN adds the shortcut, then applies one more activation: $y = \text{ReLU}(x + F(x))$ where $F$ itself contains its own internal BatchNorm/ReLU. Pre-activation instead moves normalization and activation to BEFORE each weight layer inside $F$, so the shortcut path is added to a value with NO activation immediately following the addition; this keeps the shortcut a cleaner, more purely linear path across many stacked blocks, which empirically improves optimization and generalization at extreme depths (over 100 layers), because the post-activation version's final ReLU-after-addition slightly distorts the identity path every block.

When residuals might not help, and how they change optimization dynamics: in shallow networks the vanishing-gradient problem residuals solve barely exists in the first place, so the added shortcut mostly just adds a bit of redundant capacity without a clear optimization benefit; and when introducing residuals into a NEW architecture, a common practical guideline is to initialize the last layer inside each residual branch close to zero (so the block starts close to the identity function), which avoids the compounding instability that many simultaneously-active, large residual branches can otherwise cause early in training.

Worked example
For a 50-block residual network where each block's own Jacobian $\partial F/\partial x$ has entries of typical magnitude 0.3 (deliberately small, to illustrate the point), the identity-augmented Jacobian per block is approximately $I + 0.3$, so a product of 50 such terms stays on the order of $1.3^{50}$ rather than collapsing; contrast this with a plain (non-residual) stack of 50 layers each with Jacobian magnitude 0.3, whose product is $0.3^{50}\approx 10^{-26}$, an effectively total vanishing of the gradient by the time it reaches the earliest layer.

Trade-offs & pitfalls
A common confusion is treating the shortcut as making the block's OWN weights unnecessary; the shortcut only guarantees a gradient path exists, not that $F$ learns anything useful, so residual networks still need the same careful initialization, normalization, and learning-rate tuning as any deep network, just with the vanishing-gradient failure mode specifically addressed.

Direct answer: When the stated goal is intangible, choose short-term proxy metrics that plausibly sit on the causal path to the real goal, and pair them with a longer-term outcome metric that is at least directionally related to the goal, even if imperfect, then validate the proxy periodically with a direct (often qualitative or survey-based) check rather than trusting the proxy blindly forever.

Structured elaboration

• Short-term proxies: pick behaviors that a reasonable person would expect to correlate with the intangible goal, chosen for being measurable quickly. For "more meaningful social interactions," candidates include reply depth (a back-and-forth exchange rather than a single message), time between reciprocal messages (faster mutual engagement), or the diversity of people a user regularly interacts with (breadth of relationships, not just volume).
• Long-term outcome proxies: pick something that would plausibly move if the intangible goal is genuinely being achieved over a longer horizon, such as retention specifically among users who show high short-term-proxy activity versus those who do not, or a periodic survey question closely worded to the actual goal ("I feel more connected to people I care about because of this app").
• Validating the proxy: periodically run the direct check (a survey, a qualitative study) against the short-term proxy to confirm the proxy still tracks the real goal, since proxies can drift or be gamed even when chosen thoughtfully; if the proxy and the direct check diverge, trust the direct check and revise the proxy.
• Being honest about the limits: state clearly, when reporting on this feature, that the proxy metrics are proxies, not the goal itself, so stakeholders do not over-interpret a proxy movement as proof the intangible goal was achieved.

Worked example: For a feature intended to help people have "more meaningful social interactions," the team picks reciprocal-reply rate within 24 hours as the short-term proxy and 90-day retention among high-reciprocal-reply users as the long-term outcome proxy. Six months in, the team runs a survey asking users directly whether the app helped them feel more connected to people they care about, and finds the survey response correlates well with the reciprocal-reply proxy (users high on the proxy report feeling more connected at meaningfully higher rates than users low on it), which validates continuing to use the proxy as the day-to-day operating metric while reserving the survey as a periodic sanity check rather than something to run constantly.

Trade-offs and pitfalls: The main risk is picking a proxy that is easy to measure but only weakly related to the real goal (raw message volume, for instance, correlates poorly with "meaningful," since spam-like or transactional messaging can inflate volume without any of the intended value), and then optimizing hard against that proxy, which can actively make the real goal worse (a feature that maximizes message volume might crowd out fewer, higher-quality exchanges). The other pitfall is never running the direct validation check at all, so a proxy that quietly stopped tracking the real goal (or never did) goes uncaught indefinitely.

Direct answer

Do not treat the metric win and the UX risk as opposing bets. Before building anything, agree with design and product on the primary success metric and on explicit guardrail metrics chosen specifically to catch the kind of harm the primary metric would not see, then validate cheaply with a prototype or a small qualitative test before committing to a live experiment sized to detect both.

Structured elaboration

Agree on what "good" means before anyone builds

The primary metric, say a conversion or engagement number, tells you if the change works on its own terms. Guardrail metrics are chosen specifically because they would catch harm the primary metric is blind to, such as task completion, return usage a week later, or support-ticket volume. Naming guardrails upfront, with agreed thresholds, prevents "we'll know it if we see it" arguments after the fact.

Validate cheaply before going live

A clickable prototype or a small moderated usability session can surface confusion or trust issues that the metric alone cannot catch, at a fraction of the cost of a live experiment. This is not a substitute for the experiment, it is a cheap filter that catches the worst ideas before they reach real users.

Run a bounded experiment, not a full rollout

Start with a small slice of traffic, watch both the primary metric and the guardrails, and decide the stopping rule, meaning what result on which metric ends the test, before the test starts, not after you see the numbers.

Decide and communicate together

If the primary metric improves but a guardrail moves the wrong way, that is a real finding, not a technicality to explain away. Whether to ship, iterate, or drop the idea is a joint call between design, product, and whoever owns the guardrail metric, made against the thresholds agreed upfront.

Worked example

Design proposes reordering a list of recommended items to increase click-through rate. The concern is that users may have learned to expect a stable, predictable order, and reordering it could hurt their ability to quickly find what they are looking for on repeat visits, something click-through rate would not show because a user can click more and still be more frustrated.

Before building, the group agrees the primary metric is click-through rate, and the guardrails are task completion rate (did the user's search end in the outcome they were after) and a return-usage check at one week out. A moderated usability test with a handful of participants on a clickable prototype surfaces that new users find the reordered list fine, but a couple of returning participants mention it "looks different" and take longer to find what they normally click first. That is a signal, not a stop sign: the team ships the change to a small slice of traffic, watches both metrics for an agreed window, and only expands the rollout if task completion holds steady alongside the click-through gain.

Trade-offs and pitfalls

Over-instrumenting every change with a full guardrail suite slows teams down and trains people to skip the process for anything that feels small. Guardrails should be chosen deliberately for the specific risk in question, not applied as a blanket checklist.

The sharpest failure mode is agreeing on guardrails in principle but not on thresholds, so when a guardrail moves slightly, the debate about whether it is a real regression happens after the data is already in and someone has already committed emotionally to shipping. Fixing the threshold before the test removes that fight.

Direct answer

When labels are delayed by weeks (or months), monitor with proxy signals that don't need the label at all, use partial/weak labels as an early read, and design your backtest windows to account for the delay explicitly rather than treating a too-recent window as if it were fully labeled.

Structured elaboration

1. Proxy signals: input/feature drift (needs zero labels), model confidence and score-distribution shifts, and business proxies that arrive faster than the target label itself (a dispute-rate or manual-review-flag rate that correlates with eventual fraud, arriving days before a confirmed-fraud label does).
2. Weak/partial labels: if a fast, noisier signal correlates with the eventual true label (an automated system's provisional flag before human review confirms it), use it as an early, lower-confidence read rather than waiting for full confirmation: explicitly labeled as provisional in any dashboard so nobody mistakes it for the confirmed metric.
3. Active labeling: for genuinely sparse or expensive labels, prioritize which examples get labeled first using uncertainty or diversity sampling, so the label budget goes toward the examples most likely to reveal a real problem rather than being spent uniformly.
4. Backtest window design: any backtest window has to EXCLUDE the most recent period where labels haven't had time to fully mature: computing a "recent accuracy" metric over a window still receiving late-arriving labels systematically understates true performance (the immature window looks worse than it eventually will), which is exactly the kind of artifact that causes an unnecessary panic or an unnecessary retrain if you don't account for it.
5. Statistical care to avoid overreacting: treat the maturing window's metric as a moving target rather than final: either exclude it entirely from decision-making until it's matured, or explicitly model the expected maturation curve (how much does the metric typically shift as more labels arrive) and adjust the immature reading accordingly rather than reacting to a raw, incomplete number.

Worked example

Ground-truth label collection itself matters here: for a system relying on implicit feedback (a user's later action confirming or denying a prediction) rather than explicit labels, human-in-the-loop review can supply a faster, if noisier, confirmed label for a SAMPLE of predictions: trading full population coverage for speed, which is often the right trade when the alternative is waiting the full natural delay for every single example. For a metric with a known ~90-day full-maturation curve (as in a churn-prediction context), a surrogate early-warning metric (7-day retention as an early proxy for 90-day churn) lets you catch a real regression within days rather than three months, at the cost of the surrogate being an imperfect stand-in that occasionally disagrees with the eventual full-delay signal.

Trade-offs & pitfalls

Every proxy in this answer trades speed for fidelity, and the discipline that keeps this from becoming a house of proxies nobody trusts is periodically validating each proxy AGAINST the eventual true label once it does arrive: if a proxy's historical correlation with the true label turns out weak or drifting, it needs to be recalibrated or retired, not left running indefinitely on the assumption it still means what it used to.

Start by clarifying objectives: preserve the teacher’s cross-modal representations and task behaviour in a smaller/multimodal student. You want the student to (a) match teacher outputs (task-level), (b) align intermediate cross-modal embeddings (representation-level), and (c) keep alignment between modalities (cross-modal signals).

Recommended loss terms and signals:

• Task (logit) distillation: softened cross-entropy between teacher logits zT and student logits zS with temperature T. L_task = KL(softmax(zT/T) || softmax(zS/T)).
• Feature alignment: L_feat = ||ProjT(fT) − ProjS(fS)||^2 applied at key layers. Use small projection heads to map different dims.
• Cross-modal contrastive loss: InfoNCE between teacher’s vision embeddings and student’s language (or fused) embeddings to preserve cross-modal retrieval/alignment capabilities.
• Attention/activation matching: L_att = MSE between normalized attention maps or per-head activations to convey where the teacher “looks.”
• Token-level / patch-level distillation: for vision→multimodal, match per-patch/class-token embeddings so fine-grained spatial cues transfer.
• Auxiliary reconstruction: decode student embeddings back to teacher modality (e.g., reconstruct teacher image features) to retain modality-specific info.
• Ground-truth supervision: combine with original loss L_gt so student doesn’t overfit teacher mistakes. Total loss = αL_gt + βL_task + γL_feat + δL_contrastive + εL_att + ζL_recon.

Practical notes:

• Use projection heads and temperature tuning; weight terms via validation. If teacher and student modalities differ, train cross-modal adapters that translate embeddings before matching. For transfer-across-tasks, include task-specific heads and multi-task KD: distill logits and representations for each task, or use task-agnostic contrastive signals to keep general features.
• Curriculum: start with representation/contrastive losses, add task logits later to stabilize.
• Watch for negative transfer: monitor teacher errors, allow student to rely on ground truth when teacher is wrong (e.g., label smoothing or confidence-thresholded distillation).

This combination preserves both task performance and cross-modal alignment during compression.

Direct answer

Run for a whole number of full weekly cycles, decide before looking at any data how a holiday inside that window will be handled, and hold traffic allocation balanced by day-of-week (and by region and time zone if the test spans them) rather than trusting that a single aggregate 50/50 split will average out. When a weekday-positive, weekend-negative pattern shows up later, treat it as a hypothesis to falsify with three specific checks, real heterogeneity, a traffic-composition artifact, or noise, rather than reading the raw split at face value.

Structured elaboration

Duration and analysis window

Run for at least two, ideally three or more, full 7-day cycles. A partial week biases the pooled result toward whichever days happen to be over-represented, and a single week does not let you separate a real weekday/weekend pattern from that week's idiosyncrasies. If a holiday falls inside the planned window, decide up front, before seeing any results, between two options: exclude the holiday period from the primary analysis window and report a "typical week" estimate, or explicitly include it and report a distinct holiday-period estimate. Choosing between those two after looking at which one produces a better-looking result is a form of after-the-fact window selection and should be avoided; pre-register the choice in the analysis plan.

Traffic allocation and balance across time and geography

Stratify random assignment by day-of-week, and by region or time zone if the rollout spans them, so the same proportion of each arm is exposed every day and in every zone rather than relying on an aggregate split that could hide a skew. For a multi-region or multi-time-zone test, anchor "day" and "week" boundaries to each user's local time rather than a single server or UTC clock; otherwise one region's weekend gets miscounted against another region's weekday, and verify the treatment-to-control ratio stays constant across regions and hour-of-day buckets individually, not just in the combined total. Aggregate balance can look fine while a specific region or time window is quietly imbalanced, and that imbalance is exactly what later gets mistaken for a day-of-week effect.

Modeling the temporal structure instead of ignoring it

Rather than computing one pooled treatment effect and hoping seasonality washes out, fit day-of-week (and holiday, and region, if relevant) as explicit terms: outcome ~ treatment + day_of_week + treatment:day_of_week + region. This is standard regression-formula shorthand: ~ means "model the left-hand outcome using the terms on the right," so this line reads as "predict the outcome from the treatment, the day type, and the region," and treatment:day_of_week is an interaction term, a piece that lets the treatment's effect itself differ by day type rather than assuming it is the same on weekdays and weekends. The interaction term is what actually tells you whether the treatment effect differs by day type, instead of a single pooled number that could be hiding it.

Investigating a weekday-positive, weekend-negative split

Three checks, run in this order:

1. Is it real? Fit the treatment-by-day-type interaction term from the model above and check whether it is distinguishable from a null effect. This is one specific comparison, not a license to slice every available dimension until something looks significant; keep the interaction pre-specified as part of the analysis plan for exactly this reason.
2. Is it a traffic-composition artifact? Check whether the user mix itself differs by day type: a different device split, acquisition channel, or new-versus-returning ratio on weekends than weekdays. Re-run the interaction model with that covariate added and interacted; if the day-type interaction shrinks toward zero once the segment mix is controlled for, the apparent weekday/weekend split was really a segment-level pattern wearing a calendar label. Also check whether the rollout itself was staggered mid-week (a ramp that reached full exposure partway through the window) or whether an assignment-pipeline issue caused the treatment:control ratio to drift on certain days; both produce a day-type-looking artifact that has nothing to do with actual weekday or weekend behavior.
3. Is it noise? Compare the confidence interval on each day-type's estimate rather than the point estimates alone. Weekend traffic is frequently a fraction of weekday traffic, so a "negative" weekend estimate often carries a wide interval that comfortably contains the weekday estimate.

Worked example

Suppose the weekday arm has 8,000 users per group with control conversion 10.0% and treatment conversion 10.6% (a +0.6 percentage point delta), and the weekend arm has 2,000 users per group (lower weekend traffic) with control conversion 10.0% and treatment conversion 9.4% (a -0.6 percentage point delta). This is exactly the pattern in the question: positive on weekdays, negative on weekends.

Standard error of each delta, using $SE = \sqrt{\frac{p_c(1-p_c)}{n} + \frac{p_t(1-p_t)}{n}}$:

Weekday: $SE_{wd} = \sqrt{\frac{0.10\times0.90}{8000} + \frac{0.106\times0.894}{8000}} = 0.00481$, so the weekday delta's 95% interval is roughly $-0.34\text{pp}$ to $+1.54\text{pp}$, which already crosses zero.

Weekend: $SE_{we} = \sqrt{\frac{0.10\times0.90}{2000} + \frac{0.094\times0.906}{2000}} = 0.00936$, so the weekend delta's 95% interval is roughly $-2.43\text{pp}$ to $+1.23\text{pp}$, also crossing zero.

Testing whether the two deltas actually differ from each other: $z = \frac{0.006-(-0.006)}{\sqrt{0.00481^2+0.00936^2}} = \frac{0.012}{0.01052} \approx 1.14$, well under the 1.96 threshold for a two-sided 5% test. Both individual intervals already contain zero, and the two deltas are not statistically distinguishable from each other. With these particular sample sizes, the weekday-positive-weekend-negative pattern is fully consistent with noise, before ever needing to invoke a real behavioral difference or an artifact.

Trade-offs & pitfalls

• Trusting the point estimate over the interval. A sign flip between two point estimates feels meaningful; whether it survives a formal comparison of the two deltas, as above, is what actually determines whether there is anything to explain.
• Deciding the holiday treatment after seeing results. Choosing whether to include or exclude a holiday period based on which choice produces the preferred outcome is a subtle form of p-hacking through window selection, even when no single test is repeated.
• Assuming aggregate balance implies balance everywhere. A day-of-week or region-level imbalance can hide inside an aggregate 50/50 split and later masquerade as a real seasonal effect.
• Over-correcting into paralysis. Not every day-type split needs a full forensic investigation; reserve the three-check process for patterns that would actually change a rollout decision, and size the investigation to the stakes.

Direct answer. A systematic root-cause plan for a production model's accuracy drop needs to check, in a deliberate order, whether the world the model sees at serving time still matches the world it was trained on: data drift and label shift first (since those are both common and relatively cheap to check), then feature-importance and upstream schema changes, then serving-versus-training skew, before assuming the model itself has somehow degraded.

Structured elaboration. Data drift check: compare the distribution of each input feature in recent production traffic against the training-time (or a recent stable baseline) distribution, using a distribution-comparison test appropriate to the feature type. Label shift check, where ground truth is available with some delay: has the base rate or distribution of the TARGET itself changed, independent of any feature drift, since a model calibrated for one label distribution can look worse purely because reality's label mix shifted. Upstream schema or transformation changes: did a feature-computation pipeline change (a unit conversion, a default-value change, a join that started producing nulls) in a way that alters what the model receives without anyone touching the model itself. Serving-versus-training skew: does the exact same input, computed through the serving pipeline, produce the same feature values as it would through the training pipeline, since a subtle mismatch between the two (different libraries, different rounding, different handling of missing values) is a surprisingly common, hard-to-spot cause. The decision criteria: retrain if the drift is broad, gradual, and reflects a genuine, likely-persistent shift in the population; roll back if the timing cleanly matches a specific deploy and reverting it restores prior behavior; collect more data first if the signal is still ambiguous and the cost of a wrong retrain or rollback is high.

Worked example (drift check executed in Python). Comparing a production feature (avg_order_value) between a recent sample and its training-time baseline using a Kolmogorov-Smirnov test on the numeric distribution and a chi-square test on a categorical region feature: the numeric feature shows a clear, statistically significant shift (Kolmogorov-Smirnov, KS, statistic 0.31, p near zero, consistent with the production mean rising from roughly 50 to 58), while the categorical feature shows no significant shift (chi-square p=0.07, mix essentially unchanged). That combination pinpoints the likely driver precisely: something changed the DISTRIBUTION of order values reaching the model (a pricing change, a shift in which customer segment is transacting), not the categorical mix of where those customers are from, which narrows the retrain-versus-investigate decision considerably.

Trade-offs and pitfalls. Running many per-feature drift tests at once raises a multiple-comparisons concern (some features will look 'significant' by chance alone); apply a correction (like Bonferroni) or treat a single flagged feature among dozens with appropriate skepticism, and prioritize investigating features with both a statistically significant AND a practically large shift, not just a technically significant one.

Direct answer

Triage by urgency and impact first, protect your own deliverables with an explicit, communicated time-box, and convert repeat-pattern questions into reusable artifacts so future requests don't all cost you 1:1 time. Prioritization alone doesn't scale past a certain number of mentees; reusable resources are what let personalized-feeling mentoring keep up as the queue grows.

Triage and scaling approach

Triage each request on three axes. Is it blocking (them or someone downstream) versus a growth request with slack. How long would it actually take to unblock: a quick answer versus a real session. Is this a shape of question you've answered before, which is a signal to build something reusable rather than repeat yourself.

Route, don't just prioritize. Not everything needs to be you specifically. A growth-oriented question might be better answered by a peer with more direct expertise, freeing your time for things only you can unblock.

Time-box and communicate the SLA out loud. "I can give you twenty minutes now on the blocking piece; let's put the design question on tomorrow's slot" sets expectations honestly instead of leaving people guessing whether they've been deprioritized.

Build reusable async artifacts for repeat patterns. When you notice you've answered a variant of the same question more than once, that's the signal to invest in a recorded walkthrough, a short playbook, or an FAQ instead of repeating the synchronous session a third and fourth time. This is a genuinely different lever from prioritization: it lets you scale personalized-feeling help without your 1:1 time growing linearly with the number of people asking.

Maintain the artifacts deliberately. A playbook or recording that goes stale is worse than not having one, because people trust it and get misled. Whoever owns it, you or a rotating owner, needs a cadence to revisit and refresh it, not a one-time write-and-forget.

Worked example

You're juggling your own deliverable alongside three mentees asking for time at once: one is genuinely blocked, one has a growth-oriented design question with no real time pressure, and one is asking a version of a question you've now answered several times before. You give the blocked person a focused twenty minutes to unblock them. You schedule the design question for a defined slot the next day rather than squeezing it in now. And instead of walking the third person through it live again, you point them to an existing recorded walkthrough, or if one doesn't exist yet, you record a short one this time specifically because you can already tell it'll come up again.

Trade-offs and pitfalls

Treating every request as equally urgent burns you out and, worse, under-serves the person with the actually urgent need, because everyone gets a diluted amount of attention instead of the right amount going to the right place.

Over-investing in artifacts nobody maintains creates a different failure: a stale playbook actively misleads people and erodes trust faster than simply not having documentation and telling people to ask.

Prioritizing strictly by who's loudest or most urgent can systematically starve quieter mentees who don't escalate assertively. It's worth periodically checking who you haven't heard from, not just responding to who's asking.

If you find yourself using "I'll make you a doc" as a polite way to avoid ever giving someone real synchronous time, that's usually a sign the mentee queue has outgrown what one person can reasonably carry, and it's a resourcing conversation to raise with your own manager, not something to keep absorbing indefinitely.

When you have per-class F1 scores and need to report one number, the two common ways to combine them are macro F1 and weighted F1 (a third option, micro F1, works differently: it pools all the TP/FP/FN counts across classes first and then computes one F1, rather than averaging per-class F1s).

• Macro F1: average the per-class F1 scores with every class weighted equally, regardless of how many examples that class has. Formula: F1_macro = (F1_class1 + F1_class2 + ... + F1_classN) / N.
• Weighted F1: average the per-class F1 scores, but weight each class by its support (how many true examples of that class exist). Formula: F1_weighted = sum over classes of (support_class / total_examples) * F1_class. Common and rare classes contribute in proportion to how often they occur.

Worked example: a 5-class classifier over 1,000 patients, where condition E is rare (only 20 patients) but dangerous to miss.

Macro F1 = (0.95 + 0.92 + 0.90 + 0.85 + 0.40) / 5 = 4.02 / 5 = 0.804

Weighted F1 = (4000.95 + 3000.92 + 2000.90 + 800.85 + 20*0.40) / 1,000
= (380 + 276 + 180 + 68 + 8) / 1,000 = 912 / 1,000 = 0.912

If I only reported weighted F1 (0.912), it looks like a strong, reliable model. Macro F1 (0.804) exposes that one class, the rare and dangerous condition E, is performing badly (0.40 F1), because macro treats all 5 classes as equally important instead of letting the 400-patient class A drown it out. Since condition E is rare but especially costly to miss, weighted F1 would hide exactly the failure that matters most here.

So for this scenario I'd report macro F1 as the headline metric (or at minimum report both macro and the per-class F1 for the high-stakes rare condition), because picking weighted F1 alone would make a model with a dangerous blind spot look excellent.