Lyft Product Manager (Associate PM) Interview Preparation Guide - Entry Level

Direct answer

An empathy map is a lightweight visual tool teams use to synthesize what a user says, thinks, does, and feels, in order to build shared understanding and surface design or product opportunities together, rather than relying on any one person's interpretation of the research.

Pre-work (sent 48 hours before)

Share a short persona or a recent customer vignette along with 2 to 3 real user quotes or metrics. Ask each participant to review it and bring one customer anecdote or data point of their own. Prepare a shared whiteboard with a 2x2 empathy-map template and a written agenda.

60-minute workshop plan

• 0 to 5 min, kickoff: state the purpose, the expected outputs (top 3 insights, 3 possible experiments), roles, and the timebox.
• 5 to 20 min, individual silent brainstorm: each person adds 3 to 5 sticky notes per quadrant using the shared persona or a real customer quote.
• 20 to 40 min, share and cluster: each participant reads 1 to 2 highlights per quadrant while the facilitator groups similar notes into clusters and a scribe labels each cluster with its supporting evidence.
• 40 to 50 min, insight generation: small mixed groups (designer, engineer, customer success) each pick one cluster and answer what problem it indicates, how it affects the business or user outcome, and what hypothesis or experiment follows. For example, if the customer-success representative brings in three separate tickets about a payment failing with no explanation, that cluster becomes the group's pick, and the hypothesis becomes that a vague failure message, not the underlying fraud rule, is driving repeat checkout abandonment.
• 50 to 60 min, decide next steps: each group presents one insight and one next action (prototype, analytics check, or user interview), and the product manager captures priorities, owners, and a timeline.

Roles during the session

• Facilitator (product manager): keeps time, keeps scope, synthesizes.
• Designer: surfaces empathy patterns, proposes UX experiments.
• Engineer: calls out technical constraints and feasibility.
• Customer success: brings the customer's voice, flags severity and urgency.
• Scribe: documents clusters, evidence, and action items.

For a two-sided or marketplace product, add a merchant-facing product manager or a growth representative so the map captures both sides of the transaction, not only the buyer's experience.

Sample prompts for each quadrant

• Says: "What exact phrases or quotes have customers used?" For example, "It takes too long to set up."
• Thinks: "What might they be worried about or assuming but not saying?" For example, "Is this secure? Will this break my workflow?"
• Does: "What observable actions do they take?" For example, retries setup, contacts support, abandons the flow.
• Feels: "What emotions underlie the behavior?" For example, frustrated, anxious, relieved.

Converting results into product insights

Synthesize the clusters into 3 prioritized problems, each backed by evidence (quotes, frequency, and severity as flagged by customer success). For each, write a clear hypothesis: "If we do X, then metric Y will improve by roughly Z." Map each hypothesis to a concrete next step, a quick prototype test, an analytics event change, or a targeted interview. Log outcomes against the roadmap or backlog, tag tickets with the empathy cluster they came from, assign an owner, and set a measurable target. Share a one-page summary with stakeholders and run at least one validation step within two weeks.

Trade-offs and pitfalls

A 60-minute session generates directional hypotheses, not proof; treat every "insight" from the workshop as a candidate for validation, not a finished decision.

Direct answer

I would build the screener in two layers: a short set of eligibility questions up front that filters for the right target user, kept short so it does not cost valid participants time or drop-off, and a set of fraud and data-quality signals that run partly during screening and partly after the session. Some fraud, like a professional survey-taker gaming your screener or the same person applying under a second identity, only shows up once you look at behavior, not at what someone typed into a form.

Structured elaboration

• Sequencing. Eligibility questions first (behavior or role questions that determine fit and can disqualify immediately), then a harder-to-game open-ended or scenario question, then logistics and demographics last, so you do not spend effort collecting details from someone who is about to be disqualified anyway.
• What makes you reject someone:
  1. Fails an explicit eligibility gate, for example does not use the product or does not hold the target role.
  2. Gives a generic, non-specific answer to an open-ended question, a common sign of a professional respondent giving a canned answer that could apply to any product (straight-lining, answering in a way that shows no real engagement with the specific question).
  3. Matches a duplicate signal, for example the same email, payment detail, device, or IP address as a previous respondent in this or a recent study.
  4. Shows an identity or location inconsistency, for example a stated location that does not match an IP-derived location, or demographic answers that contradict each other within the same screener.
  5. Completes a multi-minute screener in a few seconds, a marker of a bot or someone speed-clicking through without reading.
• Post-session and post-payment review path. Some fraud is only visible once the session happens, for example a respondent who clearly does not know the product despite passing the screener. Define an escalation path: flag the session, exclude its data from analysis even if payment has already been sent (protecting the incentive budget is not worth contaminating the dataset), and add the identifying signal to a suppression list so the same person cannot pass your screener again in a future study.
• Balancing speed against specificity. Keep only true disqualifiers mandatory; make everything else optional or branching so a long form does not cost you real target users. Track how long genuine target users actually take to complete the screener and trim questions if it is creating drop-off.

Worked example

A 4-question screener for a study on a budgeting app, targeting people who have used a named competitor app in the last 30 days:

1. Eligibility (mandatory): "Which of these apps have you used in the last 30 days?" with a list that includes the real target app, real competitors, and one decoy option (a made-up app name). Qualify only if a real expected item is checked and the decoy is NOT checked; checking the decoy is a classic sign of a rushed or professional respondent selecting everything.
2. Open-ended (harder to game): "What's the most annoying part of tracking your spending right now?" Reject vague, non-specific answers ("it's fine," "nothing really") that suggest low engagement or a scripted response.
3. Logistics (mandatory): availability window and device.
4. Demographics (optional, non-gating): collected but never used to disqualify.

Automated post-submit checks then compare email, device, and IP against a suppression list from prior studies, and flag any submission completed in under roughly 20 seconds if the screener is estimated to take about 90 seconds to complete honestly.

Trade-offs & pitfalls

A decoy question only works if it is something a genuine target user would clearly recognize as false; test it internally first so it does not accidentally trip up real participants. Trying to catch every possible fraud signal up front directly fights the speed goal, so it is worth prioritizing the two or three signals with the highest catch rate for your specific panel or platform rather than building an exhaustive screener nobody will finish.

Direct answer. Cohort analysis groups users by a shared starting point, most commonly the week or day they signed up, and tracks how each group behaves over time relative to that starting point. Cohort segmentation is the more general practice of slicing any analysis by these groups (or other shared attributes) instead of only looking at blended, cross-sectional averages. The reason this matters for root-cause work is that a blended metric can hide exactly the story you need: if overall retention drops, cohort analysis tells you whether it's because NEW cohorts are behaving differently (a product or acquisition-quality problem) or because an OLD cohort's behavior changed at a specific point in time (a change that hit everyone at once, like an outage or a pricing change).

Structured elaboration. Two concrete ways cohort slicing finds a root cause:

1. New-cohort-only drop. If you plot retention curves for several consecutive signup cohorts and only the most recent 1-2 cohorts show a lower curve while older cohorts look normal, the cause is almost certainly something that changed for NEW users specifically: an onboarding flow change, a shift in acquisition channel or campaign mix bringing in lower-intent users, or a broken signup-time event.
2. Cross-cohort simultaneous drop. If every cohort's retention curve, regardless of signup date, dips at the SAME calendar date, the cause is an event that hit all users at once: an outage, a pricing change, a UI regression, or a tracking break, not anything about who signed up.

Cohort dimensions worth defining beyond signup date: acquisition channel or campaign (isolates marketing-driven quality shifts), first-feature-used (isolates onboarding-path effects), platform or device (isolates a client-specific regression), and geography (isolates a market-specific or regulatory cause). The choice of cohort window (daily vs. weekly) is a bias/noise trade-off: daily cohorts pinpoint the exact day something changed but each cohort has fewer users and a noisier curve; weekly cohorts are more stable but blur the exact day.

Worked example. A subscription product's week-1 retention has been declining for a month. Plotting retention curves by signup week: cohorts from weeks 1-3 all show the SAME retention curve when re-aligned to days-since-signup, but week 4's cohort sits visibly below the others starting from day 0. That pattern (only the newest cohort is different, and it's different from day 0, not from a later day) points at something about the SIGNUP EXPERIENCE itself for that week, not a mid-lifecycle event; checking the release log finds an onboarding-flow redesign that shipped exactly at the start of week 4.

Trade-offs and pitfalls. A common mistake is reading a cohort chart from left to right instead of top to bottom: comparing where curves stand relative to each other at the SAME days-since-signup value (top to bottom) diagnoses a cohort-quality difference, while comparing what happens to one curve over calendar time can mix up product effects with market effects. Also watch for survivorship bias in very recent cohorts: the newest cohort hasn't had time to reach later retention windows yet, so an apparent 'improvement' in a young cohort's early numbers can just be incomplete data, not better retention.

Direct answer
Mentoring someone through their first postmortem means pairing them with a real, ideally low-stakes incident, giving specific feedback on structure and tone rather than just 'good job,' and modeling the blameless framing yourself before expecting them to reproduce it independently.

Structured elaboration

• Pick the right first incident. A moderate-severity, reasonably contained incident is a better first assignment than either a trivial one (nothing to learn from) or a highly political, multi-team, high-visibility one (too much pressure for a first attempt).
• Give feedback on structure. Check whether the timeline is objective and evidence-backed rather than reconstructed from memory, whether root cause is separated from contributing factors, and whether action items are specific and owned rather than vague aspirations.
• Give feedback on tone, with concrete examples. Point out any sentence that names a person rather than a system gap, and show, don't just tell, how to rewrite it: 'the engineer forgot to run the checklist' becomes 'the checklist has no automated enforcement, so a required step could be skipped.' Seeing the before-and-after side by side teaches the skill faster than an abstract rule.
• Have them facilitate a real meeting, with you as backup, not the lead. Reading about facilitation and doing it live under mild pressure are different skills; be present to redirect gently if the discussion drifts toward blame, but let them run it.
• Follow up on whether the action items actually happened. Closing the loop on whether their first postmortem's action items got implemented and verified teaches the full lifecycle, not just the writing exercise.

Worked example
A junior engineer is assigned to lead the postmortem for a minor, contained caching bug that caused stale data for about ten minutes. Before the meeting, the mentor reviews their draft timeline and flags one sentence ('the developer pushed an untested change') to rewrite as a system-focused observation about the deploy process lacking a required test gate for cache-invalidation logic specifically. During the meeting, the junior engineer facilitates; the mentor stays quiet unless the discussion drifts, at one point gently redirecting a comment that started to focus on who wrote the original caching code. Afterward, feedback covers three things: the timeline was strong and evidence-based, the action item ('add a test for cache-invalidation edge cases') was specific and well-owned, but the root cause and contributing factors weren't clearly separated in the writeup, which is worth practicing next time. Three weeks later, the mentor checks whether the test was actually added and merged, closing the loop rather than treating the writing exercise as the end of the mentorship.

Trade-offs and pitfalls
The most common mistake is giving only high-level praise or criticism ('good postmortem' or 'needs work') without specific, actionable examples the person can apply next time. A second is the mentor taking over facilitation when things get slightly awkward instead of letting the mentee work through it with light support, which prevents them from actually building the skill.

Direct answer

The situation: Sales wants a custom feature to win a big deal, Engineering worries about cost and risk, and Finance wants proof it pays back. As the product manager, I would run one structured decision process instead of three bilateral negotiations: a written one-page brief from each party in advance (for example Sales: contract value, written condition of signing, close date, odds with and without the feature; Engineering: best/likely/worst estimate, displaced work, maintenance cost; Finance: threshold and what evidence releases budget), a single decision meeting, options that stage the money and the risk, and an escalation path with a date if we are still stuck. What I ask each party to bring is the evidence that would change their own mind.

Structured elaboration: what each party brings

• Sales: the contract value and term, written confirmation that the feature is a condition of signing, the close date, honest odds with and without the feature, and whether other customers would want it.
• Engineering: a range (best, likely, worst), the recurring maintenance cost of a custom fork (a private copy of the product code for one customer that must be kept in step with the main version), which roadmap items get displaced, and the alternative of building it as a configurable feature (one setting-driven version other customers can also use, instead of code for one customer).
• Finance: the return threshold (for example a payback period, meaning how many months until the money earned repays the cost, and a minimum margin), how one-time and recurring costs are counted, and what evidence would release budget, including whether it can be released in tranches (separate instalments of the budget, each released only when a condition is met).

Running it

Circulate the three briefs 48 hours ahead. In a 45-minute meeting, agree the shared yardstick first, then compare the options: (a) build the custom version, (b) build it as a configurable feature, (c) staged funding, releasing the first tranche only when a contract is signed (the signed order form must commit the customer to the feature and its delivery date, because if the feature were a precondition for any signature the contract could never precede the build), (d) ask the customer to fund the build through a one-off engineering fee or a minimum-term commitment, (e) decline.

Worked example (illustrative numbers)

Build: 10 engineer-weeks at $4,000 per engineer-week = $40,000. Engineering says the custom code adds about one week of extra maintenance per quarter: 4 x $4,000 = $16,000 a year. Year-one cost = $56,000.

Year-one cost is the right denominator here because the build is a one-off and the maintenance recurs, so year one is the first period in which both are paid and it is the period the payback is judged over. Deal: $400,000 annual recurring revenue (ARR, the yearly subscription value) at a 70% gross margin (the share left after the direct cost of serving the customer) = $280,000 a year. Sales says the chance of closing is 80% with the feature and 20% without, so the extra expected margin = (0.80 - 0.20) x $280,000 = $168,000 a year. Return = $168,000 / $56,000 = 3.0 times. Payback: $168,000 / 12 = $14,000 a month, so $56,000 / $14,000 = 4 months. Finance's remaining question is the displaced roadmap work, which the numbers above leave out. Sizing it: if the build pushes out 4 engineer-weeks of other planned work (4 x $4,000 = $16,000), year-one cost becomes $72,000, the return falls to $168,000 / $72,000 = about 2.3 times, and payback to $72,000 / $14,000 = about 5.1 months. The case still holds, but the honest figures are the ones to bring to the meeting.

If the three still cannot agree

Escalate the decision, not the argument. Send the executive sponsor a one-page memo. A skeleton: "Decision needed by Friday. Options: (a) to (e). Recommendation: (c), release tranche one ($20,000, half the build) on signature. Cost of delay: the 80% close odds slip each week past the close date. Finance would agree if the contract is signed first; Sales would agree if work starts before signature on a written commitment." (The tranche split here is illustrative.) Tell all three parties beforehand what is in it. To make the two conditions above compatible, "signed" means a signed order form or letter of intent that names the feature, the delivery date and the price; engineering starts on the strength of that document, and if the customer will not sign even that, the deal is not yet real enough to fund and the feature is not built. Set a decision date. The recommended fallback is option (c): release the first tranche on a signed contract, so Finance gets proof and Sales gets speed.

Trade-offs and pitfalls

• A custom fork can quietly become permanent; add an exit or review date.
• Do not accept sales odds without evidence.
• Do not let Finance's proof requirement freeze a time-limited deal: propose staged spend.

Given funnel counts, identifying which step to prioritize requires computing each step's own conversion rate first, then reasoning about which is furthest from a realistic benchmark, since the step with the biggest absolute number drop isn't always the one with the worst underlying conversion rate.

Worked example

For a checkout flow with product-page visits = 10,000, add-to-cart = 1,500, checkout started = 600, purchase completed = 300:
$\text{Visit} \to \text{Cart}: \frac{1{,}500}{10{,}000} = 15\%$
$\text{Cart} \to \text{Checkout}: \frac{600}{1{,}500} = 40\%$
$\text{Checkout} \to \text{Purchase}: \frac{300}{600} = 50\%$

The biggest absolute count drop happens between visit and cart (8,500 users lost), but that's expected at the top of any funnel where most visitors are just browsing; the visit-to-cart rate of 15% is not unusual for e-commerce. The step most worth investigating first is Cart to Checkout: a 40% drop-off between adding something to a cart (an expressed purchase intent) and even starting checkout is a much larger loss of ALREADY-INTERESTED users than is typical, and is often attributable to fixable friction (unexpected shipping costs revealed at that step, a confusing 'proceed to checkout' button, or account-creation requirements) rather than a fundamental lack of interest.

Why this step, not simply the biggest raw number

A user who has already added an item to their cart has demonstrated real intent; losing 60% of them at the very next step is a much stronger signal of a fixable, high-leverage problem than the visit-to-cart drop, which mostly reflects normal top-of-funnel browsing behavior that's expensive and slow to move. Prioritizing by 'which step has the worst conversion RELATIVE TO a realistic expectation for that stage,' not by raw volume lost, is what a senior analyst does differently from a junior one here.

Trade-offs and pitfalls

Without an external benchmark or historical baseline for what a 'normal' conversion rate looks like at each stage, this reasoning is a judgment call, not a certainty; the right next step is validating the hypothesis with qualitative research (session recordings, exit surveys at the checkout-start step) before committing significant engineering effort to a fix.

Experiment spillover in marketplaces occurs when treatment affects not only treated units but also control units (e.g., buyers or sellers), violating SUTVA and biasing ATE estimates. For example, promoting prices to treated sellers can change buyer behavior platform-wide, or routing more buyers to treated sellers deprives control sellers of demand.

Why bias arises:

• Spillover creates interference: control outcomes shift toward treatment, attenuating or reversing measured effects.
• It can induce network effects (matching, prices, reviews) that confound causal attribution.

Empirical strategies to detect spillover from logs:

1. Temporal lead/lag analysis: compare pre- and post-assignment trends for control vs treated. If control outcomes change only after treatment rollout and track proximity to treated activity (e.g., increases in purchase latency or drop in orders near treated sellers), that indicates spillover.
2. Exposure/gradient tests: construct an exposure metric per control unit from logs (e.g., fraction of their neighbors or matched users who are treated, or routed impressions originating from treated cohort). Regress outcomes on exposure controlling for baseline covariates—significant exposure coefficients signal interference.
3. Placebo population / geographic holdouts: use segments unlikely to be affected (distant geos or unrelated product categories) as falsification checks; divergence suggests spillover.

Design-level mitigations:

1. Cluster randomization: randomize at the cluster level that captures interference (e.g., marketplace regions, seller cohorts, buyer segments) so interference is contained within clusters; analyze with cluster-robust inference.
2. Buffering / stratified holdouts: create buffer zones or exclude immediate neighbors from control (e.g., reserve nearby sellers as separate clusters) to reduce cross-group interactions; alternatively, use saturation experiments where whole groups receive varying treatment intensities to estimate direct vs spillover effects.

Trade-offs and practical notes:

• Clustering reduces power; choose cluster granularity to balance bias vs variance.
• Exposure regression helps quantify spillover when redesign isn’t feasible; combine with sensitivity analyses.
• Pre-register spillover metrics and monitor continuously in logs (impressions, matches, prices, messages) to detect unexpected interference quickly.

Direct answer

Pick a small set of weighted criteria before scoring anything, typically impact, effort, UX risk, and engineering complexity, score each validated concept against them with a shared rubric, and treat a close score as a signal to break the tie with the cheapest available validation step rather than more debate, since the research budget is exactly the constraint that framework needs to respect.

Structured elaboration

Why these criteria

• Impact: does the concept move the outcome the feature exists to affect.
• Effort: combined design and build cost, which is what actually translates the budget constraint into the scoring.
• UX risk: how much learnability or error risk the concept introduces, especially for a less-tested interaction pattern.
• Engineering complexity: distinct from effort; a concept can be low design effort but high build complexity, or the reverse, so score them separately.

Weighting
Weight the criteria to reflect the constraint actually driving this decision. With a limited research budget, impact and UX risk deserve more weight than usual, since a full study would normally be what de-risks those two, and this decision has to substitute a lighter process for that.

Scoring mechanics
Define a 1 to 5 rubric per criterion before anyone scores (5 is most favorable on that criterion, including for effort and complexity, so a low-effort, low-complexity concept scores a 5 there). Score independently first, then discuss any large deltas between scorers rather than averaging blindly, since a big disagreement usually means people are scoring against different assumptions.

Handling a near-tie
Decide the tie threshold before scoring, for example anything within a small band counts as effectively tied given the imprecision of the exercise. When two concepts land inside that band, do not keep debating the matrix; spend the limited research budget on the cheapest signal that resolves the single riskiest assumption each concept depends on, such as a short unmoderated test or a fast internal expert review, rather than a full study on either.

Worked example

Weights, decided before scoring: Impact 0.35, UX risk 0.25, Engineering complexity 0.25, Effort 0.15 (sums to 1.00). Two concepts, scored 1 to 5, 5 most favorable:

A 0.05 gap on a 5-point scale falls inside a reasonable tie threshold (say, anything within 0.1 points), so this is a tie, not a winner. Concept A's strength is UX risk, Concept B's strength is impact. Given the budget, the tie-break is a short unmoderated test aimed specifically at Concept B's biggest open question (whether its higher-impact interaction is actually learnable without guidance), since that is the one assumption a full study would normally have resolved and the matrix cannot settle on its own.

This holds at larger scope, too. For a broader multi-segment enterprise feature with the same lean budget, the same weighted approach applies; add a segment-coverage or per-segment risk check as an additional criterion rather than rebuilding the framework, and keep the same divergence-convergence (generating many options broadly, then narrowing them down) -stakeholder-review cadence so the extra segments do not silently expand the research ask past what the budget allows. The same logic also applies one level up, to deciding between a full redesign and an incremental improvement: redesign options typically score higher on impact ceiling, incremental options typically score higher on effort and risk, and which one wins depends on which criterion the current constraint (cost, timeline, or how much risk the team can absorb) weights most heavily.

Trade-offs and pitfalls

• Setting weights after seeing scores, even unconsciously, turns the matrix into a way to justify a pre-existing favorite rather than a decision tool; commit to weights first.
• A score to two decimal places is not more true than an honest "this is close"; the matrix's job is to force an explicit conversation about trade-offs, not to produce a number precise enough to hide behind.
• Skipping the tie-break step and defaulting to seniority or whoever argues loudest defeats the entire point of building the matrix.
• Document the rejected concept and the specific condition that would justify revisiting it (new data, a changed constraint), so the decision does not have to be re-litigated from scratch later.

Direct answer

RICE is a scoring method (a prioritization framework, meaning a repeatable way to rank work) that gives every candidate item one number. Reach is how many people or events the item touches in a fixed period. Impact is how much it moves each of them toward the goal. Confidence is how sure you are of those two guesses. Effort is the work required, in person-months (one person working for one month). I would run it as a single-cycle pilot: agree the goal, make different people supply different inputs, publish the ranked list with the evidence behind each number, and treat the score as the start of the decision conversation, not the verdict. The point is to replace "who argued loudest" with "whose reasoning can everyone see".

$\text{RICE score} = \frac{\text{Reach} \times \text{Impact} \times \text{Confidence}}{\text{Effort}}$

The score reads as "expected impact per person-month of work", so a cheap, well-evidenced item can beat a large, speculative one.

How I would run the planning cycle

1. Fix the yardstick first. Pick one goal metric for the cycle (for example, weekly active teams) and one reach window (customers per quarter, never a mix of per month and per quarter). Impact only means something relative to a goal.
2. Collect candidates. Put 15 to 20 items on one list, one line each. Every requester's item, including the loudest one's, gets scored the same way.
3. Source each input from the person or data best placed to know it:
   • Reach: product analytics (distinct users who touched the affected flow last quarter). For a brand-new surface, use the size of the target segment. Owner: PM with an analyst.
   • Impact: a coarse scale (3 massive, 2 high, 1 medium, 0.5 low, 0.25 minimal, the scale Intercom published with RICE), set by the PM and designer against the goal metric and anchored on past launches or experiments. The requester does not set their own.
   • Confidence: from an evidence ladder: 100% means tested data or an experiment, 80% means some evidence plus similar past launches, 50% means opinion only. Below 50% is a moonshot.
   • Effort: the engineers who will build it, in person-months, including design, testing and data work. Not the PM.
4. Calibrate. Sort the list, then spend 30 minutes sanity-checking the top and bottom five. If the order feels wrong, find which input is wrong and change it with a written reason. Never edit the final score to fit a preference.
5. Decide and publish. Share the ranked list with the evidence column. Items that must happen regardless (a legal deadline, a signed commitment) are tagged "mandatory" in the open so they are not smuggled in.
6. Review after shipping. Compare the predicted impact with what happened, and adjust how the team scores Impact next cycle.

The same walk-through in other domains: for a BI team, reach is the number of decision-makers who use the report. For data engineering, reach is the downstream pipelines or consumers fed, and effort includes backfills and monitoring. For an AI feature, confidence comes from offline evaluation results, and effort includes evaluation and monitoring work, not just training.

Worked example

Reach is per quarter and effort is in person-months for all three:

Saved search: 12,000 x 0.5 x 0.9 = 5,400, divided by 1 person-month gives 5,400. CSV export: 8,000 x 1 x 0.8 = 6,400, divided by 2 gives 3,200. SSO: 1,500 x 3 x 0.5 = 2,250, divided by 3 gives 750.

Where the scores will mislead me

• Strategic bets look bad. SSO scores lowest, but if enterprise deals depend on it, reach (current users) understates its value. RICE does not see strategy, deadlines or dependencies, so those need their own conversation.
• Small effort numbers dominate. Dividing by 1 rewards anything with an optimistic estimate, and small estimates are the ones most often wrong.
• False precision. 5,400 versus 3,200 looks decisive, but Impact is a guess on a five-point scale. Treat scores within roughly 20% of each other as a tie (a rule of thumb I would choose and agree with the team, not part of RICE itself), and remember the band is a floor, not a ceiling. Most steps on the Impact scale (0.25, 0.5, 1, 2, 3) are a doubling (0.25 to 0.5, 0.5 to 1, 1 to 2), though 2 to 3 is only 1.5x. The leader's Impact is 0.5, so one step up changes its score by 100% and one step down by 50%: if Saved search were really 0.25 its score would be 12,000 x 0.25 x 0.9 / 1 = 2,700, below CSV export's 3,200. So the 69% gap between 5,400 and 3,200 (a ratio of 1.69) is smaller than the doubling a single Impact step can cause, and a one-step-down error reverses the order. A gap that size is a signal worth discussing, not a proof, so test the Impact guess behind the leader before locking the order.
• Gaming. People inflate confidence to move their item up. The evidence ladder and a rule that requesters cannot score their own item are the defences.
• Reach bias. Broad, shallow features beat deep ones for a small, valuable segment.

Direct answer

The call between continued growth subsidy and a shift toward retention and profitability should be made on incrementality, not on headline growth: run randomized or geo-holdout experiments to isolate the true lift from subsidies versus retention investment, then compare each dollar's incremental lifetime value (LTV, the discounted future contribution a customer generates) against its cost, expressed as the ratio of incremental LTV to customer acquisition cost (CAC, the cost to acquire one customer) and as payback period. The switch signal is when incremental LTV-to-CAC on continued subsidy spend goes flat or declines (a rising share of orders coming from promotions without growth in net active customers) while a controlled retention-investment test shows a larger or comparable return at lower cost.

Structured elaboration

1. Isolate incrementality, not correlation. Compare a subsidy-treatment group against a true control (a geo-holdout or randomized withhold) over a horizon long enough to observe repeat behavior, at least 60 to 90 days. Vanity metrics like total order growth conflate incremental customers with ones who would have converted anyway.

2. Segment by cohort and channel. Promo elasticity differs by segment (new users, lapsed users, price-sensitive geographies); a blended number hides where subsidies still work and where they've stopped.

3. Model retention investment the same way. Run a parallel controlled test of retention levers (loyalty credits after N orders, service-quality improvements) against a control, and measure the same incrementality: repeat-order probability lift and time-to-second-order, not just satisfaction scores.

4. Compute the decision metrics. For each treatment, compute contribution margin per order, incremental LTV per acquired or retained customer, CAC (fully loaded, including the promo cost itself), and payback period.

5. Watch for the saturation signal. A rising percentage of orders coming from promotions without a corresponding rise in net active users is the clearest sign that subsidy is buying repeat orders from existing users rather than genuinely growing the customer base, at which point the marginal dollar of subsidy spend is close to pure cost.

Worked example

All figures below are stated assumptions for the illustration, not measured results.

Current subsidy-heavy acquisition channel:

$$\text{CAC} = \$18/\text{customer}, \quad \text{monthly churn} = 12\%, \quad \text{contribution margin/order} = \$2.00, \quad \text{orders/month} = 3.2$$ $$\text{Expected customer lifetime (months)} = \frac{1}{0.12} \approx 8.33$$ $$\text{Monthly contribution per customer} = 3.2 \times \$2.00 = \$6.40$$ $$\text{LTV (undiscounted)} = 6.40 \times 8.33 \approx \$53.33$$ $$\text{LTV/CAC} = 53.33 / 18 \approx 2.96, \qquad \text{Payback} = 18 / 6.40 \approx 2.8 \text{ months}$$

Hypothesized retention-investment scenario (same contribution margin, CAC redirected to loyalty spend and cut to $9, churn tested and found to fall to 8% in a controlled cohort):

$$\text{Expected lifetime} = \frac{1}{0.08} = 12.5 \text{ months}$$ $$\text{LTV} = 6.40 \times 12.5 = \$80, \qquad \text{LTV/CAC} = 80/9 \approx 8.9, \qquad \text{Payback} = 9/6.40 \approx 1.4 \text{ months}$$

If a controlled retention experiment genuinely produces that churn improvement, it triples LTV/CAC and roughly halves payback compared with continued subsidy spend, which is the kind of before/after gap that justifies shifting spend, provided the churn improvement is confirmed experimentally rather than assumed.

Trade-offs and pitfalls

The biggest pitfall is treating a before/after comparison as causal: some of the "incremental" orders from subsidies would have happened anyway (cannibalization), and a short retention-experiment window can overstate the effect before novelty wears off. A second pitfall is adverse selection: subsidized users are sometimes structurally lower-quality (deal-seeking, low organic repeat intent), so comparing their LTV against a retention-focused cohort without controlling for acquisition channel overstates the case for either side. The senior move is to keep both experiments running in parallel with real holdouts rather than making the call from a single dashboard snapshot, and to set the switch threshold (for example, a specific LTV/CAC floor or payback ceiling) before the results come in, not after.