Lyft Product Manager Interview Preparation Guide - Junior Level (1-2 Years)

Direct answer

Three biases that commonly distort problem framing: confirmation bias (favoring evidence that supports a theory you already hold), anchoring (over-weighting the first number or explanation you encounter), and availability bias (treating a vivid, memorable example as more representative than the actual data supports). Each has a specific, checkable mitigation, which is what makes naming them useful rather than just a caution.

Structured elaboration

Confirmation bias: in product analytics, this shows up as running a segmentation cut, seeing a pattern that matches your existing theory, and stopping there without checking whether an equally plausible alternative cut would explain the data just as well. Mitigation: before looking at the data, write down what pattern would DISPROVE your leading theory, not just what would confirm it, and specifically check for that pattern.

Anchoring: in product analytics, this shows up when the first stakeholder to weigh in ("I bet it's the pricing page") shapes the entire subsequent investigation, even after data arrives that's ambiguous between that explanation and others. Mitigation: generate your list of candidate hypotheses before hearing anyone's initial theory, or at minimum before looking at any data, so the list isn't anchored on whoever spoke first.

Availability bias: in product analytics, this shows up when one vivid customer complaint (a detailed, emotionally compelling support ticket) gets treated as representative of a broad pattern, when it may in fact be a single outlier. Mitigation: before acting on a vivid individual example, check its frequency against the full data; it may be real and worth acknowledging, but shouldn't set the scope of the response on its own.

Worked example

A stakeholder reports "customers hate the new pricing page," backed by one detailed, well-written complaint email. Applying the availability-bias mitigation: check the actual data first, support-ticket volume mentioning pricing, before and after the page changed, and the site's overall conversion rate at that step. If ticket volume mentioning pricing is flat and conversion at that step is unchanged, the vivid complaint, while real, isn't representative of a broader pattern, and treating it as one would misdirect the team's investigation toward a problem that isn't actually widespread.

Trade-offs and pitfalls

These mitigations take real discipline to apply consistently, particularly under time pressure, when writing down a falsifiable prediction before looking at data, or deliberately delaying hearing a senior stakeholder's initial theory, can feel like an unnecessary extra step; the value shows up specifically in the cases where the bias would otherwise have sent the investigation in the wrong direction, which by definition you can't always tell in advance. The other pitfall is using "that could just be confirmation bias" as a reflexive dismissal of a genuinely well-supported finding, rather than reserving the label for cases where the mitigation check (looking for disconfirming evidence, checking frequency against the full data) actually reveals a gap.

A concise B2B persona schema, with evidence per field

• Name and role: title, seniority, and decision-making power. Evidence: CRM job titles, account segmentation, customer interviews.
• Primary goals and success metrics: what this person is trying to achieve. Evidence: product analytics tying feature usage to outcomes, sales discovery notes, interviews.
• Key tasks and workflow: core tasks, tools used, handoffs to other roles. Evidence: contextual interviews, session recordings, product telemetry (funnels).
• Pain points and blockers: specific frustrations and constraints. Evidence: support tickets, NPS (net promoter score) verbatim comments, usability-test transcripts.
• Buying and evaluation criteria: what features, security posture, and return on investment they need to see. Evidence: win/loss interviews, procurement documents, requests for proposal (RFPs, formal documents a buyer sends asking vendors to bid).
• Technical context and data maturity: stack, integrations, and data literacy. Evidence: onboarding surveys, integration logs, customer-success calls.
• Organizational constraints and stakeholders: budget cycles, compliance requirements, and the decision chain. Evidence: account plans, contract terms, stakeholder interview maps.

A single page for two audiences
Order the fields so the top third, name and role, goals, pain points, serves a PM skimming for context in a meeting, and the middle and bottom, tasks and workflow, technical context, org constraints, serves a designer working through flows in detail. Both audiences read the same page; neither needs a separate version, because the depth increases as you move down rather than requiring a second document.

Why these sources
Interviews and contextual research capture motivation and workflow; analytics and CRM data validate frequency and scale; support and sales conversations surface high-signal pain points and buying reasons. Combining qualitative and quantitative evidence is what makes each field defensible rather than assumed.

Keeping personas up to date
Instrument the canonical signals behind each field (product events, account attributes) and run a quarterly automated check that flags drift, for example a shift in role distribution or a usage pattern that no longer matches the stated goals. Maintain the page in the design-system repo with a visible change log, sync with customer success and sales monthly, and run a lightweight re-validation, 5 to 8 interviews plus an analytics snapshot, after any major product release. Treat every field as a hypothesis with a stated confidence level, and prioritize re-research wherever confidence is low or the underlying metric has moved.

Requirements & constraints:

• Reduce duplicated engineering effort and improve reuse without hurting short-term delivery velocity.
• Migration must be incremental, low-risk, observable, and reversible.
• Support multiple product teams with minimal blocking; preserve roadmap commitments.

High-level approach:

1. Establish a Platform Team (core owners) + Product Teams as platform consumers. Platform provides well-scoped APIs, SDKs, infra, and SLAs.
2. Phased rollout across capabilities (horizontal slices): start with non-critical, high-duplication areas (e.g., auth, logging, metrics) then core business flows.

Phased rollout plan:

• Phase 0 (Preparation, 4–8 wks): inventory duplicated capabilities, define contract-first API standards, SLAs, versioning policy, and migration playbooks. Set observability & feature flags.
• Phase 1 (Pilot, 6–12 wks): pick 1–2 pilot product teams with high duplication, low external dependency; build platform MVP for one capability; migrate one small feature end-to-end.
• Phase 2 (Expand, 3–6 months): onboard 3–5 teams progressively, add additional capabilities, implement shared libraries/CLI.
• Phase 3 (Stabilize, ongoing): hardened platform SLAs, chargeback/pricing model, deprecate legacy implementations.

Pilot team selection:

• Criteria: willing partner, representative tech stack, feature with measurable usage, low regulatory risk.
• Responsibilities: close collaboration, early feedback loops, dedicated support from platform team, migration timeline in sprint plans.

Metrics to monitor (leading + lagging):

• Delivery velocity: cycle time, lead time for change (per team) — track weekly.
• Blockers: number & time-to-resolve platform-caused interrupts.
• Platform adoption: % of services using platform, API calls, SDK installs.
• Quality: incident rate attributable to platform vs. product code, mean time to recovery (MTTR).
• Developer experience: NPS, time-to-first-success (onboarding), number of support tickets.
• Business impact: duplicate effort hours saved, cost per feature.

Mitigation & stabilization steps:

• Backward-compatible contracts + versioning and deprecation windows.
• Feature flags & strangler pattern: run platform and legacy in parallel; switch per-user/feature gradually.
• “Platform SLOs” and a fast-path support channel (office hours + pager) for pilot teams.
• Dedicated migration engineers/embedded platform liaisons for critical product teams.
• Stop-the-line rollback plan per migration: pre-approved rollback criteria, automated scripts to revert.
• Prioritize platform work in roadmap only when measurable unblock/velocity improvements are clear.
• Weekly governance: triage board with PMs, engineering leads, platform to review metrics and unblock escalations.

Communication & governance:

• Publish migration roadmap, SLAs, and runbooks. Weekly syncs, monthly retrospective, public changelog.
• Incentivize uptake: recognize product teams who migrate, show saved effort metrics.

Expected outcome:

• Reduce duplicated effort gradually while keeping velocity stable by controlling scope, using pilots, monitoring leading indicators, and maintaining fast rollback/support paths.

Direct answer

Don't try to average everyone's position into a compromise nobody's happy with. Ground the negotiation in the shared outcome, make the trade-offs between options explicit with evidence, and force a real decision (with an owner and a documented rationale) within a fixed timeframe. A compromise sticks when people can see why it was chosen, not just that it split the difference.

Structured elaboration

1. Reframe around outcome, not position. Ask each stakeholder what success looks like for them, not what they want built. Two stakeholders who seem opposed on the "what" often agree on the "why," which is where the real compromise lives.
2. Bring evidence, not opinions. Gather whatever is available and relevant: usage data, cost/effort estimates, prior incidents, qualitative feedback. A room full of opinions negotiates forever; a room with a shared set of facts converges faster.
3. Make trade-offs visible. Lay out 2-3 real options with their costs and benefits side by side, instead of a single proposal to accept or reject. People compromise more easily when they're choosing between concrete alternatives than when they're being asked to give up a specific ask.
4. Use a structured negotiation move. Propose a balanced default option first, then invite each side to request a bounded concession from it, rather than starting from each side's maximal ask and negotiating down. Time-box the discussion so it doesn't drift into re-litigating the same points.
5. Document the decision and name an owner. Write down what was decided, why, who owns it, and when it will be revisited. If the group truly can't converge, escalate with a specific recommendation rather than an open question, so the escalation itself doesn't become another unresolved debate.
6. Build in a review point. Treat the agreement as provisional and testable, not permanent. A short follow-up (after the next milestone, or a fixed number of weeks) to check whether the compromise is actually working keeps people bought in because they know it isn't final and unappealable.

Worked example

Three stakeholders disagree on scope for a feature: one wants the full version shipped now, one wants it deferred a quarter, one wants a stripped-down version shipped immediately. Instead of negotiating "how much scope," the facilitator asks each what outcome they're protecting: the first is protecting a customer commitment, the second is protecting engineering capacity for other work, the third is protecting the team's ability to learn before over-investing. That reframing surfaces a real option none of them had proposed: ship a narrow version that satisfies the customer commitment, explicitly scoped as a first iteration, with the deferred work logged and re-prioritized at the next planning cycle. The decision, the scope boundary, and the re-prioritization date are written down and shared with all three stakeholders.

Trade-offs & pitfalls

• Pitfall: false compromise, where everyone gets a token piece of what they asked for and the result satisfies no one's actual underlying need.
• Pitfall: skipping documentation. An undocumented "agreement" gets re-argued the moment someone's memory of it differs.
• Pitfall: treating consensus as required. Some decisions need a single accountable owner to make the call after input, not unanimous agreement, especially under a deadline.
• Senior differentiator: designing the forcing function (a default option, a timebox, a named decision owner) instead of facilitating an open-ended discussion indefinitely. That's what turns "several people who each want something different" into an actual decision.

A strong executive status update should answer four questions quickly: are we on track, what changed, what is at risk, and what do you need from me?

I’d include:

• Overall status: green, yellow, or red
• Progress against key milestones
• Delivery confidence and why it changed
• Top risks or blockers
• Decisions, escalations, or support needed
• Business impact, such as launch timing, customer value, or revenue risk

For a non-technical audience, I’d translate technical progress into business language. Instead of saying “the API refactor is 80% done,” I’d say “the backend work needed to support launch is mostly complete, which reduces release risk.”

I’d keep details at the right altitude: enough context to make a decision, not a deep implementation report. If leadership wants more detail, I’d offer an appendix or separate follow-up. The goal is clarity, not completeness.

Direct answer. Position for mid-market buyers (roughly 100 to 1,000 employees, with a small IT team and no dedicated admin staff) as "the product that gets a team live in days and covers 80% of what the incumbent does, without enterprise overhead." Lean on time-to-value (how long until a customer gets its first real benefit) plus ease of operation, and concede both the lowest price and the longest feature list.

Why this position. Positioning means choosing which customers you target, which alternative you are compared against, and the one reason you win. Each competitor owns a corner:

• Price leader: the vendor competing on lowest price; it wins buyers who treat the category as a commodity (all products seen as interchangeable). Competing here means a margin race you lose.
• Feature-rich incumbent: wins buyers with complex needs and staff to configure it. Mid-market teams pay for depth they never use.
• Enterprise vendor: wins on security reviews, customization and procurement fit, all of which add setup cost mid-market cannot absorb.

The gap is a buyer who needs the core job done well, has no implementation team, and will not tolerate a six-month rollout.

What to lean on (differentiation). Pick one primary claim and make it provable: a guided setup that reaches the first useful result in a short, measurable time, plus self-serve administration. Support it with two or three proof points (customer references at similar size, a published onboarding timeline you actually meet). Illustrative claim: "median 5 working days from signed contract to first live team," measured on your last 20 customers and published, against an incumbent rollout that commonly takes about three months (verify the competitor figure with customers who switched).

What to concede, explicitly.

1. Lowest price: sell on total cost, meaning everything the buyer pays in year one, not the sticker price. Illustrative: ours $30k with no extras; the incumbent $20k plus $40k of consultants plus half an admin at $100k a year ($50k) = $110k. Apply the same basis to our side: if our product also needs a tenth of an admin's time ($10k), our year-one total is $40k and the gap of about $70k survives; a multi-year view should add each side's renewal and admin costs the same way.
2. Feature breadth: the incumbent's long tail of configuration stays theirs.
3. Deep enterprise controls (custom roles, on-premises deployment): lose those deals on purpose.

Test. A good position can be disagreed with. If "we are simple and powerful" would suit any competitor, it is not a position.

What would change the call. If win/loss interviews (calls with buyers about why they chose or rejected you) show mid-market buyers actually choose on integrations, lead with integrations instead.

Situation: During a simultaneous global launch I see engagement metrics (DAU, session length, conversion funnel) gradually dropping in one region while others are stable. As PM I lead a fast, prioritized investigation and coordinate mitigations.

Investigation steps (first 60–120 minutes)

• Triage: confirm anomaly is real (compare to baseline and recent trend, rule out dashboard bugs or sampling changes).
• Scope: identify affected segments (country, OS, app version, carrier, time windows).
• Mobilize: assemble SWAT with engineering (infra, backend, mobile), SRE, analytics, localization, payments, and ops.

Data and logs to request

• Telemetry: raw event logs for key events (session_start, page_view, purchase) with timestamps and client metadata.
• Crash/error logs: mobile SDK crashes, native errors, JS exceptions.
• API/server logs: latency, 5xx/4xx rates by endpoint and region.
• CDN and cache logs: edge error rates and hit/miss ratios for region.
• Payment gateway logs: failed transactions, decline codes, TTLs.
• Network telemetry: packet loss/timeout rates from regional probes / CDNs.
• Feature-flag & release logs: which builds/flags rolled out to that region.
• Localization content: missing strings/assets, date/time/number formats.
• Third-party integrations: auth, analytics, ad SDKs health.

Possible root causes to evaluate

• Localization: missing/failed localized assets causing UI breaks or content not rendering.
• Infra/CDN: regional PoP outage, increased latency, cache misconfiguration causing timeouts.
• API/backend: region-specific routing misconfiguration, rate-limiting, dependency failure.
• Payment gateways: region-specific PSP outage or new decline patterns (currency, KYC).
• Releases/feature flags: bad build pushed to region or rollout mis-targeting.
• Regulatory/blocking: ISP or governmental blocking, DNS misconfiguration.
• Device/OS: a popular local device/OS combo with a regression.
• Measurement: SDK sampling/ingestion bug misreporting lower events.

Immediate mitigations (short-term)

• Rollback or pause: pause regional feature flag or rollback recent release for that region.
• Route traffic: shift traffic to healthy backend instances or alternative CDN PoPs.
• Increase retries/timeouts: temporarily relax rate limits or increase client timeouts where safe.
• Payment fallback: enable alternate PSP or allow cached/manual payment flow.
• Communication: notify customer support and regional marketing with guidance; add banner if critical.
• Observability: add targeted logs/metrics and a dashboard for quick monitoring.
• Hotfix prioritization: if localization missing assets, push high-priority static assets to CDN.

Follow-up

• Root cause postmortem with timelines, RCA, preventive actions (tests, preflight region checks, runbooks).
• Add regional synthetic monitoring, canary releases per region, and guardrails in rollout process.

This approach balances speed (mitigate impact) and rigor (collect evidence for durable fixes) while coordinating cross-functional teams.

Direct answer

Address it directly, privately, and with specific examples, not a general impression. Name the pattern you've observed (cutting people off, dismissing an idea without engaging with it) as a fact with two or three concrete instances, separate that from any comment on their technical judgment, and ask them to try one specific, observable change in the next discussion. Waiting for the pattern to become undeniable, or only correcting it in the room in the moment, both let the behavior calcify into how the team works.

Structured elaboration

A few things make this land instead of bouncing off.

Come with specifics, not a vibe. "You dismiss people" invites a denial. "In Tuesday's design review, [person] proposed an approach and you said 'that won't scale, next' without asking a follow-up question" is a fact. Have two or three dated instances ready before you start the conversation, or it turns into a disagreement about whether the pattern is even real.

Name their technical judgment as genuinely valuable, honestly, not as a softening tactic before the real message. Someone technically strong often assumes rigor and helpfulness are the same thing; the goal of the conversation is not to make them doubt their judgment, it's to separate evaluating an idea from shutting down the person who raised it.

Describe the effect on the room, specifically. The useful detail isn't "people feel bad," it's the behavioral change: junior engineers have started sending ideas to this person privately to get a pre-blessing before raising them in the group, or people have stopped proposing alternatives in meetings this person attends. That's the actual cost: you're losing the ideas, not just the feelings.

Model the alternative concretely. Ask them to try one specific, checkable habit rather than "be nicer": restate the other person's point in their own words before responding to it, or ask one clarifying question before saying no. A vague ask to "be more inclusive" gives them nothing to actually do differently in the next meeting.

Separate the private conversation from the room dynamic. The conversation alone won't fix a review culture where junior voices have already learned to go quiet. You likely also need to change the format itself (asking less senior people to share their view before the senior voice weighs in, for instance), or the old pattern reasserts itself even after this person genuinely tries to change.

Worked example

In a platform team's design reviews, a senior engineer had a habit of interrupting less experienced engineers mid-proposal with a flat rejection and no follow-up question. Over a few review cycles, two engineers stopped presenting ideas live and instead sent their proposals to this senior engineer beforehand to get informal sign-off. Their manager noticed the shift in who was speaking in reviews, gathered two specific instances with dates, and had a private conversation: opened with genuine respect for the engineer's technical judgment, described the two instances verbatim, and named the actual cost, that people were routing decisions around the room instead of through it. The engineer hadn't realized that's what was happening and pushed back that they were just being direct about weak proposals. The manager didn't argue the point of technical merit, they separated it out: being right about a proposal's flaws and cutting someone off before they finish are two different things, and asked the engineer to try restating a person's point before responding to it in the next review. The manager also started asking more junior engineers to present first in the next few reviews, rather than relying on the private conversation alone to shift the room.

Trade-offs and pitfalls

The common wrong turn is accepting their defense that this is just technical rigor and the real problem is other people being thin-skinned. That framing can be true in isolated cases, but if the pattern is consistent and it's changing who speaks up, the effect on the team is the thing you're accountable for regardless of whether each individual instance was, on its own technical merits, a fair critique.

The opposite wrong turn is over-correcting the person into silence. The goal isn't to make a technically sharp reviewer stop giving hard feedback, candor about weak proposals is valuable and losing it is its own cost. Be specific that the ask is about how they engage with the person, not that they should stop disagreeing.

If you don't manage this person directly, for example you're an architect or a peer lead working with them rather than their manager, you can and should still have the direct conversation, but your leverage for follow-through runs through their actual manager, not you. Say so plainly rather than implying a consequence you can't deliver.

Finally, a private conversation with no change to the room itself is fragile: if the meeting format still rewards whoever speaks loudest or most senior first, the behavior tends to resurface under pressure even from someone who genuinely intended to change. Pair the conversation with a structural fix, not just a private ask.

Direct answer

Net expansion is +$17,000 of MRR per month (about $204,000 of ARR), and ARPU across the whole base rises by $1.70, from $30.00 to $31.70 (+5.67%).

Terms

• MRR (monthly recurring revenue): the subscription revenue billed per month. Base MRR here is 10,000 x $30 = $300,000.
• ARPU (average revenue per user, here per customer per month): MRR divided by customers.
• Expansion revenue: extra recurring revenue from customers you already have (add-ons, upgrades, more seats).
• Contraction revenue: recurring revenue lost when a customer stays but pays less (a downgrade). It is different from churn, where the customer leaves entirely and all its revenue goes.

Calculation

Whole-base ARPU: ($300,000 + $17,000) / 10,000 = $31.70, a change of $17,000 / 10,000 = +$1.70. Cross-check by group: 1,000 x $50 + 300 x $20 + 8,700 x $30 = $50,000 + $6,000 + $261,000 = $317,000.

Useful ratios against the $300,000 starting MRR: gross expansion is 6.67%, contraction is 1.00%, net is 5.67%. Ignoring churn (none is given), net revenue retention would be 105.67%. Annualised, $17,000 x 12 = $204,000 of ARR.

Assumptions

1. The 10% and 3% groups do not overlap (nobody both adds an add-on and downgrades).
2. No customers leave in the period; with churn the retention figure would be lower.
3. ARPU is averaged over all 10,000 customers, so it measures the base, not just those who changed. Among the add-on buyers ARPU is $50, among downgraders it is $20.

Pitfalls and judgement

• Reporting only the gross $20,000 hides $3,000 of leakage. Always show gross expansion, contraction and the net.
• A rising ARPU can come from mix (a few heavy buyers) without a broad improvement: 87 percent of customers (8,700 of 10,000) did not change, so the whole-base mean is carried by the 13 percent who did. Check the distribution as well as the mean.
• Contraction is an early warning for churn. A downgrade cohort is a good list for customer-success follow-up.

Direct answer

A strong candidate names the underlying job before jumping to metrics or a fix. Jobs-to-be-Done, or JTBD, describes what job a product is hired to do for the user, independent of any specific feature. A personalized feed's real job usually isn't "show me content," it's closer to "let me feel caught up and informed without spending time I don't have filtering through noise," and every pain, metric, and hypothesis that follows should trace back to that job.

Primary job, pains, and outcomes

Primary job: help me quickly find news that's relevant and trustworthy so I can feel informed and act on it (share, discuss, decide), without wasting time filtering out noise myself.

Three user pains: overload, where the feed surfaces too many similar or irrelevant stories and the user has to filter manually; staleness or low relevance, where stories feel generic or out of step with what the user actually cares about right now; and trust and diversity, where users worry the feed shows a narrow slice of viewpoints or content of uncertain credibility, undermining the "informed" part of the job.

Three measurable outcomes:

Those three are the outcomes you are trying to move. The third pain, trust and diversity, needs a fourth measure of a different kind: a guardrail, a number you are trying to hold flat rather than push up, because relevant-engagement rate can rise precisely by narrowing the feed. Without one, the metric set reports a win in exactly the case where the product got worse on a pain you named yourself. Two concrete, cheap options:

• Source-set breadth per user: the count of distinct sources a user engaged with over a rolling 28 days, reported as the median across users. Ship-blocking condition: it must not fall relative to the control group.
• Periodic trust survey: a single question ("does this feed show you a fair range of viewpoints?") sampled to a small slice of users each week, tracked as a trend rather than a one-off reading.

Pick one and pre-commit its threshold before the experiment starts, or it becomes something to argue about after the results are in.

Hypothesis and validation

One concrete hypothesis: adding a short, visible "why you're seeing this" line next to unfamiliar or non-followed sources will reduce the perceived-noise pain and raise relevant-engagement rate, because users are more willing to engage with an unfamiliar story once they understand why it appeared.

Before building: run a quick concept test with a clickable mockup or a manually curated stand-in for the real feature, with a handful of representative users, watching whether the explanation changes their willingness to engage with unfamiliar stories, and asking whether it reads as helpful or as an excuse.

After building: run a controlled experiment comparing the feed with and without the explanation, measuring relevant-engagement rate, return frequency, and the diversity guardrail over a few weeks, paired with a short survey on perceived trust and relevance, so you can tell whether any lift comes from genuinely better understanding rather than novelty.

Size that experiment before you run it rather than after. Detecting a 3 percentage point lift on a 22% baseline relevant-engagement rate, at the usual 80% power and 5% significance level, needs roughly 3,100 users per arm. A three-arm test (control plus two phrasings) therefore needs about 9,400 users, and a smaller test that comes back "no measurable lift" has told you nothing, since it could not have detected the effect you were looking for either way.

Worked example

Suppose the pre-build concept test with 10 to 12 users shows most, 8 of the 11 users tested, say the explanation makes them more willing to try an unfamiliar story, while a couple, 2 of the 11, say it reads as an excuse for showing them something they didn't want. That mixed signal is still useful: it tells you the wording matters as much as the feature itself, so the post-build experiment should test at least two phrasings, not just presence versus absence. Suppose that post-build experiment then runs for three weeks at about 3,100 users per arm: the better-performing phrasing lifts relevant-engagement rate from a baseline of 22% to 25% and return frequency from 2.1 to 2.3 sessions per week, the weaker phrasing shows no measurable lift on either metric, and median source-set breadth holds flat against control in both arms. That last part is what lets you ship: the engagement lift did not come from quietly narrowing the feed.

Trade-offs and pitfalls

Chasing relevant-engagement rate alone risks over-fitting the feed to what a user already likes, quietly recreating the trust and diversity pain even while the engagement number rises, which is exactly why the diversity guardrail has to be pre-committed with a threshold rather than added as a post-hoc explanation. A concept test with a handful of users can catch an obviously bad explanation but can't tell you the real effect size, so treat it as a filter before the real experiment, not a substitute for it. And return frequency is a slow-moving metric; don't expect it to move before the shorter-term engagement metric does, and don't panic if it lags behind.