Growth, Activation and Retention Questions

Direct answer

Seven segments below, each with a hypothesis and one success metric. The method: choose segments by behaviour first (it predicts value and is actionable), add demographic or geographic cuts only where the hypothesis needs them, and rank them by size x expected lift x ease before testing. Every experiment uses a randomised holdout within the segment (a random share of that segment who do not get the change, kept as the comparison group).

Segments, hypotheses, and metrics (consumer app, illustrative)

Count check: seven segments, each with one hypothesis and one metric.

How to prioritise

Score each: (segment size) x (plausible lift) x (value per converted user) / effort. Illustrative arithmetic with 100,000 new users: segment 1 is 40% of them (40,000 users) with a plausible lift of 3 points, so 40,000 x 0.03 = 1,200 extra converted users, each worth 20 USD of first-year gross profit, or 24,000 USD. Segment 3 is 2% (2,000 users) with a lift of 5 points: 2,000 x 0.05 = 100 extra users, each worth 200 USD (a power user who upgrades), or 20,000 USD. In users, segment 1 wins 1,200 to 100; in value they are close (24,000 against 20,000). Now divide by effort: if segment 1 takes 6 engineer-weeks and segment 3 takes 3, value per week is 4,000 against 6,667, so segment 3 ranks first. Compare in expected paying or retained users and their value, not in percentage points. Run the top two or three first, and use low-effort tests to learn about the others.

Experiment hygiene

• Randomise within a segment into treatment and control; do not compare segment to segment.
• Fix the metric, the minimum detectable effect (the smallest lift the test is sized to detect reliably, for example 2 points) and the duration in advance; run at least one full weekly cycle.
• Watch for overlap: a user in segments 1 and 4 should be assigned to only one test at a time.
• Guardrails (metrics that must not get worse while the target metric improves): notification opt-out rate, uninstall rate, crash rate.

Trade-offs and pitfalls

• Demographic targeting can be ethically and legally sensitive (minors, protected attributes, regional privacy rules); use behaviour where it works equally well, and take advice from legal and privacy teams before targeting by age.
• Small segments produce noisy results; if a test would take six months to reach significance (a result clear enough that chance alone is an unlikely explanation), pool segments or choose a larger effect. Pooling means testing two similar small segments together as one group to get enough users, at the price of a vaguer hypothesis, because the measured lift is an average over both.
• Do not overfit: a segment that exists only in this quarter's data is not worth a standing programme.

Direct answer

Expansion nudges work when they fire on a behavioural signal that the user is already getting value and is bumping into a limit, not on a calendar. I would build three: a capacity nudge (the workspace is nearly out of seats or quota), a collaboration nudge (a user keeps working with people who are not in the plan), and a capability nudge (a user repeatedly reaches for a feature the current plan does not include). Each is shown to the person who can act on it, capped in frequency, and measured against a holdout group (users deliberately shown nothing) so we know it created upgrades instead of just claiming credit for ones that would have happened anyway.

(Terms: a seat is one licensed user on an account. PLG, product-led growth, means the product itself drives acquisition and upgrades. An A/B test randomly splits users into groups that see different versions.)

The three nudges

Avoiding pestering users who are not ready

1. Eligibility before triggers. Only accounts that have reached the product's activation point (the first valuable action, for example creating and sharing a first report) are eligible. A user in week one is still learning; an upgrade prompt is noise.
2. Respect who decides. A prompt that names a price shown to someone without purchasing power creates frustration. Route members to a request flow instead of a paywall.
3. Frequency caps and memory. At most one expansion prompt per account per 14 days. A dismissal suppresses that nudge for 30 days; a second dismissal suppresses it for 90 days. Never show a nudge during an open support ticket or in the session after an error.
4. Show value before price. Each nudge leads with the user's own data or the specific blocked action, and the price is secondary.
5. Easy exits. One-click dismiss, and a "do not show again" setting that is respected.

Measuring that it works (illustrative numbers)

Hold out 10% of eligible accounts from each nudge. Primary metric: upgrade or seat-add rate within 30 days of eligibility. Guardrails (metrics that must not get worse): 28-day retention, dismiss rate, support tickets mentioning billing, and downgrade rate within 60 days (an upgrade that reverses is a bad nudge). If 1,000 eligible accounts are split into 900 treated and 100 held out (the 10% holdout), and 81 treated accounts expand (9%) versus 5 held-out accounts (5%), the lift is 4 percentage points, not 9: the other 5 points would have happened anyway. The seat-add revenue from only that 4-point lift is what to count.

Trade-offs and pitfalls

• Timing beats copy. Most of the lift comes from the trigger being right (the moment the user hits the wall), not from clever wording.
• Over-nudging teaches blindness. Banner fatigue lowers click rates on every later message, including important ones.
• Gaming the trigger. If the seat-cap prompt fires at 80% usage, some admins will hoard seats; test 60%, 80% and 100% thresholds.
• What would change my call: if the dismiss rate climbs above the conversion rate or 28-day retention drops in the treated group, I would drop that nudge to a passive, settings-page placement.

Direct answer

I would first decide who is worth chasing, then segment by why they left, send different messages through the channels they already accept, make a narrow and conditional offer rather than a blanket discount, and measure the programme against a randomised holdout: a random slice of lapsed customers who receive nothing, so you can see how many return anyway. It "paid for itself" if the incremental contribution (profit after variable costs) from customers who came back only because of the programme exceeds its cost.

1. Who is worth chasing (re-qualification)

Drop or deprioritise before any message goes out:

• Customers who left for reasons we will not reverse: payment fraud, abuse, a chargeback (a payment dispute raised with the bank), or a company that closed.
• Anyone who unsubscribed or has no consent for the channel.
• Customers who never reached value (never activated). They may be better served by an onboarding offer than a win-back.
• Accounts whose cost to serve (support, infrastructure and payment fees) exceeded their contribution.

Then rank the remainder by prior value (tenure, usage depth, plan) and recency of lapse.

2. Segments by reason for leaving

Example messages, three segments:

• Price: "Last spring you paused your plan. We now have a plan at $9/month for the features you used most. Your saved projects are still here."
• Dormant: "Your workspace has been quiet since March. Here is a five-minute setup for the reports most teams start with, and your data is untouched."
• Missing feature: "You asked us for calendar sync. It shipped on 12 August. Reconnect and see your old schedules inside it."

Channels: email first (consent-based), in-app message the moment they log in, push only if still opted in, and paid retargeting audiences (lists of past customers uploaded to an ad platform so they see your ads) if the platform and privacy rules allow, with a suppression list (people to exclude, such as those who have already returned) so they do not see ads. Cap contact at three touches over three weeks.

3. Offers: what I would and would not do

• Would: a one-time, time-limited offer for the price segment only; a free re-activation that restores saved data; a downgrade or pause instead of a goodbye.
• Would not: a blanket discount to everyone, an offer to the missing-feature segment (it does not address their reason, and it gives away margin), or a deep discount without an expiry. Offering a discount teaches customers that cancelling earns one (the discount-precedent risk: next quarter's active customers learn to cancel and wait). Keep the offer rule public to the team, one per customer per 12 months.

4. Did it pay for itself

The discount goes only to the price segment, so I would measure that segment directly (the other segments get no discount and cost only send time; they are read the same way against their own holdouts). Hold out a random 10% of the eligible price-segment list who get nothing. Illustrative numbers: 10,000 eligible, 9,000 treated and 1,000 held out. Reactivation in 30 days: 4.0% treated, 1.5% held out. Contribution per reactivated customer: $12 monthly price x 80% gross margin (the share of revenue left after the direct cost of serving the customer; take the real figure from finance) x 5 months retained (an assumption here; in practice take it from how long previously reactivated customers stayed) = $48. The offer is one month at 50% off ($6 given up per reactivator). Fixed programme cost (design, send, team time): $1,500.

treated, rate_t, rate_h = 9000, 0.04, 0.015
reactivated = treated * rate_t            # 360
would_return = treated * rate_h           # 135 returned anyway
incremental = reactivated - would_return  # 225
value_each = 12 * 0.80 * 5                # 48 dollars contribution
discount_cost = reactivated * 6           # discount goes to ALL 360 who return
cost = discount_cost + 1500
print(reactivated, would_return, incremental)
print(round(incremental * value_each), round(cost), round(incremental * value_each - cost))

This prints 360.0 135.0 225.0 and 10800 3660 7140: 225 incremental customers worth $10,800, against $3,660 of cost, for a net of +$7,140. Note the discount cost covers all 360, including the 135 who would have returned anyway: that is the real price of an untargeted offer, and why I restrict it. The result depends most on retention months per reactivator (5 here): at 2 months the value falls to $19.2 per customer and incremental contribution to $4,320, which is only $660 above the $3,660 cost. So track 90-day survival of the reactivated, not just the reactivation.

Pitfalls

• Counting every return as programme credit. The holdout shows 135 would have returned anyway.
• Treating a day-30 return as success when many reactivated customers churn again in a month.
• Messaging someone who left over a billing dispute with a "we miss you" email.

Direct answer

I would treat the survey as a map of hypotheses, not a verdict: weight each reason by the revenue behind it, test it against what the same customers actually did in the product before leaving, split each reason into a fixable cause, size the fix against its cost, and sequence the plan by recoverable revenue per unit of effort. I would doubt the answers where they are self-reported, incomplete, or hide the real cause: "no return on investment" often means the customer never became successful with the product, and "price" is often a polite way to say "not worth it".

Step 1: weight by money, not by headcount

Illustrative figures: 300 churned customers. The stated reasons are lack of return on investment 33% (99 customers), product fit 25% (75), support problems 20% (60), and price or competition 22% (66). Counts add to 300. Now attach average monthly recurring revenue (MRR, in USD) per customer, which differs by reason: say 400, 250, 150 and 200.

Headcount says return on investment is a third; revenue says it is about half, and support drops from 20% to about 11%. The ranking can change with revenue weights, so check it.

Step 2: validate each reason with behaviour

For each reason, find the product evidence you would expect, using the 30 to 90 days of usage before they left:

• "No return on investment": did these customers reach the first valuable outcome? If most never activated, the real problem is onboarding, not the product's value.
• "Product fit": which missing features or use cases do they mention, and are they a distinct segment that sales should not have signed?
• "Support problems": ticket counts, time to resolution, and whether the issue was a bug (engineering) or a how-to question (documentation).
• "Price": did usage collapse before the price complaint, in which case price is a symptom? Compare with customers of similar usage who stayed at the same price.

If a cluster of same-industry customers churned within one quarter, run a structured root-cause analysis (RCA): timeline of changes, what they share (a regulation, a competitor entering, an integration that broke), then the same comparison against retained customers in that industry. Example (illustrative): 12 dental clinics churn in one quarter; the timeline shows all 12 used one scheduling integration that broke in March, while retained clinics used a different one. That points to a fixable integration defect, not to price or fit.

Step 3: turn it into a prioritised plan

For each cause that survives validation, estimate recoverable MRR (lost MRR x share plausibly fixable), cost and time to fix, and confidence. Illustrative inputs and ordering (fixable share, confidence and effort are assumptions to replace with your own):

So: first, the onboarding programme (largest dollar share); second, qualification changes for fit; third, support quality and documentation fixes; price tests last, because discounting costs margin and should follow the evidence on value. Each item gets an owner and a measure (for example early-life churn, ticket resolution time) and a holdout to prove the effect.

What would make me doubt these answers

• Response bias: exit surveys are answered by a minority, usually the most vocal. Compare responders with non-responders on size, plan and usage.
• Social desirability: customers pick safe reasons ("price", "budget") rather than "we never got it working".
• Single choice forced: real churn has several causes; a drop-down hides the combination.
• Timing and recall: reasons given months later are rationalised.
• Coding noise: two people may categorise the same free-text comment differently; check inter-rater agreement by having two people code the same 50 comments independently and measuring how often they choose the same category.
• Mismatch with behaviour: the strongest warning sign is a stated reason that the logs contradict.

Pitfalls

Prioritising by the biggest percentage rather than the biggest recoverable revenue; offering discounts to customers who say "price" without checking usage; building a plan from the survey alone.

Direct answer

I would treat the eight weeks as three overlapping channels, each with a main window: in-product messaging leads, at the moment of need (main push in weeks 1 to 3, then lighter prompts for non-adopters through week 8); email follows (a launch email in week 1, waves aimed at people who have not yet tried the feature in weeks 2 to 6, a final reminder in weeks 7 to 8); and human outreach goes only to a short list of high-value accounts that stalled (identify them in weeks 2 to 3, contact them from week 4 to week 8). I would judge each channel's contribution with randomised holdouts (a randomly chosen group deliberately left out of a channel, as a comparison), not last-click attribution, which credits whichever message came last.

The plan

Principles: send nothing to users who have already adopted; suppress emails for anyone who dismissed the in-product prompt twice; and cap total contact per user so the three channels do not stack on the same day. Human time is the scarce resource, so it goes to accounts where an adopter is worth the most and where there is a signal of intent (started but did not finish).

Example wording

• In-product, at the moment of need: "Sending this report to your team every Monday? Schedule it in two clicks." [Schedule it] [Not now]
• Email, non-adopters: subject "Stop re-sending the weekly report by hand". Body: "Teams that schedule their report save about 20 minutes each Monday (an illustrative claim: use a figure measured from your own users). Here is how in three steps: [link]."
• Human outreach, a stalled high-value account: "Hi Priya, I saw your team set up scheduling last week but not finish it. Can I do 15 minutes with you on Thursday and get it running on your real report?"

Judging each channel's contribution

Last-touch attribution would credit the email that was open when a user happened to try the feature, even if they were going to anyway. Use holdouts: the group left out shows what happens without the channel, and incremental adopters are the extra adopters the channel caused, the gap between the groups. Illustrative figures:

• Email: 20,000 eligible users; 2,000 randomly held out from email. Adoption by week 8 is 6.0% in the holdout and 7.5% among the 18,000 who got email. Lift is 1.5 points, so incremental adopters from email are 18,000 x 0.015 = 270.
• Human outreach: from a list of 200 qualifying accounts, randomly assign 100 to outreach and 100 not. Suppose 38 of the outreach accounts adopt and 24 of the others (38% versus 24%): 14 incremental adopters. At 45 minutes each, 100 accounts take 75 hours, so about 5.4 hours per incremental adopter. Whether that is worthwhile depends on the value of an adopter; for a premium feature with a recurring price it may be.
• In-product: keep a small persistent holdout (a group that sees no in-product prompts for all eight weeks, for example 5%); the difference is the in-product contribution, which usually carries most of the volume. Same 20,000 eligible users: 1,000 are held out of prompts and 19,000 see them. By week 8, 2.0% of the holdout have adopted (20 people, who still got email and any outreach) against 7.5% of the prompted group (1,425 people). The lift is 5.5 points, so incremental adopters from in-product are 19,000 x 0.055 = 1,045. That is about four times the email figure of 270 and dwarfs the 14 from outreach, which is why in-product usually carries most of the volume.

Because channels overlap, the holdouts measure each channel's lift on top of the others, so they will not sum exactly to the total. For example, someone who saw a prompt and then an email may have adopted because of the two together, and each holdout credits that person to its own channel. Say so in the report instead of forcing them to.

Reporting

Weekly: eligible, exposed, tried, adopted and repeated, by channel and by segment; plus guardrails (unsubscribe rate, support tickets, any increase in cancellations from over-contact). At week 8: incremental adopters and cost per incremental adopter by channel, and what changes for the next launch.

Pitfalls

• Letting human outreach go to the loudest accounts rather than the highest-value stalled ones.
• Launching all channels at once so none can be isolated; the plan above staggers them (in-product prompts start in week 1, human outreach waits until week 4, and email after the launch message goes only to non-adopters), so each channel's holdout can show its own lift.
• Counting a single use as adoption. Track repeat use at day 14 or 30 as the adoption standard.