User Retention & Engagement Questions

Direct answer

Frame the SMB-vs-mid-market churn gap as three or four concrete, testable hypotheses about WHY the two segments differ (not "SMBs churn more" as a single vague hypothesis), pick the metric that most directly reflects the mechanism each hypothesis proposes, size the test against the SMB segment's actual traffic (which is usually smaller than mid-market's, a real constraint), and build in risk controls specific to churn experiments, since a failed intervention on an already at-risk segment can accelerate the exact outcome you're trying to prevent.

Structured elaboration

Turning "SMBs churn more" into testable hypotheses. A senior answer names multiple, DISTINCT candidate mechanisms rather than one: (1) SMBs are more price-sensitive, so churn concentrates around renewal/billing events; (2) SMBs have less internal capacity to onboard properly, so churn concentrates in the first 30-60 days from incomplete setup; (3) SMBs' business needs change faster (staff turnover, pivoting business models), so a meaningful share of "churn" is really the customer's business itself changing rather than dissatisfaction with the product; (4) SMBs get less proactive customer-success attention than mid-market's dedicated account managers, so problems that would be caught and resolved for mid-market accounts go unaddressed for SMBs. Each of these implies a DIFFERENT intervention and a different metric to watch, which is why naming them separately matters more than naming the segment gap itself.

Key metrics, matched to each hypothesis. For (1) price sensitivity: churn rate segmented by proximity to a billing/renewal event, and price-elasticity from any past discount experiments. For (2) onboarding capacity: 30/60-day onboarding-completion rate and its correlation with 90-day retention, segmented by segment. For (3) business-change churn: cancellation-survey reason codes, split into "product dissatisfaction" versus "business no longer needs this" buckets, since these have completely different implications for what an experiment could even fix. For (4) support attention: support-ticket response time and resolution rate by segment, and whether accounts with a slower first response churn at a higher rate.

Sample sizing considerations specific to SMB. SMB accounts are typically far more numerous but smaller-value than mid-market accounts, so a churn-rate experiment on SMB usually has MORE statistical power available per dollar of ARR at risk than the equivalent mid-market experiment would, which argues for running the SMB experiment with real statistical rigor rather than skipping straight to a low-n pilot; use the standard two-proportion sample-size approach (baseline SMB churn rate, MDE the team considers meaningful, alpha and power) to size the test properly, same mechanics as any other proportion-based test, just applied to the SMB churn baseline specifically rather than assumed to be too small a population to test.

Risk controls, specific to a churn intervention on an at-risk segment. A guardrail metric on the CONTROL group's churn rate (to catch a broken control experience, not just measure the treatment); a hard stop-loss rule (if the treatment arm's churn rate is trending meaningfully WORSE than control at an early interim look, the team commits in advance to stopping rather than waiting out the full duration, since an intervention that backfires on an already fragile segment is actively harmful, unlike most feature experiments where a null or mildly negative result is just a missed opportunity); and a clear rollback plan for whatever specific change (a new onboarding flow, a pricing change, a support-tier change) is being tested, since SMB accounts that experience a broken or confusing version of the product during the test are exactly the accounts most likely to churn regardless of which arm caused it.

Worked example

Say cancellation-survey data (hypothesis 3's evidence source) shows SMB cancellations are 55% "product dissatisfaction" and 45% "business no longer needs this," while mid-market's split is 80%/20% the other way. This single breakdown reframes the whole problem: nearly half of SMB churn may not be addressable by ANY product intervention, since "the business closed" or "we pivoted" isn't something an onboarding flow or a pricing change can fix. A realistic experiment targets the addressable 55% specifically (say, an improved 30-day onboarding flow aimed at reducing early product-dissatisfaction churn), and the sample-size calculation and success metric should be scoped to that addressable population, not the full SMB churn number, or the experiment will be systematically underpowered to detect a real effect that only ever applied to part of the base.

Trade-offs and pitfalls

• Treating the segment gap as one hypothesis instead of several is the most common mistake, since it leads directly to one generic intervention (usually "improve onboarding") that may only address a fraction of the actual gap, exactly as the worked example shows.
• Sizing the experiment against the full SMB population when the true addressable population is smaller produces an experiment that looks adequately powered on paper but is underpowered for the actual effect it's trying to detect, since a chunk of the population (business-change churn) is not moveable by the intervention at all.
• Skipping the stop-loss guardrail because "it's just a test" ignores that this segment is already churning at an elevated rate; an intervention that unintentionally makes onboarding worse, tested without an early-stopping rule, can measurably accelerate revenue loss on a segment the business can least afford to lose further.

Direct answer

For each campaign exposure, check whether the user had been silent for 30+ days immediately BEFORE that specific send, and if so, whether they showed any activity within 30 days AFTER it; the reactivation rate is the distinct-user count satisfying both conditions divided by the distinct-user count eligible (silent 30+ days before some send), with a user who receives multiple campaign exposures counted once in each set regardless of how many sends they were exposed to.

Structured elaboration

Eligibility, defined per campaign send, not per user in the abstract. A user is only a valid denominator member relative to a SPECIFIC send if they had genuinely gone quiet (no events activity) for at least 30 days immediately before that send's sent_at. This has to be computed per (user, campaign) pair via a correlated lookup (MAX(occurred_at) WHERE occurred_at < sent_at), not per user overall, because a user's eligibility can differ across two different sends they received at different points in their lifecycle.

Returned, within 30 days of the specific send. For each eligible (user, send) pair, check for any event with occurred_at strictly after sent_at and within 30 days of it. This is an EXISTS check, not a join that could multiply rows.

Multiple campaign exposures, handled without double-counting. A user who receives two sends (say, because the first didn't trigger a return) can be ELIGIBLE for both sends independently, and could in principle "return" relative to either. The denominator, eligible_users, is a DISTINCT set of user_ids across all their eligible sends, so a user eligible twice still counts once. The numerator, returned, groups by user_id and requires at least one of their eligible sends to have a qualifying return, so a user who reactivates after their SECOND send (having failed to reactivate after their first) still correctly counts as one reactivated user, not zero (from the failed first attempt) or two (double-counted across both attempts).

Edge cases. A user silent 30+ days before EVERY send they received but never returning after any of them is eligible and correctly not counted as reactivated. A user who was NOT actually 30+ days silent before a given send (they'd been active recently) is excluded from eligibility for that specific send, even if they happen to receive a campaign anyway (a broad-blast campaign that also reaches recently-active users should not inflate the reactivation-rate denominator with users who were never actually dormant). A user who never received any campaign at all never appears in the calculation, by construction of starting from the campaigns table.

Worked example

CREATE TABLE users (user_id INTEGER);
CREATE TABLE events (user_id INTEGER, occurred_at TEXT, event_type TEXT);
CREATE TABLE campaigns (user_id INTEGER, sent_at TEXT);

INSERT INTO users VALUES (1),(2),(3),(4),(5);
INSERT INTO events VALUES
 (1,'2025-11-01','open'), (1,'2026-01-10','open'),           -- u1: dark ~65d, returns 5d after send: reactivated
 (2,'2025-11-01','open'),                                     -- u2: dark ~65d, never returns: NOT reactivated
 (3,'2025-11-01','open'), (3,'2026-01-20','open'),            -- u3: 2 sends, returns after the 2nd only
 (4,'2026-01-01','open'), (4,'2026-01-15','open'), (4,'2026-01-20','open'),  -- u4: never goes dark: ineligible
 (5,'2025-11-01','open'), (5,'2026-01-25','open');            -- u5: dark, but never sent a campaign: ineligible
INSERT INTO campaigns VALUES (1,'2026-01-05'), (2,'2026-01-05'), (3,'2025-12-01'), (3,'2026-01-15');

WITH last_activity_before AS (
    SELECT c.user_id, c.sent_at,
        (SELECT MAX(e.occurred_at) FROM events e WHERE e.user_id=c.user_id AND e.occurred_at<c.sent_at) AS prior_activity
    FROM campaigns c
),
eligible AS (
    SELECT user_id, sent_at FROM last_activity_before
    WHERE prior_activity IS NOT NULL
      AND CAST(julianday(sent_at)-julianday(prior_activity) AS INTEGER) >= 30
),
returned AS (
    SELECT DISTINCT el.user_id FROM eligible el
    WHERE EXISTS (
        SELECT 1 FROM events e WHERE e.user_id=el.user_id AND e.occurred_at>el.sent_at
          AND CAST(julianday(e.occurred_at)-julianday(el.sent_at) AS INTEGER) <= 30
    )
)
SELECT
    (SELECT COUNT(DISTINCT user_id) FROM eligible) AS eligible_count,
    (SELECT COUNT(*) FROM returned) AS reactivated_count,
    ROUND(1.0*(SELECT COUNT(*) FROM returned)/(SELECT COUNT(DISTINCT user_id) FROM eligible), 4) AS reactivation_rate;

Executed against SQLite. Output:

eligible_count | reactivated_count | reactivation_rate
3              | 2                 | 0.6667

Per-send diagnostic (executed separately, same dataset): u1's single send (prior_activity 2025-11-01, gap 65 days) returned=1; u2's single send (same gap) returned=0; u3's FIRST send (2025-12-01, gap exactly 30 days) returned=0 (next activity was 2026-01-20, 50 days later, outside the window), but u3's SECOND send (2026-01-15, prior_activity still 2025-11-01 since no activity happened in between) returned=1 (2026-01-20 is 5 days later). u4 and u5 never appear in eligible at all (u4 has no 30+ day gap before any send it doesn't receive; u5 has no campaign row). eligible_users = {1,2,3} = 3; returned = {1,3} = 2; rate = 2/3 = 0.6667, matching the query's own output and confirming u3 is counted exactly once despite having two campaign exposures, only one of which qualified.

Complexity

For $n$ campaign sends and $m$ events per user on average, each correlated subquery in last_activity_before and the EXISTS check in returned costs roughly $O(m)$ in the naive form shown, giving overall roughly $O(n \cdot m)$; at production scale this is rewritten with a window function (LAG/LEAD over each user's events ordered by time, or a range-JOIN against a pre-sorted events table) to let the engine use sorted/indexed access instead of a per-row correlated lookup.

Edge cases

• A user with a campaign sent but ZERO prior events at all (a brand-new user mistakenly included in an outreach list): prior_activity is NULL, correctly excluded from eligible via the IS NOT NULL filter, since "reactivation" is meaningless for a user with no prior activity to reactivate FROM.
• A user whose gap is exactly 30 days (the boundary case, u3's first send in the worked example): included in eligible under a >= 30 definition; whether the boundary is inclusive or exclusive is a definitional choice that should be stated explicitly, since it can shift edge-case users in or out of the denominator.

Trade-offs and pitfalls

• Computing eligibility per USER rather than per SEND is the most common mistake in this pattern, and it breaks exactly the multi-exposure case the question asks about: a user's dormancy status can genuinely differ across two sends spaced months apart, so eligibility has to be evaluated fresh for each send, not decided once for the user overall.
• The naive correlated-subquery form shown is not the production-scale answer; on a table with millions of campaign sends this needs the window-function rewrite noted under Complexity to avoid a slow, expensive query.
• This query measures raw reactivation, not INCREMENTAL reactivation; some fraction of the "returned" users might have come back on their own even without the campaign, and only a randomized holdout (a comparable group of eligible, equally-dormant users who received no campaign) can isolate the campaign's true causal lift over that baseline.

Direct answer

Treat "10% DAU lift" as an unverified claim until it survives a specific checklist: a proper statistical test against the right baseline (not just an eyeballed percent change), confirmation the comparison group was actually randomized or held out, a check for seasonality/day-of-week/holiday effects, a sample-size/power sanity check on BOTH groups being compared, a bot/fraud screen, and a business-sense check on whether the resulting numbers are even plausible.

Structured elaboration

Statistical tests to run. If a true randomized holdout exists (some dormant users received the email, a comparable random subset did not), a two-proportion test (or a difference-in-means test on DAU counts) comparing next-day activation rates between the two groups is the right test, since it isolates the email's effect from anything else happening that same day. If no holdout exists and the comparison is purely before/after (yesterday vs. today, or this week vs. last), the claim is much weaker by construction: any test run against a single before/after pair cannot separate the campaign's effect from every other thing that changed between those two specific days.

Checking for proper randomization or holdout. Confirm the holdout group was assigned BEFORE the campaign, not carved out after the fact from users who happened not to open the email (an after-the-fact "control" group self-selected by non-engagement is not a valid comparison, since people who don't open marketing emails differ systematically from those who do, independent of any campaign effect). A sample-ratio mismatch check (are the two groups actually close to the intended split size) is a fast, cheap way to catch a broken randomization.

Seasonality and holiday effects. Compare the "lift" against the SAME weekday in prior weeks, not a flat multi-day average, since day-of-week swings in DAU are often larger than a marketing campaign's typical effect; also check whether the campaign date coincides with any other event (a product release, a seasonal pattern, a competitor's outage) that could independently explain a DAU bump.

Sample size and power adequacy. Check power on BOTH the treatment group (was it large enough to detect a real 10% lift at reasonable confidence) AND, less obviously, the BASELINE used for comparison, since a baseline built from very few historical data points can have an artificially small variance that makes ordinary noise look like a large, significant effect.

Bot/fraud detection. Screen the "activated" users for anomalous patterns (a burst of near-identical session timestamps, activity from a narrow IP range, accounts with no other product history) that would indicate automated or fraudulent activity rather than genuine reactivation, especially relevant for any campaign involving a reward or incentive that could attract bad-faith automated claims.

Business sanity checks. Does the magnitude of the claimed lift make sense given the campaign's actual reach (a 10% DAU lift from an email sent to a small fraction of the total user base would require an implausibly high response rate); does the claimed lift persist beyond one day, or does it evaporate immediately (a one-day-only spike is a much weaker business result than a lift the team can show sustains for a week).

Worked example

Fourteen days of pre-campaign DAU with a realistic day-of-week pattern (weekdays higher, weekends lower):

import statistics

baseline_dau = [
    5000, 5200, 5100, 5300, 4900, 3800, 3600,   # week 1: Mon..Sun
    5150, 5250, 5050, 5350, 4950, 3750, 3650,   # week 2: Mon..Sun
]
weekday_labels = ["Mon","Tue","Wed","Thu","Fri","Sat","Sun"] * 2

mean_all = statistics.mean(baseline_dau)
print(f"naive 14-day baseline mean = {round(mean_all,1)}")

wednesday_baseline = [d for d, lbl in zip(baseline_dau, weekday_labels) if lbl == "Wed"]
wed_mean = statistics.mean(wednesday_baseline)
wed_sd = statistics.pstdev(wednesday_baseline)
print(f"Wednesday-only baseline: values={wednesday_baseline} mean={wed_mean} sd={wed_sd}")

observed_dau = 5610  # the campaign's "next day" (a Wednesday)
naive_pct_lift = round(100*(observed_dau-mean_all)/mean_all, 1)
print(f"naive lift vs flat 14-day mean = {naive_pct_lift}%")

z_vs_weekday = (observed_dau - wed_mean) / wed_sd
print(f"z-score vs Wednesday-only baseline = {round(z_vs_weekday,2)}")

Output (actually executed):

naive 14-day baseline mean = 4717.9
Wednesday-only baseline: values=[5100, 5050] mean=5075 sd=25.0
naive lift vs flat 14-day mean = 18.9%
z-score vs Wednesday-only baseline = 21.4

Two things this demonstrates. First, the naive flat-baseline comparison and the properly day-of-week-adjusted comparison give DIFFERENT answers (18.9% naive lift vs. a z-score of 21.4 against the Wednesday-specific baseline), so which one is "the" 10% claim matters and should be stated explicitly, not left implicit. Second, and more subtly, the Wednesday-only baseline's standard deviation (25.0) is suspiciously tiny, because it is built from only 2 historical Wednesdays; that small-sample baseline manufactures an enormous, seemingly ultra-significant z-score (21.4) that should NOT be trusted at face value, exactly the "sample size and power adequacy" checklist item applied to the comparison baseline itself, not just the treatment group. A rigorous validation would insist on a longer historical baseline (or, better, a genuine randomized holdout from the campaign itself) before accepting either number.

Trade-offs and pitfalls

• Accepting a percent-change claim without asking "against what baseline, computed how" is the single most common way an unvalidated metric claim gets through; the worked example shows the same underlying data supports two very different-looking numbers (18.9% vs. a 21-sigma effect) purely from baseline choice.
• A tiny historical baseline (as with the 2-Wednesday sample above) is dangerous precisely because it looks rigorous (it produces a z-score, a seemingly quantitative, disciplined-looking output) while actually being built on far too little data to trust its variance estimate.
• Even after every checklist item passes, a before/after comparison without a genuine randomized holdout can never fully rule out a coincidental concurrent cause; the checklist reduces risk substantially but a true experiment (or at minimum a matched-control comparison) is the only way to close that gap completely.

Direct answer

Track short-term delivery metrics (opens, clicks) only as diagnostic signals that the email itself worked mechanically, and track medium- and long-term DOWNSTREAM metrics (did the user actually return and DO something in the product, and did they stay active weeks later) as the metrics that actually define campaign success; open and click rates can rank two campaigns in the opposite order from what genuinely matters if the campaign that "performs" better on opens is optimizing for an eye-catching subject line rather than for durable value.

Structured elaboration

Why open and click rate are structurally misleading as SUCCESS metrics, not just noisy. An open or a click measures curiosity about the SUBJECT LINE or the EMAIL COPY, which is a genuinely different thing from measuring whether the underlying reactivation actually worked. A subject line optimized purely to maximize opens (urgency language, an aggressive discount headline) can systematically attract users who are curious but not genuinely re-engaging, while a duller subject line that points directly at real product value can attract fewer but more durably reactivated users. The two metrics are measuring different constructs, so a campaign can win on one and lose on the other by design, not by chance.

The short-term-to-long-term metric ladder to track instead:

1. Delivery/mechanical health (delivered, bounced, spam-complained): confirms the email actually reached inboxes; a necessary but not sufficient signal.
2. Open and click rate: confirms the SUBJECT LINE and CALL TO ACTION worked; useful for diagnosing the email itself, not for judging the campaign's business value.
3. Day-1 (or next-session) reactivation rate: did the user actually come back and do something in the product, the first real behavioral signal.
4. Day-7/day-30 retention of reactivated users: of the users who came back, how many are STILL active weeks later, the metric that actually reflects whether the campaign produced durable value rather than a one-time visit.
5. A holdout-based incremental lift on the day-30 retention number specifically (comparing the campaign's recipients against a matched or randomized holdout who received no email), since without a control group even day-30 retention of "reactivated" users can be confounded by users who would have come back anyway.

Guardrails. Track unsubscribe rate and spam-complaint rate alongside the funnel above; a campaign that maximizes opens via aggressive subject lines can simultaneously increase unsubscribes, quietly shrinking the pool of users reachable by FUTURE campaigns even if this one campaign's headline numbers look good.

Worked example

Illustrative, pinned scenario: two subject-line variants sent to 1,000 dormant users each. Variant B uses a flashier discount-focused subject line; Variant A points directly at a genuine feature update.

variant_A = {"sent": 1000, "opened": 220, "clicked": 90, "returned_day1": 150, "still_active_day30": 95}
variant_B = {"sent": 1000, "opened": 410, "clicked": 260, "returned_day1": 240, "still_active_day30": 40}

def rate(n, d): return round(100 * n / d, 1)

for name, v in [("A", variant_A), ("B", variant_B)]:
    print(f"Variant {name}: open={rate(v['opened'],v['sent'])}% click={rate(v['clicked'],v['sent'])}% "
          f"day1_reactivation={rate(v['returned_day1'],v['sent'])}% "
          f"day30_of_sent={rate(v['still_active_day30'],v['sent'])}%")

Output (actually executed):

Variant A: open=22.0% click=9.0% day1_reactivation=15.0% day30_of_sent=9.5%
Variant B: open=41.0% click=26.0% day1_reactivation=24.0% day30_of_sent=4.0%

Variant B wins decisively on every top-of-funnel metric (open rate nearly double, click rate nearly triple), and would be declared the "better" campaign by anyone reading only opens and clicks. But Variant A produces MORE than double B's day-30-retention-of-sent (9.5% vs. 4.0%): B pulled in a larger volume of curious but shallow returns that mostly did not stick, while A's smaller, more targeted response converted into durable reactivation at a much higher rate. Judging the campaign on open/click alone would have selected the objectively worse campaign by the metric that actually reflects business value.

Trade-offs and pitfalls

• The reversal shown above is not a hypothetical edge case; it is the exact failure mode this question is testing for. Any campaign metric that rewards attention-grabbing rather than genuine value (open rate, click rate, even raw day-1 reactivation count without a downstream check) can be gamed by copy that maximizes short-term curiosity at the expense of the outcome that matters.
• Downstream metrics take longer to observe, which is real friction: a team under pressure to report results quickly may default to open/click because they are available same-day, even knowing they can mislead; the fix is not to abandon them (they remain useful DIAGNOSTIC signals for the email's mechanics) but to explicitly label them as leading indicators of email quality, not proxies for campaign success.
• Without a holdout group, even the day-30 number can overstate the campaign's true incremental effect, since some fraction of "reactivated" users might have returned on their own regardless of the email; a rigorous read of a reactivation campaign needs the incremental lift over a comparable no-email control, not just the raw reactivated-user retention rate.

Direct answer

A low WAU/MAU (weekly-active-to-monthly-active) ratio means most of a product's reachable monthly base is showing up rarely, not weekly; the three buckets to check are a genuine PRODUCT problem (the product doesn't warrant weekly use, or something is discouraging return visits), an ACQUISITION problem (recent acquisition brought in a large low-intent cohort that inflates MAU without ever forming a habit), or a DATA problem (the "active" event definition or a bot/duplicate-account issue is distorting one side of the ratio but not the other).

Structured elaboration

Hypothesis 1: Product genuinely doesn't warrant weekly use, or something discourages return visits. Some products are legitimately monthly-cadence by nature (subscription billing review tools, a monthly report generator); for those, a low WAU/MAU is not a problem, it is a correct reflection of the use case, and the diagnostic is really "am I comparing this product's ratio against the right baseline" rather than "why is this broken." For products meant to be used more often, low WAU/MAU can mean users complete a task, get what they need, and don't return until the NEXT time they need that same task, i.e. shallow, transactional usage rather than habitual usage.
Diagnostic: segment DAU/WAU/MAU by FEATURE, not just overall; if one or two features drive nearly all activity and the rest of the product sits unused, that's evidence the core loop itself doesn't currently give users a reason to come back more than once in a while.

Hypothesis 2: Acquisition problem, a recent low-intent surge diluting the ratio. A large recent signup wave (a marketing campaign, a viral moment, a free-tier promotion) can add many users to the MAU denominator who were never going to become weekly users, mechanically dragging WAU/MAU down even if the PRE-EXISTING user base's habitual behavior hasn't changed at all.
Diagnostic: compute WAU/MAU segmented by cohort age (users acquired more than 90 days ago vs. users acquired in the last 30 days); if the ratio is healthy for the older cohort and low only for the newest cohort, the issue is acquisition mix and onboarding for new users, not a regression in the core product for people who have already formed a habit.

Hypothesis 3: Data issue in the "active" event definition, or a bot/duplicate-account distortion. If the event counted as "MAU-qualifying" is much broader or looser than the event counted as "WAU-qualifying" (e.g., MAU includes passive events like a push notification received, while WAU only counts an explicit in-app action), the ratio compares two structurally different definitions of activity rather than the same definition measured at two frequencies, which will produce a low ratio regardless of real user behavior. Similarly, a batch of bot accounts or test accounts that fire a single MAU-qualifying event once a month (but never a WAU-qualifying weekly action) inflates MAU without ever appearing in WAU.
Diagnostic: confirm both DAU and MAU are computed from the SAME event definition, just aggregated over different windows; then check the account list contributing to MAU-but-not-WAU for an anomalous concentration of accounts with suspicious signup patterns (same IP, same signup burst, no other product usage at all).

Worked example

A B2B analytics tool reports DAU=800, WAU=3,000, MAU=15,000, so DAU/MAU=5.3% and WAU/MAU=20%, both on the low end for a tool meant for regular weekday use. Running the three diagnostics: segmenting by feature shows 70% of WAU comes from a single "dashboard view" action while three other core features are used by fewer than 5% of MAU, supporting Hypothesis 1 (shallow, single-feature usage). Segmenting by cohort age shows users acquired more than 90 days ago have WAU/MAU=34%, meaningfully healthier than the aggregate 20%, supporting Hypothesis 2 (a recent acquisition wave, confirmed separately to be a free-trial promotion launched 3 weeks earlier, is diluting the aggregate ratio). Checking the MAU-but-not-WAU account list shows no anomalous concentration, ruling out Hypothesis 3 for this product. The combined read: the low aggregate ratio is real but partly a temporary acquisition-mix artifact layered on top of a genuine, pre-existing shallow-usage pattern concentrated in one feature.

Trade-offs and pitfalls

• Comparing WAU/MAU against a generic industry benchmark without first confirming the product's expected cadence is a common mistake; a genuinely monthly-cadence product will always look "bad" against a benchmark calibrated for daily-cadence products, and chasing that number up is chasing the wrong goal.
• Running only ONE of the three diagnostics and concluding the whole story is a real risk, since the worked example above shows two of the three hypotheses can be simultaneously true and interacting (acquisition dilution stacked on top of genuinely shallow habitual usage); segmenting by feature alone or by cohort age alone would each have told only part of the story.
• Fixing the metric definition (Hypothesis 3) is sometimes mistaken for fixing the product. If DAU and MAU turn out to be using inconsistent event definitions, correcting that will change the reported ratio without changing any real user behavior; that correction is necessary for the metric to be trustworthy, but it is not itself a retention win and should not be reported as one.