Metric Frameworks and Goal Alignment Questions

Understand how to choose, define, and apply metric frameworks that align product work to company objectives. Topics include common frameworks such as Acquisition, Activation, Retention, Revenue, Referral as well as selecting a single North Star metric that represents overall business success. Candidates should be able to define metrics at multiple levels including feature level, product level, and business level; distinguish leading indicators from lagging indicators and explain how leading metrics predict lagging outcomes; decompose a North Star into measurable submetrics and team level signals that teams can influence directly; set measurable targets and success criteria; and explain why a given metric is the most appropriate North Star for a particular business model. Practice scenarios include choosing metrics for feature launches, improving conversion or retention, reducing friction in checkout flows, and increasing engagement or virality, and describing how those metrics map to business outcomes and Objectives and Key Results.

MediumTechnical

53 practiced

Attribution question (medium): Compare last-click, first-click, and multi-touch attribution models for allocating marketing spend. For a two-sided marketplace with long conversion windows, recommend an approach and justify it from a metrics and business perspective.

Sample Answer

Situation: We need to choose how to credit marketing touchpoints for a two-sided marketplace with long conversion windows so spend is allocated to channels that truly drive growth.

Recommendation (short): Use a data-driven multi-touch attribution (MTA) model — specifically a time-decay or Markov-chain/algorithmic attribution — combined with regular incrementality/experimentation to validate and calibrate results.

Why not first- or last-click:- First-click: Overweights upper-funnel channels (awareness) and ignores later conversion drivers (retargeting, pricing).- Last-click: Overweights last-touch (ads, email), undercounts branding and earlier discovery; especially misleading with long conversion windows where the last touch may be a minor nudge.

Why multi-touch:- Distributes credit across the buyer’s full journey, capturing both discovery and conversion assists.- Time-decay or data-driven Markov models reflect that earlier touches matter but closer touches often have higher marginal influence.- For long windows, incorporate survival analysis / decayed weights by time since touch so stale exposures get less credit.

Metrics & business alignment:- Optimize on incremental GMV and contributor-level margin, not just clicks or attributed conversions. Use LTV:CAC by channel and cohort to capture long-term value on both sides of the marketplace (buyers and suppliers).- Track channel-level incremental ROAS from experiments (geo, holdout) to validate MTA outputs.

Operational steps:1. Instrument event-level touch and conversion data, unify IDs across devices/sessions.2. Build attribution pipeline: compute paths, apply time-decay or Markov attribution, produce channel LTV, CAC, and margin.3. Run periodic holdout experiments to measure incrementality and recalibrate model.4. Report channel KPIs (incremental GMV, LTV:CAC, payback period) to finance/marketing for budget allocation.

Trade-offs:- Algorithmic MTA is more complex and needs clean data; experiments are costly but essential to avoid misallocation.- Combine deterministic matching where available (user IDs) with probabilistic modeling.

This approach balances accurate credit assignment for long funnels while grounding decisions in incrementality and business impact.

MediumTechnical

52 practiced

Experiment design (medium): Explain the concept of Minimum Detectable Effect (MDE) and outline how to compute a sample size for an A/B test when baseline conversion is 5%, desired power is 80%, and significance is 5%. You do not need to calculate numbers precisely but describe the formula components and assumptions.

EasyTechnical

33 practiced

Explain DAU/MAU ratio: what it measures, how to compute it, and what typical pitfalls are when interpreting it. Provide one example where a high DAU/MAU ratio could be misleading.

Sample Answer

DAU/MAU (Daily Active Users / Monthly Active Users) measures short-term engagement as the fraction of monthly users who use the product on an average day. It’s a simple proxy for “stickiness” or how often monthly users return.

How to compute:- DAU = count of unique users active on a given day.- MAU = count of unique users active in the preceding 28–31 days (commonly 30).- DAU/MAU = DAU ÷ MAU (express as a decimal or percent).

SQL example:

sql

-- DAU on 2025-11-01 and MAU for Oct 2025
WITH dau AS (
  SELECT COUNT(DISTINCT user_id) AS dau FROM events WHERE event_date = '2025-11-01'
),
mau AS (
  SELECT COUNT(DISTINCT user_id) AS mau FROM events WHERE event_date BETWEEN '2025-10-02' AND '2025-11-01'
)
SELECT dau, mau, dau::float / mau AS dau_mau_ratio FROM dau, mau;

Pitfalls when interpreting:- Window mismatch: inconsistent MAU window (28 vs 30 vs 31 days) changes the ratio.- Bots, test accounts, or shared accounts inflate DAU or MAU.- Cohort mixing: comparing different user cohorts (e.g., a newly acquired cohort vs. organic) can mislead.- Product cadence: apps used weekly (finance, travel) naturally have low DAU/MAU but strong engagement.- Sampling/definition differences across reports — ensure consistent active-user definition.

Example of misleading high DAU/MAU:A productivity app forces daily check-ins via a mandatory daily prompt. Many users open the app once to dismiss the prompt — DAU rises but real engagement (time spent, features used, retention) is low. The high DAU/MAU masks superficial activity; combine with other metrics (session length, retention cohort curves, task completions) before concluding success.

Actionable advice:Always standardize definitions, filter non-human users, segment by cohort/product usage pattern, and triangulate DAU/MAU with retention and qualitative signals.

HardTechnical

32 practiced

Perverse incentives (hard): Describe three ways metric definitions can create perverse incentives when teams optimize them, and propose guardrail mechanisms (definition changes, monitoring, incentives) to prevent gaming. Use concrete product examples.

Sample Answer

Situation/Context (brief): As a data analyst advising product and ops teams, I often see well-intentioned metrics create perverse incentives when teams optimize narrowly. Below are three concrete examples and practical guardrails (definition changes, monitoring, incentives).

1) Metric: Daily Active Users (DAU) for a social appPerverse incentive: Team drives trivial “engagement” (push notifications, auto-refresh, clickbait) to boost DAU without increasing meaningful retention or value.Guardrails:- Definition: switch to “meaningful DAU” (actions with intent, e.g., post/comment/share) or weight actions by value.- Monitoring: add complementary metrics — 7/30-day retention, session quality score, user satisfaction surveys — and alert on divergence (DAU up while retention down).- Incentives: tie rewards to composite KPI (weighted score of meaningful DAU + retention) and require A/B test evidence showing long-term uplift before productizing growth hacks.

2) Metric: Checkout Conversion Rate for e‑commercePerverse incentive: Product reduces form fields or hides fees to inflate conversion but increases returns, disputes, or churn.Guardrails:- Definition: measure “net conversion” (completed, non-refunded purchases after 30 days) and include return rate in the denominator.- Monitoring: funnel health dashboard with refund rate, chargebacks, LTV of cohorts who converted after changes; set automated anomaly detection.- Incentives: tie compensation to long-term revenue per cohort (3–6 month LTV) rather than immediate conversion.

3) Metric: Customer Support Average Handle Time (AHT) for SaaSPerverse incentive: Agents rush calls or close tickets prematurely to hit AHT targets, degrading satisfaction.Guardrails:- Definition: pair AHT with “first-contact resolution” and CSAT; prefer “effective handle time” that weights resolved tickets higher.- Monitoring: random QA audits of closed tickets, NPS/CSAT trend monitoring, and system flags for re-opened tickets.- Incentives: performance scorecards that penalize reopens and low CSAT, reward quality+efficiency trade-offs.

Conclusion: As an analyst, I recommend always defining metrics to reflect long-term value, instrumenting orthogonal health metrics for monitoring, and designing incentive structures that reward holistic outcomes — not just single-number improvements.

EasyTechnical

29 practiced

Briefly describe what cohort analysis is and why it is useful for measuring retention. Given a table of user events, what basic SQL approach would you take to compute a 30-day retention cohort (users who returned on day 30 after signup)? You do not need to write full SQL, but outline the steps and key functions you would use.

Unlock Full Question Bank

Get access to hundreds of Metric Frameworks and Goal Alignment interview questions and detailed answers.

Join thousands of developers preparing for their dream job.