Growth and Product Metrics Analysis

Analysis skills specific to growth and product contexts: interpreting funnel metrics, cohort and retention analyses, attribution of acquisition versus activation, detecting seasonality and external event impacts, and diagnosing conversion or engagement changes. Candidates should be able to form hypotheses about what drove changes, propose targeted follow up analyses or A B tests, and identify which additional metrics are needed to evaluate unit economics and growth efficiency.

40 questions

Feature Success Measurement

Focuses on measuring the impact of a single feature or product change. Key skills include defining a primary success metric, selecting secondary and guardrail metrics to detect negative side effects, planning measurement windows that account for ramp up and stabilization, segmenting users to detect differential impacts, designing experiments or observational analyses, and creating dashboards and reports for monitoring. Also covers rollout strategies, conversion and funnel metrics related to the feature, and criteria for declaring success or rollback.

45 questions

Customer Experience and Data Driven Thinking

Covers the ability to understand and improve customer experience using quantitative and qualitative evidence. Interviewers look for candidates who analyze user behavior and funnel metrics, identify drop off points, use experiments or controlled tests to validate hypotheses, and balance data signals with user research and empathy. This topic includes awareness of data quality and measurement limitations, selecting appropriate success metrics, interpreting results responsibly, and using insights to prioritize and influence product or process changes that improve customer outcomes. Candidates should show structured thinking about measurement, trade offs when data is incomplete, and how to communicate data driven recommendations to technical and non technical stakeholders.

40 questions

AARRR Growth Framework & Metrics

Understand the AARRR model (Acquisition, Activation, Retention, Revenue, Referral) as a mental model for thinking about growth. Know how to map growth problems to specific stages: acquisition challenges differ from retention problems. Learn standard metrics for each stage (CAC, LTV, activation rate, retention cohorts, viral coefficient). Practice identifying which stage is most relevant for a given problem.

40 questions

Metric Hierarchies & Leading/Lagging Indicators

Learn the difference between lagging indicators (revenue, retention cohorts) and leading indicators (signups, feature adoption, content views). Understand that leading indicators enable faster feedback loops. Practice building metric cascades: how does North Star break down into team-level metrics? How do leading metrics predict lagging outcomes?

0 questions

Segmentation & Dimensionality in Metrics

Learn to think about how metrics vary across dimensions: user segment, geography, traffic source, device type, etc. Practice deciding which dimensions are critical to track separately. Understand why slicing metrics reveals insights (e.g., desktop vs. mobile retention may differ significantly).

40 questions

Growth Constraints and Diagnosis

Covers methods and frameworks for diagnosing why product or business growth is slowing or stalling and for identifying the highest impact constraints to address. Candidates should be able to distinguish demand side issues from product side issues and monetization or retention problems, use funnel based thinking to map conversion and drop off points, analyze acquisition channels for cost and quality, evaluate activation and engagement metrics, and quantify retention and churn drivers. Emphasis is on root cause analysis techniques such as cohort analysis, funnel decomposition, experiments and instrumentation, hypothesis driven problem solving, and prioritization of constraints by impact and effort to guide strategy. For senior and staff levels include deeper diagnostics that connect metrics to underlying causes such as go to market execution, product experience, onboarding flows, pricing models, and market size or awareness limitations.

0 questions

Feature Success and A/B Testing

How you'd measure success of a specific feature launch. Setting up experiments or A/B tests. Understanding statistical significance and sample sizes at a basic level. Interpreting results and deciding when to ship, iterate, or kill a feature.

0 questions

Experimentation Roadmap and Learning

Covers how individual experiments and tests are planned and sequenced within a broader experimentation program to generate reliable, actionable learning. Topics include hypothesis formulation, experimental design, metrics and success criteria, prioritization frameworks, resource allocation, and sequencing to enable sequential learning and knowledge accumulation. Also covers managing a portfolio of experiments, deciding when to stop or scale treatments, instrumentation and data quality considerations, handling confounders and bias, documenting results, and ensuring learnings are transferred into product and business decisions. Candidates should be able to explain trade offs between speed and statistical rigor, how to structure experiments to maximize organizational learning, and how to align experiments to strategic objectives.

0 questions

Experimentation Velocity and Iteration Mindset

Demonstrate a bias toward rapid experimentation and continuous iteration. At junior level, this means showing comfort with speed-over-perfection thinking: running small, fast experiments to learn quickly rather than lengthy planning cycles. Discuss how you prioritize learning speed, discuss experiments that 'failed' but taught you valuable lessons, and show examples of iterating rapidly based on data. Mention tools and processes that enabled experimentation velocity (e.g., running 3-4 tests per week, using no-code testing tools, rapid prototyping). Show that you view marketing as a series of controlled experiments rather than campaigns executed once.

0 questions

Funnel Analysis and Conversion Tracking

Product analytics practice focused on analyzing user journeys and measuring how well a product or website converts visitors into desired outcomes. Core skills include defining macro and micro conversions, mapping multi step user journeys, designing and instrumenting event level tracking, building and interpreting conversion funnels, calculating step by step conversion rates and drop off, and quantifying funnel leakage. Candidates should be able to segment funnels by cohort, acquisition source, channel, device, geography, or user persona; perform retention and cohort analysis; reason about time based attribution and multi path journeys; and estimate the impact of optimization levers. Practical competencies include implementing tracking, validating data quality, identifying common pitfalls such as missing events or incorrect attribution windows, and using split testing and iterative analysis to validate hypotheses. Candidates should also be able to diagnose root causes of drop off, create mental models of user behavior, run diagnostic analyses and experiments, and recommend prioritized interventions and product or experience changes with expected outcomes and measurement plans.

0 questions

Impact Driven Mindset

Approach and habits that prioritize measurable impact over activity. Topics include defining success criteria and hypotheses, using data to compare and prioritize initiatives, selecting work with the highest expected business return, balancing short term wins and long term investments, time boxing experiments and minimum viable solutions to learn quickly, and communicating impact oriented choices to stakeholders. Candidates should be ready to show examples of how they set impact goals, measured results, and redirected effort based on outcomes.

0 questions

Marketing Funnel Optimization and Experimentation

Using data and controlled experimentation to improve conversion across marketing funnels. Subjects include mapping the funnel and instrumenting events and metrics for each stage, identifying bottlenecks, framing hypotheses and prioritizing experiments by impact, designing and running A B tests and controlled experiments with appropriate sample size considerations, segment and personalization strategies, post experiment analysis including effect size and significance interpretation, attribution considerations, and implementing technical or workflow changes to capture gains. Candidates should be able to describe concrete experiments, metric definitions, and how findings influenced product or marketing decisions.

0 questions

Experimentation Methodology and Rigor

Focuses on rigorous experimental methodology and advanced testing approaches needed to produce reliable, actionable results. Topics include statistical power and minimum detectable effect trade offs, multiple hypothesis correction, sequential and interim analysis, variance reduction techniques, heterogenous treatment effects, interference and network effects, bias in online experiments, two stage or multi component testing, multivariate designs, experiment velocity versus validity trade offs, and methods to measure business impact beyond proximal metrics. Senior level discussion includes designing frameworks and practices to ensure methodological rigor across teams and examples of how to balance rapid iteration with safeguards to avoid false positives.

51 questions

A and B Test Design

Designing and running A and B tests and split tests to evaluate product and feature changes. Candidates should be able to form clear null and alternative hypotheses, select appropriate primary metrics and guardrail metrics that reflect both product goals and user safety, choose randomization and assignment strategies, and calculate sample size and test duration using power analysis and minimum detectable effect reasoning. They should understand applied statistical analysis concepts including p values confidence intervals one tailed and two tailed tests sequential monitoring and stopping rules and corrections for multiple comparisons. Practical abilities include diagnosing inconclusive or noisy experiments detecting and mitigating common biases such as peeking selection bias novelty effects seasonality instrumentation errors and network interference and deciding when experiments are appropriate versus alternative evaluation methods. Senior candidates should reason about trade offs between speed and statistical rigor plan safe rollouts and ramping define rollback plans and communicate uncertainty and business implications to technical and non technical stakeholders. For developer facing products candidates should also consider constraints such as small populations cross team effects ethical concerns and special instrumentation needs.

40 questions

A/B Testing and Optimization Methodology

Discuss your approach to designing, running, and analyzing A/B tests (randomized controlled experiments) to optimize a product or business metric. Cover experiment design fundamentals: forming a testable hypothesis, choosing the unit of randomization, selecting a primary metric plus guardrail and secondary metrics, and estimating sample size and statistical power. Explain how you interpret results (p-values, confidence intervals, statistical versus practical significance) and avoid common pitfalls (novelty effects, peeking, SUTVA violations, confounding, seasonality). Discuss how you prioritize testing opportunities and build a testing roadmap. Ground your answer with concrete examples from your own experience, whether that is testing content elements (headlines, messaging, CTAs, visual design), conversion flows (checkout, signup), pricing, or feature rollouts.

40 questions

Experimentation and Product Validation

Designing and interpreting experiments and validation strategies to test product hypotheses. Includes hypothesis formulation, experimental design, sample sizing considerations, metrics selection, interpreting results and statistical uncertainty, and avoiding common pitfalls such as peeking and multiple hypothesis testing. Also covers qualitative validation methods such as interviews and pilots, and using a mix of methods to validate product ideas before scaling.

45 questions

Hypothesis and Test Planning

End to end practice of generating clear testable hypotheses and designing experiments to validate them. Candidates should be able to structure hypotheses using if change then expected outcome because reasoning ground hypotheses in data or qualitative research and distinguish hypotheses from guesses. They should translate hypotheses into experimental variants and choose the appropriate experiment type such as A and B tests multivariate designs or staged rollouts. Core skills include defining primary and guardrail metrics that map to business goals selecting target segments and control groups calculating sample size and duration driven by statistical power and minimum detectable effect and specifying analysis plans and stopping rules. Candidates should be able to pre register plans where appropriate estimate implementation effort and expected impact specify decision rules for scaling or abandoning variants and describe iteration and follow up analyses while avoiding common pitfalls such as peeking and selection bias.

40 questions

Metrics Selection and Diagnostic Interpretation

Addresses how to choose appropriate metrics and how to interpret and diagnose metric changes. Includes selecting primary and secondary metrics for experiments and initiatives, balancing leading indicators against lagging indicators, avoiding metric gaming, and handling conflicting signals when different metrics move in different directions. Also covers anomaly detection and root cause diagnosis: given a metric change, enumerate potential causes, propose investigative steps, identify supporting diagnostic metrics or logs, design quick experiments or data queries to validate hypotheses, and recommend remedial actions. Communication of nuanced or inconclusive results to non technical stakeholders is also emphasized.

40 questions

Trade Offs Between Metrics and Guardrails

Rarely does a feature improve all metrics simultaneously. Discuss trade-offs: optimizing for engagement might reduce conversion if users spend time but don't buy. Recommend a primary metric (what you're optimizing for) and guardrails (metrics you monitor to avoid unintended consequences). For example: 'Primary metric is checkout conversion rate. Guardrails: average order value shouldn't decline, and page load time shouldn't exceed 3 seconds.' This balanced approach shows mature analytical thinking and prevents tunnel vision.

36 questions

Experiment Design and Execution

Covers end to end design and execution of experiments and A B tests, including identifying high value hypotheses, defining treatment variants and control, ensuring valid randomization, defining primary and guardrail metrics, calculating sample size and statistical power, instrumenting events, running analyses and interpreting results, and deciding on rollout or rollback. Also includes building testing infrastructure, establishing organizational best practices for experimentation, communicating learnings, and discussing both successful and failed tests and their impact on product decisions.

0 questions

Metric Frameworks and Goal Alignment

Understand how to choose, define, and apply metric frameworks that align product work to company objectives. Topics include common frameworks such as Acquisition, Activation, Retention, Revenue, Referral as well as selecting a single North Star metric that represents overall business success. Candidates should be able to define metrics at multiple levels including feature level, product level, and business level; distinguish leading indicators from lagging indicators and explain how leading metrics predict lagging outcomes; decompose a North Star into measurable submetrics and team level signals that teams can influence directly; set measurable targets and success criteria; and explain why a given metric is the most appropriate North Star for a particular business model. Practice scenarios include choosing metrics for feature launches, improving conversion or retention, reducing friction in checkout flows, and increasing engagement or virality, and describing how those metrics map to business outcomes and Objectives and Key Results.

40 questions

Experimentation Strategy and Advanced Designs

When and how to use advanced experimental methods, and how to prioritize experiments to maximize learning and business impact. Candidates should understand factorial and multivariate designs, interaction effects, blocking and stratification, sequential testing and adaptive designs, and the trade-offs between running many factors at once versus sequential A/B tests in terms of speed, power, and interpretability. The topic includes Bayesian and frequentist analysis choices, techniques for detecting heterogeneous treatment effects, and methods to control for multiple comparisons. At the strategy level, candidates should be able to estimate expected impact, effort, confidence, and reach for proposed experiments, apply prioritization frameworks to select experiments, and reason about parallelization limits, resource constraints, tooling, and monitoring. Candidates should also be able to communicate complex experimental results, recommend staged follow-ups, and design experiments to answer higher-order questions about interactions and heterogeneity.

40 questions

User Retention and Engagement

Comprehensive coverage of strategies and tactics used to retain and reengage users or customers, deepen engagement, and build healthy communities that drive long term value. Topics include diagnosing the root causes of churn through cohort analysis and retention curve analysis, defining and tracking core metrics such as churn rate, retention rate at key intervals, reactivation rate, cohort lifetime value, and engagement metrics including daily active users and monthly active users. Candidates should be able to identify at risk segments using behavioral segmentation and propensity modeling, prioritize levers, and design targeted reengagement and lifecycle campaigns such as email sequences, win back offers, incentives for lapsed users, referral and loyalty programs, content recommendation, and personalized messaging and notifications. Product levers include onboarding and activation flow optimizations, habit forming engagement loops, recommendation systems, and community activation programs including events, moderation, governance, and community health monitoring. Candidates should also demonstrate experiment design and iterative A B testing, proper instrumentation and analytics, cross functional collaboration with engineering, design, and marketing, and the ability to measure and interpret both short term campaign metrics such as open and click rates and longer term outcomes such as retention curves and changes in lifetime value. Interviewers may probe segmentation and personalization strategies, prioritization frameworks, trade offs between acquisition and retention, and examples of optimizations and their measurable impact.

0 questions

Growth & Business Optimization Topics

Growth and Product Metrics Analysis

Feature Success Measurement

Customer Experience and Data Driven Thinking

AARRR Growth Framework & Metrics

Metric Hierarchies & Leading/Lagging Indicators

Segmentation & Dimensionality in Metrics

Growth Constraints and Diagnosis

Feature Success and A/B Testing

Experimentation Roadmap and Learning

Experimentation Velocity and Iteration Mindset

Funnel Analysis and Conversion Tracking

Impact Driven Mindset

Marketing Funnel Optimization and Experimentation

Experimentation Methodology and Rigor

A and B Test Design

A/B Testing and Optimization Methodology

Experimentation and Product Validation

Hypothesis and Test Planning

Metrics Selection and Diagnostic Interpretation

Trade Offs Between Metrics and Guardrails

Experiment Design and Execution

Metric Frameworks and Goal Alignment

Experimentation Strategy and Advanced Designs

User Retention and Engagement