InterviewStack.io LogoInterviewStack.io

Metric Monitoring and Segmentation Questions

Focuses on continuous monitoring of product or business metrics and analyzing changes through segmentation. Core skills include instrumenting reliable tracking, building dashboards, setting up anomaly detection and alerts, and segmenting metrics by cohorts such as new versus returning users, platform, geography, customer tier, or acquisition channel. Candidates should be able to identify which segments drive overall metric changes, distinguish signal from noise, and perform root cause analysis considering feature launches, bugs, seasonality, or external factors. Familiarity with cohort analysis, funnel analysis, threshold and anomaly rules, and communicating actionable insights to stakeholders is expected.

HardTechnical
33 practiced
Compute the minimum sample size per variation needed to detect a 5% relative uplift in conversion with a baseline conversion of 10%, using 80% power and a 5% two-sided alpha. Show the formula with z-scores, intermediate calculations, and the final per-variation sample size. Discuss how required sample size changes for smaller segments and when correcting for multiple tests.
MediumTechnical
33 practiced
Given events(user_id, event_type, amount numeric, channel varchar, occurred_at date), write a SQL query to compute week-over-week revenue change by channel for the last two complete weeks and return the top 5 channels contributing to any overall revenue drop. Explain assumptions about attribution windows, null amounts, and users belonging to multiple channels.
HardSystem Design
39 practiced
How would you monitor metrics that are segmented by a high-cardinality dimension such as product_id (millions of values)? Discuss storage strategies, pre-aggregation, approximate algorithms (e.g., HyperLogLog, Count-Min), and how to surface anomalies for long-tailed segments in a meaningful way.
HardTechnical
32 practiced
A sudden discrepancy appears between totals in the event warehouse and the billing system for the same period. Describe a methodical approach to detect whether a pipeline bug introduced a systematic bias (e.g., missing country X), quantify the bias, patch the pipeline, backfill corrected data, and communicate impact to stakeholders including revenue estimates.
MediumTechnical
64 practiced
Propose a concrete algorithm combining rolling percentiles and EWMA to set dynamic alert thresholds for a metric with weekly seasonality and occasional spikes. Describe parameter choices (window lengths, smoothing alpha), how to handle missing days, and how to prevent alert flapping with transient breaches.

Unlock Full Question Bank

Get access to hundreds of Metric Monitoring and Segmentation interview questions and detailed answers.

Sign in to Continue

Join thousands of developers preparing for their dream job.