OKR and Metric Definition Questions

Covers translating strategic objectives into measurable key results and operational metrics that drive the right behaviors. Topics include writing clear objectives and specific, measurable key results, distinguishing leading and lagging indicators, defining primary metrics versus guardrail metrics, selecting absolute versus relative targets, avoiding perverse incentives, and ensuring metric hygiene through reliable instrumentation and data quality checks. Also addresses setting appropriate targets and time horizons, monitoring cadence, dashboard design and alerting for metric deviations, and using metrics to inform prioritization and continuous improvement without encouraging gaming.

EasyTechnical

74 practiced

Describe the difference between leading and lagging indicators and provide three concrete examples of each for an e-commerce product. For one leading indicator you choose, explain how you would validate that it actually predicts the lagging business outcome (for example, purchases).

MediumTechnical

134 practiced

You wake up to an alert: one key metric dropped 18% overnight. Outline your triage checklist as a Data Analyst to determine whether the issue is instrumentation, ETL/data pipeline, a product regression, or a real business change. Provide proposed quick checks you would run and the expected evidence each check would produce.

Sample Answer

First, stay calm and follow a prioritized triage checklist: verify, localize, test, escalate. For each step I list a quick check and what evidence would point to instrumentation, ETL, product regression, or real business change.

1) Verify the alert- Quick check: open dashboard and rerun the metric for the affected window and previous windows.- Evidence: If dashboard query returns same drop → real signal. If dashboard shows normal values → alerting/monitoring rule false positive (instrumentation/alerting).

2) Confirm data freshness & ETL health- Quick check: check pipeline/job status (Airflow/DBT logs) and row counts in raw/staging/final tables (SELECT COUNT(*) WHERE date = ...).- Evidence: failed/restarted jobs, missing rows, delayed timestamps → ETL issue.

3) Compare to upstream sources / raw events- Quick check: query raw events (ingest table) for event counts by hour/day and compare to previous period.- Evidence: raw events drop → likely upstream product/traffic change; if raw events intact but aggregated metric wrong → aggregation bug (ETL/transform).

4) Validate instrumentation- Quick check: sample recent events for required fields (SELECT ... WHERE event_time > X AND (field IS NULL OR field = '')) and check SDK/agent versions or recent deploys to analytics lib.- Evidence: missing/NULL fields or sudden change in event schema → instrumentation break.

5) Segment and sanity-check cohorts- Quick check: split metric by dimension (country, platform, browser, user cohort) and run percentage change per segment.- Evidence: drop limited to one platform/region → product regression or localized outage; uniform drop across all segments → business-level change.

6) Check recent deploys & incidents- Quick check: review deployment logs, release notes, and incident Slack/ops pages for releases around the timestamp.- Evidence: matching deploy + failing tests or new feature affecting metric → product regression.

7) External / business context- Quick check: check marketing campaigns, pricing changes, promotions, holidays, competitor news, or payment gateway outages.- Evidence: correlated campaign end or external outage → real business change.

8) Quick sanity analytics- Quick check: compute leading indicators (traffic, new users, conversion funnel head counts) with simple SQL aggregations.- Evidence: traffic drop → business change; conversion step drop with traffic stable → product/UX regression.

9) Reproduce and isolate with smaller queries- Quick check: run the exact aggregation SQL used by the metric on raw data and compare intermediate aggregates.- Evidence: mismatch between raw/aggregated and dashboard → transform bug.

10) Communicate & next steps- Quick check: summarize findings (what checked, evidence) and flag severity; open incident ticket and notify engineers/product if ETL/instrumentation/product issue; if business change, prepare deeper analysis.- Evidence: clear ownership assigned; triage path (rollback, patch, further analysis) defined.

Typical quick SQL examples:- Row counts: SELECT event_date, COUNT(*) FROM raw_events WHERE event_date >= '2025-11-19' GROUP BY event_date;- Field health: SELECT event_date, COUNT(*) FILTER (WHERE important_field IS NULL) FROM raw_events GROUP BY event_date;- Segment delta: SELECT country, 100.0 * (metric_today - metric_yesterday)/metric_yesterday FROM metrics_by_country WHERE date IN (...);

Expected mapping summary:- Instrumentation: missing fields, SDK changes, event schema drift- ETL: failed jobs, missing partition/day, aggregation errors- Product regression: correlated deploys, segment-limited drops, funnel stage failures- Real business change: traffic/unique users down, external events, campaign timing

This checklist gives a fast path to isolate root cause and identify who to involve and which remediation (fix pipeline, rollback deploy, or deeper business analysis) is required.

MediumTechnical

66 practiced

Describe how frequently OKRs and their underlying metrics should be reviewed at company, team, and individual levels. Provide a cadence recommendation (daily, weekly, monthly, quarterly) and explain which metrics belong at each cadence and why.

MediumSystem Design

73 practiced

Design a concise weekly executive OKR dashboard for a product organization that must show 5 top Key Results, trends, and alerts. Describe the visual elements, the data sources and refresh cadence, top-line interactions for executives, and an alerting policy for when a KR deviates materially.

Sample Answer

Requirements & scope: a single-page weekly executive OKR dashboard that surfaces the top 5 KRs, their current status vs target, recent trend, and immediate alerts with one-click drilldowns and commentary.

Visual elements (concise, top-to-bottom):- Header: week range, last refresh timestamp, owner contacts.- KPI row (5 compact cards): KR name, current value, target, % to goal, RAG pill (green/amber/red), delta vs prior week.- Trend row: small sparklines under each KR showing 12-week trend + slope indicator and % weekly change.- Trend chart (center): stacked area or line showing leading indicators (2–3 drivers) aligned to the most critical KR selected.- Alerts & notes panel: active alerts, root-cause summary, owner, action items.- Interaction: click KR card to open modal with time-series, cohort breakdown, top contributing segments, and raw table export.

Data sources & refresh cadence:- Primary: warehouse OLAP tables (product events, revenue, adoption_metrics) queried via SQL.- Secondary: CRM/CS for account outcomes, finance for revenue targets.- ETL: nightly batch (Airflow) + incremental streaming for high-velocity metrics.- Dashboard refresh: full refresh weekly with near-real-time (15–60 min) for KPI tiles; timestamp visible.

Data pipeline & quality:- Pre-aggregation in warehouse (materialized views) for performance.- Automated data quality checks (row counts, nulls, anomaly checks) run in ETL; failure flags prevent refresh and surface in header.

Top-line executive interactions:- One-click “What changed?” shows top 3 contributors to delta (by segment).- “Simulate impact” slider to model % improvement on a leading metric and projected KR change.- Subscribe/unsubscribe to alerts and add notes/action items that persist.

Alerting policy:- Two-tier policy: 1. Soft alert: deviation >10% vs expected weekly trajectory or one-week drop beyond 2x historical STD — notify KR owner and product lead via email/Slack digest. 2. Critical alert: deviation >25% or sustained 3-week decline — page exec sponsor, create incident ticket, require 24‑48h remediation plan.- Smart rules: use seasonally-adjusted time-series anomaly detection (Prophet/ARIMA or EWMA) to avoid false positives; incorporate business calendar (promos, releases).- Each alert includes context: baseline, driver breakdown, recommended next steps, and link to drilldown.

Metrics to display on each alert: absolute gap to target, velocity (week-over-week), contribution by segment, confidence score.

MediumTechnical

123 practiced

List a comprehensive metric hygiene checklist that you would apply to a new critical metric before it is published in dashboards or used for OKRs. Include technical checks, documentation requirements, ownership, and monitoring rules.

Unlock Full Question Bank

Get access to hundreds of OKR and Metric Definition interview questions and detailed answers.

Join thousands of developers preparing for their dream job.