Technology Strategy and Business Alignment Questions

Direct answer

A north star metric is the single measure that best captures the value customers get from the product and that the business grows from. It changes engineering priorities by giving every team the same question to ask of any roadmap item: which input to this metric does it move, and by how much? It settles arguments because it replaces opinion about what matters with a shared yardstick.

What a good one looks like

• It reflects delivered customer value, not just activity (successful orders per week, not registered accounts).
• It leads revenue rather than lagging it: it moves first, within weeks, while revenue (a lagging measure) only confirms the result months later.
• It breaks into input metrics (smaller measures that add up to the north star and that individual teams can move, such as checkout success rate, search-to-cart rate, delivery on time).
• It is understandable by engineers, not just finance.

How it changes technical priorities

1. Each team maps its work to an input metric and states the expected effect before the quarter starts.
2. Reliability targets (service-level objectives, or SLOs, which are the reliability level you promise; a guardrail metric is a second number that must not get worse while you chase the first, such as refund rate or support tickets) are set on the user journeys that feed the metric, such as checkout, rather than on every service equally.
3. Platform and tooling work is justified by the input metric it speeds up or protects.
4. Some work does not move the metric at all (security, legal compliance, preventing future outages). It is kept in a protected capacity share and is not asked to prove itself with the metric.

Example that settled an argument (illustrative)

A marketplace's north star is successful orders per week. The platform team wants to rebuild an internal reporting dashboard. The payments team wants to fix intermittent checkout failures. Both are reasonable.

Numbers: 200,000 checkout attempts a week, 4% fail technically, which is 8,000 failures. Halving the rate to 2% removes 4,000 failures. If about half of those buyers give up rather than retry, that is roughly 2,000 more successful orders a week. The dashboard serves a handful of internal analysts, so its link to the metric is indirect and unquantified. The payments fix went first. The mapping was written down like this (checkout rate and expected effect are illustrative):

The dashboard was not rejected: it was asked to state which input it speeds up, and it came back with a case about fraud review time.

Pitfalls

• Choosing a vanity metric (a number that looks impressive and rises easily but does not reflect customer value, such as sign-ups) that rises while value falls.
• Choosing revenue alone: too slow to guide a team's weekly choices.
• Gaming: teams optimise the number in ways customers dislike, so pair it with guardrail metrics.
• Treating the metric as the only legitimate goal and starving long-term engineering health.

Direct answer

I would treat "p95 (the response time that 95% of requests beat) dropped from 800 ms to 300 ms" as a technical result and test whether it caused a business change. The cleanest way is a randomized experiment; if that is impossible, a careful before-and-after with controls for confounders. I would present the effect size with its uncertainty, not a single headline number.

How to show it moved a business outcome

1. State the chain: faster pages, fewer abandoned sessions, higher conversion or retention. Name which metric you expect to move and by roughly how much.
2. Prefer an A/B test: serve the fast path to a random half of users using a feature flag (a switch that turns a change on for chosen users without redeploying) and compare conversion between halves. Randomization removes most confounders.
3. If you already shipped to everyone: compare periods with controls (difference-in-differences: compare how much the changed group moved before versus after against how much an unaffected group moved over the same dates, so shared trends cancel out), or use gradual rollout stages as natural experiments.

What could confound it

• Seasonality, a marketing campaign, pricing changes, or other releases in the same window.
• Mix shift (the audience composition changes): more desktop than mobile users in the after period.
• Selection: slow sessions were the ones abandoning before the change, so the sessions you now keep are different.
• Novelty effects (users react to anything new, and the bump fades) and a small sample.

Worked example (illustrative numbers, computed)

Control: 20,000 sessions, 600 conversions (3.0%). Fast path: 20,000 sessions, 640 conversions (3.2%). That is +0.2 percentage points, a 6.7% relative lift (0.2 / 3.0). The question is whether 0.2 points is a real effect or just random wobble ("noise"): two identical groups will rarely convert at exactly the same rate.

Terms in plain words:

• z-score: the observed difference divided by its typical random wobble. The bigger it is, the less likely chance alone explains the gap.
• p-value: if the change did nothing, the chance of seeing a gap at least this big by luck. Small (under about 0.05) is convincing; large is not.
• 95% interval: the range of true differences that the data cannot rule out.
• Statistical power (80%): the chance the test spots a real effect of the size you care about. The effect size here is that 0.2-point gap.

pooled rate  = (600 + 640) / 40,000                  = 3.1%
typical wobble (standard error) = sqrt(0.031 * 0.969 * (1/20000 + 1/20000)) = 0.00173 (0.173 points)
z = 0.002 / 0.00173                                  = 1.15
p-value (two-sided) for z = 1.15                     = about 0.25
95% interval = 0.2 +/- 1.96 * 0.173 points           = -0.14 to +0.54 points

A p-value of 0.25 means a gap this size would appear by luck about one time in four even if the change did nothing, so this result is within noise, and the interval covers zero (no effect) as well as a useful gain. To see a real 0.2-point difference at a 3% baseline reliably, you need more sessions. The standard sample-size formula for comparing two rates, with a 5% false-alarm rate and 80% power, gives:

n per arm = (1.96 + 0.84)^2 * (0.03*0.97 + 0.032*0.968) / 0.002^2  = about 118,000 sessions

With 100,000 per arm at the same rates (3,000 vs 3,200 conversions), the wobble shrinks to 0.0775 points, so z = 0.2 / 0.0775 = 2.58, p is about 0.01, and the interval is +0.05 to +0.35 points: now convincing. The lesson for leadership: report "likely small positive, interval X to Y", and say how much more traffic would settle it.

Presenting to product leadership

Lead with the business metric and its interval, then the technical cause, then the decision you want (keep investing in latency or stop). Show a chart of the two arms, not a slide of p95 numbers.

When many initiatives ship at once

Use a global holdout (a small group of users who get none of the new changes, kept as a permanent baseline) to measure the combined effect; use separate feature flags and randomization per initiative so each effect can be separated; stagger releases; and avoid claiming credit by timing alone. If experiments are impossible, fit a regression (a statistical model that estimates each initiative's separate contribution to conversion, given which users saw which initiatives) and say clearly that it is observational.

Direct answer

By platform I mean the shared internal foundation (build and deploy tooling, common services, data and infrastructure) that product teams build on, as opposed to the customer-facing features themselves. I would not ask executives to approve "three years of platform work". I would ask them to approve a staged, gated investment tied to outcomes they already care about, with the first stage cheap and reversible, and with stop criteria (the pre-agreed results that would make us halt or change course) agreed in advance. Being possibly wrong is handled by design, not by confidence.

Framing in business terms

• Name the business problem: for example, the time from idea to customer is slowing, outages are costing customers and engineers' time, and the roadmap itself is at risk because every feature gets harder.
• Say what the company gets: faster delivery of features later, fewer incidents, ability to enter markets the current system cannot support.
• Say what it costs: roadmap capacity, in plain terms. If the program takes 30% of engineering capacity for 3 years, that is 0.3 x 36 months = 10.8 months of capacity: about a year of delay spread over 3 years, not a year of nothing. Name the roadmap items that wait, for example "the loyalty-points feature and the second-region launch move out; the checkout redesign and the compliance deadline stay".

Sample opening of the pitch: "Today an idea takes about 20 days to reach customers, and a growing share of each team's week goes to working around the old base. I am asking for 30% of engineering for three years in stages. The first stage costs one team for 90 days. If lead time has not moved by month 12, we stop."

Evidence to bring

• Trend data you can show: lead time (idea to production), how much of each team's time goes to workarounds, incident counts and customer-facing impact, and examples of roadmap items that took far longer than they would have on a cleaner base.
• A comparison: what happens if we do nothing, with the same metrics projected.
• Left to the finance partners: the detailed financial model, which they build from these figures.

What I ask of them

1. A decision on the direction and an executive sponsor (a senior leader who owns the outcome, clears obstacles and defends the budget).
2. Protected capacity (a fixed share of engineering time that is not taken back when a feature gets urgent, not whatever is left).
3. Agreement on the gates (checkpoints where leaders decide to continue, change or stop), say at months 6, 12 and 24, each with a measurable test. Illustrative gates: month 6, the pilot team deploys without manual steps; month 12, idea-to-production lead time falls from 20 days to 12; month 24, a majority of teams run on the platform and the share of engineering time lost to workarounds drops from 30% to 15%.
4. A commitment on which roadmap items will wait and which will not.

A rough payback check (illustrative)

Using the numbers above, the program costs 30% of capacity for 36 months. Per engineer that is 0.3 x 36 = 10.8 months. If the month-24 gate is met and workaround time falls from 30% to 15% of engineering time, that frees 15% of capacity, or 0.15 x 12 = 1.8 months per engineer per year. 10.8 / 1.8 = 6 years to recover the cost from workaround time alone, counted from when the saving starts. The 15% saving is only reached at the month-24 gate, so on the calendar the break-even from this one benefit is later still, which strengthens the point below. So that benefit by itself does not carry the case. The case has to rest on the other outcomes: lead time from 20 to 12 days (features reach customers 40% sooner, 8 / 20), fewer incidents, and markets the current system cannot support. I would put a number on each with the finance partners and say plainly which one the investment depends on, so executives can judge the claim that matters, and so the month-12 gate tests that one.

Handling the risk that I am wrong

• Start with a thin slice (the smallest end-to-end piece that tests the idea for real): one team or product on the new platform within about 90 days, to test the assumption cheaply.
• Make the gates real: for example, if the pilot team's lead time has not improved by the agreed amount by month 12, we stop or change course, and that is written down now.
• Keep each stage independently useful, so stopping halfway still leaves value.
• Invite the strongest opposing view and put it in the proposal.

Pitfalls

Hiding the delay, arguing from technical purity, and asking for unconditional trust. What would change my call: if the pilot shows the benefit is much smaller than predicted, I recommend scaling back.

Direct answer

I would give executives a one-page view per quarter with the same four columns for each squad: the squad's own business outcome against its own target, delivery health, quality, and a one-line narrative. The fairness trick is to compare each squad to its own target and its own trend, never to rank squads on raw numbers, because a checkout squad and a platform squad cannot be measured by the same revenue metric. The view exists to drive investment and unblocking conversations, not performance rankings.

What goes on it

1. Outcome: one business metric the squad can influence, with a target set at the start of the quarter.
2. Delivery health: deployments (releases of a change so that real users can reach it) and median lead time (from code merged to running in production), shown as trend.
3. Quality: change failure rate = failed deployments / total deployments (a failed deployment is one that needed a rollback, meaning going back to the previous version; a hotfix, meaning an urgent small fix released outside the normal schedule; or an incident), and a reliability measure such as incident minutes.
4. Narrative: two sentences from the squad lead on risk, confidence and the one thing they need.

Illustrative one-page view (all numbers illustrative)

The other two columns of the same page, with the same illustrative squads (the deployment counts are the denominators used above):

For a lower-is-better metric the score is target divided by actual, so under 100% always means behind: Platform is 20 / 24 = 83%, not 120%. For higher-is-better rows it is actual divided by target (3.1 / 3.4 = 91%).

Making it fair

• Percent of own target and trend, not cross-squad ranking. Platform's outcome is internal speed because its customers are other squads.
• Show deployment counts but never target them; a squad shipping 60 tiny changes is not better than one shipping 30 meaningful ones.
• Annotate context: a squad that lost two engineers or inherited a legacy service gets a note beside its number, for example "Onboarding: two of six engineers moved to an incident team in week 3".
• Agree definitions with the squads once and publish them, so nobody is surprised. Definitions are the usual place for quiet drift, for instance one squad counting a hotfix as a failed deployment and another not.

Conversations it should drive

• Checkout is at 91% and onboarding at 90%, with onboarding's failure rate rising from 8.7% to 11.4%: ask what quality investment or scope cut would help, rather than demand speed.
• Search beat target: should it get more funding or has the metric stopped being stretching?
• Platform is 4 minutes over its 20-minute target (83% score): whose roadmap pays for the fix?

Trade-offs and pitfalls

• Small counts are noisy: 5 failures out of 44 deployments moving to 6 out of 44 is not a trend; read two or three quarters.
• Executives will want a single score; resist it, because a blended number hides exactly the context that makes comparison fair.
• If squads start tuning numbers, rotate a reviewer from outside the squad to audit definitions each quarter: they sample five deployments per squad and check each was classified (failed or not) by the published rule.

Direct answer

I would build the roadmap backwards from business outcomes the company needs over three years, convert each outcome into a capability the platform must have, and attach to every phase a trigger (a measurable condition) that says when to start it. Executives get progress as outcomes and risk headroom, never as a count of technical tasks.

Structure

1. Growth arithmetic. Ten times the users in three years is about 2.15 times per year, or roughly 6.6% growth per month. Users double about every 10.8 months.
2. Outcome ladder. Translate growth into things the business cares about: the product stays fast and available at peak, onboarding a large customer takes weeks not quarters, new regions or segments can launch, and the engineering organisation can still ship at a steady pace with several times more people.
3. Phases, each with an outcome, a capability, a trigger and an executive measure. The table is illustrative.

Proven capacity is the highest load the system has actually been measured to handle in a load test while still meeting its speed and error targets, not a guess from hardware specs. Headroom is the gap between today's peak load and that ceiling.

4. Re-architecture triggers. Pick a condition that leaves enough lead time. If a major redesign needs six months and growth is 6.6% a month, load grows about 1.47 times in that span, so the work must start at or below roughly 68% of proven capacity. The arithmetic: growth of 6.6% a month compounds to 1.066^6, about 1.47, over six months, so a system at 68% today (1 / 1.47) reaches 100% exactly when the redesign finishes. A trigger of 60% leaves about eight months before the ceiling, because load needs ln(1/0.6) / ln(1.066), about 8 months, to climb from 60% to 100%. Under doubling traffic that is the signal, not a calendar date.
5. ML platform tied to geographic expansion. An ML platform is the shared tooling for training, deploying and monitoring machine-learning models. If expansion into a region needs local models or local data handling (for example, a model that must be trained and served inside that region because customer data cannot leave it), the ML platform work belongs in phase 3 with the expansion as its justification, not as a standing investment.
6. Developer productivity. Express it as a time-to-ship target the business can feel (for example, a change reaches customers within a day of approval), and show it as phase 4's measure.

Reporting to executives

A one-page quarterly view: each phase with status against its outcome, the headroom number (load as a share of proven capacity), the next trigger and how far away it is, and one decision needed from them. Two-way: they can see why something is early or late.

Trade-offs and pitfalls

• Time-based phases ignore that growth may come slower or faster. Triggers adapt.
• A phase with no named outcome is a technology wish. I would cut it.
• Pre-building for ten times too early wastes effort; building late risks outages.
• What would change my plan: if growth tracks at half the forecast for two quarters, the triggers push phases out automatically.