Technology Strategy and Business Alignment Questions

Direct answer

I would move the architecture team from approving designs to making the right path the easy path. Standards that truly must hold become automated guardrails and a small number of mandatory rules. Everything else becomes recommended patterns, with a quick, advisory conversation instead of a gate. After a year I would judge it by whether product teams move faster and whether the standards are actually still met.

Step 1: Understand why it feels like a gate

Interview six or so product teams and look at review history: how long does a review wait, how late in the project does it happen, how many reviews end in a change? A gate is usually slow, late and applied to everything equally.

Step 2: Evolve the model

1. Tier by reversibility (how easily the decision can be undone) and blast radius (how much can break, and for how many teams or customers, if it goes wrong). For example, a team choosing a charting library is easy to undo and affects only that team, so it needs no review. A team creating a new store of customer addresses that other teams will read is hard to undo and affects many, so it gets a short review. A guardrail is a rule enforced automatically (for example, storage must be encrypted: a deployment that creates unencrypted storage is blocked automatically, with a message naming the one setting to change) so nobody has to ask. Easy-to-reverse, local decisions need no review. Hard-to-reverse or cross-team decisions (data ownership, shared interfaces, anything touching customer data) get a short review.
2. Build the paved road. Ready-made templates and reference designs that already satisfy the standards. Teams that use them skip review.
3. Embed architects with teams. Closer to an enabling team in the Team Topologies sense (Team Topologies is a published model of how to organise software teams; an enabling team's job is to help others adopt good practice), rather than a separate approval body.
4. Write decisions down. Short ADRs (architecture decision records: a page stating context, decision and consequences) so teams can decide and architects can review afterwards. A sample: Context: Orders and Billing both need customer addresses. Decision: Orders owns the address record and Billing reads it through Orders' interface. Consequences: one source of truth, but Billing now depends on Orders being available, so Billing caches addresses for a few minutes.
5. Keep the escape hatch. Teams may deviate with a documented reason, and the exception is reviewed.

Step 3: Worked example (illustrative)

Today every one of 40 designs a quarter goes to the review board. If paved-road designs and easy-to-reverse changes cover 60%, 24 skip the board and 16 receive a focused review. Reviewers spend the same effort on fewer, harder decisions.

Step 4: Knowing after a year

• Median time from design submission to decision (target agreed up front).
• Share of designs that used a paved road or needed no review.
• Compliance with the mandatory standards, measured automatically.
• Number of exceptions and their outcomes.
• Incidents traced to a missed standard.
• A short survey of product teams: is architecture helping or blocking? Ask the same questions as at baseline.

Pitfalls

• Removing review without guardrails lets standards erode.
• Paved roads that nobody maintains become the new bottleneck.
• Measuring only speed hides erosion; measuring only compliance hides friction. Use both.
• What would change my call: if compliance falls while speed rises, tighten the guardrails, not the gate.

Direct answer

I would treat "p95 (the response time that 95% of requests beat) dropped from 800 ms to 300 ms" as a technical result and test whether it caused a business change. The cleanest way is a randomized experiment; if that is impossible, a careful before-and-after with controls for confounders. I would present the effect size with its uncertainty, not a single headline number.

How to show it moved a business outcome

1. State the chain: faster pages, fewer abandoned sessions, higher conversion or retention. Name which metric you expect to move and by roughly how much.
2. Prefer an A/B test: serve the fast path to a random half of users using a feature flag (a switch that turns a change on for chosen users without redeploying) and compare conversion between halves. Randomization removes most confounders.
3. If you already shipped to everyone: compare periods with controls (difference-in-differences: compare how much the changed group moved before versus after against how much an unaffected group moved over the same dates, so shared trends cancel out), or use gradual rollout stages as natural experiments.

What could confound it

• Seasonality, a marketing campaign, pricing changes, or other releases in the same window.
• Mix shift (the audience composition changes): more desktop than mobile users in the after period.
• Selection: slow sessions were the ones abandoning before the change, so the sessions you now keep are different.
• Novelty effects (users react to anything new, and the bump fades) and a small sample.

Worked example (illustrative numbers, computed)

Control: 20,000 sessions, 600 conversions (3.0%). Fast path: 20,000 sessions, 640 conversions (3.2%). That is +0.2 percentage points, a 6.7% relative lift (0.2 / 3.0). The question is whether 0.2 points is a real effect or just random wobble ("noise"): two identical groups will rarely convert at exactly the same rate.

Terms in plain words:

• z-score: the observed difference divided by its typical random wobble. The bigger it is, the less likely chance alone explains the gap.
• p-value: if the change did nothing, the chance of seeing a gap at least this big by luck. Small (under about 0.05) is convincing; large is not.
• 95% interval: the range of true differences that the data cannot rule out.
• Statistical power (80%): the chance the test spots a real effect of the size you care about. The effect size here is that 0.2-point gap.

pooled rate  = (600 + 640) / 40,000                  = 3.1%
typical wobble (standard error) = sqrt(0.031 * 0.969 * (1/20000 + 1/20000)) = 0.00173 (0.173 points)
z = 0.002 / 0.00173                                  = 1.15
p-value (two-sided) for z = 1.15                     = about 0.25
95% interval = 0.2 +/- 1.96 * 0.173 points           = -0.14 to +0.54 points

A p-value of 0.25 means a gap this size would appear by luck about one time in four even if the change did nothing, so this result is within noise, and the interval covers zero (no effect) as well as a useful gain. To see a real 0.2-point difference at a 3% baseline reliably, you need more sessions. The standard sample-size formula for comparing two rates, with a 5% false-alarm rate and 80% power, gives:

n per arm = (1.96 + 0.84)^2 * (0.03*0.97 + 0.032*0.968) / 0.002^2  = about 118,000 sessions

With 100,000 per arm at the same rates (3,000 vs 3,200 conversions), the wobble shrinks to 0.0775 points, so z = 0.2 / 0.0775 = 2.58, p is about 0.01, and the interval is +0.05 to +0.35 points: now convincing. The lesson for leadership: report "likely small positive, interval X to Y", and say how much more traffic would settle it.

Presenting to product leadership

Lead with the business metric and its interval, then the technical cause, then the decision you want (keep investing in latency or stop). Show a chart of the two arms, not a slide of p95 numbers.

When many initiatives ship at once

Use a global holdout (a small group of users who get none of the new changes, kept as a permanent baseline) to measure the combined effect; use separate feature flags and randomization per initiative so each effect can be separated; stagger releases; and avoid claiming credit by timing alone. If experiments are impossible, fit a regression (a statistical model that estimates each initiative's separate contribution to conversion, given which users saw which initiatives) and say clearly that it is observational.

Direct answer

I would trace each five-year goal into the technical qualities it demands, make each major technical choice cite the goal it serves, record those choices in decision records, and take the traceability map to product, engineering and sales framed in terms each cares about. The point is that a reviewer can ask of any design decision, "which business goal is this for, and what did we give up?"

Process

1. Clarify the goals with the owners. Turn each goal into something testable: "international expansion" becomes "sell in these regions, in local currency, with local data rules, by these dates".
2. Derive architectural drivers (the requirements that most shape the design) by turning each goal into testable statements. For example: "open a new region in under 3 months", "release any service independently several times a week", and "reproduce any customer's invoice from stored records".
3. Make decisions against the drivers and write each as a short architecture decision record (ADR): context, options, decision, consequences. (Michael Nygard's original ADR template is title, status, context, decision, consequences; an options section is a common extension.)
4. Stage the work (build in phases): build the minimum for the first stage, keep reversible options open for later stages.
5. Define fitness measures (regular checks, ideally automated, such as time to open a new region) that tell you later whether the decision is serving the goal.

Artifact: strategy-to-technology map

Other artifacts: a short list of architecture principles, meaning short rules that guide later decisions (for example "billing events are append-only", meaning they are added but never edited or deleted, so history can always be reconstructed), a staged roadmap with decision points, and a risk and assumptions register.

Worked decision (illustrative)

Where should customer data live? Sales says some prospective customers in one region will only sign if their data stays in that region. That requirement points to per-region data stores. The cost is higher operating complexity and harder cross-region reporting, accepted because it supports the expansion goal. The ADR records that cost and a trigger to revisit it. Filled in (illustrative):

ADR-003: Store customer data in per-region data stores
Context: sales has two prospects who will sign only if data stays in their region;
         the five-year goal includes opening 4 regions.
Options: (1) one global store; (2) per-region stores; (3) global store with encryption.
Decision: option 2.
Consequences: higher operating cost and harder cross-region reporting;
              a new region needs its own store, so the under-3-months driver holds only if
              region setup is automated (a first manual region is estimated at 4 months,
              so automating it is part of this decision).
Revisit when: more than 8 regions are live, or reporting delays hurt sales.

Getting buy-in

• Product: show the sequencing (for example "the billing foundation ships in stage one, which is why the self-serve upgrade page lands a quarter earlier"), which features ship earlier because of the foundations, and which wait.
• Engineering: involve leads in writing the principles, give them the paved path (the default, well-supported way to build and deploy that teams are encouraged to follow) and the reasoning, not a mandate.
• Sales: state what they can promise and when, in plain terms. A sentence they can say: "From stage two you can quote in euros and pounds with customer data stored in the EU; Japan and local tax filing come at stage three, so please do not promise those before then."
• All three: agree up front on how scope disputes are resolved.

Pitfalls

Designing all five years of capability now (overbuilding) or ignoring later goals so the first design blocks them. Staging solves both.

Direct answer

A north star metric is the single measure that best captures the value customers get from the product and that the business grows from. It changes engineering priorities by giving every team the same question to ask of any roadmap item: which input to this metric does it move, and by how much? It settles arguments because it replaces opinion about what matters with a shared yardstick.

What a good one looks like

• It reflects delivered customer value, not just activity (successful orders per week, not registered accounts).
• It leads revenue rather than lagging it: it moves first, within weeks, while revenue (a lagging measure) only confirms the result months later.
• It breaks into input metrics (smaller measures that add up to the north star and that individual teams can move, such as checkout success rate, search-to-cart rate, delivery on time).
• It is understandable by engineers, not just finance.

How it changes technical priorities

1. Each team maps its work to an input metric and states the expected effect before the quarter starts.
2. Reliability targets (service-level objectives, or SLOs, which are the reliability level you promise; a guardrail metric is a second number that must not get worse while you chase the first, such as refund rate or support tickets) are set on the user journeys that feed the metric, such as checkout, rather than on every service equally.
3. Platform and tooling work is justified by the input metric it speeds up or protects.
4. Some work does not move the metric at all (security, legal compliance, preventing future outages). It is kept in a protected capacity share and is not asked to prove itself with the metric.

Example that settled an argument (illustrative)

A marketplace's north star is successful orders per week. The platform team wants to rebuild an internal reporting dashboard. The payments team wants to fix intermittent checkout failures. Both are reasonable.

Numbers: 200,000 checkout attempts a week, 4% fail technically, which is 8,000 failures. Halving the rate to 2% removes 4,000 failures. If about half of those buyers give up rather than retry, that is roughly 2,000 more successful orders a week. The dashboard serves a handful of internal analysts, so its link to the metric is indirect and unquantified. The payments fix went first. The mapping was written down like this (checkout rate and expected effect are illustrative):

The dashboard was not rejected: it was asked to state which input it speeds up, and it came back with a case about fraud review time.

Pitfalls

• Choosing a vanity metric (a number that looks impressive and rises easily but does not reflect customer value, such as sign-ups) that rises while value falls.
• Choosing revenue alone: too slow to guide a team's weekly choices.
• Gaming: teams optimise the number in ways customers dislike, so pair it with guardrail metrics.
• Treating the metric as the only legitimate goal and starving long-term engineering health.

Direct answer

I would not reject it outright and I would not adopt it alone. I would adopt it as a leading signal (an early measure that tends to move before the result we care about, such as retention or revenue, so it helps steer but does not prove success on its own) only if I can pair it with measures that reveal gaming, and I would define it so that gaming it requires doing the thing we actually want. The rule of thumb behind this is Goodhart's law, popularly phrased by Marilyn Strathern as: when a measure becomes a target, it ceases to be a good measure.

Step 1: Decide whether to adopt

Ask:

1. What behaviour do we want, and how directly does this number reflect it?
2. How would a customer game it, and what would they gain? If gaming has no payoff, the risk is small.
3. Can I redefine it so the cheap trick no longer counts?

Example (illustrative): the KPI is "reports created per account", meant to show engagement. A customer can run a script that makes hundreds. Redefine it as "reports viewed by at least two different users within seven days". Now gaming means getting colleagues to read the reports, which is the outcome we wanted.

Step 2: Guardrail measures

A guardrail measure is a second number watched alongside the KPI to catch harm or distortion:

• Retention of the same accounts, not only activity.
• Support tickets and complaints per account.
• Share of the KPI that comes from the top few accounts.
• Quality indicators of what was counted (do the reports get opened?).

Also: do not tie pay or targets to the raw number, and report it with its guardrails every time.

Step 3: Detect gaming after launch

Look at distributions (the full spread of values across accounts: how many sit near 10, how many near 150) rather than the single average. Worked example with illustrative data: 1,000 accounts, of which 986 create about 10 reports (9,860 in total) and 14 create about 150 (2,100). The average looks healthy at about 12 per account, but those 14 accounts (1.4%) produce roughly 17.6% of all volume. Then check whether those 14 also have views, retention and tickets in line with the others. If their reports are never opened, that is gaming.

Other checks: sudden step changes (a jump in the number right after the KPI was announced, instead of a gradual climb), usage at odd hours or in fixed intervals, accounts with high counts and low value on every other measure. Review the segment monthly and ask customer success what they see.

Step 4: Respond

Fix the definition first, not punish customers. A message to a flagged account might read: "We noticed your account created 150 reports last week and few were opened. We changed how this measure counts so it reflects reports your colleagues read. Is something in the product making the old count hard to avoid, or can we help you get more out of the reports?" Separately, if the KPI is also part of a contract or incentive, the response differs and needs legal and sales input.

Trade-offs and pitfalls

• A KPI so guarded that it takes months to compute is useless.
• Dropping every gameable metric leaves nothing measurable.
• What would change my call: if the payoff for gaming is large (money, status) and I cannot redefine it, I would refuse to adopt it.