Facilitation, Consensus Building and Decision Driving Questions

Direct answer

Pre-work decides most of the outcome. Before the meeting I interview both stakeholders, ask each to define their metric precisely and bring a sample query or demo on the same data, and agree a decision rule. In the 30 minutes I compare the candidates against that rule, decide, and publish the agreed definition as a short written artifact both can see themselves in. (The lead metric is the one headline KPI, a key performance indicator, that the team is judged on; other metrics play supporting roles.)

Pre-work

1. 1:1 with each stakeholder: what decision does your KPI drive, and what would convince you the other metric is better?
2. Write the candidate definitions in the same format: name, formula, time window, data source.
3. Evidence on the same data: each brings a sample query or dashboard demo. For retention, a historical check is useful: which candidate best predicts customers still active 90 days later? To run it, take customers who joined six months ago, compute each candidate for their first 30 days, and compare against whether they were still active at day 90. Illustratively, if 150 of 200 customers with 3 or more active weeks in month one were still active at day 90 (75%) versus 30 of 100 with fewer (30%), that candidate separates retained customers well.
4. Decision rule (proposed in advance): the lead metric must be (a) tied to retention, (b) movable by the team within a quarter, and (c) hard to game (it cannot be pushed up by actions that do not help customers, such as discounting or relabelling accounts). Each criterion needs a minimum score of 2 out of 5, and then the higher total leads; where a candidate scores low on one criterion, the other metric covers the gap (for 90-day retention, which reads out late, weekly usage is the early signal). The other metric becomes a guardrail (a metric that must not get worse) instead of being dropped.

Agenda (30 minutes)

Total: 30 minutes.

Illustrative scoring during the 8 to 20 slot, each criterion scored 1 to 5 aloud, with the stakeholders agreeing each number:

The totals are close, so the group reads the result as: retention is the lead metric because it measures the outcome itself, and weekly active usage becomes the guardrail and the early-warning signal.

Facilitation techniques: "say it back" before rebutting; scoring each candidate against the three criteria aloud; naming what each person gains (their metric becomes the guardrail).

Example conflicts and how the data decides

• Revenue versus long-term engagement as the lead KPI: revenue moves fast but can rise by discounting and lose customers later; engagement moves slower but tracks retention. A common outcome is engagement or retention as the lead metric with revenue retention as a guardrail.
• Last-click versus first-touch conversion: suppose a customer clicks an ad on Monday, an email on Wednesday and buys on Friday. Last-click credits the email; first-touch credits the ad. Run both definitions on the same quarter's data, show how credit shifts between channels, and pick one for the stated purpose (acquisition decisions favour first-touch; closing-funnel optimisation favours last-click), keeping the other as a reported secondary.

Documenting the decision

A one-page definition. A filled example: Name: 90-day customer retention rate. Meaning: the share of customers who are still active 90 days after they sign up. Formula: customers active on day 90 / customers who signed up in the cohort (for example 180 / 300 = 60%, a cohort in which the 200 customers with 3 or more active weeks in month one and the 100 with fewer were pooled). Window: day 90 after signup, cohorts by signup month. Source: the billing and product events tables. Owner: named person. Known limits: ignores customers who left and returned. Review date: next quarter. In general the page holds: metric name, plain-English meaning, exact formula, time window, data source, owner, known limits, and review date. Both stakeholders get a line recording what they contributed and the guardrail they own.

Trade-offs and pitfalls

• Choosing a metric without agreeing a rule first becomes a vote on personalities.
• A lead metric that no team can influence invites gaming or apathy.
• Skipping the written definition lets each team compute it differently.

Direct answer

The goal of the half-day (240 minutes) is to leave with one written definition, one canonical query and one named owner for each contested metric, plus a lightweight process for changing them later. I would make the disagreement concrete before debating it: have all 8 teams compute the metric on the same handful of sample accounts and compare answers. Definitions argued in the abstract produce vague compromises; definitions tested against real cases expose exactly where teams differ. A canonical query is the one agreed piece of code or logic that every report must reuse for that metric.

Structured elaboration

Pre-work (one week before): each team submits the definition and query it uses for its top 3 metrics and nominates one person with authority to agree for that team. The facilitator then groups the up to 24 submissions (8 teams x 3 metrics) by metric name and picks the six to resolve live: the ones where at least two teams give different definitions and the number reaches executives or customers. The rest are settled in writing afterwards, using whatever the session decides.

The blocks sum to 240 minutes.

Worked example: "churned customer" on six sample accounts, as of 31 March

Three definitions are in use: Finance counts a customer as churned if the subscription ended in March; Product counts a customer as churned if there has been no login for 30 days or more; Customer Success counts customers who gave notice in March.

Finance counts 3 (A, D, E), Product counts 2 (A, C), CS counts 3 (A, B, D). Totals alone hide that the sets differ. The agreed outcome: churn = subscription end date within the month, owned by Finance; "dormant" (no login 30 days) and "notice given" become separately named leading indicators owned by Product and CS. The disagreement is resolved by giving each idea its own name.

Escalation policy: if a metric is not agreed within its 10 minutes, the named metric owner decides at the end of the session; if the owner is contested, the head of data governance (the person accountable for company-wide data rules) decides within 48 hours.

Governance so decisions stick

• Every metric has one owner, a version number and a change request (a short form asking to change a definition). A filled-in example:

Metric: churned customer          Current version: v1.0
Requested change: count a customer as churned when the subscription end date falls in the month (v2.0)
Why: Finance and Product report different churn counts for the same month
Affected: board dashboard, CS weekly report, 2 forecasting models
Expected effect: March count moves from 2 (login-based) to 3
Owner: Finance      Decision date: 15 April

• Changes are reviewed on a fixed monthly cadence. A breaking change is one that moves a number people already publish (like the churn example above), so it gets a notice period and the old name is kept as a labelled alias: the old metric name still works but is tagged "deprecated, now means v2.0" until dashboards are migrated.
• Enforcement in the pipelines: dashboards read from the canonical query or a shared metrics layer (one governed place where each metric is defined once, so every dashboard asks that place instead of writing its own query). An automated reconciliation test recomputes the metric from the canonical query each night and compares it with the number each report shows. Example result: canonical churn for March is 3; the Product retention dashboard shows 2 (it uses the login-based definition, which the six-account table counts as A and C), so the test flags that dashboard as drifted and names its owner. The CS report, which counts notices given, also shows 3, the same total as canonical churn, so a test that compares only the totals would pass it even though its set (A, B, D) differs from the canonical set (A, D, E). That is why the test should also compare the account lists (which accounts are counted), not just the totals: it would then flag the CS report as measuring a different thing and it would be renamed to the "notice given" indicator.
• The glossary lives next to the code, with the six-account table above kept as the worked example.

Trade-offs and pitfalls

• Eight teams means eight vetoes. Pre-agree the decision rule or the day becomes a negotiation.
• Do not redefine metrics silently afterward; restated history needs an announcement.
• Skipping the sample-case step produces definitions that read fine and still disagree on edge cases.

Direct answer

I would reframe the debate from "how much to log" to "which decisions need which data, at what sensitivity, for how long, at what cost". I would get the facts from each function first, separate options by data type, propose privacy-respecting alternatives, and have a named owner decide with Legal's sign-off recorded.

Facilitating the conversation

1. Frame: the goal is enough data to make product decisions, with the minimum personal data, within budget and the law. A decision owner is named (usually Product with Privacy or Legal holding a veto on compliance: the right to block any option that breaks the law, whatever its other merits). Retention periods in the table below are illustrative.
2. Get the facts per function:
   • Product: the specific questions the data must answer (for example, funnel questions: at which step of sign-up or checkout users drop off, and which feature is used in what order), and how soon.
   • Infra: event volume, storage and processing cost, and what retention would cost.
   • Privacy/Legal: which fields are personal data, what consent was given (the permission users agreed to for specific uses of their data), which rules apply (data minimisation means collecting only what a stated purpose requires), and retention limits (how long data may be kept before it must be deleted).
3. Classify the data: anonymous aggregate counts, pseudonymous user IDs (a stable ID that is not a name; under GDPR such data is still personal data if it can be linked back to a person using other information, so Recital 26 treats it as personal data and it still needs a lawful purpose, retention limit and Legal's view), and direct personal data. The trade-off differs for each.
4. Offer alternatives that serve the need: aggregate counters instead of raw event streams; sampling a fraction of users; replacing identifiers with random IDs; keeping raw events short-lived and aggregates long-lived; or doing the aggregation on the device so raw data never leaves (the app counts locally, for example "opened export 3 times this week", and uploads only that count).

Cost arithmetic (illustrative)

Suppose 50 million events per day at about 1,500 bytes each: 75 GB per day. Thirty days of raw retention is 2,250 GB (2.25 TB). Sampling 10 percent of events cuts that to 225 GB. To turn this into money, assume (illustratively) 0.50 per GB for ingesting and indexing logs: 2,250 GB x 0.50 = 1,125 per month for full raw logging, against 225 GB x 0.50 = 112.50 per month at 10 percent sampling. Plain storage at an assumed 0.02 per GB-month is only 45 versus 4.50, so the real cost driver is ingestion and querying, which is worth knowing before arguing about retention. This turns "too expensive" into a number the group can weigh against what Product loses by sampling.

A proposed outcome

Product gets the funnel questions answered, Infra's cost drops, Privacy's risk is smaller.

Closing and documenting

Record the decision, the data collected, purpose, retention, owner, Legal's sign-off and a review date. Where Legal has the final word on compliance, make that explicit rather than negotiating it.

Trade-offs and pitfalls

• Treating the dispute as cost versus privacy hides the real question: what decisions the data supports.
• Do not accept "we might need it later" as a reason to collect personal data.
• Sampling can bias small segments; say so in the record.