Knowledge Sharing and Team Enablement Questions

Direct answer

Interviewing an expert one analyst at a time does not scale because the knowledge stays in the expert's head and in each analyst's private notes. I would convert the expert's know-how into shared, maintained assets (a glossary of business terms, a log of the quirks that explain the numbers, and definitions embedded in the data layer, meaning the tables and views that queries are built from), using structured sessions where analysts capture the knowledge and the expert only reviews it. The test of success is a new analyst who can answer common domain questions without booking the expert.

Step 1: find what to capture first

Mine the last few months of analyst questions (Slack, tickets) to rank recurring domain questions. Capture those first; they cost the experts the most time.

Step 2: structured elicitation (tacit knowledge is what experts know but rarely say)

• Shadowing: an analyst sits in a real month-end close (the monthly process where finance finalises the books) and records the steps and judgment calls.
• "Walk me through this number" sessions: the expert explains one metric from source to report while the session is recorded and an analyst writes it up.
• Ask about exceptions: "What always surprises new people about this number?"

Step 3: turn it into durable assets

Step 4: keep it alive

The expert reviews for 15 minutes rather than authoring from scratch; entries carry an owner and a review date tied to the close cycle; domain office hours and a short embedded rotation (an analyst spends a week with finance) refresh the knowledge.

Worked example (a quirks-log entry)

Why Jan revenue on the dashboard differs from finance
- Definitions: bookings = total value of contracts signed; billings = amount invoiced to the customer; recognised revenue = the share of that money the company has earned so far under accounting rules.
- Annual contracts are invoiced at signing but recognised evenly over 12 months.
- Example: a 12,000 annual contract signed on Jan 1.
  Bookings in January: 12,000. Billings in January: 12,000. Revenue recognised in January: 12,000 / 12 = 1,000.
  Difference between billings and revenue in January: 11,000.
- Rule: dashboards labelled "revenue" must use recognised revenue. Billings and bookings are separate metrics with their own labels, never called "revenue".

An analyst who reads this no longer needs to ask why the two reports disagree, and the same rule can be encoded as a data test (an automated check, as defined above) that fails if a "revenue" metric is computed from invoice date, so the trap is caught without anyone remembering the note.

How to measure it

• Expert interruptions per week (fewer over time).
• A check: give a new analyst five common domain questions and see how many they answer from the assets alone.

Trade-offs and pitfalls

• Asking experts to write documentation themselves rarely happens; let analysts write and experts correct.
• Docs that are not tied to the data layer drift away from what the queries do.
• What would change my call: if one expert holds unique, high-risk knowledge and is leaving soon, do intensive recorded sessions first and clean them up afterward.

Direct answer

Make writing the hypothesis the entry ticket to an analysis: no query runs until the request states the decision it serves, a hypothesis, and what result would change the decision. Back it with a short review before the work begins, a scorecard after it ends, and leadership that celebrates refuted hypotheses. To show it took hold, track the share of analyses with a hypothesis recorded before the first query, and look for a healthy refutation rate and decisions that actually changed.

What I would put in place

1. A request template with five fields: the decision, the hypothesis ("we think the drop is concentrated in mobile checkout"), the evidence that would support it, the evidence that would refute it, and what we would do in each case.
2. A 10-minute hypothesis review before work starts, with a peer asking "what would prove this wrong?"
3. A post-analysis scorecard: confirmed, refuted or inconclusive, plus what we learned. Refuted results are logged as valuable.
4. Coaching in the work itself: leads ask "what is your hypothesis?" in stand-ups and reviews, not in a training deck.
5. Guarding against HARKing (hypothesising after results are known, meaning writing the hypothesis to match what you found): the hypothesis field is timestamped and compared to the first query in the query log.

Worked example (illustrative numbers)

A ticket: "Why did trial signups drop?" becomes:

Decision: whether to pause the paid campaign. Hypothesis: the drop is concentrated in mobile users arriving from the campaign. Supports: mobile share of the drop above 70%. Refutes: mobile drop close to desktop drop. If supported: pause mobile spend. If refuted: look at pricing page changes.

Adoption measure, defined precisely:

Month 1: 6 of 24 closed analyses, which is 25%. Month 3: 20 of 25, which is 80%. Companion signals: refutation rate and the count of analyses that changed a decision.

There is no magic target for refutation rate, but a healthy record is neither near 0% nor near 100%. Example scorecard for month 3 (25 closed analyses, 20 with a hypothesis logged first): among those 20, 8 confirmed, 9 refuted, 3 inconclusive, so 9/20 = 45% refuted. If the same team showed 0 refuted out of 20 for several months, I would audit the hypotheses for vagueness or after-the-fact writing.

Example scorecard row: "Ticket 412 | Hypothesis: drop concentrated in mobile campaign traffic | Result: refuted (mobile drop 12%, desktop 11%) | Learned: not a channel problem, check pricing page | Decision changed: no mobile pause".

Trade-offs and pitfalls

• Mandating the template can produce box-ticking ("hypothesis: X will change"). Audit five tickets a month for whether a result could really have refuted the hypothesis.
• Exploratory work has no hypothesis yet. Allow a labelled "exploration" track with a time limit, or analysts will fake hypotheses. Example: "Exploration, 2 days: profile the new events table to see what is there", which ends in a written hypothesis for the next step or a decision to stop.
• Speed complaints are real: keep the template to five short lines.
• The rate proves habit, not quality. Pair it with outcomes such as decisions changed and a peer-rated sample of analyses.

Direct answer

Knowledge sharing is how what one person knows becomes usable by others on the team; knowledge transfer is the deliberate version, moving a specific body of knowledge from someone to someone (for example, when an owner changes). The useful distinction is between explicit knowledge (can be written down and looked up) and tacit knowledge (know-how gained by doing, hard to write down). Teams need both, and they move through different practices.

Explicit versus tacit

Practices you should expect to see working

• Docs live next to the work: updated in the same pull request (proposed code change) as the code, each with an owner and a last-reviewed date.
• Code review that explains the why, not only "change this".
• Blameless postmortems (incident reviews that examine causes, not culprits) that produce written actions.
• On-call shadowing and rotations so more than one person has handled each system.
• Regular demos, brown-bags (informal lunchtime talks) and office hours (a fixed slot where anyone can ask an expert questions).

Team flavours:

• Backend microservice team: each service has an owner, an ADR trail, an API contract document (a written spec of what the service accepts and returns), and a shadow on the on-call rota (the rotating schedule of who responds to alerts).
• SRE (site reliability engineering) team: runbooks tested during game days (rehearsed failures), handoff notes at every on-call shift change, postmortem review meetings.
• BI (business intelligence) team: recorded dashboard walkthroughs, a shared glossary of metric definitions, weekly office hours for the people who read the dashboards.

Worked example: first 90 days after a launch

A silo is an area only one person understands, so if they leave or are unavailable the work stalls. The day-90 silo review lists each area of the system and counts who could handle it alone; any area with one name gets a pairing plan, for example two more people take turns as second pair of hands.

Trade-offs and pitfalls

• Over-relying on documents for tacit knowledge produces a large, stale wiki; over-relying on conversation produces a team that cannot survive turnover.
• Sharing that depends on goodwill alone fades under deadlines; give it calendar time and owners.
• Count sharing by outcomes (fewer repeated questions, more people able to handle an area), not by attendance.

Direct answer

Make "competent" and "using it" observable, start with a small real use case rather than a course, put a pair of early adopters in front, and give everyone else a short guided path with pairing and office hours. Then decide explicitly when the new way becomes the default, and track real use (work done through the tool), not attendance.

Steps

1. Define the goal in behaviours. For a new deployment pipeline: "every engineer has shipped a change through it without help, and the old path is retired." Without this, adoption stays vague.
2. Spike first (week 1): a spike is a short, time-limited experiment to learn the unknowns rather than deliver a feature. Two volunteers try it on a low-risk service, write a one-page guide and list the gotchas. That converts unknowns into materials.
3. Weekly milestones (example 6 weeks): week 1 spike, week 2 guide and demo, weeks 3-4 pairs each ship a small change, week 5 everyone ships one, week 6 retire or freeze the old path (freeze means allow no new work on it and keep it only for emergencies).
4. Materials: a one-page "how to do the 3 common tasks", a troubleshooting list, and a recorded 20-minute demo. Keep them in the repository so they get fixed.
5. Pair sessions and office hours: the most confident person pairs with the least confident for the first change.
6. Make using it the path of least resistance: templates, defaults, and a deadline for turning off the old way (with a rollback plan, meaning a way to switch back quickly if the new path fails).

KPIs (key performance indicators, the numbers you track) that show real adoption (worked example, illustrative)

A team of 8. Track:

• People who have shipped a change through it: 2 of 8 by week 2 (the two spike volunteers), 8 of 8 by week 4 (four pairs each ship a small change with both people hands-on), matching the week plan.
• People who shipped one unaided (no pair partner): 2 of 8 at week 4 (the volunteers), then 8 of 8 by the end of week 5 when everyone ships alone; week 6 is buffer for stragglers and for the retire-or-freeze decision on the old path.
• Share of eligible deploys through the new tool (a deploy is a release to production; eligible ones are those the new tool can technically handle, with approved exceptions excluded). Numerator: deploys via the new pipeline; denominator: all eligible deploys in the week. For example, week 3 has 14 of 40, so 14 / 40 = 35 percent; week 6 has 38 of 40 = 95 percent.
• Time to complete the standard task compared with the old way, for example 25 minutes to ship a one-line fix on the old path versus 10 on the new, timing the same task for two people.
• Support questions per week (should peak then fall) and rollbacks to the old way.

Longer arc (six-month upskilling)

If the tool needs deeper skill (for example an operational discipline, meaning a habit for running live systems reliably, such as incident reviews), extend: month 1-2 basics on real work, month 3-4 each person leads a task, month 5-6 they teach a newer teammate. Measure with adoption targets such as "80 percent of eligible work through the tool by month 4", which you set with the team.

Pitfalls

• Mandating before it works: the team bypasses it.
• Counting logins or attendees. Someone can attend three sessions and never use it.
• One expert becomes the single point of help; rotate the pairing so knowledge spreads.
• No end date for the old way, so both run forever at double cost.

Direct answer

Work backwards: pick the tasks mid-level engineers must be able to do afterwards, write the assessment that proves it, then choose weekly content and hands-on labs that lead there. Decide what to teach from evidence (a skills baseline, incident and review data, manager input), keep each week to one or two objectives, and assess with a capstone (a final hands-on task on data like the real thing) plus follow-up on real work.

Deciding what to teach

1. Audience and task analysis: interview a few engineers and their managers: which tasks do people currently get wrong or avoid? Run a short pre-assessment to find the starting level.
2. Outcomes as verbs: "diagnose a slow job from its execution details", not "understand performance"; likewise "rewrite a noisy alert so it fires only on customer-visible symptoms" beats "know about alerting".
3. Cut ruthlessly: anything not tied to an outcome goes to a reference list.

Structure (worked example: a 6-week observability course)

Observability means being able to understand what a running system is doing from its logs, metrics and traces (records of a request as it crosses services).

Other examples using the same skeleton:

• Spark tuning (Spark is a distributed data processing engine). Week 3 objective: read an execution plan (the engine's step-by-step description of how it will run the job) and find the costliest step. Lab: open the plan for a slow job. Week 4 objective: spot skewed data (a few keys hold most rows, for example one customer has 60 percent of all orders, so one worker does most of the work). Lab: find the skewed key. Week 5 objective: fix a slow join (a join between two large tables that is slow because data must move across machines). Lab: rewrite it and compare run times.
• Model interpretability (techniques that explain why a model made a prediction). Objective: explain a model's predictions on a realistic dataset and state where the explanation stops being trustworthy. Lab: explain three predictions, including one the model got wrong.

Assessing that people can do it

• Capstone on a task they have not seen, scored on a rubric (a scoring guide): correct diagnosis, evidence used and clarity are each scored 0 to 2, so 6 is the maximum. For example, diagnosis 2 (names the real cause), evidence 1 (cites one graph but not the log line), clarity 2 gives 5 of 6. Pass at 4 or more with at least 1 on diagnosis, because a clear write-up of the wrong cause should not pass.
• Lab checks each week so problems surface early.
• Follow-up at 30-60 days: did they apply it on real work? A manager or peer review of one artifact they produced.
• Course feedback to improve the next run.

Pitfalls

• A lecture-heavy course with a quiz at the end: people can recite, not do.
• Labs that use toy data, so nothing transfers.
• Packing too much per week; two objectives is plenty.
• Never asking what happened after the course ended.