Knowledge Sharing and Team Enablement Questions

Direct answer

Instrument at two horizons and trust behaviour over opinion. If starting small, begin with four core signals: the no-notes knowledge check, the day-90 re-test, contributor spread, and help tickets; add the rest only if you will act on them. Short term (days to weeks): a knowledge check taken without notes and lab completion. Longer term (one to six months): retention re-tests, help-desk tickets and documentation search queries on the topic, and what people actually do in the codebase (time to first merged change, spread of contributors, use of the taught practice). Distrust anything that measures attendance or satisfaction, and treat incident recovery time as a noisy, confounded signal (many things besides learning also move it). Note that transfer means people actually using what they learned on real work, which is what the longer-term signals are for.

Signals, defined precisely

Where each signal comes from (instrumentation)

• Knowledge check and re-test: a closed-book quiz form or learning-system quiz with a time limit, using the same scenario bank rephrased.
• Help tickets: the ticket tool, with a topic tag or label on each ticket, plus usage counts from the tool's own logs.
• Docs search: the documentation site's search logs joined with the ticket tool.
• Time to first merge and contributor spread: git history and the code host's author data.
• Documentation freshness: last-edited dates from the docs platform.
• MTTR: incident tracker start and resolved timestamps.

Worked example (illustrative)

Training on "deploy with the new pipeline", 20 people.

• Day 0 check: 18 of 20 pass, 90%.
• Day 90 re-test: only 14 respond and 11 of them pass. 11 / 14 = 78.6%. But 6 did not respond, so if none of them would have passed it could be as low as 11 / 20 = 55%. The honest statement is a range from 55% (no non-responder would have passed) to 85% ((11 + 6) / 20, all of them would have passed), with 79% being only the responders' rate, and I would chase the non-responders.
• Contributor spread: before, 3 of 10 engineers had changed pipeline code in the past 90 days (0.3). After, 6 of 10 (0.6).
• Help tickets: 30 tickets on 600 uses is 5 per 100 uses; after, 18 tickets on 900 uses is 2 per 100. Usage rose, so per-100 is the fair measure, but sample a few tickets to check people did not just stop asking.
• Docs search: 200 topic searches, 140 ended without a follow-up ticket = 70%; after, 180 of 200 = 90%.
• Time to first merge: new starters took a median of 12 days before and 8 days after (with a revert-rate check).
• Documentation freshness: 24 of 60 pages edited in the last 6 months = 40%; a year later 42 of 60 = 70%.
• MTTR: four incidents of 30, 45, 50 and 215 minutes average (30+45+50+215)/4 = 340/4 = 85 minutes, but the median is (45+50)/2 = 47.5 minutes; one outage swamps the mean, which is why it is a weak signal.

What I would distrust

• Satisfaction scores and attendance: they measure a pleasant afternoon.
• Immediate pass rate: it measures recall, not transfer to real work.
• Lab completion alone: copying steps is not understanding.
• MTTR: with a handful of incidents a quarter, one long outage swamps the average, and many other things move it.

Trade-offs and pitfalls

• Metrics get gamed (tiny merges to hit time-to-first-merge), so pair with a quality guard such as revert rate (the share of merged changes later undone).
• Ticket counts can fall because people stopped asking. Check by sampling.
• Decide the signals before the training, and instrument only what you will act on.

Direct answer

With six months I would sequence the transfer as a gradual handover of responsibility, from watching to co-leading to leading with the expert observing, and focus on transferring how decisions are made, not just what was decided. Documents are a by-product; the goal is that the successor can make good calls without the expert in the room.

What actually has to move

Decisions and their reasons, the stakeholder map (who depends on this work and what each of them cares about) and relationships, the risks only the expert knows about, and the judgement about trade-offs. Start with a risk register (a list of the things that could go wrong or that only the expert knows, with owner and status) listing the top items of that kind.

The six-month sequence

How to transfer judgement rather than documents

• Decision replays: show the successor an old problem without the outcome, let them propose a solution, then compare with what was actually chosen and why.
• Pre-mortems: before a decision, ask "assume this failed in a year; why?"
• Narrated decisions: the expert explains what they weighed, including what they rejected.
• Safe practice: start with reversible, low-cost decisions.

Rotation schedule and continuity signals

Put the successor into the on-call rotation (the schedule of who responds first to production alerts) or the design-owner rotation (the schedule of who is accountable for each design review's outcome) from month 3. Signals that the transfer is working:

• Share of design reviews led by the successor (target: 0 in month 1, rising to all by month 5).
• Number of decisions made without escalation.
• Times per month the expert is consulted (should trend down).
• Stakeholders confirm they go to the successor first.

Worked example (illustrative)

If 8 design reviews occur a month, the plan for reviews the successor leads is: month 1 = 0 (shadowing), month 2 = 0 (curating ADRs, not yet leading), month 3 = 2 (25 percent, the co-lead stage), month 4 = 6 (75 percent, the lead stage, with the expert taking only the 2 highest-stakes reviews), month 5 = 8 (all of them). If the successor led only 3 in month 4 instead of 6, that signals the handover is behind and I would extend month 4 rather than stepping the expert out on schedule.

Trade-offs and pitfalls

• The expert unconsciously holds on, or the successor is left alone too early; both fail the transfer.
• Handing over only documents loses the reasoning.
• What would change my call: if the expert's departure date moves earlier, compress by transferring the top items in the risk register first and postponing the rest.

Direct answer

Rank services by two things multiplied together: how much the business would hurt if the service stayed broken (criticality) and how few people could actually fix it (concentration). Concentration is the "bus factor" idea: the number of people who could disappear (leave, get sick, go on holiday) before nobody left can safely change or restore the system. I would collect cheap, objective signals from version control, the incident record and the docs, sanity-check the top of the list with a direct question to the team, and then spend my limited effort only on the top few.

Signals to use

Plain-language notes on the caveats: a squash merge combines a whole branch into one commit credited to whoever merges it; a pair commit credits one author although two people worked on it; a drive-by fix is a one-off small edit by someone who does not otherwise work on the service; paged means the alert reached that person's phone as the on-call responder; a tier list is the team's agreed ranking of services by importance (for example, tier 1 = payments, sign-in). Without an agreed tier list, ask the team to rank the services once before scoring.

# Commits per author for one service over the last 12 months
git shortlog -sn --no-merges --since="12 months ago" -- services/payments-ledger/
# Who last touched it, newest first
git log --no-merges --format='%an %ad' --date=short -- services/payments-ledger/ | head -n 20

What the flags do: -s prints only a summary count per author, -n sorts by count (highest first), --no-merges skips merge commits, --since limits the time window, and everything after -- restricts the search to that directory. head -n 20 keeps the first 20 lines. Illustrative output of the first command for payments-ledger (240 commits in total):

   204  Priya Nair
    36  Tom Weber

204 / 240 = 85 percent top share, and 2 distinct authors, which are the two numbers in the worked example below.

Scoring rule

Concentration score (0 to 3), one point each: top author has 60% or more of commits; fewer than 3 active contributors; no runbook that is current (a runbook counts as current if it was edited in the last six months or someone other than its author has followed it successfully; otherwise it is stale). Criticality: tier 1 = 3, tier 2 = 2, tier 3 = 1.

risk = criticality x concentration

The rule is deliberately crude: it is a triage list, not a measurement. It scores only the three signals that come straight from version control and the docs, because they are cheap to compute for every service. Commit staleness and incident involvement take manual digging, so I use them afterwards to check the top of the list and to break ties, not to score every service.

Worked example (illustrative services and numbers)

Ranking: payments-ledger (9), search-indexer (6), admin-ui (3), report-exporter (2), notifications (0). I would treat the first two now. Note that notifications is tier 1 yet last, because many people already know it, and admin-ui is fully concentrated but low stakes, so it can wait.

Validate and act

• Ask the top five "who could restore this alone at 3 a.m.?" The git data suggests candidates, the answer is the truth.
• Tie-break equal scores by pages per quarter: the service that actually breaks needs knowledge sooner.
• Treatment is proportional: runbook plus a second on-call for the top two, a lighter pairing plan for the rest, and an explicit decision to accept the tail.

Trade-offs and pitfalls

• Commit counts measure typing, not understanding; the incident record measures operating. Prefer both.
• Signals get gamed (splitting commits, one-line edits to fake contributors), so keep them as triage inputs rather than performance targets.
• A monorepo (one repository holding many services) or generated code (files written by a tool rather than a person) inflates one author; exclude generated paths.
• I would flip the ranking for a fast-approaching departure or leave: a known exit date raises that person's services above everything.

Direct answer

Time-to-productivity is the elapsed time from a new hire's start date until they meet an agreed bar of independent, useful work. Because "productive" is vague, I would define the bar as a few observable milestones, measure each from an existing system, and report the day the last milestone is met. I would use it to judge the onboarding process (everything that brings a new hire up to speed: access, training, a buddy, first tasks), not to rate the individual.

Defining it for a team

Three milestones and their sources:

1. Time to first production change: days from start to the first change merged and deployed. Source: version control and deployment history, collected automatically.
2. Time to independent delivery: days until the hire finishes a first "standard" ticket (pre-tagged by someone else as typical size) without needing more than ordinary review. Source: ticketing labels and review history, confirmed by the manager.
3. Checklist completion: at 30, 60, 90 day check-ins, hire and manager mark each competency (owns a ticket, reviews others' changes, handles an alert) as "independent", "with help" or "not yet", and record the date each competency was first seen independent (from the ticket, review or alert that showed it). The completion day is the date of the last competency, not the check-in date, so it can fall between check-ins. Source: a simple form or the learning system.

Time-to-productivity = the day on which all three are met (the maximum of the three milestone days).

Worked example

Three hires, milestone day counts:

Read the checklist column as dates logged this way: hire A's last competency was seen on day 55 and confirmed at the day-60 check-in, hire C's on day 48 (confirmed at day 60), and hire B's only at the day-90 check-in. Median for independent delivery is 38 days (values 33, 38, 71); median time-to-productivity is 55 days (48, 55, 90). Note hire B: fastest to a first merge at 3 days, slowest to independence at 71. That is exactly why first-merge alone misleads: a typo fix counts the same as a real feature.

Guarding against gaming

• Hire rushes trivial tickets: the "standard" tag is set by someone other than the hire.
• Manager ticks boxes early to look good: sample reviews and check the milestone against real tickets.
• Pressure on people: report cohort medians, never use it as a personal performance score.
• Team comparisons: teams with harder systems will ramp slower; compare a team with its own past cohorts.

Trade-offs and pitfalls

• With small numbers, medians beat means; a single slow hire changes the mean a lot.
• A short time is not automatically good: hires who were rushed may need rework later; watch quality signals (review rework, incidents caused).
• Slow ramp usually points at the system (access delays, stale docs, no buddy), so read the number with a diagnostic conversation.

Direct answer

I would let the newcomers do the writing and the expert do the correcting, because an overloaded expert asked to author documentation becomes the bottleneck the question warns about. I would capture the knowledge that carries the most risk first (how to run, deploy and roll back (undo a release by restoring the previous version); the dangerous areas; the top failure modes), protect the expert with fixed office hours and a public question log, and make contributions safe with tests and clear review rules so people can start changing code within the first weeks.

Step 1: capture what matters, in risk order

1. How to run and deploy, and how to roll back.
2. A one-page map: main components, data flow (how data moves between them), where the risky areas are ("changes here need the expert").
3. Runbooks (step-by-step operating guides) for the top three recurring incidents, taken from tickets and chat history.
4. Decision records (short notes of why a choice was made) for surprising design choices.
   For a data science team the equivalent artifacts are a data dictionary (what each column means and where it comes from), pipeline runbooks, and model cards (one-page descriptions of a model's purpose, training data, evaluation and known limits), each with a named owner.

Step 2: protect the expert's time

• A fixed budget, for example two 60-minute office hours a week plus about two hours of asynchronous review; anything beyond goes to the queue.
• A public "questions for the expert" channel; every answer is copied into the docs as an FAQ entry, so each question is paid for once.
• Record the first architecture walkthrough (90 minutes, once) instead of repeating it.

Step 3: make contribution safe

• Characterization tests: tests that record what the code does today, so a change that alters behaviour is caught even where nobody knows the intent. A minimal example in Python (runnable as written), where the expected value 17.05 was obtained by running today's code, not from a specification:

def calculate_total(prices, tax_rate):
    return round(sum(prices) * (1 + tax_rate), 2)

# characterization test: records current behaviour (10.00 + 5.50 = 15.50, plus 10% tax)
assert calculate_total([10.00, 5.50], 0.10) == 17.05

For an analyst or data scientist, the same idea is to save today's row count and total for a report over a fixed date range, and alert when a change to the pipeline alters them without a known reason.

• A CODEOWNERS file (a file in the repository that names required reviewers for each folder or file path), with the expert required only on the risky paths; everything else reviewed by trained peers.
• A list of small starter tasks: a typo fix in the docs, then a config change, then a small feature.

Worked example (illustrative)

Team of five inherits a billing service; the expert has 4 hours a week. Week 1: recorded walkthrough (1.5 hours) and a newcomer writes the deploy runbook while following it. Weeks 2 to 3: two office hours a week (2 hours) plus review (2 hours) fit the 4-hour budget; newcomers write the map and the first two incident runbooks. Weeks 4 to 6: starter tasks merged with peer review, with the expert only on payment paths. Signal of success: time to a newcomer's first merged change, and the number of questions that could be answered from the docs.

Trade-offs and pitfalls

• Documentation written by the expert alone is slow and reflects what they think is obvious; newcomer-written docs expose the real gaps.
• Big documentation projects go stale; write what a task needs, at the moment it is needed.
• Tests around unclear behaviour can lock in bugs; label them as characterization tests, not specifications.