Documentation and Knowledge Management Questions

Direct answer

For about 2,000 engineers I would build a small set of well-understood components around docs-as-code (docs in the repos), a central index for search, identity-driven access, CI checks, and an analytics layer, with lightweight governance built on owners and review dates. Keep simple: one authoring format, one search engine, no bespoke editor, and no machine-learning features until search logs prove a need.

Terms

• Docs-as-code: docs stored as Markdown in repos and reviewed like code.
• CI (continuous integration): automated checks on each change.
• ACL / RBAC: access control lists / role-based access control (permissions granted by role or group).
• SSO: single sign-on, one login for all tools.
• Staleness: a doc likely out of date because time passed or its code changed.
• Content gap: a question people search for that has no good page.
• Front matter: a small metadata header at the top of a Markdown file (owner, type, sensitivity, last_reviewed).
• Documentation guild: a small volunteer group with one representative per major area that maintains shared standards. It advises and sets templates, unlike a central team that writes or approves everything.
• Federating ownership: each team owns its own docs instead of one central team owning all of them.
• Directory-to-group mapping: the company's employee directory (who reports to which team) is used to create the user groups that permissions follow, so a reorg updates access automatically.
• Boost: nudging certain results higher in search ranking. Drift: a doc slowly becoming wrong as the code changes.

Sizing (illustrative assumptions, not measurements)
Suppose 2,000 engineers in about 250 teams, each team owning roughly 20 pages: about 5,000 pages. With a review every 6 months on average, that is about 830 reviews a month, or 3 to 4 per team per month, which is why review dates must be automated rather than chased by hand. If each engineer searches about 3 times a week, that is roughly 6,000 searches a week, a small load that one managed search engine handles comfortably. This is the reasoning behind keeping the platform simple.

Components

Governance

• Ownership: every page has an owner team; reorganizations transfer ownership via a directory-to-group mapping.
• Review cadence: reviewed within its window (for example quarterly for runbooks, longer for concepts).
• Standards: a short style guide and templates, maintained by a small documentation guild with representatives from major areas rather than a central gatekeeping team.
• Escalation: unowned or repeatedly stale critical pages go to the engineering director for that area.

Analytics that drive action

• Usage: the top viewed and searched pages get quality investment first.
• Gaps: queries with zero results or immediate re-search become a backlog for owners.
• Staleness: pages past review date, grouped by team, in a monthly leadership report.
• Onboarding: time for a new hire to complete first tasks, and which pages they visit.

Global scale extras

• Multi-language: keep English as the source of truth, translate the top pages (by usage) rather than everything, and label translations that lag the source.
• Personalized ranking: boost by team, region and role from directory data. Simple boosts beat learned models here.
• Page-level access across business units: group-based ACLs with an audit log; keep the restricted set small so defaults stay open.
• Optional machine learning summarization: defer until search analytics show people struggle with long pages, run it as a pilot on non-sensitive content, and require it to cite its sources. The justification test is a measurable improvement in time to answer against the baseline, and a cost within budget.

Where I deliberately keep it simple

• One authoring format instead of supporting many.
• Off-the-shelf search and hosting instead of building.
• Permissions by group, not per-person exceptions.
• Automated freshness signals instead of manual audits.
• No custom ranking algorithm at launch.

Worked example (illustrative)
A payments engineer changes a service's timeout config. CI notes that the linked runbook was not touched, and the owner adds a one-line update in the same pull request. Meanwhile analytics shows "how to rotate keys" returns no results, and the security guild writes the missing page. Both are cheap loops driven by data.

Trade-offs and pitfalls

• Strict CI gates breed workarounds; use warnings first, and block only for the critical rules.
• Analytics can mislead: high views may mean confusion, so pair with feedback.
• Federating ownership scales better than a central team but needs the guild for consistency.
• What would change my call: heavy regulation would raise access and audit requirements, and a smaller organization could skip the guild and analytics dashboards.

Direct answer

Do not pick a winner org and delete the other. Run a time-boxed reconciliation: inventory both sets, sort every conflict into a small number of types, let a named decider resolve true conflicts through a new decision record that supersedes (never erases) the old ones, archive the old material read-only with redirects, and make one index the only place anyone is told to look. Start with the documents that hurt during an incident or a deploy, not the ones that are merely ugly.

Terms used below

• ADR (architecture decision record): a short dated document recording one decision, the options considered, and why one won. It has a status such as Proposed, Accepted or Superseded.
• Runbook: step-by-step instructions for an operational task, such as failing over a database.
• Authoritative version: the one copy a reader is entitled to trust and follow.
• Terraform: a tool that describes cloud infrastructure (servers, networks, databases) in code files. Ansible: a tool that configures existing servers from scripts.
• Jenkins and GitHub Actions: two competing services that automatically build and test code when it changes.
• OAuth: a standard way for one service to be granted limited access to another without sharing passwords, either on a user's behalf or, for service-to-service authentication as in the table below, as a short-lived access token issued to the calling service itself.
• Tier-1 service: a service whose outage pages people or stops revenue (tier 2 matters but degrades gracefully; tier 3 is internal or low impact).
• Staging: a safe copy of the production environment used to test changes.
• History-preserving import: copying a repository so every past change and author is kept, not just the latest files.
• Redirect: an old web address automatically forwarding to the new page. Stub: a tiny placeholder page that only points to the new location.
• Sunset date: the date after which something stops being supported. Carve-out: a deliberate exception to a new rule. Claim window: a fixed period in which owners may say "this page is still needed" before it is archived.

1. Inventory and triage (weeks 1-2)

List every ADR, runbook and standards page from both orgs with owner, service, last-verified date and format. Then classify each conflict:

In practice you expect many conflicts to fall into the first, second or fourth type (this is an expectation, not a measured figure; your own inventory will show the real split). Spend the debating effort only on true conflicts.

2. What happens to conflicting decisions

• Each true conflict gets a reconciliation ADR with one named decider (the architect or engineering lead who owns the affected system, not a committee vote). It lists both original ADRs, the criteria (cost, migration effort, who runs it, security posture), the choice and a migration deadline.
• The losing ADR is not deleted. Its status becomes Superseded by M-0004 and it keeps its text. Reason: engineers will ask "why did the other org do it that way?" for years, and the reasoning is often still valid for a corner case.
• If a decision cannot be settled quickly, record it as Both accepted, sunset date TBD with an owner. An explicit temporary state beats silent divergence.
• Rule for the interim: whichever ADR governs new work is the reconciliation ADR; existing systems follow the old one until migrated.

3. Preserving history

• Import each org's docs repo into a read-only archive keeping full version history (for git repos, a history-preserving import rather than copy-paste).
• Keep original numbers and add an org prefix: A-0007 and B-0007 cannot collide, and old links in code comments and tickets still resolve. New reconciliation ADRs written after the merger use the prefix M- (M for merged company), for example M-0004.
• Old wiki URLs get a redirect to the new page, or, if the page was archived, to the archive copy with a banner.

4. Runbook consolidation

Pair runbooks by service. For each pair, keep the one that was most recently exercised or used in a real incident, then merge in the steps the other had that the kept runbook lacks, in a single new template. Where two runbooks disagree on a command, test it in staging before choosing. Tier-1 services (paging, revenue path) first.

5. Telling engineers where the truth lives

• One documentation portal or repo with a front-page index: "Authoritative docs for service X".
• Every page shows a status banner (Authoritative, Superseded, Archived) and a last-verified date.
• Every old page becomes a stub: "Moved. Do not follow this copy." with a link.
• Pager alerts and CODEOWNERS files (a repo file that maps paths to owning teams) point to the new runbook URLs. This matters more than any announcement, because engineers follow the link in the alert.
• Announce once, with the top ten most-searched pages, then rely on redirects.

Worked example

Org A's A-0012 says "Postgres on managed cloud". Org B's B-0031 says "self-hosted MySQL on VMs". The merged company has two production databases for one product. The database architect decides: Postgres managed for all new services, MySQL kept for two legacy services until a migration ticket closes. Result: new ADR M-0004 (Accepted), B-0031 marked Superseded by M-0004 (legacy carve-out until migration), and the two db-failover runbooks merged into one Postgres runbook plus a short MySQL appendix that carries a removal date.

Trade-offs and pitfalls

• Reformatting everything to one template up front is a trap: it burns months on pages nobody reads. Migrate tier-1 now and everything else "on touch" or by archiving after a claim window.
• Letting both orgs keep their own portal "for now" is how the duplicates return.
• A committee that seeks consensus stalls; one decider with a written rationale is faster and leaves a record.
• What would change my call: if one org's systems are being retired within months, archive that org's docs instead of reconciling them.

Direct answer

Treat "stale or forgotten" as several independent signals, score each 0 to 1, combine them with fixed weights into a 0 to 100 staleness score per runbook, and set the action threshold by how critical the service is. Add a hard override: if a real incident said the procedure was wrong, act regardless of the score. Avoid alert fatigue by sending weekly digests by default and opening a ticket only past the threshold, capped per owner.

Terms. A runbook is a step-by-step procedure for an operational task. An SRE (site reliability engineer) team runs production and relies on runbooks during incidents. A tier ranks a service's criticality (tier 1 is paging or revenue-critical). A paging alert is an alert that wakes someone up. Drilled means rehearsed on purpose before a real emergency. A dry run executes the steps against a safe copy with no real effect. The inventory is the current list of live hosts and services (from your configuration database). A digest is one summary message instead of many alerts. To snooze is to silence a reminder until a set date. An archive candidate is a runbook that looks safe to retire. To deduplicate is to keep one item where there would be several.

Inputs (each scaled 0 to 1, where 1 is worst)

Why these shapes and weights:

• Capped at 2, then halved. Age is measured against the review interval. A runbook exactly at its interval scores 1 before halving, so 0.5 after. Capping at 2 means anything more than twice overdue is simply maximally stale (1.0 after halving) rather than letting a 10-times-overdue page dominate the score.
• Proved wrong weighs most (0.30) because it is direct evidence from a real incident. Dead references (0.20) are hard, checkable facts. Age and unexercised (0.25, 0.15) are only proxies. Owner gone (0.10) is a flag more than a measurement.

Weights are a judgement. Calibrate them by backtesting, meaning you replay the past: compute scores as they stood last quarter, then check them against what actually went wrong afterwards. Illustrative example: 4 runbooks were flagged wrong in this quarter's incidents. If 3 of those 4 were among last quarter's top-scoring runbooks, the score caught 75%. If only 1 of 4 was, raise the weight on dead references or add a signal, and re-check.

When to act

Tier 1 opens a ticket at 45 or above, tier 2 at 60, tier 3 at 80. Tier 1 opens earliest because a stale runbook there costs the most during an outage, and its review interval is shortest (90 days, versus 180 and 365). These numbers are starting values you tune from the backtest, not laws. Below the threshold the runbook appears only in the weekly digest. Override: one or more "proved wrong" flags opens a ticket whatever the score.

Runnable computation (pinned data, prints the result)

from datetime import date

TODAY = date(2026, 9, 28)
WEIGHTS = {"age": 0.25, "unexercised": 0.15, "wrong": 0.30, "dead_refs": 0.20, "owner": 0.10}
# tier -> (score that opens a ticket, review interval in days)
TIERS = {1: (45, 90), 2: (60, 180), 3: (80, 365)}

# name, tier, last_reviewed, last_exercised (used or drilled), uses_in_90d, flagged_wrong_in_90d,
# dead_refs, total_refs, owner_active
RUNBOOKS = [
    ("db-failover",      1, date(2026, 2, 10), date(2026, 3, 1),  1, 1, 3, 10, True),
    ("cache-flush",      2, date(2026, 8, 20), date(2026, 9, 10), 2, 0, 0,  6, True),
    ("legacy-batch-job", 3, date(2025, 6, 1),  date(2025, 6, 1),  0, 0, 4,  5, False),
    ("cert-rotation",    1, date(2026, 8, 30), date(2026, 9, 1),  0, 0, 1, 12, True),
    ("queue-backlog",    2, date(2026, 3, 15), date(2026, 8, 2),  3, 0, 1,  8, True),
]

def score(tier, reviewed, exercised, uses, wrong, dead, total, owner_active):
    interval = TIERS[tier][1]
    parts = {
        "age": min((TODAY - reviewed).days / interval, 2) / 2,
        "unexercised": min((TODAY - exercised).days / interval, 2) / 2,
        "wrong": (wrong / uses) if uses else 0.0,
        "dead_refs": dead / total,
        "owner": 0.0 if owner_active else 1.0,
    }
    return 100 * sum(WEIGHTS[k] * v for k, v in parts.items()), parts

def action(tier, s, wrong):
    if wrong >= 1:
        return "OPEN TICKET (incident said it was wrong)"
    return "open ticket" if s >= TIERS[tier][0] else "weekly digest only"

print(f"{'runbook':<17}{'tier':>4}{'score':>7}  action")
for name, tier, rv, ex, uses, wrong, dead, total, owner in RUNBOOKS:
    s, _ = score(tier, rv, ex, uses, wrong, dead, total, owner)
    print(f"{name:<17}{tier:>4}{s:>7.1f}  {action(tier, s, wrong)}")

Output:

runbook          tier  score  action
db-failover         1   76.0  OPEN TICKET (incident said it was wrong)
cache-flush         2    3.5  weekly digest only
legacy-batch-job    3   52.5  weekly digest only
cert-rotation       1    7.9  weekly digest only
queue-backlog       2   18.6  weekly digest only

Check one by hand. db-failover (tier 1, interval 90 days): reviewed 230 days ago, so age is capped at 1.0. Last exercised 211 days ago, also capped at 1.0. It was used once and flagged wrong once, so wrong is 1.0. Dead references are 3 of 10, so 0.3. Owner is active, so 0.

$100 \times (0.25 \times 1 + 0.15 \times 1 + 0.30 \times 1 + 0.20 \times 0.3 + 0.10 \times 0) = 76.0$

Now a low row. cache-flush (tier 2, interval 180 days): reviewed 39 days ago, so age is 39/180 = 0.217, halved to 0.108. Last exercised 18 days ago, so 18/180 = 0.1, halved to 0.05. It was used twice with no wrong flags, has no dead references and an active owner, so those are 0.

$100 \times (0.25 \times 0.108 + 0.15 \times 0.05) = 3.5$

Recent review plus recent use gives a near-zero score, so it stays out of the ticket queue.

Also note legacy-batch-job scores 52.5, which is below tier 3's threshold of 80, so it stays in the digest. Its dead references and missing owner make it an archive candidate, not something to nag anyone about.

Avoiding alert fatigue

• Default channel is one weekly digest per team, sorted by score.
• Tickets, not pages: a stale runbook is never a 3 a.m. alert.
• Cap open stale-runbook tickets per owner (for example three) so a backlog does not flood them.
• Deduplicate: one ticket per runbook, updated, not re-created.
• Snooze with a reason and an expiry date, so silence is a decision.

How it gets gamed, and the guard

• Rubber-stamp reviews: someone bumps the date without reading. Guard: a review counts only if it includes a checked step (a dry run or a drill signed by someone other than the author).
• Deleting dead links to zero the reference score: guard by tracking the reference count as well.
• Archiving a runbook to remove it from the list: guard by requiring that a paging alert never points to an archived page.

Pitfall. A single number hides the reason. Show the top contributing signal next to the score so the owner knows what to fix.

Direct answer

Treat the migration as content curation first, tooling second: inventory everything, decide keep, merge or retire using a simple scoring rule, put owners and metadata on every kept page BEFORE moving it, migrate in waves ordered by operational criticality, leave redirects behind, and judge success by whether people find answers faster, not by how many pages moved.

Terms

• Inventory: a spreadsheet listing every doc, where it lives, who wrote it and when it last changed.
• Redirect: an old link that automatically forwards to the new location.
• Cutover: the moment the new system becomes the official source.
• Runbook: step-by-step instructions for operating or fixing a system.

Phase 1: Inventory and triage (weeks 1-2)
Crawl the wikis, shared drives and repos, and add the informal sources: chat channels and personal notes. Record for each item: location, owner, last edit, views if known, and which system it describes.

Decide keep, merge or retire with a short rule:

Worked example (illustrative arithmetic)
Inventory finds 400 items. Triage results: 120 keep, 90 merge (they collapse into 30 pages), 190 retire. The new knowledge base receives 120 + 30 = 150 pages, 37.5 percent of the original. Every item still has a recorded fate in the inventory, so the numbers reconcile: 120 + 90 + 190 = 400.

Phase 2: Structure, ownership, findability

• Target structure: a few top-level areas (for example Services, Runbooks, Processes, Onboarding), not a mirror of the org chart.
• Tags: a small controlled list (team, system, lifecycle, audience); no free-form tags at launch.
• Every page has an owner team and a last-reviewed date; pages without an owner are not migrated.
• Search: index titles, headings and tags, boost pages by verified freshness, and test with a list of the 20 most common real questions.

Phase 3: Migrate in waves, by operational criticality
For knowledge that lives only in chat channels and personal notes, rank by "what hurts if it is missing at 3am": on-call runbooks and recovery steps first, then setup and onboarding, then background context. Capture chat knowledge by turning a decision or fix into a proper page (with the thread linked), not by dumping the export. Set a measurable onboarding-speed goal, for example that a new engineer completes the first supervised on-call shift using only the knowledge base, and measure the median days before and after.

Phase 4: Old material and cutover
Freeze the old wikis to read-only with a banner pointing to the new location, keep redirects for a defined period, archive (do not delete) retired items for legal or historical reasons, then switch off after traffic to old links falls.

Success measures

• Search: share of searches with a click and no immediate re-search, and zero-result queries falling.
• Coverage: percent of critical systems with an owned, current runbook.
• Freshness: percent of pages reviewed within their window.
• Onboarding speed (above) and fewer "where is the doc for X" questions in chat.

Trade-offs and pitfalls

• Migrating everything "just in case" copies the rot into the new home.
• No owners means the new system decays as the old one did.
• A big-bang cutover is riskier than waves, but waves prolong two sources of truth, so set a firm end date.
• What would change my call: a small team with 50 pages should skip scoring and simply re-read them.

Direct answer

Keep documentation as files in the code repositories (docs-as-code), build them into one site on every merge, and index the built content in a search service that filters results by the searcher's permissions. Ownership lives in the repo (a CODEOWNERS file), review is the normal pull request, and freshness is enforced by automated checks and review dates. That is enough for a modest scale; do not build a custom platform.

Terms

• Docs-as-code: writing docs in plain text (Markdown) inside the repository and reviewing them like code.
• CODEOWNERS: a repository file naming who must approve changes to given paths.
• CI (continuous integration): automated checks that run on every change.
• ACL (access control list): a rule on who may view an item.
• Audit log: a record of who accessed or changed what.
• Front matter: a small metadata header at the top of a Markdown file (owner, type, sensitivity, last_reviewed) that tools can read.
• Facets: clickable filters in search results, such as by team or page type. Typo tolerance: still finding "kubernetes" when someone types "kuberentes".
• Link unfurl: the preview card a chat tool shows when someone pastes a link. Lineage: a record of which data sources feed which datasets or reports.
• Model card: a short standard document describing what a machine learning model is for, its training data and its limits. Metric spec: a written definition of a business metric.

Architecture (simple)

Repos (Markdown + front matter) -> CI build and checks -> Static docs site
                                         |
                                         v
                              Search index (with access groups per page)
                                         ^
   User -> single sign-on -> search API filters hits by user's groups

Search choices

{ "title": "Checkout runbook", "body": "...", "allowed_groups": ["payments-oncall"] }
query: text = "checkout runbook" AND allowed_groups contains any of (caller's groups)

The filter is part of the query, so a non-member gets zero hits back, not hits that are hidden afterwards.

Authoring and review workflow

1. Author edits Markdown in the repo, using a template (runbook, architecture decision record or ADR, how-to guide) with front matter: owner, type, sensitivity, last_reviewed.
2. Pull request triggers CI: link checker, formatting, required front matter, spelling, and a check that code-linked pages changed when the code did (or a note explains why not).
3. CODEOWNERS approval merges, and the site rebuilds.

Ownership model
Every page has an owner team via CODEOWNERS or front matter. A weekly report lists pages past their review date, sent to the owner team. Orphans (owner team deleted) escalate to the engineering manager.

Keeping content current

• Review-by dates and a staleness banner after the date.
• CI flags pages whose linked code has changed since last review.
• Usage data: heavily used but stale pages get priority.
• Deleting is a feature: archive deprecated pages with redirects.

Common variations (optional extras, rarely the first question in an interview; same skeleton, different fields)

• Machine learning (ML) or data organization: add model cards and runbooks as typed pages, restrict sensitive ones by group, and keep an audit log of reads and edits. Auto-generated metadata (schemas, lineage) merges with human-written pages, and CI keeps the two consistent.
• A wiki serving about 100 teams: add versioned templates for runbooks and ADRs (a template for architecture decision records), a tag list with a steward, ranking by freshness and usage, and Slack and GitHub integrations (link unfurls, a bot posting stale-page reminders).
• BI artifacts: metric specs and dashboard templates as typed records with owner, definition and access, searchable alongside the rest.
  I would keep all of these at component level, adding a variant only when a team needs it.

Worked example (illustrative)
An SRE edits the checkout service runbook in the repo. CI fails because the front matter (the metadata header) lacks an owner, so she adds one. The reviewer from the owning team approves, the site rebuilds, and the index updates with group "payments-oncall". A contractor outside that group searches "checkout runbook" and sees no hit. Three months later the owner gets the stale-page report.

Trade-offs and pitfalls

• Docs-as-code excludes non-engineers unless there is an editing path (a web editor over the repo), so plan one.
• Permission syncing between repos and the index is the hardest part. Test it with a deny case.
• Consistency versus speed: generated metadata is fast but shallow, human docs deeper but stale. Label which is which.
• What would change my call: heavy non-technical contributors favor a hosted wiki with an export to the repo.