Documentation and Knowledge Management Questions

Direct answer

Decide from criteria tied to the actual problem (people cannot find or trust knowledge), not from a preference for building. My default recommendation for most organizations is to buy or adopt a hosted product for the reading and editing experience, and keep the parts that must live with the code (reference docs, runbooks tied to services) in the repository. Build only if a specific requirement rules every product out, because a knowledge platform is a long-lived product that needs an owner, on-call, and roadmap forever.

Terms used below

• Docs-as-code: writing docs as Markdown files in the same Git repository as the code, reviewed by pull request like code.
• Data residency: a rule about which country or region your data may be stored in. Data sovereignty is the stricter legal version: the data must stay under one country's laws.
• Group sync: automatically copying your company's user groups (for example "SRE team") into the tool so page permissions follow them.
• Audit log: a tamper-resistant record of who viewed or changed what. Retention: how long content and history are kept before deletion.
• Per-seat pricing: paying per user per month, so cost grows with headcount. Exit (lock-in): how hard it is to leave, mainly whether you can export everything cleanly.
• Self-hosted open-source: free software you install and run yourself, so you own the operations work.

Criteria and weights (for a mid-sized engineering organization)

Weights depend on context; the point is to write them down before demos, so the vendor with the best presentation does not set them.

Worked example: SRE organization, three options

Score each option 1 to 5 per criterion (5 is best), multiply by the weight, and add up. The options: a hosted wiki product, repo-based docs, and a newer all-in-one workspace (a hosted tool combining notes, docs and small databases). Illustrative scores for this SRE org:

For repo docs, code linking is 5 so 0.15 x 5 = 0.75, while search is 2 so 0.20 x 2 = 0.40. Across all content the wiki wins (3.90). But the weights assume one blended pile of content. Runbooks and service docs are a different class: they change with code and need review, so re-weight for them (code linking 0.35, search 0.15, permissions 0.10, compliance 0.10, cost 0.15, migration 0.10, integrations 0.05, which again sum to 1.00) and recompute: repo docs 3.60, wiki 3.35, all-in-one 3.20. Repo docs win that class.

That is why the recommendation is a hybrid: repo-based docs for runbooks and service docs, plus the wiki for broad non-code material (policies, onboarding, meeting notes). The matrix picks the best tool per content class, and a hybrid is defensible when two classes rank differently. Then sanity check the ranking against your gut feeling, and if they disagree a criterion is probably missing.

Build versus buy

• Build makes sense only with a hard constraint: data may not leave your environment, a deep integration no product offers, or scale where per-seat pricing is prohibitive.
• Buy wins when time to value matters and the team cannot staff a product. Include the 3-year staffing cost of build: at least one engineer's ongoing attention.

What would change my answer

• A strict data-sovereignty rule pushes to self-hosted open-source or build.
• A pilot showing search relevance below a threshold (say fewer than half of the 20 test questions land in the top three) disqualifies that product regardless of price.
• If exit is hard (no clean export), demote that product unless the discount is large.

Defending the decision

Run a two-week pilot with real content and real users, publish the scored matrix and the assumptions, and record an ADR (architecture decision record, a short document capturing a decision, options and reasons) so the decision can be revisited when assumptions change.

Direct answer

Do not pick a winner org and delete the other. Run a time-boxed reconciliation: inventory both sets, sort every conflict into a small number of types, let a named decider resolve true conflicts through a new decision record that supersedes (never erases) the old ones, archive the old material read-only with redirects, and make one index the only place anyone is told to look. Start with the documents that hurt during an incident or a deploy, not the ones that are merely ugly.

Terms used below

• ADR (architecture decision record): a short dated document recording one decision, the options considered, and why one won. It has a status such as Proposed, Accepted or Superseded.
• Runbook: step-by-step instructions for an operational task, such as failing over a database.
• Authoritative version: the one copy a reader is entitled to trust and follow.
• Terraform: a tool that describes cloud infrastructure (servers, networks, databases) in code files. Ansible: a tool that configures existing servers from scripts.
• Jenkins and GitHub Actions: two competing services that automatically build and test code when it changes.
• OAuth: a standard way for one service to be granted limited access to another without sharing passwords, either on a user's behalf or, for service-to-service authentication as in the table below, as a short-lived access token issued to the calling service itself.
• Tier-1 service: a service whose outage pages people or stops revenue (tier 2 matters but degrades gracefully; tier 3 is internal or low impact).
• Staging: a safe copy of the production environment used to test changes.
• History-preserving import: copying a repository so every past change and author is kept, not just the latest files.
• Redirect: an old web address automatically forwarding to the new page. Stub: a tiny placeholder page that only points to the new location.
• Sunset date: the date after which something stops being supported. Carve-out: a deliberate exception to a new rule. Claim window: a fixed period in which owners may say "this page is still needed" before it is archived.

1. Inventory and triage (weeks 1-2)

List every ADR, runbook and standards page from both orgs with owner, service, last-verified date and format. Then classify each conflict:

In practice you expect many conflicts to fall into the first, second or fourth type (this is an expectation, not a measured figure; your own inventory will show the real split). Spend the debating effort only on true conflicts.

2. What happens to conflicting decisions

• Each true conflict gets a reconciliation ADR with one named decider (the architect or engineering lead who owns the affected system, not a committee vote). It lists both original ADRs, the criteria (cost, migration effort, who runs it, security posture), the choice and a migration deadline.
• The losing ADR is not deleted. Its status becomes Superseded by M-0004 and it keeps its text. Reason: engineers will ask "why did the other org do it that way?" for years, and the reasoning is often still valid for a corner case.
• If a decision cannot be settled quickly, record it as Both accepted, sunset date TBD with an owner. An explicit temporary state beats silent divergence.
• Rule for the interim: whichever ADR governs new work is the reconciliation ADR; existing systems follow the old one until migrated.

3. Preserving history

• Import each org's docs repo into a read-only archive keeping full version history (for git repos, a history-preserving import rather than copy-paste).
• Keep original numbers and add an org prefix: A-0007 and B-0007 cannot collide, and old links in code comments and tickets still resolve. New reconciliation ADRs written after the merger use the prefix M- (M for merged company), for example M-0004.
• Old wiki URLs get a redirect to the new page, or, if the page was archived, to the archive copy with a banner.

4. Runbook consolidation

Pair runbooks by service. For each pair, keep the one that was most recently exercised or used in a real incident, then merge in the steps the other had that the kept runbook lacks, in a single new template. Where two runbooks disagree on a command, test it in staging before choosing. Tier-1 services (paging, revenue path) first.

5. Telling engineers where the truth lives

• One documentation portal or repo with a front-page index: "Authoritative docs for service X".
• Every page shows a status banner (Authoritative, Superseded, Archived) and a last-verified date.
• Every old page becomes a stub: "Moved. Do not follow this copy." with a link.
• Pager alerts and CODEOWNERS files (a repo file that maps paths to owning teams) point to the new runbook URLs. This matters more than any announcement, because engineers follow the link in the alert.
• Announce once, with the top ten most-searched pages, then rely on redirects.

Worked example

Org A's A-0012 says "Postgres on managed cloud". Org B's B-0031 says "self-hosted MySQL on VMs". The merged company has two production databases for one product. The database architect decides: Postgres managed for all new services, MySQL kept for two legacy services until a migration ticket closes. Result: new ADR M-0004 (Accepted), B-0031 marked Superseded by M-0004 (legacy carve-out until migration), and the two db-failover runbooks merged into one Postgres runbook plus a short MySQL appendix that carries a removal date.

Trade-offs and pitfalls

• Reformatting everything to one template up front is a trap: it burns months on pages nobody reads. Migrate tier-1 now and everything else "on touch" or by archiving after a claim window.
• Letting both orgs keep their own portal "for now" is how the duplicates return.
• A committee that seeks consensus stalls; one decider with a written rationale is faster and leaves a record.
• What would change my call: if one org's systems are being retired within months, archive that org's docs instead of reconciling them.

Direct answer

A month of heavy use followed by a sharp fall is usually a novelty spike hitting a product that does not fit people's daily work: the content was thin or stale, search did not find things, or nothing sent people back. So diagnose from data first (search behaviour, who left, what they wanted), then interview a handful of lapsed users, and only then choose fixes. Plan in four steps: quick fixes within two weeks, habit hooks within six weeks, governance within a quarter, and a review of the platform choice last, with metrics defined before you start so success is not just "more page views".

Terms

• Novelty spike: a burst of use because something is new, not because it is useful.
• Launch cohort retention: of the people who used it in the launch month, the share still using it now.
• Zero-result search: a search that returned nothing.
• Habit hook: a place in people's normal daily tools that sends them to the wiki without them having to remember it.
• Chat link previews: when a wiki link pasted in chat expands to show the page title and first lines.
• Bounce: a visit where the person leaves after one page without doing anything else.
• Synonyms (in search): telling the search engine that different words mean the same thing.

Step 1: Diagnose (week 1)

Metric definitions (numerator, denominator, window)

• Weekly active readers = distinct engineers who opened a page or searched in a 7-day window, divided by engineers eligible to use it.
• Search success rate = searches followed by a click on a result within the same session, divided by all searches.
• Zero-result rate = searches returning nothing, divided by all searches.
• Freshness = pages viewed in the window that were verified within their review interval, divided by pages viewed.

Illustrative worked example. Suppose 200 engineers. Launch month: 150 weekly active (75 percent). Now: 60 (30 percent). Of 1,000 searches last week, 320 returned nothing (32 percent) and 400 led to a click (40 percent). Sort the 1,000 searches into three groups: 320 returned nothing (32 percent), 280 returned results but nobody clicked (28 percent), and 400 led to a click (40 percent). So 600 searches, six in ten, ended without a click, and only 40 percent succeeded. Roughly half of the failures (320 of 600) mean missing content or a vocabulary mismatch, and the other 280 mean results were shown but looked irrelevant or stale. Both are search problems, so search is the main leak and search fixes come before governance.

Step 2: Prioritized plan

1. Quick fixes (weeks 1-2): create or redirect pages for the top zero-result queries, add synonyms (for example "on-call" and "pager"), fix or archive the 20 most-viewed stale pages, and add chat link previews so wiki links are useful where people already talk.
2. Habit hooks (weeks 2-6): link runbooks from alerts, link onboarding checklists, and make "put it in the wiki" the default answer in team channels.
3. Governance (weeks 4-12): each space has an owner, each page a review-by date, stale pages get a visible "unverified" banner then archive, and there is a simple contribution guide with a page template.
4. Review the platform choice last. Replacing the tool is expensive and rarely the root cause; only reopen it if search relevance cannot be fixed.

Measuring success

Target weekly active readers recovering to a stated level (say back toward 60 percent: that is a judgment call, not a law. Launch was 75 percent, and part of that was curiosity, so recovering about half of the gap between today's 30 percent and the launch peak of 75 percent (a 45-point gap, so about 22 points back), roughly 52 percent, is the floor and 60 percent is a stretch. Agree the number with the sponsor up front) sustained for eight weeks, search success rising, zero-result rate falling, and the freshness ratio rising. Also track contribution (pages edited by non-authors).

Pitfalls

• Raw page views are easy to game and inflated by bots or bounces, so pair them with search success and freshness.
• A mandate ("everyone must use the wiki") gets compliance, not trust.
• What would change my call: if interviews show the content lives in a different tool people already prefer, integrate with that instead of competing.

Direct answer

Set a small, tiered minimum standard (a few artifacts every service, pipeline or model must have), keep the docs in the same repository as the code and change them in the same pull request (PR), and enforce the standard with automation and defaults rather than meetings and sign-offs. Freshness comes from ownership plus a review date on every doc, not from goodwill. The test of "not bureaucracy" is that following the standard is the easiest path and the checks are run by tooling.

Structured elaboration

1. The standard itself: small and tiered

• Tier 1 (anything on-call, customer-facing or shared): a README (the front-page file of a repo: what it is, who owns it, how to run it, where to ask), a runbook (step-by-step instructions for responding to an alert or failure), and a decision record (a short note recording what was chosen, the options rejected and why) for hard-to-reverse choices.
• Tier 2 (internal tools, experiments): README only.
• ML and data assets: a model card (short document stating what a model is for, its training data, known limits and owner) or a dataset datasheet, plus a data dictionary (what each column means, its unit and source). For analytics projects add required metadata: owner, refresh schedule, source tables and last-reviewed date.
• One template per artifact, pre-filled, with the required fields marked. Templates are what stop inconsistent deliverables and the rework that follows.
• The ML and data items are optional add-ons for teams that ship models or datasets, not part of the core README, runbook and decision record set.

A minimal template skeleton, so the required fields are concrete (the CI check simply fails if a heading is missing):

README.md            RUNBOOK.md                 decision record
# <service>          # Alert: <alert name>      # Decision: <statement>
Owner: <team>        Severity: <P1-P3>          Status: proposed | accepted | superseded
Last reviewed: <d>   Symptoms: <what you see>   Context / Options / Consequences
## What it does     Diagnosis: <steps>          Owner and review date
## Run locally      Fix or rollback: <steps>
## Ask for help     Escalate to: <who>

2. Governance as one framework (owners, templates, review cadence, tooling, incentives)

• Templates: the pre-filled skeleton above is the only format accepted for each artifact, stored in one template repository, so the CI check has a fixed set of required headings to test.
• Owners: every doc names a team owner (a CODEOWNERS file, which routes PR review to the named owners, plus a service catalog entry, meaning a central list of every service with its owner, docs link and on-call contact). An unowned doc is treated as deletable.
• Review cadence: each doc carries a last_reviewed date. Tier 1 is reviewed at least every 6 months, and also whenever the system changes shape.
• Tooling: docs-as-code (markdown in the repo, reviewed like code), a CI (continuous integration) job that checks required headings, broken links and stale dates, and one searchable site built from all repos.
• Incentives: doc work counts in performance and promotion criteria, authors are credited by name, and "a new hire followed the README and it worked" is celebrated. Punishment alone breeds checkbox docs.

3. Keeping standards current as work changes

• A PR template asks "docs updated or not needed, and why". Changes to alerts, APIs or schemas require touching the linked doc (a CI rule can map code paths to doc paths).
• A bot opens a ticket on the owner when last_reviewed passes its limit. Two missed cycles moves the doc to archived rather than leaving a confident but wrong page.
• The standard itself is versioned and reviewed twice a year by a small docs working group (a few volunteers from different teams who own the standard), and any rule that nobody can justify with an incident or onboarding pain is removed.

4. Measuring compliance and handling non-compliance

• Coverage = Tier 1 assets with all required docs / all Tier 1 assets. Freshness = docs reviewed inside their window / docs required.
• Escalation ladder: bot nudge, ticket on the owner, visible on the team's monthly ops review, and only for Tier 1 a launch-readiness blocker, meaning the service cannot go live until its Tier 1 docs exist. Nothing else is gated.

Worked example

One team owns 15 Tier 1 services. The dashboard shows 12 have a README and runbook, and 9 of those are inside the 180-day review window.

coverage  = 12 / 15 = 80%
freshness =  9 / 15 = 60%   (9 of the 15 required sets are fresh)

The team lead sees two numbers and six named services to fix (three with no docs, three with docs past their review window), not a lecture. The runbook and decision record carry the same Last reviewed field as the README, so the CI stale-date check and the 6-month cadence apply to every Tier 1 artifact, not only the README. Note freshness is computed against the required 15, so a missing doc cannot hide behind a fresh one.

Rollout at scale (500 engineers, ten teams, two regions), phased over 12 months

• Months 1-2: pilot with two willing teams, build templates and the CI check, fix what the pilot hates.
• Months 3-5: add four teams, stand up the single search site and the service catalog, run a 2-hour training per engineer (500 x 2 hours = 1,000 engineer-hours, which is 25 engineer-weeks at 40 hours, a small fraction of the organization's yearly capacity).
• Months 6-9: remaining four teams; migrate only Tier 1 legacy docs (archive the rest, do not migrate everything).
• Months 10-12: switch on the launch-readiness gate for Tier 1, publish the first compliance report, review the standard.
• Rough cost, in people rather than invented dollars: a docs platform owner and a tooling engineer (about 2 full-time), plus one steward per team (the team's named docs champion who helps peers and chases reviews) at about 10% time (ten teams is about 1 full-time equivalent). Two regions: one site, async PR review so no time zone waits on another, and one regional steward each.

Trade-offs and pitfalls

• Too many required docs produces empty template filling. Cut the list until each item has a story of an incident or onboarding delay it prevents.
• Central mandate without team stewards makes the standard feel imposed. Stewards give teams a peer to ask.
• Gating everything on compliance encourages gaming (docs written to satisfy a linter). Gate only Tier 1 and spot-check quality by having a new joiner try the README.
• What would change my approach: for a 15-person team I would skip the catalog and the phases and adopt README, runbook and a PR checkbox in one week.

Direct answer

An assumption log is a living list of the things a project is treating as true without having proved them. A useful entry is specific, falsifiable (written so a test could prove it wrong), owned, backed by stated evidence, and has a date by which it must be validated or dropped. It differs from a risk log (possible future events) and a decision log (choices already made). You update it continuously, in the routines the team already runs, and at every milestone.

Structured elaboration

What makes an entry useful

• Statement: concrete and testable. Bad: "users want reports". Good: "at least half of active admins export a report weekly".
• Owner: the one person who will chase the answer.
• Evidence and confidence: what we know now (source, sample size) and how sure we are (high / medium / low).
• Impact if wrong: what breaks, so we test the expensive ones first. For a data analyst: "if exports are not CSV, the ingestion script must be rewritten, about two weeks of rework". For a business analyst: "if the sponsor has not approved the budget, the scope in the requirements document is void".
• Validation method and date (expiry): how we will test it and when it must be settled. An untested assumption without a date is just an opinion.
• Status: open, validated, invalidated, retired. Plus a link to the decisions that depend on it.

How it differs from a risk log and a decision log

They connect: an assumption is an input to a decision, and an invalidated assumption usually becomes a risk or an issue.

When to update

• In daily scrum (the short daily team check-in), ask one question: did anything block or change an assumption?
• In sprint planning (the meeting where the team picks the work for the next one to two weeks), scan open assumptions and pull validation work in as tickets.
• When new evidence arrives, and at milestones (agreed checkpoints such as "design approved") or phase gates (formal go/no-go reviews between project stages). Surfacing it through the project lifecycle reduces ambiguity, because people stop building on unstated beliefs.
• On the expiry date, when the owner must validate, extend with a reason, or invalidate.

After a discovery call (an early meeting with a customer or stakeholder to learn their needs)
Capture assumptions and open questions the same day. For each, assign an owner (the person best placed to find the answer, not the note taker) and a date. Flag any item that touches contracts, data privacy or regulation for legal review, and any item that would change scope for escalation to the sponsor (the executive who funds the project and must approve scope changes). Example: "the customer assumes we will migrate 10 years of history" is escalated because it could double the effort.

Worked example

If A1 is invalidated (a customer sends 2 GB files), it becomes a risk entry ("upload pipeline may fail on large files") and triggers a decision record about chunked uploads.

Trade-offs and pitfalls

• Vague entries ("system will be fast") cannot be tested. Rewrite until a test exists.
• Logs die when nobody owns them: name one owner for the log itself and review it in planning.
• Do not merge it into the risk log. Mixing them loses the "how do we validate this?" discipline.