Technical Writing and Documentation Questions

Approach

Python is the better choice over Bash here, because correct heading extraction needs a small amount of state (are we inside a fenced code block, a code sample wrapped in triple backticks?) plus Unicode-safe file reading (it copes with non-English characters such as é or 日本語 instead of crashing).

Core versus optional. The core ideas are the fence flag, the sorted walk and the duplicate-anchor counter. Setext headings, front matter and the complexity note are extras you can add later. Plan:

1. Walk the repo, keep every .md file, skip .git, node_modules, virtual environments and hidden directories, and skip the output file itself so re-runs do not index their own output.
2. Sort paths so the output is deterministic (the same input always gives the same output) and diff-friendly (a re-run changes only lines that truly changed, so version control shows a small diff).
3. For each file, read line by line, toggle a "in fenced code" flag on lines that start with three backticks or three tildes (so # comment lines inside code are not mistaken for headings), and match ATX headings (the #-prefixed kind: # Title, ## Subtitle, ### Section).
4. Print directories as nested bold entries, each file as a link, and each heading as a nested link using a GitHub-style anchor (an anchor is the #section-name part of a link that jumps to a heading). Duplicate headings in one file get -1, -2 suffixes, as GitHub does. A slug is the link-friendly form of a heading, for example Getting Started becomes getting-started.

Code

#!/usr/bin/env python3
"""Build TOC.md from the Markdown headings (levels 1-3) of every .md file under a repo."""
import re
import sys
from pathlib import Path

SKIP_DIRS = {".git", "node_modules", ".venv", "venv", "__pycache__"}
FENCE = re.compile(r"^\s{0,3}(`{3}|~{3})")
HEADING = re.compile(r"^ {0,3}(#{1,3})\s+(.+?)\s*#*\s*$")


def slugify(text):
    """GitHub-style anchor: lowercase, drop punctuation, spaces to hyphens."""
    text = re.sub(r"[`*_]", "", text.strip().lower())
    text = re.sub(r"[^\w\s-]", "", text)
    return re.sub(r"\s", "-", text)


def headings(path):
    """Yield (level, title, anchor) for one file, skipping fenced code blocks."""
    seen, fence = {}, None
    for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
        m = FENCE.match(line)
        if m:
            fence = None if fence == m.group(1) else (fence or m.group(1))
            continue
        if fence:
            continue
        h = HEADING.match(line)
        if h:
            title = h.group(2)
            base = slugify(title)
            n = seen.get(base, 0)
            seen[base] = n + 1
            yield len(h.group(1)), title, base if n == 0 else f"{base}-{n}"


def build(root, out_name="TOC.md"):
    root = Path(root)
    files = sorted(
        p for p in root.rglob("*.md")
        if p.name != out_name
        and not any(part in SKIP_DIRS or part.startswith(".") for part in p.relative_to(root).parts[:-1])
    )
    lines, shown_dirs = ["# Table of contents", ""], set()
    for f in files:
        rel = f.relative_to(root)
        for depth, d in enumerate(rel.parents[::-1][1:]):   # each ancestor dir, top-down
            if d not in shown_dirs:
                shown_dirs.add(d)
                lines.append("  " * depth + f"- **{d.name}/**")
        base = "  " * (len(rel.parts) - 1)
        lines.append(f"{base}- [{rel.name}]({rel.as_posix()})")
        stack = []   # open ancestor levels, so a skipped level (h1 then h3) still nests one step
        for level, title, anchor in headings(f):
            while stack and stack[-1] >= level:
                stack.pop()
            stack.append(level)
            lines.append(f"{base}{'  ' * len(stack)}- [{title}]({rel.as_posix()}#{anchor})")
    (root / out_name).write_text("\n".join(lines) + "\n", encoding="utf-8")
    return len(files)


if __name__ == "__main__":
    n = build(sys.argv[1] if len(sys.argv) > 1 else ".")
    print(f"indexed {n} files")

Worked example (run it exactly as shown)

mkdir -p demo/docs/api demo/.git
printf '# Acme\n\n## Install\nRun it.\n' > demo/README.md
printf '# Guide\n\n## Setup\n~~~bash\n# not a heading\n~~~\n\n## Setup\n\n#### Too deep\n' > demo/docs/guide.md
printf '# Auth\n### Tokens ###\n' > demo/docs/api/auth.md
echo '# hidden' > demo/.git/x.md
python3 make_toc.py demo && cat demo/TOC.md

Output:

indexed 3 files
# Table of contents

- [README.md](README.md)
  - [Acme](README.md#acme)
    - [Install](README.md#install)
- **docs/**
  - **api/**
    - [auth.md](docs/api/auth.md)
      - [Auth](docs/api/auth.md#auth)
        - [Tokens](docs/api/auth.md#tokens)
  - [guide.md](docs/guide.md)
    - [Guide](docs/guide.md#guide)
      - [Setup](docs/guide.md#setup)
      - [Setup](docs/guide.md#setup-1)

Note that # not a heading inside the code fence and #### Too deep (level 4) are both absent, the second "Setup" got the -1 anchor, the trailing ### in ### Tokens ### was stripped, and the .git/x.md file was ignored.

Reading the two regular expressions

FENCE:

^\s{0,3}(```|~~~)

• ^ start of the line; \s{0,3} up to three spaces or tabs of indentation.
• The bracketed group matches either three backticks or three tildes, captured as group 1. The code remembers which one opened the block and only a matching one closes it.

HEADING:

^ {0,3}(#{1,3})\s+(.+?)\s*#*\s*$

• ^ {0,3} start of line, up to three spaces.
• (#{1,3}) one to three # characters, captured: the count is the heading level. Four hashes fail to match, which is why #### Too deep is dropped.
• \s+ at least one space, so #NoSpace is not a heading.
• (.+?) the title, captured lazily (as short as possible) so it does not swallow the closing hashes.
• \s*#*\s*$ optional spaces, optional closing hashes as in ### Tokens ###, then the end of the line.

Key points

• rel.parents[::-1][1:] gives each ancestor directory top-down (dropping the empty root), so each directory is printed once, preserving structure.
• The stack makes nesting follow the heading hierarchy even if a document jumps from # to ###.
• Links are relative to the repo root where TOC.md is written, so they work on GitHub.

Complexity

Big-O notation describes how work grows with input size. Time is O(total size of all Markdown files) to read them, plus O(F log F) to sort F file paths (F is the number of files; sorting a list of F items takes a bit more than F steps). Memory is O(F + H) for the sorted path list and the output lines (H is total headings), and the seen-anchor dictionary is per file.

Edge cases and limits

• Setext headings (a title underlined with === or ---) are not handled. Add a regex for the line above such underlines if the repo uses them.
• Anchor slugs follow GitHub's rules only approximately (for example, non-ASCII titles and some punctuation differ across renderers). Verify on your docs platform.
• Headings inside HTML blocks or front matter (a YAML block at the top of a file) are ignored or could be misread. Skip a leading --- block if you use it.
• Unreadable or non-UTF-8 files are read with errors="replace" so one bad file does not abort the run.

Direct answer

Build a scheduled job that gives every document a staleness score from several signals, maps each document to an owner, and sends owners a short, capped, weekly digest instead of one alert per document. No single signal is trustworthy (an old doc may still be correct, a fresh edit may have been a typo fix), so combine them, tune thresholds so precision beats recall (nudges you send should nearly always be deserved, even if that means missing some stale docs), and measure the outcome so the system earns trust. The goal is fewer, better nudges.

Terms in plain words

• False positive: the system flags a doc that is actually fine. False negative: it misses a doc that really is wrong.
• Precision: of the docs you flagged, the share that truly needed attention. Flag 10, 8 needed work: precision is 80%.
• Recall: of all docs that truly needed attention, the share you flagged. If 20 needed work and you caught 8, recall is 40%.
• "Precision beats recall" means preferring fewer, more accurate flags over catching everything.
• Embedding similarity: turning text into lists of numbers so that similar meaning gives similar numbers. It is a heavier alternative to matching names, and the worked code below does not use it.
• Normalised to 0-1: rescaled so 0 is "no problem" and 1 is "worst case". Saturates means it stops growing at 1 (a doc older than a year scores the same as one exactly a year old).

Signals and what each really tells you

Age alone is the weakest signal. Drift plus broken references is the strongest, because it says something concrete changed.

Scoring, weights and thresholds (run with pinned data)

Each signal is normalised to 0-1, then combined with weights (drift 0.35, age 0.30, broken references 0.25, owner inactive 0.10). Traffic does not raise the score. It decides the action: unread stale docs go to archive review, read ones go to an owner.

from datetime import date

TODAY = date(2026, 9, 1)   # pinned so the run is reproducible

# name, last_doc_edit, code_commits_to_linked_paths_since, broken_refs, views_90d, owner_active
DOCS = [
    ("payments-runbook",  date(2025, 3, 1),  18, 3, 420, True),
    ("onboarding-guide",  date(2026, 7, 20),  2, 0, 900, True),
    ("legacy-batch-job",  date(2024, 1, 10),  1, 1,   0, False),
    ("api-gateway-setup", date(2026, 2, 1),  12, 1, 150, False),
    ("style-guide",       date(2025, 6, 1),   0, 0,  60, True),
]

def score(last_edit, drift, broken, owner_active):
    age_n = min((TODAY - last_edit).days / 365, 1)   # 0..1, saturates at one year
    drift_n = min(drift / 20, 1)                     # 20+ code changes since the doc moved = max
    broken_n = min(broken / 3, 1)                    # dead links or renamed symbols the doc mentions
    gone = 0 if owner_active else 1
    return round(0.30 * age_n + 0.35 * drift_n + 0.25 * broken_n + 0.10 * gone, 2)

def action(s, views):
    if s >= 0.40 and views == 0:
        return "archive review"
    if s >= 0.60:
        return "nudge owner"
    if s >= 0.40:
        return "weekly digest"
    return "ignore"

for name, edit, drift, broken, views, active in DOCS:
    s = score(edit, drift, broken, active)
    print(f"{name:18} score={s:.2f} views={views:4} -> {action(s, views)}")

It prints:

payments-runbook   score=0.86 views= 420 -> nudge owner
onboarding-guide   score=0.07 views= 900 -> ignore
legacy-batch-job   score=0.50 views=   0 -> archive review
api-gateway-setup  score=0.57 views= 150 -> weekly digest
style-guide        score=0.30 views=  60 -> ignore

Tracing one score by hand (payments-runbook, run on 2026-09-01):

age    = 549 days / 365 = 1.50, capped at 1   -> 0.30 x 1   = 0.300
drift  = 18 commits / 20 = 0.90               -> 0.35 x 0.9 = 0.315
broken = 3 refs / 3 = 1.00                    -> 0.25 x 1   = 0.250
owner active, so gone = 0                     -> 0.10 x 0   = 0.000
total = 0.865

The hand total is exactly 0.865 but the program prints 0.86. That is floating-point rounding: computers store 0.865 as 0.86499999999999999..., so round(..., 2) rounds down. The action does not change (0.86 and 0.87 are both above the 0.60 line), but expect this kind of one-cent difference when you check scripts by hand.

Two thresholds are in play. 0.60 is the alerting line (a direct nudge to the owner). The 0.40 to 0.59 band is not an alert: those docs only appear in the capped weekly digest, and if nobody reads them they go to archive review instead.

Reading the result: payments-runbook is old, has 18 code changes since, three broken references, and is viewed often, so it gets a direct nudge. onboarding-guide is heavily read but fresh, so nothing happens (popularity alone never triggers a nudge). legacy-batch-job scores 0.50 but nobody reads it, so the right action is archive review, not an update. api-gateway-setup is in the middle band and its owner is inactive, so it goes to the digest and reassignment.

Mapping documents to owners (fallback chain)

1. owner in the doc's front matter (the metadata block at the top of the file).
2. The CODEOWNERS entry (file that assigns responsible people or teams by path) for the doc.
3. The owners of the code paths the doc links to.
4. The last substantive author (skip bulk reformat commits).
5. The team channel as a last resort, and a person is never left unassigned.

If the owner has left, reassign automatically to their team and say so in the nudge.

False positives, false negatives and alert fatigue

• A false positive is nudging an owner about a doc that is fine. A false negative is missing a doc that is wrong. Here false positives cost more, because each wasted nudge teaches people to ignore the next one, so set the alerting threshold high (0.60) and accept some misses.
• Cap the digest (for example five items per owner per week), highest score first, and never nudge the same doc twice in 30 days.
• Give every nudge two one-click answers: "Still accurate" (records a review date and resets the clock) and "Snooze 60 days". Both feed the score.
• Add a review-by date in front matter for docs where the owner knows they are stable.
• Track precision: of the nudges sent, what fraction led to an edit or a "still accurate" confirmation versus being ignored? If ignored nudges climb, raise the threshold or drop a signal.

Review workflow for candidates

Weekly job writes the candidate list to a dashboard and posts each owner's digest. Owner triages within two weeks: update, confirm, snooze, or archive. Untouched items escalate once to the team lead, not the whole channel. Archived docs get a banner and redirect rather than deletion.

Trade-offs and pitfalls

• Weights are judgement calls. Start simple, sample twenty flagged docs by hand, and adjust from what you find.
• Access logs can be gamed or skewed by bots, so use them to prioritise, never to declare a doc correct.
• Do not auto-edit or auto-archive without a human step, because a wrong automatic change destroys trust faster than a missed nudge.

Direct answer

I would judge documentation by what happens after someone reads it: do they find what they need, act on it, and stop asking people? So I collect a few behavioural signals (search that fails, questions that repeat, ramp time for new hires), one or two opinion signals (a page rating and a short survey), and I would distrust raw page views, which go up when docs are good and also when they are confusing.

Signals and how to collect them

MTTR by runbook tag, with made-up numbers: incidents with a runbook took 45, 30 and 60 minutes (mean 135/3 = 45); incidents without one took 90, 120 and 75 (mean 285/3 = 95). The 50-minute gap is a hint, not proof, since the incident types may differ.

Worked example (illustrative data, run it)

# Tiny, made-up docs-effectiveness data (illustrative). Each row is one week.
weeks = [
    # (page_views, searches, searches_with_no_click, tickets, tickets_where_answer_was_in_docs)
    (900, 200, 90, 40, 14),
    (1500, 210, 60, 33, 9),   # after rewriting the top pages
]
for i, (views, searches, no_click, tickets, in_docs) in enumerate(weeks, start=1):
    print(f"week {i}: views={views}",
          f"| search abandonment={no_click / searches:.0%}",
          f"| avoidable tickets={in_docs / tickets:.0%} ({in_docs}/{tickets})")

# Onboarding signal: days from start date to first merged pull request
before = sorted([21, 24, 19, 30, 26])
after = sorted([14, 16, 12, 20, 15])
median = lambda xs: xs[len(xs) // 2]
print("median days to first merged PR:", median(before), "->", median(after))

Output:

week 1: views=900 | search abandonment=45% | avoidable tickets=35% (14/40)
week 2: views=1500 | search abandonment=29% | avoidable tickets=27% (9/33)
median days to first merged PR: 24 -> 15

Reading it: after rewriting the top pages, search abandonment fell from 45% (90 of 200 searches) to 29% (60 of 210), avoidable tickets from 35% (14 of 40) to 27% (9 of 33), and the median time to first merged PR from 24 to 15 days. Note that page views rose too; on their own, views could not have told us whether that was good.

Which popular measure I distrust: page views

Views measure traffic, not usefulness. A confusing page produces repeat visits and more views. A great page that answers in one glance also produces views but no follow-up. Other gameable ones are "number of pages written" and "words". Guard by pairing every activity measure with an outcome measure (views with search abandonment; docs written with avoidable tickets).

Quantitative versus qualitative KPIs (key performance indicators, the few numbers you track to judge success) and the response when one drops

Quantitative: the rates above. Qualitative: free-text feedback, and watching a new hire try a task. When a KPI drops, look at the pages contributing most, read the comments, run a 20-minute test with someone who has not seen the page, then fix and re-measure a few weeks later. Also link the improvement to an outcome such as incident time to resolve: compare incidents where the runbook was used with those where it was not.

Pitfalls

Small samples (say which counts are behind a percentage), changing two things at once, and treating a correlation as proof. Say the numbers back a claim about direction, not exact cause.

Direct answer

A README is the front page of a repository, so I order it by the questions a newcomer asks in sequence: what is this, is it the right thing, how do I run it, how do I use it, who owns it. For an internal microservice that order ends with operating and owning it. For a public library it starts with installing and using it, and adds versioning and contribution rules, because the readers are strangers who cannot walk over and ask.

Internal microservice README, in order

1. Name, one-line purpose, status and owner: "billing-events: turns payment webhooks into internal events. Owner: payments team, #payments-help."
2. What it does and does not do: two short lists. Stops people using it for the wrong job.
3. Quick start: clone, install, run locally, and how to see a request work, in copy-paste commands.
4. Configuration: every environment variable, default, and whether it is a secret.
5. API and contracts: link to the API spec or schema, plus one request and response example.
6. Run the tests: the one command.
7. Deploy and operate: where it runs, dashboards, alerts, the runbook link (a step-by-step guide for handling each alert or failure), and its SLA (service-level agreement, the availability and response promise to its users).
8. Dependencies and architecture: upstream services (the ones that send it data) and downstream services (the ones that consume its output), plus one small diagram.
9. Where the rest lives and how to contribute: design docs, ADRs (architecture decision records, short notes on why a choice was made), change process.

Rule of thumb: the first screen plus quick start must get a new engineer to a working local run without reading further.

# billing-events
Turns payment webhooks into internal `payment.settled` events.
**Owner:** payments team | **Status:** production | **Runbook:** link

## Quick start
    make install && make run
    curl localhost:8080/health

## Configuration
| Variable | Default | Secret |
|---|---|---|
| `WEBHOOK_SIGNING_KEY` | none | yes |

What changes for a public library

The internal reader can ask a colleague; the public reader can only read, so the library README must be complete at first contact.

Worked example: adapting the README for an ML repo and a multi-team service

• An ML repository for analysts and new hires keeps the same order but adapts the middle: what the training code does, how to run training on a small sample, how to run the containerised inference (the model packaged in a container so it runs identically anywhere), and what the CI (continuous integration, automated checks on each change) verifies.
• A service used by several teams adds an onboarding block: ownership, the SLA, review cadence for changes, and how to request a new capability.
• Documenting a new module or public API for maintainers and new hires: usage examples for the common cases, a diagram of how the parts connect, the schema or API contract, migration notes for anyone upgrading, and one line saying where longer docs live so the README stays short.

A concrete excerpt for the ML repository (illustrative):

# churn-model
Predicts which subscribers may cancel in the next 30 days. Owner: data-science team.

## Try it in 10 minutes
    make sample-data      # downloads a 5,000-row sample
    make train-small      # trains in a few minutes on a laptop
    make predict ROW=17   # prints one prediction and its score

## What CI checks
Lint, unit tests, and one training run on the sample. It does not check full-data accuracy.

The multi-team onboarding block is a short list: "Owner: payments team. SLA: 99.9% monthly availability. Changes reviewed by two owners, weekly. To request a capability, open a ticket with the label payments-request."

Trade-offs and pitfalls

• A README that grows into a manual is skipped. Link out for depth.
• Commands that were never run are the top cause of distrust. Copy them from a working session.
• Do not put secrets or environment specifics that go stale in it; link to the source of truth.

Direct answer

Docs-as-code means treating documentation the way a team treats source code: plain-text files (usually Markdown) stored in Git next to the code, changed through pull requests (proposed changes that a teammate reviews before they merge), checked by automated tests in CI (continuous integration), and built into a website on every merge. If you have not used it, say so plainly and describe how you would pilot it; the shape of the answer below works either way.

What it looks like

payments-service/
  src/retry.py
  docs/
    index.md
    runbook-timeouts.md   # a runbook: step-by-step guide an on-call engineer follows when something breaks
  mkdocs.yml        # site config for a static site generator

A change to the retry behaviour is one pull request containing the code in src/retry.py and the edit to docs/runbook-timeouts.md. The reviewer sees both together, and the merge publishes the site.

How a team adopts it

1. Pilot with one repository and one team, not the whole company.
2. Pick a static site generator (a tool that turns Markdown files into a website, for example MkDocs or Docusaurus) and publish from the main branch.
3. Add cheap automatic checks first: Markdown lint (a tool that flags formatting mistakes such as a skipped heading level), a link checker (a tool that follows every link and reports ones that lead nowhere), and a build that fails on errors. A failed link check prints something like this (illustrative):

docs/index.md:12: broken link 'runbook-timeout.md' (file not found; did you mean runbook-timeouts.md?)

4. Change the habit: add a line to the pull request template (the pre-filled text every new pull request starts with), such as "Docs updated, or not needed because ...". A filled-in line reads "Docs updated: docs/runbook-timeouts.md" or "Docs not needed because: internal refactor, no behaviour change". Also name a docs owner (the person responsible for the accuracy of a folder, often recorded in a CODEOWNERS file) for each folder.
5. Move only the pages people use, starting with the runbooks and the quick start (the short getting-started page a new user follows first). Leave the archive where it is.
6. Lower the barrier for non-Git users: an "edit this page" link on every page, so a product manager or support engineer can fix a typo in the browser.

Main benefit

Docs change in the same pull request as the behaviour they describe, so they are reviewed, versioned and released together. A reader can also look at the docs as they were at any past release. That is what keeps them from going stale silently.

Main cost

Contributors who do not live in Git face friction. Support staff, product managers and analysts either stop editing or file requests that engineers turn into pull requests, which recreates the bottleneck. The mitigations are the in-browser edit link, a light-touch review rule for docs-only changes, and a tool that shows a live preview. A second, smaller cost is upkeep of the pipeline itself: someone owns the build and the link checker, and broken builds must be fixed quickly or people will start ignoring them.

Pitfalls

• Moving everything and fixing nothing: a migrated pile of stale pages is still a pile of stale pages. Delete before you migrate.
• Making the checks noisy: a flaky link check (one that fails on some runs and passes on others for reasons unrelated to your change, such as a third-party site being briefly down) trains people to click past failures. Block on internal links and build errors, and treat external links as advisory.
• No owner: docs in the repository still rot if nobody is responsible for a folder.