Technical Writing and Documentation Questions

Direct answer

A docstring (the text in triple quotes right under a function's signature, shown by help tools and editors) should let a caller use the function without reading its body: what it does in one line, what goes in and comes out including shapes, what can go wrong, and anything surprising. Inline comments should explain why a line exists, never repeat what it does. Below I document a small function that ranks features by how much they influenced a model's predictions.

The function, documented

def mean_abs_attribution(attributions, feature_names, top_k=None):
    """Rank features by how strongly they moved predictions, on average.

    Takes per-row feature attributions (for example SHAP values, which split
    each prediction into one signed contribution per feature) and returns the
    mean of the absolute contribution for each feature, largest first.

    Args:
        attributions: list of rows. Each row is a list of floats with one value
            per feature, in the same order as feature_names. Rows are
            predictions, columns are features.
        feature_names: list of str, one name per column.
        top_k: keep only the k highest-ranked features. None keeps all.

    Returns:
        list of (name, score) tuples sorted by score descending. Ties keep the
        original column order.

    Raises:
        ValueError: if attributions is empty, or a row length differs from
            len(feature_names).

    Notes:
        Signs are discarded, so a feature that pushes predictions up for some
        rows and down for others still ranks high. This says how much a
        feature matters, not in which direction.
        Cost is O(rows * features) time (work grows in proportion to the number of values, so twice the rows takes about twice as long). The values themselves are only as
        exact as the method that produced them: sampling-based SHAP is an
        approximation and varies with its random seed (the starting value of a random number generator; a different seed gives slightly different numbers).
    """
    if not attributions:
        raise ValueError("attributions is empty")
    n_cols = len(feature_names)
    totals = [0.0] * n_cols
    for row in attributions:
        if len(row) != n_cols:
            raise ValueError(f"row has {len(row)} values, expected {n_cols}")
        for j, value in enumerate(row):
            totals[j] += abs(value)  # abs first: +0.4 and -0.4 must not cancel
    n_rows = len(attributions)
    ranked = sorted(zip(feature_names, (t / n_rows for t in totals)),
                    key=lambda pair: -pair[1])  # sorted() is stable (equal items keep their order), keeps ties in order
    return ranked[:top_k] if top_k is not None else ranked


rows = [[0.4, -0.1, 0.0], [-0.4, 0.3, 0.0], [0.4, 0.2, 0.0]]
ranked = mean_abs_attribution(rows, ["age", "income", "tenure"], top_k=2)
print([(name, round(score, 2)) for name, score in ranked])

Running it prints [('age', 0.4), ('income', 0.2)]. For age the three absolute values are 0.4, 0.4, 0.4 (mean 0.4). For income they are 0.1, 0.3, 0.2 (mean 0.2). tenure is 0.0 and is cut by top_k=2.

Why the docstring looks like this

• First line states the purpose. Shapes are spelled out (rows are predictions, columns are features) because that is the mistake callers passing tables of numbers make.
• Assumptions and edge cases: same column order as feature_names, empty input and ragged rows (rows of unequal length) raise ValueError, ties keep column order.
• Notes carry what a reader cannot infer: signs are discarded, cost is O(rows x features), and SHAP values (contributions that split one prediction across features) may come from a sampling method that is approximate and seed-dependent.
• Which parts are core: the summary line, Args, Returns and Raises are what every function needs. Notes is optional, used only for facts that change how far a caller can trust the result, such as discarded signs, cost and approximation.
• Inline comments justify decisions: abs before summing (otherwise +0.4 and -0.4 cancel) and why sorted keeps ties in order.

README note for analysts (three lines beside the code): "mean_abs_attribution ranks features by average influence, ignoring direction. Input: one row per prediction, one column per feature. If you need direction, use the signed mean instead."

Two comments I would push back on in review

1. totals[j] += abs(value) # add absolute value to total. It restates the code. Ask for the reason or delete it.
2. # TODO: handle NaN later. NaN (not-a-number) is the special value that marks a missing or invalid result. No owner, no ticket, no behaviour stated. Either handle NaN now, or document today's behaviour in the docstring and link a ticket.

Pitfalls

• A stale comment is worse than none. When behaviour changes, edit the docstring in the same PR (pull request, a proposed change for review).
• If a comment must explain what tangled code does, first rename or extract a function.

Direct answer

A notebook is a lab diary: it records what was tried, in the order it happened. A post is an argument: it states one conclusion and shows only what supports it. My editorial job is to find the conclusion, decide who needs it, and cut everything else, then check every claim I keep against the notebook by rerunning it.

Editorial steps

1. Interview the author (15 minutes) and read the notebook end to end. Ask "what did you find and what should the reader do about it?" Write the answer in one sentence. That is the headline.
2. Name the audience and the decision. Engineers want what changed and how to use or integrate it; product managers want impact, risk, confidence and next step. If both, lead with the result and put method below the fold.
3. Sort every section into three piles: keep (supports the headline), move to an appendix or link (needed to reproduce), delete (dead ends, debugging, repeated plots). Dead ends survive only if they change what the reader should believe (for example "we tried X and it did not help, so do not retry").
4. Rerun the notebook top to bottom in a clean environment so every number and chart is the one you publish. Hidden state and stale outputs are common.
5. Choose 2-4 figures. Each gets a title that states the takeaway ("Recall rises for new customers"), labelled axes, and units. Redraw plots that were made for exploring.
6. Translate the jargon. Define the metric (for example, recall: the share of real positives the model finds), give the baseline, and state uncertainty honestly, including sample size.
7. Write to a fixed skeleton: result and recommendation, why it matters, what we did (short), evidence (figures), caveats, next steps, link to the notebook and the data snapshot.
8. Cut again and have the author fact-check it. They approve accuracy; you own clarity. Then test it on one engineer and one PM who were not involved: can each say the conclusion back?

Worked example (illustrative excerpt)

Headline: Adding support-ticket counts to the churn model finds more at-risk customers among new accounts, at the cost of a slower nightly build.
Recommendation: Ship it for new accounts; hold for older accounts until we re-test in Q4.
Evidence: (Figure 1) recall by account age; (Figure 2) build time before and after.
Caveats: The new-account group is small, so the gain has wide uncertainty.
Reproduce it: link to the notebook, commit and data snapshot.

Of the roughly 500 lines, what survives is the two figures and about five lines of setup; the feature-engineering experiments live behind the link.

Trade-offs and pitfalls

• Cutting can distort: do not drop a caveat because it complicates the story.
• Do not paste code into the post unless the audience will run it.
• The author must approve; the editor's polish should never change a claim.

Direct answer

Comments should explain what the code cannot say for itself: why a choice was made, a constraint that is not visible, and a trap for the next editor. They should leave out what the code already says, history that version control (git) records, and anything you would not want public. The test is: "would a competent reader be surprised or misled without this line?"

What belongs in a comment

• Why: the reason for a choice that looks arbitrary.
• Constraints and hidden coupling: things that break if changed.
• Pointers: where the evidence or longer explanation lives.
• Warnings: ordering, leakage (information from validation data sneaking into training, which makes scores look better than they are), units.

What to leave out

• Restating the line, commented-out old code, jokes, and author or date notes (git blame, a command showing who last changed each line and when, answers those).
• Vague TODOs with no owner or ticket.
• Secrets and internal-only details.

Good and bad, from a model training script

# BAD: says what the code says
epochs = 20  # set epochs to 20
optimizer.zero_grad()  # zero the gradients
# lr = 1e-3  # old value

# GOOD: says why
# Split by user_id, not by row. The same user's events in both train and
# validation would leak and make the validation score look better than reality.
train_users, val_users = split_users(users, seed=SEED)

# 3e-4 was the largest rate that kept loss stable in the sweep,
# see experiments/lr_sweep.md.
learning_rate = 3e-4

Terms in the snippet: a gradient is the signal that tells the model how to adjust, and zero_grad clears it before each step. Validation data is the held-out portion used to test the model. A seed is a fixed starting value so random choices repeat. The learning rate is how big a step the model takes when adjusting, and 3e-4 means 0.0003. A sweep is a set of runs trying several values of one setting, so "largest rate that kept loss stable" means the biggest tried value where training did not blow up.

The bad comments add noise, and the commented-out lr invites confusion about which value is live. The good ones save a future reader from re-deriving a decision, or from "fixing" the split back to per-row, which would silently inflate results. If the good comments are long, the reason belongs in a short doc that the comment links to.

Comments that help new contributors onboard

Comments explain local reasoning. The README carries the map a newcomer needs: how to set up, how to run a small smoke training run (a very short run that only proves the pipeline works end to end), where the data comes from, where decisions are recorded, and how to run the tests. Together they cut the questions a newcomer must ask a person.

Pitfalls

• Comments rot. A comment that contradicts the code (# uses 5 folds above k = 10) is worse than none. Update it in the same change as the code.
• If you need a comment to explain a confusing line, first try a clearer name or a small function.

Direct answer. A handoff document must capture what only the current owners know: why it was built this way, what it silently depends on, how it fails, what "healthy" looks like in business terms, and who to call. The receiving team can read the code and the schema themselves. They cannot recover intent, tribal knowledge, known landmines or agreements with stakeholders.

Terms. A runbook is a step-by-step operating guide for a routine task or alert, like a recipe. Tribal knowledge is what team members know but never wrote down. Shadowing means the receiving team watches, then does the work with the previous owners on hand. A KPI (key performance indicator) is a business number the team is judged on. A feature store is a shared table of prepared model inputs; feature drift is when those inputs change in distribution over time so the model sees data unlike its training data. A secrets manager is a locked vault for passwords and API keys. Rollback means returning to the previous working version. A label definition states exactly what counts as a churner, the thing the model predicts.

Template

1. Purpose and consumers   what it is for, who uses it, business decision it supports
2. Owners and contacts     current owner, receiving owner, escalation, stakeholders
3. Architecture at a glance diagram, inputs, outputs, schedules, hosting
4. Dependencies            upstream data sources and owners, credentials/where secrets live, downstream consumers
5. Acceptance criteria     the measurable conditions under which the receiving team accepts the handoff
6. Monitoring and KPIs     what is monitored, thresholds, and how each maps to a business KPI
7. Known issues and gotchas quirks, workarounds, past incidents, decisions NOT to revisit and why
8. Operations              runbook link, retraining/refresh cadence, on-call expectations, rollback
9. Open items and owners   research-to-production checklist with a named owner and date per item
10. Handoff timeline       shadowing period, sign-off, date ownership formally transfers

If time is short, keep these: 1 (purpose), 2 (owners), 4 (dependencies), 5 (acceptance criteria), 7 (known issues) and the runbook link from 8. The rest can follow within a sprint.

The acceptance criteria let a data scientist hand a model to ML engineers with no ambiguity ("done" means numbers, not opinion). The monitoring-to-KPI mapping serves SRE (site reliability engineering) and product together. Item 9, with a named owner and date on every line, keeps the research-to-production checklist from becoming orphaned advice.

Filled example for one item: a churn-prediction model (illustrative).

• Purpose: ranks subscribers by cancellation risk weekly; the retention team calls the top 500.
• Owners: current, Data Science lead; receiving, ML Platform on-call; escalation, Head of Retention.
• Dependencies: reads the subscriptions and support_tickets tables (owner: Data Engineering); needs the feature store job to finish by 02:00 UTC; API key stored in the secrets manager under churn-model.
• Acceptance criteria: reproduces the recorded validation score (same data version and seed) within the tolerance agreed in the handoff meeting, for example a recorded AUC (a 0 to 1 ranking-quality score) of 0.86 must reproduce between 0.855 and 0.865; scoring job completes within its agreed window on production data, for example under 45 minutes so the call list is ready before the retention team starts work on Monday; rollback tested once. The point is that the receiving team can pass or fail each line with a number, not an opinion.
• Monitoring to KPI: feature-drift alert on days_since_last_login (a leading sign the model is scoring stale behavior) maps to the KPI "retention calls that reach at-risk users"; a scoring-job failure maps to "retention team has no call list Monday".
• Known gotchas: trial accounts must be excluded or the model predicts them all as churners; the July training data has a billing migration gap, so it is deliberately skipped. Do not switch to daily scoring: the retention team can only act weekly.
• Open items: replace a manual CSV export (owner: ML Platform, next sprint); document the label definition (owner: Data Science).

Pitfalls. A handoff that is only a link to the repo; monitoring with no owner or KPI link; no shadowing period. The strongest check: ask the receiving team to run the runbook once, unaided, before sign-off.

Direct answer

Define the score as a weighted checklist of things that can be verified from the model's repository and registry, cap it by freshness, and cross-check the claims against real evaluation data so filling in a field is not enough to score. Then say explicitly that the score is a screening signal for triage, not a goal to reward.

Definition (numerator, denominator, window)

A model card is a short document stating a model's intended use, training data, evaluation results and limitations. The model registry is the shared catalog where a team records every trained model version: its owner, its version number and its evaluation runs (a saved test of that version on held-out data, with the metrics it produced). The deployment config is the file that says which version is live in production. The score for one model is:

$\text{score} = \min\!\Big(\sum_i w_i c_i,\ \text{freshness cap}\Big)$

where each check c_i is 1 if passed and 0 if not, the weights w_i sum to 100, and the cap is 60 if the card was last reviewed more than 180 days ago. Intuition: a full, verified card scores 100; an old card cannot score above 60 however complete it looks. Reading it with numbers: checks passed with weights 20 + 20 + 20 + 5 = 65, but the card is 212 days old, so min(65, 60) = 60.

Worked example (pinned date, run it)

import datetime as dt

TODAY = dt.date(2026, 9, 1)  # pinned so the result never changes

# Rubric: each item is a yes/no check a script or a reviewer can answer.
# Weights sum to 100.
RUBRIC = {"intended_use": 20, "training_data": 20, "eval_by_slice": 25,
          "limitations": 20, "owner_and_contact": 5, "version_matches": 10}

cards = {
    "fraud-v3":  {"intended_use": 1, "training_data": 1, "eval_by_slice": 1, "limitations": 1,
                  "owner_and_contact": 1, "version_matches": 1, "reviewed": dt.date(2026, 8, 10)},
    "churn-v2":  {"intended_use": 1, "training_data": 1, "eval_by_slice": 0, "limitations": 1,
                  "owner_and_contact": 1, "version_matches": 0, "reviewed": dt.date(2026, 2, 1)},
    "rank-v7":   {"intended_use": 1, "training_data": 0, "eval_by_slice": 0, "limitations": 0,
                  "owner_and_contact": 1, "version_matches": 1, "reviewed": dt.date(2026, 8, 20)},
}

def score(card):
    base = sum(w for k, w in RUBRIC.items() if card[k])
    age_days = (TODAY - card["reviewed"]).days
    stale = age_days > 180          # freshness cap: stale cards cannot score above 60
    return base, min(base, 60) if stale else base, age_days

for name, card in cards.items():
    base, final, age = score(card)
    print(f"{name}: checklist={base} final={final} days_since_review={age}")

Output:

fraud-v3: checklist=100 final=100 days_since_review=22
churn-v2: checklist=65 final=60 days_since_review=212
rank-v7: checklist=35 final=35 days_since_review=12

Check the arithmetic: churn-v2 passes intended use, training data, limitations and owner (20 + 20 + 20 + 5 = 65), fails slice evaluation and version match, and its last review was 212 days before the pinned date, so the cap of 60 applies. rank-v7 gets 20 + 5 + 10 = 35.

What data feeds it

Card files in the repository (parsed by a script), the model registry (versions, owners, latest evaluation run), and deployment config (which version is live). Aggregate the score per team on the dashboard, and show the distribution of scores and the reason for each failing check, not one average.

How to stop gaming (Goodhart's law: when a measure becomes a target, it stops being a good measure)

• Verify, don't just detect a field: the evaluation numbers on the card must match the registry's latest run; the version must match what is deployed. Copying template text scores zero.
• Sampling audit: a human reviews a random 5 percent of high scorers each quarter; a score that a reviewer disagrees with triggers a rule change.
• Freshness cap so old cards decay.
• Do not attach bonuses or punishment to the raw number; use it to route help to the teams with low scores, and publish the checks so the goal is the document's quality.
• Outcome tie-back: watch whether low-score models have more incidents or slower onboarding (new team members taking longer to become productive on that model); if the score does not predict anything, revise it.

Trade-offs and pitfalls

A checklist measures presence, not truth or clarity. Keep the number small, explain each point deducted, and accept that the score triages effort rather than proving a document is good.