Documentation and Knowledge Management Questions
Governing how teams capture and keep knowledge usable: documentation standards, ownership models (centralized, federated or dedicated team) and review cadences, incentives and culture change that get engineers to document, decision records and assumption logs, post-project reviews that preserve lessons, and knowledge-base strategy so organizational knowledge stays findable and attributed. Includes building or buying a knowledge platform, migration and consolidation plans, taxonomy, tagging and search quality, linking docs to code, capturing tacit expertise into the knowledge base, detecting and remediating stale docs across a documentation estate, and versioning and change policies for shared reports and metric definitions. Covers the governance and lifecycle of a body of documentation, not the craft of writing any single document.
What is an assumption log, and what makes an entry in one actually useful? How is it different from a risk log or a decision log, and when do you update it?
Sample Answer
Direct answer
An assumption log is a living list of the things a project is treating as true without having proved them. A useful entry is specific, falsifiable (written so a test could prove it wrong), owned, backed by stated evidence, and has a date by which it must be validated or dropped. It differs from a risk log (possible future events) and a decision log (choices already made). You update it continuously, in the routines the team already runs, and at every milestone.
Structured elaboration
What makes an entry useful
- Statement: concrete and testable. Bad: "users want reports". Good: "at least half of active admins export a report weekly".
- Owner: the one person who will chase the answer.
- Evidence and confidence: what we know now (source, sample size) and how sure we are (high / medium / low).
- Impact if wrong: what breaks, so we test the expensive ones first. For a data analyst: "if exports are not CSV, the ingestion script must be rewritten, about two weeks of rework". For a business analyst: "if the sponsor has not approved the budget, the scope in the requirements document is void".
- Validation method and date (expiry): how we will test it and when it must be settled. An untested assumption without a date is just an opinion.
- Status: open, validated, invalidated, retired. Plus a link to the decisions that depend on it.
How it differs from a risk log and a decision log
| Log | What it holds | Shape of an entry | Question it answers |
|---|---|---|---|
| Assumption log | Beliefs treated as true | Statement, evidence, validate-by date | "What are we relying on?" |
| Risk log | Possible bad future events | Event, probability, impact, mitigation | "What could go wrong?" |
| Decision log | Choices made and why | Decision, options, rationale | "What did we choose?" |
They connect: an assumption is an input to a decision, and an invalidated assumption usually becomes a risk or an issue.
When to update
- In daily scrum (the short daily team check-in), ask one question: did anything block or change an assumption?
- In sprint planning (the meeting where the team picks the work for the next one to two weeks), scan open assumptions and pull validation work in as tickets.
- When new evidence arrives, and at milestones (agreed checkpoints such as "design approved") or phase gates (formal go/no-go reviews between project stages). Surfacing it through the project lifecycle reduces ambiguity, because people stop building on unstated beliefs.
- On the expiry date, when the owner must validate, extend with a reason, or invalidate.
After a discovery call (an early meeting with a customer or stakeholder to learn their needs)
Capture assumptions and open questions the same day. For each, assign an owner (the person best placed to find the answer, not the note taker) and a date. Flag any item that touches contracts, data privacy or regulation for legal review, and any item that would change scope for escalation to the sponsor (the executive who funds the project and must approve scope changes). Example: "the customer assumes we will migrate 10 years of history" is escalated because it could double the effort.
Worked example
| ID | Statement | Owner | Evidence, confidence | Validate by | Status |
|---|---|---|---|---|---|
| A1 | Customer exports arrive as CSV, under 100 MB | Priya | One sample file, low | 10 Oct | open |
| A2 | Legal permits storing EU data in the US region | Legal contact | None yet, low. Flag: legal review | 8 Oct | open |
| A3 | Payments API supports refunds by order ID | Sam | Read API docs, high | 3 Oct | validated |
If A1 is invalidated (a customer sends 2 GB files), it becomes a risk entry ("upload pipeline may fail on large files") and triggers a decision record about chunked uploads.
Trade-offs and pitfalls
- Vague entries ("system will be fast") cannot be tested. Rewrite until a test exists.
- Logs die when nobody owns them: name one owner for the log itself and review it in planning.
- Do not merge it into the risk log. Mixing them loses the "how do we validate this?" discipline.
Business stakeholders rely on production reports and dashboards that change without notice. Create a change and versioning policy for them: how changes are classified, who approves what, how you roll back, how breaking changes are communicated, and how long old versions are kept.
Sample Answer
Direct answer
Treat each production report as a versioned product with an owner. Classify a change by its effect on consumers (cosmetic, compatible, breaking), route approvals by class, keep every change reversible, announce breaking changes ahead of time with a comparison to the old numbers, and retain the previous version for a defined period.
Terms
- Versioning: giving each release an identifier (for example v2.1) so people can tell which one they saw.
- Breaking change: one that alters what a number means or removes something people rely on.
- Rollback: restoring the previous working version.
- Deprecation: announcing something will be removed, with a date.
- Semantic versioning: MAJOR.MINOR.PATCH numbering; MAJOR for breaking changes.
- Changelog: a dated, running list of every change to a report and why it was made.
- Staging copy: a test copy of the report that runs on real data but that no business user reads.
- Validation total: a headline number (for example total revenue) that you compare between old and new versions before release.
- Metric owner: the business person who defines what a number means (for example Finance owns "revenue").
1. Classify changes
| Class | Examples | Version bump | Approval |
|---|---|---|---|
| Patch: cosmetic or bug fix that restores the intended number | Label fix, colour change, fixing a wrong join (a bug: the report was not computing what its definition already says) | x.y.Z | Report owner plus peer review |
| Minor: additive and compatible | New filter, new chart, new column | x.Y.0 | Report owner plus reviewer, notice to users |
| Major: changes meaning or removes | Redefining "active customer", changing a filter default, dropping a metric | X.0.0 | Owner plus business metric owner (for example Finance) plus data owner; 2+ weeks notice |
| A change that shifts any published number by redefinition is Major even if the screen looks identical. |
The Patch versus Major boundary: ask whether the definition changed. A Patch correction makes the number match the definition everyone already agreed on (the join was wrong, so the number was wrong), so it is announced in the changelog with the before and after values. A Major change alters the definition itself, so the same customers now count differently, and that needs the metric owner's approval and advance notice.
2. Approvals (RACI style)
The analyst who builds is responsible, the report owner is accountable, the metric owner in the business is consulted and must sign off on definition changes (consulted here means their agreement is required, not just their opinion), and affected consumers are informed. (RACI: responsible, accountable, consulted, informed.) Every class gets a second-analyst review: for Patch it is a quick peer check, and for Minor and Major it is a full review before release.
3. Rollback
- Keep the prior version deployable and the query in source control, so rollback means redeploying it, not rebuilding.
- Test in a staging copy against production data before release, and compare key totals side by side.
- Roll back if a validation total differs from the expected one, or if consumers report a wrong number. Decide the rollback owner in advance.
4. Communicating breaking changes
Announce at least two weeks ahead through the channel people actually read (email plus a banner in the dashboard), state what changes, why, when, and show old versus new values for the last three periods. Include a link to the changelog and a named contact. Run old and new versions in parallel where possible.
5. Retention of old versions
Keep the previous Major version accessible read-only for a fixed period (for example 90 days, or through the next month-end financial close, when Finance finalizes its numbers), keep definitions and change logs indefinitely, and archive the rest. The exact period should follow audit and legal requirements.
Worked example (illustrative)
The team wants "active customer" to mean a purchase within 28 days instead of 30. That alters historical counts, so it is Major (v3.0.0). Finance's metric owner approves. Users get notice with a comparison table built from the same customers (the numbers below are illustrative):
Period Old (30 days) New (28 days) Change
Last qtr 12,400 11,900 -500 (about -4%)
Prior qtr 11,800 11,350 -450 (about -4%)
The count drops because customers whose last purchase was 29 or 30 days ago no longer qualify, and the table shows that the drop is a definition effect, not a business decline. v2 remains available for 90 days, and if the Finance total looks wrong at the first close, the team redeploys v2.
Trade-offs and pitfalls
- Heavy approvals push people to silent edits, so keep patch changes light and auditable.
- Communicating only the change and not the numeric impact leads to trust loss.
- Retention costs storage and confusion, so label old versions clearly.
- What would change my call: for self-serve exploratory dashboards, a lighter policy is appropriate. Apply the full one to reports that feed decisions or external reporting.
You are asked to organize an internal knowledge base for a data platform. How would you structure the top-level categories, how would a dataset doc, a runbook and an architecture doc each be classified, and why does the taxonomy matter for people finding things?
Sample Answer
Direct answer
Organize the knowledge base around how people look for things, not around who wrote them. Use a few top-level categories, classify each document by a type (what kind of document it is), and let secondary labels such as team or system cut across. A taxonomy matters because people can only use what they can find, and a stable structure lets search, ownership and freshness rules work.
Terms
- Taxonomy: an agreed way to group and name content.
- Dataset doc: describes a table or file (columns, source, refresh, owner).
- Runbook: step-by-step instructions to operate or fix something, usually during an incident.
- Architecture doc: explains how a system is built and why.
Top-level categories for a data platform
- Getting started: onboarding, access requests, tooling setup.
- Data catalog: dataset docs, grouped by business domain (orders, payments, customers).
- Pipelines and systems: architecture docs and design decisions per system.
- Operations: runbooks, on-call guides, incident reviews.
- Standards and policies: naming conventions, quality rules, data handling.
How each document is classified
| Document | Category | Type | Key labels |
|---|---|---|---|
Dataset doc for orders_daily | Data catalog, Orders domain | dataset | owner team, refresh cadence, sensitivity |
| Runbook: "Nightly load is stuck" | Operations | runbook | system (orders pipeline), severity, on-call team |
| Architecture doc for the ingestion service | Pipelines and systems | architecture | system, status (current or proposed) |
The same page can be reached by category, type or system, but has one home so there is one canonical copy.
Why it matters (worked example, illustrative)
At 2 a.m. an engineer sees the nightly orders load fail. With a good taxonomy she opens Operations, filters by system "orders pipeline", and finds the runbook in two clicks. With no taxonomy she searches "load failed", gets forty results including old design notes and an outdated wiki page, and wastes time or follows stale steps. The taxonomy also tells people where to put new content, which prevents duplicates.
Principles
- Name categories by reader tasks (get started, fix, look up) rather than org charts, because teams reorganize and paths break.
- Keep depth to about two or three levels.
- Use one type list and reuse it everywhere so search filters are predictable.
Trade-offs and pitfalls
- Deep hierarchies hide content; flat structures with rich filters scale better.
- A misc bucket grows without limit, so review it monthly.
- Testing with real users (ask three people to find a given doc) beats debating categories.
The company uses several BI tools and the same metric gives different numbers depending on where you look. How would you design the documentation and governance so metric definitions stay consistent: central registry versus federated docs, roles and approvals, change control, enforcement, and how authors are onboarded?
Sample Answer
Direct answer
Use one central, versioned metric registry as the single source of truth for definitions of shared, company-level metrics (revenue, active user, churn), and let teams keep federated docs only for local, team-specific metrics that reference the registry. Every shared metric has a named owner, a change process with approval and notice, and enforcement through code: dashboards pull the definition (or the query) from the registry, so a number in a BI (business intelligence, dashboard and reporting) tool cannot silently diverge.
Why the numbers differ (worked example)
Say 100 orders total 1,000 in item price, plus 100 tax, and 50 was refunded. Three tools report "revenue":
- Tool A: sum of amounts charged, including tax = 1,100.
- Tool B: item price only, ignoring refunds = 1,000.
- Tool C: item price minus refunds = 950.
All three are "correct" for their own unwritten definition. The registry entry writes one down:
metric: net_revenue
definition: sum(item_price) - sum(refunds), excluding tax
grain: daily, in UTC (sales dated by order date; refunds dated by refund date since version 3)
owner: Finance Analytics
source table: orders_curated
version: 3 (changed 2026-08-01: refunds now booked at refund date)
Grain means the level of detail a metric is calculated at (here, one number per day). With that entry, only 950 is "net revenue"; the others need different names (gross_charged, item_sales). The 950 example assumes the orders and the refund fall in the same period, so version 2 and version 3 agree there. They differ when a refund lands in a later period:
Order placed 2026-07-30 for 200; refunded 2026-08-03 for 50
Version 2 (refund booked on order date): July = 150, August = 0
Version 3 (refund booked on refund date): July = 200, August = -50
The total is 150 either way, but July's published number changes from 150 to 200, which is why this needed a version bump.
Central versus federated
| Central registry | Federated docs | |
|---|---|---|
| Consistency | High | Low, drifts |
| Speed of change | Slower (review) | Fast |
| Ownership | Clear per metric | Blurred |
| Best for | Shared company metrics | Team-local metrics |
Recommendation: central for the small set of tier-1 shared metrics (the handful that drive company decisions or external reporting) (typically tens, not thousands); federated for the rest, with a rule that a local metric must not reuse a registered name.
Roles and approvals
- Metric owner (one accountable person per metric, from the business function that lives with the number).
- Data steward group (a small cross-team panel of people responsible for data quality and definitions) approves new tier-1 metrics and definition changes.
- Authors propose changes by pull request against the registry file, so history is the audit trail.
- Approval: owner plus one steward for a change; consumers listed in the entry get notified.
Change control
A definition change bumps the version, states the date it takes effect, and states impact ("historical values restated", meaning past numbers are recalculated under the new rule, or "old value frozen", meaning past numbers stay as originally published and only new periods use the new rule). Dashboards show the version. Breaking changes need a notice window before rollout, and a side-by-side period where old and new values appear together. Example: announce on 2026-07-15, show both columns in July, and switch on 2026-08-01, with a note that July net revenue moves from 150 to 200 in the example above.
Enforcement
- Prefer a semantic layer (a shared layer that holds each metric's definition, so it is written once) or governed metric definition (one place where the metric's SQL lives and BI tools query it) so tools consume rather than re-implement.
- CI check: fail if a dashboard defines a field named like a registered metric outside the registry.
- A weekly reconciliation job compares each tool's reported value for tier-1 metrics against the registry query and alerts on mismatch above a tolerance (the difference you accept, for example 0.5%). Example: the registry says 950 and Tool A shows 941; the gap is 9 / 950 = about 0.95%, above 0.5%, so an alert names the tool and the metric.
Onboarding authors
A one-page contribution guide, a template entry, a 30-minute walkthrough, and a first task of documenting one existing metric under a steward's review. Authors learn the naming rule (no silent reuse) early.
Pitfalls
A registry with no enforcement becomes another stale wiki. A registry too large to steward gets bypassed; keep tier-1 small. What would change this: if only one BI tool remains, its native governed layer may make a separate registry unnecessary.
You are introducing decision records for product and analytics decisions made under uncertainty. Draft the template you would give the team, then explain how you would get people to actually fill it in and revisit it later rather than let it rot.
Sample Answer
Direct answer
For decisions made under uncertainty, the record must capture more than the choice: it needs the assumptions, the evidence and its limits, how we will know if we were right (metrics and a date), the fallback, and who disagreed and why. I would give the team a one-page template like the one below, embed it in the places where work already happens (sprint planning, handoffs, roadmap reviews), and keep it alive with review dates that create tickets automatically.
Structured elaboration
The template
Two terms first. A pull request (PR) is a proposed change to a shared file or code that teammates comment on before it is accepted; on a non-engineering team, read it as the comment thread on a proposed change. Validate by means the date by which we check the success metric.
Decision: <one-sentence decision> ID / Date / Status (proposed | accepted | reversed | superseded)
Owner (decides): <name> Consulted: <names> Informed: <names>
Context: <what problem, what deadline, what we know>
Options considered: <option, why not chosen> (at least two)
Assumptions: A1 <statement> confidence H/M/L how we will test it
Evidence: <data links, with a note on quality and sample size>
Success metrics: <metric, threshold, when we check> ("validate by")
Risks and fallback: <what would make us reverse, and the plan B>
Dissent log: <who disagreed, their argument, link to the pull request or thread, how it was resolved>
Review date: <date> Outcome (filled later): <what actually happened>
Fields that matter most under uncertainty
- Assumptions with confidence: the decision is only as good as the beliefs under it. Marking low-confidence ones tells reviewers what to test first.
- Metrics to validate with a date and threshold: without a threshold nobody can say we were wrong.
- Stakeholders and fallbacks: who is affected and what we do if it fails, decided before we are emotionally committed.
- Dissent log (conflict log): recording disagreement is not a blame device. It preserves the reasoning of people who lost the argument, links to their pull request comments or thread, and lets a later reviewer see if the dissent was proven right.
Getting people to fill it in
- Proportionality: only for decisions above a threshold (irreversible, costly, or affecting other teams). A 15-minute fill-in, pre-populated from the ticket.
- Put it in the workflow: sprint planning asks "does this story depend on an open decision record?", and handoffs link the record so the next team inherits the assumptions.
- Owner writes it, others comment: ownership is one name.
- Visible use: open the record in the retro when the review date comes up. If people see records being used, they write them.
Keeping it from rotting
- The review date creates a calendar ticket for the owner. On review they set outcome: confirmed, partly, reversed.
- Retention: keep records permanently (small), mark them superseded rather than deleting, and each quarter review the aggregated outcomes to learn which kinds of assumptions tend to fail.
Worked example
Decision: cap the free plan at 3 projects.
- Assumption A1 (confidence: medium): users who reach a second project are more likely to upgrade. Test: compare upgrade rates for users with 1 vs 2+ projects in the event logs.
- Metric: at least 20% of new free users create a second project within 30 days (illustrative threshold). Check date: 45 days after launch.
- Fallback: the second-project metric is only a necessary precondition for the cap to bind (a cap of 3 only affects users who reach a third project), so on its own it cannot show the cap is binding. If fewer than 20% of new free users reach a second project, a cap of 3 is not what limits them, so raising it to 5 would change nothing and the cap should be dropped or replaced by a different upgrade lever. Raise the cap to 5 only if the dissent proves right, that is, if churn among students rises past the agreed threshold. Add a second metric so A1 is actually tested: upgrade rate of users with 2+ projects versus 1 project after 45 days. Also track the share of new free users who reach the 3-project limit, since that is the group the cap actually affects.
- Dissent: support lead argued the cap would raise churn among students. Logged with a link to their comment. At the review, the outcome field records what churn actually did, so the dissent is judged fairly.
Filled in for this example:
Decision: Cap the free plan at 3 projects. ID: D-014 / 2026-03-10 / accepted
Owner (decides): Head of Product Consulted: support lead, growth analyst Informed: sales, engineering
Context: Free users cost hosting money; no data on what drives upgrades.
Options considered: No cap (rejected: cost); cap at 1 (rejected: blocks evaluation).
Assumptions: A1 users who reach a second project upgrade more; confidence M; test: upgrade rate 1 vs 2+ projects
Evidence: Event log query, 3 months, about 4,000 users (small for paid conversion)
Success metrics: >= 20% of new free users create a second project within 30 days; upgrade rate 2+ vs 1 project at 45 days; validate by 2026-04-24
Risks and fallback: If <20% reach a 2nd project the cap is not binding: drop it. If student churn rises past threshold: raise cap to 5
Dissent log: Support lead: cap raises student churn. Link: comment thread on the proposal. Resolved: accepted with churn tracked.
Review date: 2026-04-24 Outcome: (filled at review)
Trade-offs and pitfalls
- A template that is too long dies. If people skip fields, cut them.
- A record with no threshold looks rigorous but cannot be wrong. Insist on a number and a date.
- Do not use the dissent log to relitigate. It records reasoning, not a running argument.
Unlock Full Question Bank
Get access to all 8 Documentation and Knowledge Management interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.