Technical Writing and Documentation Questions

Direct answer

Data skew means a few key values hold most of the rows. A distributed join (a join run across many machines, for example in Spark) sends all rows with the same key to one worker, so with skew one partition (that worker's slice of the data) gets far more of the work. The whole job then waits on that one worker. A key holding most of the rows is called a hot key. A minimal reproducible example (the smallest data and query that still shows the problem, which anyone can run) needs a tiny synthetic table with the same key distribution as the real one, the join query, and a measured expected-versus-actual.

What I include

• Data: an orders table and a customers table, generated with a fixed seed. Only the join key matters, so the only real property to preserve is its frequency shape: here one customer holds about 90% of orders.
• Query: the join with the aggregate.
• Expected versus actual: expected, rows per partition roughly equal (largest close to the average); actual, one partition holds most rows.

import sqlite3, random

random.seed(42)
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE customers (customer_id INTEGER PRIMARY KEY, region TEXT)")
db.execute("CREATE TABLE orders (order_id INTEGER PRIMARY KEY, customer_id INTEGER, amount INTEGER)")

# 50 customers; customer 0 is a bulk reseller that places 90% of the 1,000 orders
db.executemany("INSERT INTO customers VALUES (?, ?)",
               [(i, "EU" if i % 2 else "US") for i in range(50)])
rows = []
for order_id in range(1000):
    cid = 0 if random.random() < 0.9 else random.randint(1, 49)
    rows.append((order_id, cid, random.randint(5, 100)))
db.executemany("INSERT INTO orders VALUES (?, ?, ?)", rows)

print("orders per customer, top 3:")
for r in db.execute("""SELECT customer_id, COUNT(*) AS n FROM orders
                       GROUP BY customer_id ORDER BY n DESC LIMIT 3"""):
    print(" ", r)

# A distributed join sends each join key to one worker. Real engines use a hash partitioner
# (a function that maps a key to a partition number); key % 4 is a simple stand-in.
sizes = [0] * 4
for (cid,) in db.execute("SELECT customer_id FROM orders"):
    sizes[cid % 4] += 1
print("rows per partition:", sizes)
print("largest / average:", round(max(sizes) / (sum(sizes) / 4), 2))

joined = db.execute("""SELECT c.region, SUM(o.amount) FROM orders o
                       JOIN customers c ON c.customer_id = o.customer_id
                       GROUP BY c.region ORDER BY c.region""").fetchall()
print("join result:", joined)

Output:

orders per customer, top 3:
  (0, 895)
  (31, 6)
  (39, 5)
rows per partition: [921, 18, 30, 31]
largest / average: 3.68
join result: [('EU', 2357), ('US', 50949)]

Reading it: customer 0 has 895 of 1,000 orders (89.5%). With 4 partitions, partition 0 gets 921 rows (customer 0 plus others that share partition 0), which is 3.68 times the average of 250 (921 / 250), close to the worst possible ratio of 4 (the maximum, reached when all rows land on one of the 4 partitions, since that partition then holds 4 times the average). Expected is about 1.0.

What I strip out

• Rows: 1,000 instead of millions. Skew is about proportions, not size.
• The other columns and tables that the join does not touch, and personal data (real customer IDs become integers). Synthetic means generated data, not a copy of the real table.
• Unrelated filters, unrelated jobs, and cluster-specific configuration.

How I know the strip-down is safe

After each removal I re-run the check and confirm the ratio is still far above 1. If removing something makes it disappear (for example, spreading the hot key), that part was essential and goes back in. Running this on the same data gave real numbers. The second line comes from adding this to the script above, which scatters the hot key's orders by mixing each order id into the key before partitioning:

spread = [0] * 4
for order_id, cid, _ in rows:
    spread[(cid * 31 + order_id) % 4] += 1
print(spread, round(max(spread) / 250, 2))   # [234, 256, 261, 249] 1.04

keys only (drop amount and the customers table): ratio 3.68  -> safe removal
hot key spread (each order id mixed into its key): rows per partition [234, 256, 261, 249], ratio 1.04  -> skew gone, so the hot key is essential

Honest limits

The customer_id % 4 rule is a stand-in for the engine's real partitioner, so on the real engine I would add the engine name and version, the partition count, and the slow task's duration against the median, since those cannot be derived from the toy.

Pitfalls

• Uniform random data, which never reproduces skew.
• Keeping real data, which is large, sensitive and slow to share.
• Reporting only "the job is slow" without the per-partition evidence.

Direct answer

Each entry says a word but leaves the decisions that cause wrong analysis unmade. I would find what is unclear per column, ask the owner to confirm each answer, and rewrite the entries with type, constraints, precise meaning and an example. Where I cannot confirm something, I mark it as an open question, not invent it.

What is unclear

• users.id "unique identifier": unique within what (one region, all regions, one tenant)? What type? Can it be reused after deletion? Does it change if two accounts are merged? Is it the internal key or a value shown to users?
• users.status "active/inactive": the slash does not tell me whether those are the only two values. Does "active" mean verified, recently logged in, or paying? Who changes it and when? Is it stored or derived? Can it be null? Are there missing states like pending, suspended or deleted?
• users.signup_date "date": a date or a timestamp? Which time zone? Which event counts: form submission, email verification, first payment? Can it be null? What do imported or legacy accounts contain? Which format?

Rewrite (proposed; assumptions flagged for owner confirmation)

users:
  id:
    type: bigint
    constraints: primary key, not null, never reused
    description: >
      System-generated surrogate identifier for one account. Unique across
      all regions. Not the user's email and not stable across an account merge
      (the surviving account keeps its id; merged ids are recorded in
      user_merges).
  status:
    type: text
    constraints: not null
    allowed_values:
      active: Email verified and not deactivated. Says nothing about recent usage.
      inactive: User closed the account, or an administrator deactivated it. Data is retained until the deletion date.
      pending: Account created but email not yet verified (proposed; without this value an unverified, not-deactivated user fits neither active nor inactive).
    changed_by: account settings page, admin tool
    notes: Values are exactly these three; any other value is a data error.
  signup_date:
    type: date
    constraints: not null
    description: >
      Calendar date in UTC on which the account was first created (the moment
      the signup form was submitted, not when the email was verified).
    format: YYYY-MM-DD
    notes: For accounts imported from the old system, this holds the original creation date, not the import date.

What I assumed and must confirm with the owner

1. That the status values are exactly active, inactive and pending (the original snippet only showed "active/inactive"; the pending value is my proposal to cover unverified accounts), and what "active" really means (it might be defined by recent activity, in which case the rewrite is wrong and the column may be derived).
2. Whether id is reused and how merges work, including whether a user_merges record exists (I invented that name), and the retention rule for inactive accounts.
3. Which event marks signup and whether legacy accounts hold original or import dates.

Why the rewrite is unambiguous

• Every column has a type and nullability, so a query author knows whether to guard against nulls.
• The status values are enumerated and defined, so two analysts counting "active users" get the same population.
• The signup date names its time zone and its triggering event, so a daily signup chart does not shift at midnight local time.

Pitfalls

• Replacing a vague definition with a confident one that is wrong; unconfirmed content should be marked as such.
• Making "active" mean usage in a status column; usage changes daily and belongs in a derived metric.
• Forgetting to record how the definition changed, since historical values may follow an older meaning.

Direct answer

The sentence hides at least six decisions, and each one changes the numbers the model sees. "Resize to 224" does not say which side, what happens to aspect ratio, or how pixels are blended. "Normalize between 0 and 1" does not say what is divided by what. Another engineer following it literally would get a different input tensor (the array of numbers fed to the model) and a silently different model score. (The model was trained on inputs prepared one particular way; any other preparation gives it inputs it never learned from.) The rewrite must pin every choice, name the library, and give a check value.

Ambiguities

1. "224": 224 by 224, or the shorter side set to 224 (keeping aspect ratio)? If the shorter side, is there a crop, and where (center, random)?
2. Aspect ratio: squash the image to a square, or preserve the ratio and pad or crop?
3. Interpolation (the rule for blending neighbouring pixels when the size changes): nearest, bilinear or bicubic. Libraries also differ in antialiasing (smoothing before shrinking), so the same method name can give different pixels.
4. Input state: colour space and channel order (RGB versus BGR; OpenCV loads BGR by default), grayscale versus 3 channels, and integer 0-255 versus float.
5. "Between 0 and 1": divide by 255, or min-max per image (subtract the image minimum, divide by its range)? These agree only when the image happens to contain both 0 and 255.
6. Missing: dtype (float32), whether the dataset's mean is subtracted and the result divided by its standard deviation afterwards (a common extra rescaling that changes every value), tensor layout (height, width, channels versus channels first), and order of operations (resize before or after scaling).

Seeing the difference, not just asserting it

import numpy as np
from PIL import Image

# A 4x4 grayscale image with pixel values 0, 16, 32, ... 240
pixels = (np.arange(16, dtype=np.uint8) * 16).reshape(4, 4)
img = Image.fromarray(pixels, mode="L")

# "Resize to 2": same target size, three different interpolation methods
for name, method in [("nearest", Image.NEAREST),
                     ("bilinear", Image.BILINEAR),
                     ("bicubic", Image.BICUBIC)]:
    out = np.asarray(img.resize((2, 2), resample=method))
    print(name, out.flatten().tolist())

# "Normalize between 0 and 1": two different recipes
x = np.array([0, 51, 255], dtype=np.float64)
print("divide by 255:", (x / 255.0).round(3).tolist())
print("min-max of this image:", ((x - x.min()) / (x.max() - x.min())).round(3).tolist())
dark = np.array([20, 40, 60], dtype=np.float64)
print("divide by 255 (dark image):", (dark / 255.0).round(3).tolist())
print("min-max (dark image):", ((dark - dark.min()) / (dark.max() - dark.min())).round(3).tolist())

Output:

nearest [80, 112, 208, 240]
bilinear [57, 83, 157, 183]
bicubic [47, 77, 163, 193]
divide by 255: [0.0, 0.2, 1.0]
min-max of this image: [0.0, 0.2, 1.0]
divide by 255 (dark image): [0.078, 0.157, 0.235]
min-max (dark image): [0.0, 0.5, 1.0]

The same 4x4 image resized to 2x2 gives three different results depending on interpolation (for example 80 versus 57 versus 47 in the first cell). For normalization, on the image containing 0 and 255 both recipes agree, but on a dark image (20, 40, 60) dividing by 255 gives 0.078 to 0.235 while min-max stretches it to 0.0 to 1.0. A model trained on one recipe and served with the other sees a different input distribution (the overall spread of values it receives, which it never learned to handle).

Rewritten step (the values are an example; they must match what the model was trained with)

Input: an 8-bit RGB image (channel order R, G, B; convert grayscale or BGR sources first).

1. Resize so the shorter side is 256 px, keeping aspect ratio, bilinear interpolation, using Pillow Image.resize (pin the Pillow version in the lockfile, the file listing the exact version of every package).
2. Center-crop (cut a 224 x 224 window out of the middle of the resized image) to 224 x 224 px.
3. Convert to float32 and divide every value by 255.0, so values lie in [0, 1]. Do not rescale per image. No mean or standard deviation subtraction.
4. Output layout: height x width x channels (224, 224, 3); transpose only inside the model wrapper.
   Check: the provided sample image docs/sample.jpg must produce a tensor whose first row sums to the value stored in tests/expected_sum.txt, and a unit test asserts this (within 1e-3).
   Tiny illustration of the idea: a first row of pixels 0, 51, 255 divided by 255 gives 0.0, 0.2, 1.0, which sums to 1.2, so the file would contain 1.2. For the real 224 x 224 image the file holds whatever number the reference function printed when the team first ran it; anyone whose pipeline prints a different number has diverged.

Trade-offs and pitfalls

• Writing the rule is not enough: ship the preprocessing as one shared function used by both training and serving, and test it. Prose drifts, code does not.
• The most damaging bug in this family is training and serving using different pipelines (training-serving skew: the model sees differently prepared inputs in production than it saw in training, so it scores worse there with no error message). The check value catches it at review time.
• Pinning the library version matters because implementations change between releases.

Direct answer

Reproducible means a stranger, with only what you left behind, can rebuild the same inputs, run the same code in the same environment and get the same result (or a result inside a stated tolerance). The minimum I always record is five things: the exact code version, the exact data version, the environment, the randomness controls, and the question plus the result and decision. Everything else can be found from those five.

The minimum record (one per experiment)

1. Question and decision: what you were testing and what you concluded. Without this, six months later nobody knows why the run exists.
2. Code: the git commit hash (a unique ID for a saved version of the code) and whether there were uncommitted changes. A notebook name alone is not a version.
3. Data: a snapshot identifier or query plus date, row counts, and how the train/validation/test split was made. Provenance (where each input came from and how it was transformed) includes the feature pipeline: which script and commit built the features, since features are often rebuilt from a moving table.
4. Environment: Python version and a lockfile (an exact list of package versions), plus hardware where it matters.
5. Randomness: seeds (the number that starts a random generator, so shuffles and initial weights repeat) for every library used, and a note on operations that are not deterministic anyway (some GPU operations sum in a different order and vary in the last decimals).

Then the result (metric, baseline, tolerance, where the saved model or output lives).

Worked example (illustrative values)

Metric used below: recall@top10pct means rank customers by the model's predicted churn risk, flag the top 10%, and measure what share of the customers who actually churned were in that flagged group. 0.61 means 61% of real churners were caught.

experiment: churn-model-2026-09-14-run3
question: Does adding the 30-day support-ticket count improve churn recall on new customers?
code:
  repo: github.com/example-org/churn-model
  commit: 3f9a1c2
  uncommitted_changes: none
environment:
  python: "3.12.4"
  lockfile: requirements.lock (sha256 a41c...e07)
  hardware: 1x GPU, driver 550
data:
  snapshot: warehouse.churn_features@2026-09-01
  row_count: 184230
  query_or_pipeline: pipelines/build_features.py @ 3f9a1c2
  split: by customer_id hash, 80/10/10, seed 42
randomness:
  numpy_seed: 42
  model_seed: 42
  nondeterministic_ops: GPU sum order (results vary in 4th decimal)
result:
  metric: recall@top10pct
  value: 0.61
  baseline_value: 0.58
  artifact: s3://ml-artifacts/churn/run3/model.pkl
decision: Keep feature; re-check on the Q4 snapshot.

This record is a short file stored next to the run (or in an experiment tracker such as MLflow or Weights & Biases; either tool is optional, a plain YAML file committed next to the code already meets the bar). Six months later, checking out commit 3f9a1c2, installing the lockfile, and rebuilding the 2026-09-01 snapshot reproduces the run within the stated tolerance.

Documenting a reproducible investigation of a claimed segment-level failure

Suppose a stakeholder says "the model is bad for customers on the annual plan". Write the investigation so it can be re-run:

• Claim in one sentence, with who made it and when.
• Definition of the segment as code or a query (not "annual customers"), plus its size.
• Same-snapshot comparison: the model's metric on the segment versus everyone else, computed on a named snapshot, with a confidence interval (a range the true value plausibly lies in, wider for small samples) or at least the segment's row count, because a small segment gives noisy numbers.
• What was ruled out: label quality (are the churn labels for this segment correct, or do annual customers show as "not churned" only because their contract has not ended yet), leakage (a feature that accidentally contains the answer, or information from after the prediction date, so offline scores look better than reality), a train/serve preprocessing mismatch, and a change in the data feed.
• Verdict and next action, and the exact command to reproduce the table.

Filled-in segment example (illustrative)

• Claim: "the model is bad for annual-plan customers" (raised by the retention lead, 2026-09-20).
• Segment: plan_type = 'annual' on warehouse.churn_features@2026-09-01; 1,840 customers, of whom 220 churned.
• Result: the model flagged 108 of those 220 churners in its top 10%, so recall@top10pct = 108/220 = 0.49. Everyone else: 0.62.
• Uncertainty: with 220 churners, the rough 95% interval on 0.49 is about 0.42 to 0.56, which does not reach 0.62, so the gap is unlikely to be noise.
• Ruled out: labels re-checked on 50 sampled annual accounts (all correct); no leaked feature found; the same preprocessing function is used in serving.
• Verdict: real weakness. Next action: add annual-specific features and re-measure; reproduce the table with python analysis/segment_recall.py --snapshot 2026-09-01.

Trade-offs and pitfalls

• Notebooks are the usual culprit: cells run out of order and hold hidden state. Re-run top to bottom before saving, and export the final analysis as a script or clean notebook.
• Over-documenting kills the habit. Automate capture (log the commit, lockfile hash and seed from code) so the human writes only the question and the decision.
• "Reproducible" has levels: same numbers exactly, same numbers within a tolerance, or same conclusion. Say which one you promise.

Direct answer

For compliance I write a factual, complete, plain-language record of what personal data goes in, why, where it flows, who can see it, how long it is kept, and what controls exist, and I leave the legal conclusion ("this is lawful") to them. For fellow data scientists I write shorthand about features, experiments and pitfalls. The facts overlap; the reader, vocabulary, completeness and burden of proof differ.

How the two documents differ

What the compliance document contains

1. Purpose of the model, in one paragraph.
2. Personal data inventory: each field, whether it is personal data (any information relating to an identifiable person) or sensitive (special categories such as health or religion that carry stricter legal rules), its source, and why the model needs it.
3. Data flow: collection, storage, training, inference, outputs, third parties.
4. Retention and deletion: how long training data and logs live, and how a person's deletion request is handled, including the honest answer if a trained model cannot be edited and must wait for a retrain.
5. Access and security controls.
6. Model outputs and decisions: whether a person is affected by an automated decision (a decision made by software with no human in the loop, such as an automatic loan refusal), and whether a human can review it.
7. Known risks and mitigations, and open questions marked as open.

Worked example (excerpt)

"Field: postcode. Personal data: yes, can help identify a person in combination with other fields. Source: customer signup form. Purpose: regional demand feature. Stored in the feature table; access limited to the modelling team group. Retention: raw postcode deleted after 24 months by a retention job (a scheduled job that deletes data past its keep-until date; see linked ticket). The 24 months is an example value; the real period comes from the company's retention policy. Reduced to a 3-character region code before training." A data scientist note would just say "region_code, dropped postcode, re-identification risk with city" (re-identification risk: region code combined with city could narrow a record down to very few people; this is different from target leakage, where a feature carries information that would not exist at prediction time).

Pseudonymised versus anonymised: pseudonymised data has names or IDs replaced by codes, but someone holding the key or other fields can still work out who it is, so it is still personal data. Anonymised data cannot be traced back to a person by anyone with reasonable effort, which is a much higher bar. Calling pseudonymised data anonymised overstates the protection and can mean the wrong legal rules are assumed.

Pitfalls

Overclaiming ("fully compliant", "anonymised" for merely pseudonymised data), leaving gaps silently, and writing a legal argument yourself. If I do not know something, I write "unknown, owner X will confirm by date Y".