Direct answer
I would comment on the claim, not the author, ask for the evidence as a question, say what would change my mind, and mark which comments block approval. I push back when the claim drives a costly or hard-to-undo decision, accept when it is cheap to reverse and the design includes a way to check it, and move to a call when the thread has gone two rounds without converging, when the tone is tightening, or when the disagreement is about meaning (what the claim or the goal actually is) rather than about facts a benchmark could settle.
Example
The design doc says: "The new cache will cut p99 latency (the time within which 99% of requests finish) in half." There is no benchmark.
Comment drafts
- Question about the evidence: "This claim is the main reason for adding the cache, so I want to be sure of it. What workload and data size was this measured on, and what is the baseline? If you have not measured yet, could we add a quick benchmark to the doc, or mark this as a hypothesis with a plan to test it?"
- Say what would settle it: "A comparison of cache-on vs cache-off on a replay of last week's traffic (recorded real requests sent through both versions) would convince me."
- Label severity: start with "Blocking:" for the claim, and "Nit:" or "Optional:" for minor edits, so the author knows what must change.
- Acknowledge what is good: "The invalidation section (how stale cached entries get removed) is clear, and the rollout plan is thoughtful."
Push back or accept
| Situation | Choice |
|---|
| The claim justifies adding a new system or a data migration (hard to undo) | Push back until there is evidence |
| The claim is incidental to the design | Accept, and note it as unverified |
| The design includes a feature flag (a switch that turns the new behavior on for a few users first) and a measurement in the rollout | Accept, and make the measurement an explicit exit criterion (a condition that must be met before rollout continues), for example "the cache stays on only if p99 drops at least 20% on the 10% of traffic that gets it" |
When to move to a call
- After two rounds of replies that still disagree.
- When the thread is getting longer or the tone is tightening, since text hides intent.
- When the author is likely to be discouraged by a stack of comments.
- When the disagreement is about meaning, such as what "latency" or "fast enough" refers to, not about a fact a measurement could settle. A call clears up definitions quickly, while more text rarely does.
Accepting an incidental claim without a check is safe only if the thread says it is unverified, so nobody later builds on it as if it were measured.
After the call, write the outcome back into the doc ("agreed to benchmark on replayed traffic, owner Sam, due Friday") so the decision is on record.
Written feedback on a research-style draft
The same structure fits a colleague's paper draft: a short summary of what you understood the claim to be, then strengths (specific), then concerns ordered by importance (for example: the baseline is weak, the claim is stronger than the experiment supports), each with a suggested check, in a respectful tone. A concern written out: "The comparison is against a model with default settings, so the gain may come from tuning, not the new method. Could you add a baseline that gets the same tuning effort? If the gap holds, the claim is much stronger." Starting with the summary shows the author whether you understood them, which often removes half the disagreement.
