Consultative Discovery and Requirements Gathering Questions

Direct answer

"Low latency" (the delay before a system responds) or "faster" is a feeling, not a requirement. I would ask what the user is doing, where and how it is measured, which percentile counts, what load it must survive, what the current number is, and what the client would give up to get it. Different answers lead to different designs, so I would write the answer as a target with a number and a measurement point. (The service-level agreement, or SLA, is the written promise to customers about things like response time or uptime.)

What I ask, in order

1. Which journey? "Which action feels slow: page load, search, checkout, a nightly job?" A page and a batch job need different fixes.
2. Measured where? In the browser, at the server edge (the point where a request first reaches your servers), or inside the database? Each hides different delays.
3. Which percentile? A percentile such as p95 is the time within which 95% of requests finish (p99 is the same idea at 99%, so it looks further into the slow tail). The median (the middle value) hides the slow tail, meaning the slowest few percent of requests, so ask: "Is the median fine, or do the slowest few requests hurt?"
4. What is the baseline and the target? "How long does it take now, and what number would feel fast?"
5. What load? Peak requests per second, steady versus spiky, data size.
6. What does slow cost? Abandonment, penalties in the service-level agreement (SLA), a human waiting.
7. What can give? Cost, freshness of data, accuracy, features.

How answers change what I build

The first move for an interactive page is usually caching and fewer round trips, because those are cheap and fix the most common cause. The other rows apply only when the measurement points there.

Plain meanings: a content delivery network (CDN) is a set of servers near users that serve saved copies of pages. A round trip is one request and reply between the browser and a server. Queueing is requests waiting in line for a busy server. Capacity headroom is spare capacity above the expected peak. Batching is processing several items together instead of one at a time. A compressed model is a smaller version of a machine-learning model that runs faster, usually at a small cost in accuracy. An asynchronous step moves slow work out of the user's wait: the page confirms first and the slow call finishes in the background.

Related cases

• Latency and cost dual targets over several quarters: set quarterly milestones for both numbers, and say which wins on conflict.
• Cost-reduction percentage target: ask "percent of what baseline, which bill, per what unit (for example per 1,000 requests)?" A percentage of a growing total is not a target. Example (illustrative): "cut 20% of a $50,000 monthly bill" is $10,000, but if traffic grows 30% the bill would have risen by $15,000 anyway, so after the $10,000 saving it is still $55,000, higher than the $50,000 baseline, and nobody can say whether the 20% was met against the old or the new total. "Cost per 1,000 requests from $0.40 to $0.32" (also 20%) survives growth because it is a unit cost.
• Retraining time scoping: retraining means rebuilding a machine-learning model on fresh data. Ask which stage is slow (data load, training, evaluation), what triggers retraining, how often, and what slower is acceptable. Example (illustrative): "the weekly retrain takes 9 hours and must finish before Monday 6 a.m." is a testable target; "retrain faster" is not.
• Acceptable regressions: "If we hit the target but accuracy falls two points, is that acceptable?" Example (illustrative): "Up to 1 point of accuracy for a 40% faster response is fine; 2 points is not." That sentence turns the trade-off into a pass or fail limit.

Worked example (illustrative)

Client: "Make checkout faster." After the questions: "p95 of the pay-button click to confirmation page, measured in the browser, under 2 seconds at 500 requests per second peak, from about 3.5 seconds today." That is testable. If the cause is a slow third-party call, I propose an asynchronous step rather than a bigger server.

Trade-offs and pitfalls

• Quoting an average that hides the tail.
• Building before agreeing the measurement point.
• A target stricter than users can perceive costs money for no benefit.

Direct answer

I do not argue with "fast" or write it down as is. I treat it as a symptom and walk it to a number with plain questions: which action, how long now, what would be acceptable, for how many requests, and measured where. I then read the number back to the stakeholder (anyone with a stake in the outcome who can accept the requirement) to confirm.

The questions, in order (six), with the words I use

1. Which action? "Which screen or task feels slow, and who is waiting on it? Can you show me?" This turns a feeling into a specific task.
2. Baseline. "How long does it take today, roughly?" If nobody knows, I measure first, because a target needs a starting point.
3. Target by behaviour. "If it took 5 seconds, 2 seconds or under 1 second, which of those would change what you or your customers do?" Offering three values to react to is easier than inventing a number from nothing, and avoids anchoring (answers drift toward the first number heard) on a single figure I chose.
4. How many requests? "Does it need to be that quick every time, or for most requests? And when is it busiest?" This yields a percentile and a load. A p95 (95th percentile) of 800 ms means 95 of every 100 requests finish in 800 ms or less. An average can hide the slow ones: if 93 of 100 searches take 0.5 s and 7 take 20 s, the average is (93 x 0.5 + 7 x 20) / 100, about 1.9 s, which sounds fine, yet 7 in 100 users wait 20 s and the p95 shows 20 s. Those slowest requests are the slow tail.
5. Measured where? "Do we time it on our servers, in the user's browser, or on a phone over mobile data?"
6. Trade-off. "Are you willing to trade a feature, a deadline or cost for it?"

When the owner cannot say what success is

Work backwards from consequences. Ask "At what point would you phone me to complain?" (the unacceptable threshold is easier to state than the ideal), look at behaviour (abandoned searches, support complaints, rework), and compare with what rivals or internal tools do. Show them the measured baseline and ask them to point at the line where it becomes a problem.

Turning it into concrete targets, each with its test

Where the numbers come from (illustrative): if analytics show 360,000 searches in the busiest hour, that is 360,000 / 3,600 = 100 searches per second, and I ask for about 2x headroom, so 200. The 0.5% error rate comes from asking how many failed searches customers would tolerate: 0.5% is 1 in 200.

Terms in the table: real user monitoring (RUM) records actual timings from real visitors' browsers. A load test sends simulated traffic at a test copy of the system at a chosen rate while measuring speed and errors. Sustained at peak means holding that rate for a stretch (say 30 minutes), not a one-second burst. Ramped past peak means raising the rate beyond the expected peak until the system struggles, to learn its limit.

What I write down and read back

"Catalogue search returns results in 800 ms or less for 95% of searches, measured in the user's browser, while handling 200 searches per second, over a rolling 7 days. Today's baseline is 2.1 s p95." (Illustrative numbers.) I ask: "If we hit that, would you call it fast?" A yes makes it a requirement. A "not quite" shows the real number.

Pitfalls

• Accepting an average: a good average can hide a slow tail, so use a percentile.
• Fixing the number without the measurement point.
• Forgetting the test: a target nobody can check is only a wish.

Direct answer

"More reliable" is a feeling about a pattern of pain. I first find out what failures the stakeholder (the person who asked, with a stake in the outcome) has experienced, then pick one measurable indicator of user-visible success, set a target with a time window and a measurement point, check the target against history to see if it is realistic, and agree how progress will be reviewed.

Steps

1. Collect the pain. "Which incidents come to mind? What did customers or staff experience, and what did it cost?" Pull the last quarter's incident list and support tickets to confirm.
2. Choose the journey. Reliability of what? Checkout, login, nightly report? Pick the one that matters most.
3. Define the indicator (SLI, service-level indicator): the share of events that were good. Example: share of checkout requests that return a non-error response within 1 second, measured at the load balancer (the server that spreads incoming requests across machines, so it sees every request).
4. Set the target (SLO, service-level objective) with a window: 99.9% over a rolling 30 days (a window that always covers the latest 30 days, not a calendar month).
5. Check against history. If the last 90 days show 99.5%, 99.9% is a real investment, so the sponsor must know before agreeing. At 99.5%, 43,200 x 0.005 = 216 minutes of outage per 30 days were being tolerated; 99.9% allows 43, about five times less.
6. Agree what happens when the target is missed, for example pausing feature releases for reliability work. This pre-agreed rule is the error budget policy.

The arithmetic, shown

$30 \times 24 \times 60 = 43{,}200 \text{ minutes in 30 days}$

$43{,}200 \times 0.001 = 43.2 \text{ minutes}$

So 99.9% allows about 43 minutes of total outage in 30 days (the error budget). If the service needs two dependencies in series, each at 99.9%, the chain gives $0.999 \times 0.999 = 0.998001$, about 99.8%, so the target for my service cannot exceed what I depend on. Note that the 99.8% belongs to the two dependencies alone: if my own service also runs at 99.9% and all three are needed for every request, the user sees 0.999 x 0.999 x 0.999, about 99.7%. This holds when every dependency is a hard one; a cache, a fallback or graceful degradation can lift the user-facing figure above the product, which is why I ask which dependencies are truly required.

How I know afterwards it was met

• The SLI dashboard shows the SLO held over the window.
• Alerts fire when the budget is burning too fast, so we find out before the month ends. For example, if half of the 43.2-minute budget is gone after 3 days, which is 10% of the 30-day window, the burn rate is 5x (50% / 10%) and an alert fires.
• A monthly review compares incidents and support tickets before and after.
• I check the measurement itself, for instance that probes (scripted test requests sent on a schedule from outside, mimicking a user) see what users see.

Other vague phrasings: resilient, observability, better or significant

• "Make the service resilient": ask which failures it must survive (one server, a whole data centre zone, a dependency outage). Set an RTO (recovery time objective: how long recovery may take) and an RPO (recovery point objective: how much recent data may be lost) per failure, and prove them with a planned failure test (a controlled drill in which you deliberately switch off a server or zone and time the recovery).
• "Better observability": ask what question nobody could answer in the last incident. Turn it into "detect within 5 minutes, find the cause within 30" and set retention (how long data is kept) by what you need to look back at, balanced against storage cost, for example keeping detailed data for weeks and summaries longer.
• "Better" or "significant" from a non-technical stakeholder: ask "What would you see happen that tells you it is better?" Then agree a before-and-after comparison on a named measure and the smallest change that would count.

Pitfalls

• A target with no measurement point.
• Setting 99.99% because it sounds good, with no cost conversation.
• Counting server uptime while users still see errors.

Direct answer

"Zero downtime in two weeks" is two demands that may conflict. Here the migration means moving a live payment service to new infrastructure, and the cutover is the moment traffic switches from the old system to the new one. Availability is the share of time the service works for users. I would first find out what zero downtime has to protect (which users, which functions, how long an interruption is tolerable), then what two weeks is tied to, then lay out the options with their risk and cost. I would take it back as a choice, not a refusal: "Here is what is achievable by that date and what it costs to go further."

Exploring what it has to mean

• "Which services and users must stay up? Is read-only access during cutover acceptable?"
• "What is the longest interruption nobody would notice or complain about?"
• "What happens at 2 a.m. versus at peak? Are there blackout periods (times when changes are forbidden, such as month-end)?"
• "Why two weeks? A contract, an event, a licence expiry, a cost?"
• "What is the rollback requirement (the ability to switch back to the old system) if something fails?"

Feasibility, with the arithmetic

Availability numbers translate into allowed downtime. For a 30-day month (43,200 minutes):

A 99.999% payment cutover across clouds leaves about 26 seconds in a month. 99.9% is often called "three nines" and 99.999% "five nines". A single failed deployment can use that up, so the plan needs a staged cutover (moving traffic in steps rather than all at once), rollback rehearsed, and real-time monitoring.

Dependencies cap the target. A migration adds dependencies: the new cloud's services, a network link, a partner API. Suppose a client expects 99.99% overall but a dependency fails in 5% of hours (an exaggerated, illustrative figure chosen so the effect is easy to see). If every request needs that dependency, the parts are "in series": the whole works only when every part works, so availabilities multiply. 0.9999 x 0.95 = 0.949905, about 94.99%, so the dependency alone caps the system near 95%. Two independent copies reduce the failure to 5% x 5% = 0.25%, giving 99.75% (this assumes the failures are independent). So the answer is design change (redundancy, fallback, retries), not a promise.

Options I would bring back

1. Phased migration in cohorts (separate groups of users moved in turn; for 100,000 users: first 1,000, then the next 10,000, then the next 50,000, then the remaining 39,000), with the ability to roll back each cohort.
2. Read-only window (a few minutes when users can view but not change data), announced in advance.
3. Full dual-running (old and new systems both live and kept in sync, so traffic can move with no gap), which needs more time and budget.

My recommendation, for the stated two weeks: option 1 or 2 with a fixed agreed maximum interruption, and option 3 only if the VP funds more time. What would change my call: a hard legal or contractual reason for no interruption at all.

Capacity or growth uncertainty

If traffic may grow suddenly (viral growth), a capacity shortfall during the cutover is downtime too. Ask for plausible bounds, not one number. Illustrative: today 200 requests per second, low case 300, expected case 600, high case 2,000. Then agree what degrades first if the high one occurs, for example new signups are queued before payments slow down.

Taking it back to the VP

Use a one-page summary: what "zero downtime" is interpreted as, three options with date, risk and cost, my recommendation, and the decision needed by a given day. Lead with what we can deliver, then what each extra increment costs.

Trade-offs and pitfalls

• Saying "impossible", which ends the conversation.
• Promising the number without a rollback plan.
• Forgetting that dependencies set the ceiling.
• Different people reading "downtime" differently: always agree the measurement point.