Structured Problem Solving and Decomposition Questions

Direct answer

Treat "85% to 99%" as a measurement problem first and an engineering problem second. Spend the first week pinning down what "on time" means and why the late runs were late, then split the work along the causes that account for the most late runs, and give leadership a one-page plan with a baseline, a few workstreams, monthly targets and the decisions you need from them.

Structured elaboration

What to learn first (week 1)

1. The definition. A pipeline is a scheduled job that moves and transforms data. "On time" needs a deadline (say 06:00), a measurement point (job finished, or data actually visible to consumers) and a scope (all pipelines, or only the ones people depend on). Count in one unit: a pipeline-night is one pipeline's run on one night.
2. Who is hurt by lateness. Tier the pipelines. One feeding a 07:00 executive dashboard matters more than an archive job. The 99% may only need to hold for the top tier, which is a question to settle with leadership early.
3. A cause log. For every late pipeline-night in the last several weeks, record the first thing that went wrong. Rule: each late run goes in exactly one bucket, so the buckets cannot overlap.

Subproblems to split off. The split below is MECE (mutually exclusive: no run in two buckets; collectively exhaustive: every late run lands somewhere), because the first-cause rule decides placement in time order.

Worked example

All numbers are illustrative. Take 20 pipelines over 30 nights: 600 pipeline-nights. At 85% on time, 510 are on time and 90 are late. At 99%, at most 6 may be late, so about 84 of the 90 late runs (roughly 93%) must stop happening.

Cause log: started late 45 (upstream late 36, dependency chain with no slack 9 (job B waits for job A with no spare time between them, so any delay in A passes straight through)), failed 27, ran too long 18. The total is 90. So half the problem (45 of 90) is upstream timing and 30 percent (27 of 90) is failures. That ordering decides the plan: fixing run time first would attack only 20% of the lateness.

One run traced through the rule: a pipeline due at 06:00 gets its upstream data at 05:30 instead of 04:00, so it starts late, and it then also fails on a timeout. It is logged once, under started late, because that was the first thing that went wrong; the failure goes in a notes column. Counting it in both buckets would make the buckets sum to more than the 90 late runs.

Monthly targets: 91% means at most 54 late runs, 96% at most 24, 99% at most 6.

The one page for leadership (excerpt):

• Goal: tier-1 pipelines on time 99% of nights by month 3, measured as data visible by 06:00. Baseline today: 85% across all 20 pipelines (the tier-1 baseline comes from cutting the same cause log by tier, and the 99% applies to that tier-1 slice).
• Diagnosis: 90 late runs in 30 nights: 50% started late, 30% failed, 20% ran long.
• Workstreams: (1) upstream delivery agreements and fallback data, (2) failure recovery and alerting, (3) runtime on the critical path.
• Measure: weekly on-time rate per tier, plus late runs by cause.
• Milestones: 91% month 1, 96% month 2, 99% month 3.
• Asks and risks: a named owner on each upstream team; if upstream will not commit, we deliver stale-marked data on time instead of fresh data late.

The same one-page shape for a latency goal such as cutting data latency by 50% in six months. Define latency as source event to queryable data, measured at the 95th percentile (sort the runs from fastest to slowest; 95 of every 100 finish at or under this value, so it shows the slow tail rather than the typical run). Break it by stage and give each a budget. Illustrative: ingest 15, transform 30, load 10, refresh 5 minutes (60 total) becomes 8, 14, 5, 3 (30 total). Caveat: the 95th percentile of the total is not the sum of the stage 95th percentiles. With independent, light-tailed stages the sum usually overstates it, because the slowest run in one stage is rarely the slowest in every stage; with heavy-tailed stages (a few very long runs) it can understate it, so neither direction is guaranteed. Treat the stage budgets as targets and measure the end-to-end 95th percentile directly. The page then lists goal, one component per stage, a measure per stage and checkpoints roughly every month.

Trade-offs and pitfalls

• A single average hides the tail: report on-time rate per tier, not one blended number.
• Do not promise 99% before the cause log exists. After week 1, state the plan with real data.
• Upstream lateness cannot be fixed by your team alone, so the plan must contain a fallback you control.
• Fixing the loudest incident instead of the largest bucket is the common wrong turn.
• If two buckets turn out to share one cause (for example a shared cluster), say so on the page and treat it as one workstream.

Direct answer

I do not start with analysis. I start by turning the vague request into a decision for the stakeholder (the person who asked for the work or will act on its result): who has to decide what, by when, and what would change their mind. Then I break the problem into parts that do not overlap, guess which parts matter most, test the cheapest high-impact guess first, and hand back a recommendation with my confidence and the assumptions it rests on. I show the stakeholder something small early so I am never working blind for weeks.

The process, step by step

1. Frame the decision. Ask: what decision will this inform, by when, what would you do if the answer were A versus B, and what is off limits (budget, tools, timeline)? Write the result as one sentence plus a success measure (the number that tells you the problem is solved). Example framed sentence: "Decide by 14 March whether to put next quarter's budget into onboarding, pricing or acquisition; success means paid sign-ups back to 120 a quarter." A Solutions Architect (SA) facing a client who says "make it modern and faster" does exactly this: which workload, what latency or cost number, by which date, and what cannot change.
2. Decompose into MECE parts. MECE (mutually exclusive, collectively exhaustive) means no item sits in two branches and every item sits in one. The tool is an issue tree: the question at the root, a split underneath, then splits of the splits. A small one for "why are paid sign-ups down?" (illustrative):

Why are paid sign-ups down?
|-- Fewer people arrive (visitors)
|   |-- Paid traffic down
|   |-- Organic traffic down
|   `-- Other sources down (referral, direct, email)
|-- Fewer arrivals sign up (sign-up rate)
`-- Fewer sign-ups pay (paid conversion)

It is MECE as a funnel: every visitor stops at exactly one stage (left without signing up, signed up but did not pay, or paid), so the stages partition the people, and the traffic branch ends in an "other sources" leaf so no channel is homeless. It does not follow that the drop in the count belongs to one stage: both rates can fall at once, as the worked example below shows, which is why the tree tells me where to look and not how to split the loss.
3. Form and rank hypotheses. A hypothesis is a specific, testable guess ("fewer sign-ups are reaching the paid plan"). A hypothesis tree starts from one belief ("the sign-up page change hurt") and lists the facts that must be true if it holds, then tests those. An issue tree maps the whole space first; a hypothesis tree bets on one branch to save time. Rank by how much of the problem it could explain, how likely it is, and how cheap it is to test.
4. Sequence the work, cheapest first. For a data engineering (DE: building and maintaining the pipelines that collect and prepare data) request the delivery stages are request, clarify, design, build, validate, deliver, and each gets assumptions-log entries (below). The ordering rule for the analysis itself, which applies inside the clarify and validate stages: list the open hypotheses, estimate for each how much of the problem it could explain and how many hours it takes to check, and do first the one with the best explained-share per hour, preferring checks that can eliminate several branches at once (a funnel count that already exists beats a new survey). Re-rank after every result. Whatever the work type, each stage gets entries in a written assumptions log (a running list of what I assumed and who can confirm it) so a wrong assumption surfaces at a review, not after delivery. One entry: "Assumed 'sign-up' means a verified email, not a started form. Owner: growth PM. Confirm by Friday. If wrong, the 4% rate changes."
5. Test and update the tree. Drop branches the evidence rules out and split the ones it supports.
6. Recommend. Lead with the answer, give two or three supporting facts, state confidence, list assumptions, and name the next decision or action needed.

Which structuring tool, and when

Worked example

A stakeholder says: "Our new self-serve plan is not growing. Figure out what to do." (Illustrative numbers.)

• Frame: decide next quarter's investment among onboarding, pricing and acquisition, by the planning meeting in two weeks.
• Metric tree: paid sign-ups = visitors x sign-up rate x paid conversion.
• Cheapest test first: the funnel counts already exist. Last quarter: 20,000 visitors, 5% sign-up (1,000), 12% paid (120). This quarter: 20,000 visitors, 4% sign-up (800), 10% paid (80).
• Result: paid sign-ups fell from 120 to 80. Holding conversion at last quarter's 12%, the sign-up drop alone gives 800 x 0.12 = 96, a loss of 24. The conversion drop then takes 96 to 80, a loss of 16. Attribution order matters. Taking the conversion drop first instead: 1,000 x 0.10 = 100 (a loss of 20), then the sign-up drop takes 100 to 80 (another 20), which is 50/50. Because the split moves from 60/40 to 50/50 with the order, I say "sign-up is the larger or equal share" and not a precise percentage.
• Recommendation: investigate the sign-up step first (larger share), with a conversion check in parallel, and state the assumption that traffic quality is unchanged.

How I know the process is working

Three measurable outcomes: time-to-first-insight (days from request until the stakeholder has something to react to), rework (how many deliverables are redone because a requirement was misread), and follow-through (share of my recommendations actually acted on).

Trade-offs and pitfalls

• Pulling all the data first feels safe but has no stopping rule and delays the first insight.
• Frameworks are scaffolding. If the tree does not change what I do next, it is decoration.
• Unwritten assumptions are the commonest cause of rework. Write them down and send them to the stakeholder.
• Waiting for certainty is its own failure: a recommendation with stated confidence and a cheap way to be wrong beats a perfect one delivered late.

Direct answer

Accuracy is correct predictions divided by labelled predictions. Both sides of that fraction come from only four places, which is why four branches can cover it. A prediction depends on the inputs and on the deployed model and pipeline; the label it is compared with depends on the real outcome and on how labels are produced. So the number is determined by four things: the inputs the model receives, the true relationship between inputs and the real outcome, the deployed model and pipeline that turns inputs into predictions, and the process that produces labels and the metric. A fall in accuracy requires a change in at least one of them, so those four are my top-level branches. I check soundness by replaying past incidents onto the tree and by testing every leaf for a single home.

The tree (example: a spam filter, where the two classes are spam and not spam)

Terms in the leaves: a threshold is the score cut-off above which the model says "spam". Upstream means earlier in the data flow. A schema is the agreed list of fields and their types. Retraining means fitting the model again on newer data. A preprocessing library is shared code that cleans and converts raw data before the model sees it. Drift means live data or the real-world pattern has moved away from what the model learned from. Data drift is the inputs changing (A1); concept drift is the link between inputs and the true outcome changing (B1).

Tie rule that keeps leaves exclusive: classify each cause by where in the chain it originates. A broken upstream job that feeds bad values is A2, even though it also looks like data drift. A shift in the spam share is A1, since it changes the inputs, not the rule linking them to labels. Two more overlaps to settle in advance. Skew versus a changed library: if a feature is computed differently online than in training and nothing was deployed this week it is A3; if a deployed code or library change made it differ it is C3 (the change event is the origin). Traffic mix versus labelled-sample mix: if real traffic changed it is A1; if traffic is unchanged but the emails chosen for labelling changed, it is D3.

How I check the structure is sound

1. One-home test: write five real past incidents (or imagine plausible ones) and place each. If one fits two leaves, sharpen the leaf definitions. If one fits none, there is a gap.
2. Exhaustiveness argument: could accuracy move while all four factors stay fixed? If I cannot construct such a case, the four branches cover the space. Where I cannot argue it, I add an explicit "unexplained" leaf that must shrink to zero before anyone calls the investigation finished.
3. So-what test: each leaf maps to a different owner or action (data team, ML team, labelling team, analytics).
4. Peer review: ask someone who did not draw the tree to find a cause with no home.

One incident traced through the one-home test (illustrative numbers)

Daily accuracy on 1,000 labelled emails falls from 95% (950 correct) to 89% (890 correct). Investigation shows a feature called sender reputation arrives empty for 12% of emails since a data-job change last Tuesday. Placement: the model and threshold are unchanged (not C), labelling volume and rules are unchanged (not D), spammer behaviour is the same on emails with a score (not B). The inputs are wrong, and they are wrong because an upstream job broke, so the single home is A2, not A1 and not B1. Fixing the job and re-scoring the affected emails should restore accuracy; if it recovers only part of the 6 points, the remainder still has to be placed on a leaf, and an unplaceable remainder goes to the "unexplained" leaf.

Trying to break the exhaustiveness argument: random noise on a small labelled sample can move accuracy with nothing real changing. That is a measurement cause, which is why D3 includes "too small".

Same discipline on three other systems (one split each, checked with the same one-home and exhaustiveness tests used on the classifier tree)

A feature store is a service holding precomputed model inputs (features) for training and live use. A Kafka topic is a named stream of events, durably stored (the Kafka documentation likens it to a folder of events), that producers write to and consumers read from. Here a broker is a server holding the log, a partition is one slice of a topic, a sink is where a consumer finally writes the data, retries are repeated send attempts after a failure, replication lag is the delay before copies of data catch up, and cache lifetime is how long a stored value is reused before being refreshed.

Pitfalls

• Mixing "what" and "where" in one level (for example "data drift" next to "upstream job failure").
• A catch-all "Other" holding most of the cases.
• Debugging before the tree is agreed, which anchors everyone on the first plausible story.

Direct answer

Pick one story where the task was genuinely vague, you chose a small first deliverable on purpose, and the plan changed because of what you learned. STAR means situation, task, action, result. The story below is an illustrative skeleton; swap in your own facts and numbers.

Structured elaboration: how to choose the first deliverable

I pick the piece that (1) is the smallest thing that is useful end to end, (2) can be tested within about two weeks, (3) uses data or access I already have, (4) carries low risk if wrong, and (5) teaches me the most about the parts I do not yet understand. Everything else gets sequenced after it.

Worked example (STAR)

• Situation: Leadership said, "Use AI to cut our support workload." No spec, no owner, no baseline.
• Task: I owned turning that into a plan within three weeks.
• Action:
  1. Wrote a one-sentence outcome with the sponsor: fewer agent minutes per ticket on routine questions, without a drop in customer satisfaction.
  2. Hand-labelled a random sample of 200 tickets by intent (the reason the customer wrote). 54 of 200 (27%) fell into just three intents with near-scripted answers (password reset, invoice copy, plan change).
  3. Cut the work into pieces: baseline measurement, draft replies for those three intents shown to agents, then a customer-facing chatbot later. Chose draft replies as the first deliverable. Why: 27% of tickets had near-scripted answers, so drafts could be right often; a support agent (a human on the support team, not an AI agent) reviews every draft, so a wrong draft costs little; and the chatbot would have to handle all 200 tickets, including the 73% with no script, while talking to customers directly. Drafts also produce labelled feedback for later steps.
  4. Shipped to four agents in week two.
• What changed: Agents ignored the drafts. The cause was that drafts lacked the customer's account context (plan, last invoice), so agents had to look it up anyway. I added an account lookup to the draft before widening the rollout, and parked the chatbot until acceptance was healthy (we had agreed at least 50% of drafts sent after changing less than about a fifth of the text). I also learned early that the intent categories needed one more split (billing questions with and without a dispute).
• Result: The weekly rate of drafts accepted with little editing, which I had set up before launch, was 22% in week 3 (44 of 200 drafts) and 61% in week 6 (122 of 200) once context was added (illustrative figures; use your own). The sponsor approved a second phase based on that measured number, not on a demo. Be honest about what that number is: acceptance rate is a leading indicator (a proxy that should move before the real outcome), not the outcome the sponsor asked for. Agent minutes per ticket and customer satisfaction were not yet measured, so the second phase was approved on the condition that the first milestone of phase two is that baseline, with a stated bar for the next review. A good result line names the metric, the before and after values, and what decision followed.
• What I would do differently: Measure agent minutes per ticket before building anything, and sit with two agents on day one instead of week two.

The same shape in other settings

• Ambiguous security task, little documentation: first deliverable was an inventory of services holding customer data (a usable v1 in days), which then told us where to look deeper.
• Cross-team operational problem: first deliverable was a shared timeline of handoffs between teams, since nobody owned the whole picture, followed by one agreed fix per handoff.

Pitfalls

• Starting with the most interesting piece, not the most informative.
• A story with no plan change. Interviewers are probing whether you adapt.
• Inventing precise results. If you lack a number, say what you measured and how it moved.