Structured Problem Solving and Decomposition Questions

Direct answer

What stays the same: state the goal, break it into parts that can be answered or built, write down assumptions, and sequence by what you learn or deliver per unit of effort. What changes: in exploratory research the parts are questions and hypotheses whose answers are unknown, success means reducing uncertainty (including finding out it will not work), the plan is expected to change, and I order work by the riskiest assumption first. In shipping a known product the parts are deliverables, the answer is mostly known, success means meeting a spec, and I order work by dependencies and risk to delivery.

Structured elaboration

Terms in plain words: the critical path is the longest chain of dependent tasks, which sets the earliest finish date. Acceptance criteria are the checks a deliverable must pass to count as done. A time-box is a fixed amount of time after which you decide, whatever you have found. A Gantt chart is a bar chart of tasks against calendar dates. A thin vertical slice is one narrow case built end to end (data through to what the user sees) instead of one layer for every case. A technical spike is a short time-boxed experiment that answers one technical question. A kill criterion is a result agreed in advance that means stop.

What stays the same, in detail

• A written goal and a decision the work will inform.
• Explicit assumptions, so a surprise is recognisable as a surprise.
• Small steps that produce evidence before big commitments.
• A record of what was tried and why, so work is not repeated.

Worked example

Question: "Can we summarise customer support tickets automatically?"

Research track, time-boxed to three weeks:

• Q1: Is there enough signal? A baseline summariser run on 200 hand-reviewed tickets (illustrative count).
• Q2: What does "good" mean to the agents who would use it? Five agent interviews and a rubric.
• Q3: What fails? Error analysis of the worst 20 summaries.
• Kill criterion agreed upfront: if reviewers rate fewer than 30 of 50 sampled summaries (60%, an illustrative threshold agreed with the agents) as useful, stop or narrow the scope. Writing it first prevents rationalising a weak result.

The same plan as a tree, with each branch carrying its test and a result that would reject it:

Goal: automatic ticket summaries worth shipping
|-- Q1 Is there enough signal?  Test: baseline summariser on 200 reviewed tickets.
|                               Rejected if the main issue is missed in most summaries.
|-- Q2 Would agents use them?   Test: five agent interviews and a rubric.
|                               Rejected if agents say they would re-read the ticket anyway.
|-- Q3 What fails?              Test: error analysis of the worst 20 summaries.
                                Rejected if most errors share one cause that cannot be fixed.

Gate: the research ends with a go, narrow, or stop decision. If go, the work becomes the known-product shape: thin vertical slices (one ticket type end to end, behind a flag), acceptance criteria from Q2's rubric, a dependency map including privacy review of ticket text, and estimates.

Trade-offs and pitfalls

• Applying a delivery plan to research produces false precision: Gantt charts for experiments nobody can estimate.
• Applying research looseness to delivery lets scope drift. The handoff gate is the safeguard.
• Open-ended research without time-boxes expands to fill the available time.
• Hybrids are common. A technical spike inside a product project follows the research shape, time-boxed, with its output being a decision.
• What would change my approach: if the cost of a wrong guess is very high (safety, regulated), I move toward more upfront structure even in research.

Direct answer

A good milestone sequence retires the largest unknowns first, cheapest first, and makes each milestone end in something you can evaluate, not just "work done". Tell one concrete story: what was uncertain at the start, how you ordered milestones against that uncertainty, one decision where the order mattered, and one split you would change with hindsight.

Structured elaboration

Use STAR (Situation, Task, Action, Result) with the Action as a sequence:

• Situation and Task: the project, the stakes, the constraint (time, data, team size).
• Action: list the milestones in order, each with what it proved, then why that order: usually data and labels exist and are trustworthy, then a simple baseline shows there is signal, then the model is integrated and run in shadow (scored but not acted on), then it takes partial real traffic with a human override, and only then optimized.
• Result: what each milestone told you and any decision it changed (including a stop or a pivot).
• Looking back: a specific different split, with the reason.

Worked example

Example story to adapt (qualitative, meaning described in words; add your real numbers only if verifiable). Project: routing incoming support tickets to the right team with a text classifier.

Situation and Task. Tickets were routed by hand; leadership wanted automation, with two engineers and a quarter.

Action.

1. Milestone 1: data and metric. Export past tickets with the team that finally resolved them, define "correct routing" with the support leads, build a held-out test set (tickets kept aside and never used for training, so the score reflects unseen cases). Proved whether labels were usable.
2. Milestone 2: baseline. Keyword rules, then a simple text model. Proved there was signal and set the baseline (the simple version any later model must beat).
3. Milestone 3: shadow mode. The model predicted on live tickets while humans routed as usual; we compared the two daily. Proved integration worked and exposed live-data differences.
4. Milestone 4: partial traffic. The model routed only high-confidence tickets (those where it was most sure of its choice), with an easy human override.
5. Milestone 5: improvement. A stronger model only for the categories error analysis (reading the mistakes to see which categories fail and why) showed were weak.

I put data and the metric first because it was the cheapest way to learn whether the project was viable at all, and left model sophistication last since it was the most expensive and least certain to matter.

What a go or no-go looked like (illustrative numbers). Milestone 2 bar agreed with the support leads: at least 75% correct on the held-out set. Keyword rules scored 55%, the text model 78%, so go. In shadow mode the model matched the humans on 80% of live tickets and the disagreements were read one by one; a pile of them were new product names, so a retraining step was added before partial traffic. Had the model sat at 60%, the call would have been no-go until labels or categories were fixed.

Result. Each stage ended in a go or no-go decision; shadow mode caught a mismatch between historic and live ticket wording before customers were affected.

What I would split differently. I defined labels in milestone 1 with one support lead; reviewers disagreed later about borderline categories. Now I would add a small labeling-agreement check to milestone 1 (two leads label the same 50 tickets independently and you count how often they pick the same team; 41 of 50 would be 82%, and a low figure means the categories need redefining before any model work), and bring the integration work forward so it overlaps with the baseline.

Trade-offs and pitfalls

• A calendar split ("month 1, month 2") is not a milestone split; each milestone must produce evidence.
• Starting with the most interesting model is the classic wrong turn.
• A story with no failed or changed milestone sounds rehearsed. Show a decision a milestone changed.
• "Nothing" as the answer to what you would split differently signals little reflection.
• Do not claim accuracy numbers you cannot defend; describe the decision each result supported.

Direct answer

First I turn the goal into a precisely defined number. Then I split it using a rule that guarantees the pieces do not overlap and cover everything, and I pick the splitting dimension that separates owners and actions. I stop splitting when each leaf is testable, owned, measurable and decision-relevant.

Worked decomposition: "onboarding completion dropped 30%" (illustrative numbers; the monolith goal is worked through with numbers after the table of other goals)

1. Define the metric. Completion = users who finish setup within 7 days of sign-up, divided by users who start. Check what "30%" means. A relative drop from 60% to 42% is 30% relative (18 percentage points); a drop of 30 points is a different problem.
2. First split: by funnel stage. Stages are sequential, so every user stops in exactly one place (mutually exclusive) and a user who finished passed them all (exhaustive). A simplified two-stage funnel:

Check: 0.75 x 0.80 = 0.60 and 0.75 x 0.56 = 0.42. Stage 2 fell from 0.80 to 0.56, which is 30% relative, so it holds the whole drop. A real funnel has more stages.
3. Second split inside stage 2: by what the user did. (a) never reached the connect screen, (b) reached it but never attempted, (c) attempted and failed. Each user is in exactly one state. The three states cover the stage-2 users who did not finish. Users who connected successfully are a fourth state, (d), kept in the split so it stays exhaustive and the sum test passes: with illustrative counts per 1,000 users who verified their email, before the drop 800 are in (d); after the drop 560 are in (d), and the 240 extra non-completers are spread across (a), (b) and (c). Each leaf has a metric from event logs, an owner (navigation team, product design, integrations engineering) and a test (count users in the state, before versus after). "Funnel" here means the ordered steps users pass through from sign-up to finished setup, and event logs are the recorded actions (screen viewed, button clicked) that let you count users at each step.

How I decide the split

I choose the dimension where (1) branches imply different actions, (2) branches map to different owners, (3) the data already exists, and (4) the split can be checked cheaply. A split by device or customer segment is useful too, but as a second-level cut, because it does not tell a team what to fix.

The stopping rule

A leaf is done when it is:

• Testable: one analysis or small experiment answers it in days, not weeks.
• Owned: one person or team can act on it.
• Measurable: it has a metric, a baseline and a target.
• Decision-relevant: a yes or no would change what we do.

Too shallow: a leaf needs a multi-week project or has two owners. Too deep: leaves nobody owns, or whose answer would not change the plan.

The same method on other goals

The monolith goal with numbers (illustrative)

Suppose the yearly bill is 1,200k and "cut cost 20%" means a saving of 240k a year, run cost only, with one-off migration cost reported separately. First split, by cost type, is run cost 1,200k plus one-off migration cost (tracked, not counted against the 20%). Second split, run cost by module: checkout 400k, search 300k, reports 200k, admin 100k, shared infrastructure 200k (400 + 300 + 200 + 100 + 200 = 1,200). Each module is a different team's code and bill, so each has an owner. Stopping check on the leaf "reports module, 200k": testable (move it behind an interface and compare its bill before and after, in weeks), owned (the reports team), measurable (baseline 200k, target for example 140k, a 60k saving, which is a quarter of the 240k goal), decision-relevant (if the saving is small, the next module is chosen instead). Shared infrastructure fails the check as a single leaf because no one team owns it, so it is split further by the module that consumes it.

Pitfalls

• Splitting by whatever is easiest to draw, which gives branches nobody can act on.
• Mixing two dimensions at one level (stage and device), which breaks mutual exclusivity.
• Splitting until the tree looks impressive instead of until leaves are testable.

Direct answer

"Engagement" is not a metric, so first fix a definition (meaningful actions per day), then split it into drivers that multiply back to it, and map each driver to the model output and input features that move it. Start with the feed ranker's action rate per item, with guardrails, unless the data shows new-user retention is the weaker driver.

Terms: DAU is daily active users. A ranker is the model that orders items in a feed. A feature is an input signal to a model. A guardrail metric must not get worse while you optimize the main one. Cold start is the problem of recommending to a user with no history. A cohort is a group of users who started in the same period (for example, everyone who signed up in one week). Headroom is how much room a metric has to improve.

Step 1: define the target

Engaged actions per day = DAU × sessions per DAU × items seen per session × action rate per item. "Action" means a meaningful one (comment, share, reply, post), because optimizing raw clicks builds clickbait. Illustrative baseline: 1,000,000 DAU × 3 sessions × 20 items × 10% = 6,000,000 actions a day (6 per DAU). Raising the action rate from 10% to 11% is +10% relative and lifts the total to 6,600,000.

Step 2: drivers, model outputs and features

Four drivers multiply. DAU itself is a sum of activated new users, retained users and resurrected (lapsed, returned) users.

Step 3: where to intervene first

Choose by four criteria: headroom, how directly the model controls the driver, how fast a test teaches you, and risk. The ranker wins on control and speed: you can replay logged data offline (re-score past sessions with the new model and see what it would have ranked, with no live users involved), then run an A/B test (users randomly split into new versus current) in one to two weeks. Notifications risk opt-outs. Activation may have the most long-run value but needs weeks of retention data to read.

The first move: blend predicted comment and share probability into the ranking score alongside likes. Illustrative weights: score = 1 x P(like) + 3 x P(comment) + 4 x P(share) - 5 x P(hide). Item X (like 0.10, comment 0.01, share 0.005, hide 0.002) scores 0.10 + 0.03 + 0.02 - 0.01 = 0.14. Item Y (like 0.15, comment 0.002, share 0.001, hide 0.02) scores 0.15 + 0.006 + 0.004 - 0.10 = 0.06. Ranking on likes alone puts Y first; the blend puts X first, which is how you steer toward conversation without a new model, and you can change the weights in a day. Guardrails: hide and report rate (so outrage bait or spam that people dislike is caught), creator diversity, and notification opt-outs.

What flips the call: if new cohorts' day-7 retention is far below older cohorts', the cold-start model goes first.

Pitfalls

The offline metric rises while the A/B test is flat. The ranker only learns from items it showed, so keep an exploration share (a small slice of feed slots, say 5%, filled with items the model would not normally pick, so it can learn whether they are good). An example of offline-versus-A/B gap: a new model predicts comments better on logged data, but in the live test people see more of what they already like and comment no more. Targets get gamed (Goodhart's law: a measure used as a target stops being a good measure; for example, counting raw comments rewards provocative posts, which is why guardrails sit beside the target).

Direct answer

Separate two orders that are often confused: the order things must be built in (set by dependencies) and the order you learn in (set by uncertainty). Build order follows the critical path; learning order starts the riskiest unknowns early with cheap spikes (short, time-boxed experiments that answer one question and are thrown away, not production code), even if the full build comes later. Anything with no dependency between them runs in parallel.

Structured elaboration

Assumed dependencies (state them, then confirm with the team)

• Auth (who the user is) feeds payments, because a payment belongs to an identity.
• Payments produces the transactions that fraud detection scores.
• Analytics events are inputs to fraud features, and analytics can start with instrumentation early.
• Client validation (checking input in the app before sending it) depends on nothing but helps payments forms.

Critical path is the longest chain of dependent work: auth, then payments, then fraud detection. Everything off that chain has slack (time it can slip without delaying the end).

Worked example

Illustrative durations in weeks: auth 2, payments 3, fraud detection 4, analytics 2, client validation 1.

• Critical path: 2 + 3 + 4 = 9 weeks.
• Schedule on the critical path: auth weeks 1-2, payments weeks 3-5, fraud detection weeks 6-9.
• Analytics must be finished by the end of week 5, when fraud starts. It takes 2 weeks, so its latest start is week 4; starting in week 1 instead gives 4 - 1 = 3 weeks of slack.
• Client validation must be finished by the end of week 5, when payments goes live. It takes 1 week, so its latest start is week 5; starting in week 1 gives 5 - 1 = 4 weeks of slack.
• So weeks 1-2 run auth, analytics and (if capacity allows) validation in parallel.

Fastest learning. Fraud detection is last to build but the most uncertain: is there signal, how are labels obtained? Run a shallow offline analysis on historical data and a payments-provider sandbox spike in week 1, while building in dependency order.

The same method on two other problems

• Five ML workstreams, four weeks, one engineer. One engineer means no parallelism, so cut each to a thin slice and sequence by what unblocks and teaches most. For example (data pipeline, labeling, training, serving, monitoring): week 1 data plus a minimal labeled set and a baseline, week 2 a first model with offline evaluation, week 3 serving, week 4 hardening with monitoring reduced to logging inputs and predictions, full alerting deferred.
• Model quality versus a 100 ms latency budget. Latency is the time from request to response; budget it per stage, illustratively 20 for network and orchestration (the code that coordinates calls between services), 30 for feature fetch (looking up the input values the model needs), 35 for inference (the model computing its prediction), 15 for post-processing (total 100). Levers: architecture (a smaller model, or a distilled one trained to imitate a larger one), features (precompute expensive ones), caching (reuse answers for repeated inputs, at the cost of staleness), batching (grouping several requests into one model call: higher throughput, meaning more requests handled per second, but waiting to fill a batch eats budget). Measure the quality lost per millisecond saved (illustratively, a distilled model that cuts inference from 35 to 20 ms, 15 ms saved, while accuracy falls from 91% to 90%, costs about 0.07 points per ms; caching that saves 10 ms at no accuracy cost should be taken first) and judge at the 99th percentile, the slowest 1% of requests, not the average.

Trade-offs and pitfalls

• Dependency order can hide the riskiest item at the end; spikes fix that without changing build order.
• Parallel work needs people; with fewer people than branches, the critical path still goes first.
• A mocked upstream lets parallel work start, but integration surprises move to the end unless you schedule an early integration.
• What would flip the order: if fraud signal turns out to be weak in the spike, you reconsider the whole workstream before payments is built.