Direct answer
Qualitative research tells us why people struggle and what they need; it does not tell us how many people are affected or whether our specific solution fixes it. Engineering cost tells us what the answer will cost. So I would not choose between "believe the research" and "believe engineering". I would turn the recommendation into a testable claim, price the cheapest way to test it, and choose among proceed now, postpone, or test an alternative on that basis.
Process
- Restate the recommendation as a problem and a hypothesis (a testable guess about cause and fix). "Users cannot find X, which blocks Y. We believe moving it to Z fixes this." Separate the observed problem (strong, from interviews) from the proposed solution (a guess).
- Probe the evidence with the researcher. How many participants, which segments, how consistent, what did people do versus say? A handful of interviews is good at surfacing problems and weak at sizing them, so I look for a cheap quantitative check (support ticket counts, funnel drop-off at that step: the share of users who reach a step in a flow but do not continue past it) to size the problem.
- Unpack the engineering "expensive" with the engineers. Ask what drives the cost: new back-end work, a design-system change (editing the shared set of interface components that many screens reuse), migration of existing users? Ask for the smallest slice that tests the idea and a rough range, not a single number.
- Generate options at different costs. Typical ladder: a clickable prototype test (days), a partial change to the most-hit screen, a feature-flagged experiment (the change shown to a random share of users behind an on/off switch, compared with everyone else), the full redesign.
- Decide with explicit criteria: size of the problem, confidence the solution works, cost, reversibility, and what else the team would not build.
| Situation | Choice |
|---|
| Problem is large and well evidenced, a cheap fix covers most of it | Proceed now with the cheap version |
| Problem is real but sizing is unclear, full change is costly | Test alternatives first (prototype or experiment) |
| Problem is small or hits a minor segment, cost is high | Postpone and log it with the trigger that would revisit it |
Worked example (illustrative)
Eight interviews show people abandoning the setup flow at the account-linking step. Engineers estimate 10 engineer-weeks (one engineer working one week each) for the full redesign. A funnel check shows 1,000 sign-ups a month reach account linking and 600 complete it, so 400 (40%) abandon, and the problem is real. Instead of 10 weeks, we spend 1 week on a clickable prototype test with new users and 2 weeks building a simplified version behind a flag for an experiment, 3 weeks of team effort before the experiment can start. The calendar time to a read is longer: the 500-per-group sample needs 1,000 users, and only 1,000 sign-ups a month reach account linking, so the experiment runs for about a month after the build, roughly 7 calendar weeks in all (1 prototype + 2 build + about 4 running). Decide the thresholds first. Lift means the increase in completion rate compared with the control group. If the simplified version lifts completion by 10 points or more (60% to 70%, 100 extra completions a month), proceed to the full redesign or ship the simple version permanently. If lift is under 3 points, postpone and log it. Between 3 and 10 points, extend the test, because with 500 users per group the random noise is about 3 points (square root of 0.6 x 0.4 / 500 x 2 is about 0.031), so small differences cannot be trusted.
Pitfalls
- Treating "5 of 8 said it" as a measured rate. It is a clue, not a percentage.
- Letting engineering cost veto without asking for a smaller slice; or letting the research sponsor dismiss cost.
- Closing the loop poorly: tell the researcher and engineers what was decided and what evidence would reopen it.
