Customer and User Obsession Questions

Direct answer

Treat popularity as evidence of a problem, not as a specification. I would find the need behind the request, test whether the UX direction (the intended overall design approach of the product) has good evidence behind it too, and then choose between building it as asked, building a version that fits the direction, solving the need another way, or declining. Whatever I decide, I tell users what we heard, what we chose and why.

How I decide

1. Understand the need: ask what people do with it and what breaks without it.
2. Size it: how many people, how often, and how severe? Compare broad usage data with the loudest forum voices.
3. Challenge both sides: does our direction rest on evidence, or on taste?
4. Weigh the cost of contradiction: more complexity, an inconsistent experience, and extra maintenance.
5. Prefer a small test: a prototype or limited release before committing.

Worked example (illustrative)

Users of a file app keep asking for the old list view, while the team is moving to a card grid. Asking what they do shows they scan file names and sort by date. Instead of bringing back a second layout, the team adds a compact grid density showing full names plus date sorting. That serves the need and keeps the direction.

How I tell users

• Say what we heard ("many of you asked for the list view").
• Say what we decided and why, in plain terms.
• Say what we did instead and what to expect.
• Say when we will revisit, and invite feedback.

Silence reads as being ignored, which costs more trust than a clear no.

Pitfalls

Building the feature because it is loud, or declining because it is inconvenient, without testing the need.

Direct answer

Here is the shape of a good answer, which you fill with your own real story if you have one. I found something that was good for customers but cost near-term revenue, sized the real cost (usually smaller than the headline), identified what we would watch to learn the long-term effect, and got agreement for a time-boxed test with a stop condition. I commit to the customer-good option when the loss is bounded and measurable, and the harm to customers is clear.

Weighing it

1. Size the true short-term cost, net of refunds, chargebacks (a customer disputes the charge with their bank, which costs the business the money plus a fee) and support cost, not the gross headline (the top-line revenue figure before refunds and other costs).
2. Name the customer harm: who pays or suffers today, and how.
3. Choose leading indicators (early measurements that move before the long-term result does) for the long-term benefit (not just "trust will improve"): refund rate, support contacts, repeat purchase at 90 days.
4. Propose a bounded test with finance agreeing the cost ceiling (the most revenue the team accepts losing during the test) beforehand.
5. Decide the stop rule and what would flip your call.

Worked example (illustrative numbers)

A checkout shows a pre-checked $5 add-on. Assume 10,000 orders a month. With the box pre-checked, 600 orders keep the add-on: gross add-on revenue is 600 times $5 = $3,000 a month. 150 of those 600 later request a refund, so 450 orders at $5 = $2,250 a month is kept. If the box becomes opt-in, assume 200 orders choose it with almost no refunds: 200 at $5 = $1,000 a month.

Headline loss quoted by the revenue team: $3,000 minus $1,000 = $2,000 a month. Net loss after refunds: $2,250 minus $1,000 = $1,250 a month, or $15,000 a year (12 times $1,250). The $15,000 is a real cost; it is also smaller than the headline, and it is bounded. Support cost shrinks it further. At an illustrative $4 of agent time per refund request, 150 requests cost $600 a month, so the net loss is $2,250 minus $600 minus $1,000 = $650 a month, about $7,800 a year (12 times $650). I would present the refund-only $15,000 as the conservative ceiling and the support-adjusted figure as the likely cost.

The case for it is not "customers will love us": 150 refund requests a month is a measurable harm, each one a customer who felt tricked. The proposal: A/B test (two versions shown to different random groups) opt-in for one month, with the cost ceiling agreed with finance, tracking refund rate, support contacts and 90-day repeat purchase. The stop rule: if repeat purchase does not improve or support contacts do not fall, revisit.

Convincing others

Present the net cost, not the gross; let finance own the ceiling; show the customer complaints in their own words; offer a smaller first step.

Pitfalls

• Claiming long-term benefit with no mechanism to check it. "Trust" is not a number.
• Hiding the cost. State the net cost first, as a ceiling.
• Moralising. Frame as risk and customer evidence, not as ethics against colleagues.

Direct answer

Start by naming the question you are answering. Usage data (telemetry, the events a product records automatically about what users click and do) tells you what people do and how many do it, and it is hard to fool. Interviews tell you why, what is missing, and what people cannot yet do. When they disagree I look for the reason, and let the one that fits the question win.

What each is good at

Example where interviews should win

Eight customers of a project-planning tool say they export plans to a spreadsheet to share status with finance. Usage data shows nothing in that area, because the product offers nothing there: nobody can use a feature that does not exist. The workaround is invisible to telemetry and visible only in conversation. Eight customers is a hint, not proof that most users share the need, so I would test it with more conversations and by checking how often plans are exported at all.

Example where usage data should win (illustrative)

Six of eight interviewees say they would use a weekly summary email. Meanwhile a similar email already exists, sent to 5,000 users, and in a typical week about 600 of them open it (12%). Six of eight is 75% saying they would use it, against 12% actually opening a close cousin. What people say they will do is a poor predictor of what they do, and the actual behaviour is recorded for thousands of users. I would not build the new feature on six enthusiastic statements. Usage also wins when loud power users want an advanced setting that most users never reach.

How I weigh a conflict

1. Check the samples: who was interviewed and whose usage is counted?
2. Compare stated against observed behaviour: is it a "would" against a "did"?
3. Check the measurement: is the metric the right proxy (measurable stand-in) for the need? An email open rate stands in for "found it useful" only roughly, so I would also look at clicks or replies before concluding.
4. Reconcile through a cheap test: a prototype, a fake-door button (a button for a not-yet-built feature that records clicks), or a handful of follow-up conversations with people the data identifies.
5. Decide by cost: if wrong is cheap to undo, follow the stronger signal and learn.

Pitfalls

Treating interviews as votes, treating usage as the truth about needs, and ignoring that a small segment (for example, administrators) can matter far more than its headcount in the data.

Direct answer

Yes. I pick a story where I moved from caring about the elegance of the solution to caring about what the person using it was trying to do, and I show three concrete changes: what I built, what I prioritized, and what I traded away. I tell it in STAR order (Situation and Task, Action, Result), the standard structure for behavioral interview stories.

Situation and Task

I was the engineer on an ordering app used by small cafe owners at the counter during rush hour. I took pride in a clever offline sync design (the app keeps taking orders without internet and merges them with the server when the connection returns). Then I spent a shift watching an owner use the app and saw that what she dreaded was a lost order when the connection dropped, not anything about how sync worked.

Action

• Engineering decisions: I replaced my clever conflict-resolution design (the rules for what happens when two edits to the same order collide) with a simpler one that always kept both versions of an order and flagged the clash in plain words for a human to resolve in seconds. Example: the counter phone edits an order to 2 lattes while offline, a helper's tablet edits the same order to 3 lattes while offline. The clever design silently kept one and lost the other. The simple one shows "Order 41: 2 lattes or 3 lattes? Tap the right one." It was less elegant and much easier to trust.
• Prioritization: I moved a planned refactor (restructuring code without changing what it does) behind a fix that made a half-saved order visible on screen, because that failure cost the owner money in real time.
• Trade-offs: I accepted slightly clunkier code and a delayed internal cleanup to protect the owner's rush hour, and I said so openly to the team.

Result

Support complaints about lost orders dropped from about a dozen a month to two or three (illustrative; use your own measured figures), and the owners I had watched started using offline mode with confidence. I also began asking "who is on the other end of this change?" in design reviews.

Pitfall

Avoid vague "I love our users" language. The proof is a decision you made differently, and a cost you knowingly paid.

Direct answer

Rank by estimated customer pain per unit of effort, and build pain from independent evidence: how many people hit the problem (reach), how badly (severity), and how much work the fix takes (effort). Loudness is controlled by counting distinct affected users from behaviour data instead of counting messages, and the ranking is checked against qualitative evidence from interviews, support transcripts and Net Promoter Score (NPS) comments. NPS is a survey asking customers how likely they are to recommend you on a 0 to 10 scale, with a free-text reason.

Step 1: deduplicate and tag

Cluster the 150 reports into distinct underlying problems, then tag each with the area, the symptom and a severity class: 3 = loses data or blocks a core task, 2 = slows a task but a workaround exists, 1 = annoyance or cosmetic.

Step 2: measure reach from behaviour, not volume

Use telemetry (usage data the product records automatically): error logs, funnel drop-offs (the share of users who quit at each step of a multi-step flow such as checkout), usage events. Tickets overcount persistent complainers and undercount quiet sufferers. One account sending nine emails is one account. Upvotes (votes from users on a public request) are volume too, so use them only as a hint, not as reach.

Step 3: score

priority = reach (% of active users) x severity (1 to 3) / effort (engineer-days)

Enter reach as a plain number of percentage points (12% goes in as 12, 0.4% as 0.4, never 0.12 or 0.004). Plain English: more people, hurt more, for less work, ranks higher. Treat scores in the same ballpark as ties and break them with judgement.

Step 4: let qualitative evidence change the ranking (illustrative numbers)

Tag each interview, NPS comment and transcript to the same issue clusters, with explicit rules: a transcript showing lost work or abandonment raises severity to 3; one showing a trivial workaround lowers it. Record the supporting quote so the change can be audited.

Before the transcripts, C outranks B. Reading five transcripts shows users redoing hours of work after failed exports, so B moves to severity 3 and overtakes C. The loudest report, A, is cosmetic (severity 1 on the Step 1 scale) and stays last.

Conflicting requests from strategic customers

Strategic customers are the accounts the company has named as most important to win or keep. Do not override the score silently. Reserve a small explicit quota, agreed in advance, for documented churn risk (a customer likely to cancel), and judge those items on evidence (renewal date, stated blocker). Everything else competes on the score.

Checking the ranking reflects real pain

• Spot-check with a few users from the top and the bottom of the list.
• After fixing the top items, confirm tickets and funnel drop-offs for those issues fell.
• Compare who is complaining with who is affected in the telemetry.

Variation: complaints about model quality (ML-powered features)

The same scoring still applies. For a feature powered by a machine learning model, add these checks in the first 24 hours: (1) did complaints start after a model, data or prompt release (a change to the instructions given to a language model)? (2) replay saved examples to tell a regression (something that used to work and now does not) from an old weakness; (3) compare today's inputs with the training period (data drift: real inputs shift away from what the model learned); (4) rule out the product around the model (truncated text, meaning input cut off before it reaches the model; timeouts; a stale cache, meaning an old saved answer served instead of a fresh one) before blaming the model; (5) check whether complaints cluster in a segment.

Example (illustrative): 40 tickets say "the assistant summaries are wrong since last week". Check 1 shows all 40 began within two days of a prompt release. Check 2 replays 50 saved inputs and finds the new prompt drops the last paragraph of long documents, so it is a regression. Check 4 shows the same truncation in a log. The fix is to roll back the prompt, and check 2 is what caught it.

Variation: player-value scoring (games)

Weight reach by player value, for example new players (whose retention, meaning whether they keep playing, decides the game's future) versus long-tenured or paying players, as a capped multiplier so it does not recreate a revenue-ranked skew.

Example (illustrative): a bug hits 3% of players, severity 2, effort 4, so the base score is 3 x 2 / 4 = 1.5. If most affected players are paying, apply a multiplier of 1.5 (the cap): 1.5 x 1.5 = 2.25. Uncapped, a spending-weighted multiplier of 5 would give 7.5 and let paying players outrank every other issue, which is the skew the cap prevents.