Customer and User Obsession Questions

Direct answer

When you never meet customers, obsession becomes a habit of asking "who is affected, and how will they experience this?" before choosing, and of using the closest available evidence about them: support tickets, error logs, product telemetry (usage data the product records automatically), and the people who do talk to customers. The test is behaviour, not belief. A skeptical colleague should be able to point at specific choices you made differently.

What a skeptical colleague would actually see

• A user-impact line in your design notes and reviews. Who is affected, what do they see when it fails, what do they do next? A real line looks like: "Impact: about 2% of checkouts (card payments over $500) hit this path; the user sees a spinner for 30 seconds, then a generic error, and may retry and be charged twice."
• Error and failure behaviour written for the person hit by it. The message says what happened, whether their data or money is safe, and what to do next.
• Metrics tied to a customer outcome. You alert on "checkouts failing" or "reports arriving late to users", not only on server load.
• Instrumentation chosen to answer customer questions. Instrumentation means the logging and metrics you add to code so you can see what it did. You log enough (with an identifier support can look up) that a ticket can be resolved without asking the customer to reproduce it.
• Regular proxy contact. A proxy is a stand-in source: someone or something that sees customers so you do not have to. You read a sample of recent tickets, sit in on a support or sales call now and then, and ask the people who do speak to customers what surprised them.

Worked example (a backend engineer)

A payment call times out. The default behaviour returns "Error 500". The obsessed version does three things differently:

1. The message tells the user whether they were charged and that retrying is safe or not.
2. The log line carries the order identifier, so support can answer "was I charged?" in one lookup.
3. The alert fires on the failed-checkout rate rather than on CPU, so the team learns about customer pain first.

Nothing here needed a customer conversation. It needed the habit of imagining the person on the other end and checking tickets to confirm.

The same habit in other seats

• Network engineer, during an incident: decide the order of fixes by which customers are cut off (for example the region carrying the most checkouts), and write the status update in terms of what customers cannot do.
• Mobile developer: treat battery drain and crashes as customer-facing features, and check crash rate on older phones, not just your own.
• Machine learning engineer: read a sample of real model outputs and complaints, not only the offline accuracy score.
• Data engineer: choose schemas and keys so support can trace one customer's records, and keep the fields analysts need to answer "who was affected".
• Data analyst: build the dashboard for the business intelligence (BI) reader who will use it, with the customer-outcome metric first and plain labels.

Trade-offs and pitfalls

• Proxies mislead. Tickets over-represent people who bother to write in, and telemetry shows what users did, not why. Arrange some direct exposure even if it is only one call a quarter.
• Do not claim to speak for customers from inside your head; cite the ticket, the log or the person who talked to them.
• If your work never changes because of customer evidence, you are doing the same job as before with a nicer label.

Direct answer

Pick an improvement where you can draw a line from your technical change to something a customer felt: faster pages, fewer failed payments, fewer crashes. Show the before-and-after measurement from the customer's side, not just internal numbers like CPU usage, and be honest about other factors.

Structure

1. Problem as customers experienced it. A complaint, a ticket tag or an abandonment point.
2. Change. One or two sentences, in plain terms.
3. Proof it reached the customer. Use a customer-side measure and a fair comparison:
   • A real-user measurement (timing taken from actual customer sessions, for example a small script that reads the browser's built-in performance timing when the checkout form becomes usable and sends it, with the device type, to analytics) rather than only a test machine.
   • A staged rollout (release to a small share of users first) so the old and new versions can be compared in the same week. The 10% who get the new code are the rollout group; the other 90% keep the old code and act as the control group, the comparison group that lives through the same weeks and events but does not get your change.
   • A customer signal such as tickets, abandonment or retries.
4. Caveats. What else changed in that period.

Worked example (illustrative numbers)

A frontend developer notices checkout feels slow on mid-range phones. They defer loading a large recommendations script until after the payment form appears. Rollout goes to 10% of users first. The metric is the 95th percentile (p95) page-ready time: the time within which 95 out of 100 visits finish, so it shows the slow tail that customers complain about, not the average.

In the 10% phase, p95 page-ready time was 3.9 s for the rollout group against 5.7 s for the control group in the same two weeks (illustrative), which is 1.8 s or about 32% faster. Customer-side signal: support tickets tagged "slow checkout" per 1,000 active users. Suppose that was 18 per 1,000 before and 11 per 1,000 after, comparing four-week windows. That is 7 fewer per 1,000, about a 39% reduction (7 divided by 18). The honest line to add: "a marketing campaign ended in the same window, so I also compared the 10% rollout group against the control group in the same weeks, and the gap held." The control comparison is the p95 gap above: the campaign reached both groups, so it cannot by itself explain a gap between them. This holds only if users were assigned to the rollout group at random (for example by a hash of the user ID), so say that in the story. Even then it supports 'contributed to', not 'proved': the ticket drop (18 to 11 per 1,000) is a before and after comparison that the campaign ending can still distort, so also report tickets per 1,000 for the rollout group against the control group over the same weeks.

Pitfalls

• Internal metric as proof. "Server time dropped" does not show a customer noticed.
• Averages hide the tail. Report the tail or the share of affected customers.
• No comparison group. Before-and-after alone cannot separate your change from the calendar.
• Overclaiming. Say "contributed to" unless you had a clean control.

Direct answer

I would build a small weekly habit of looking directly at real user behavior and words, using the cheapest channels, and I would track a decision log so I can see whether that evidence changes what the team builds. Closeness only counts if it moves a decision.

A week-to-week routine (about 75 minutes weekly, plus a 45-minute call every other week)

Check your company's privacy rules before watching recordings or joining calls.

How to tell whether it changes decisions

Keep a one-line decision log: decision, the user evidence behind it, and whether the evidence changed the plan. Each quarter, count the decisions where user evidence changed the scope, the order or the answer. If that count stays at zero for a quarter, the habit is a ritual.

Worked example (illustrative)

A data engineer owns a pipeline that feeds a dashboard. Ticket reading shows several users saying "the numbers are old at 9am". Shadowing a call shows managers opening the dashboard for a morning meeting before the refresh finishes. The team's plan was to make the job 20% faster. The evidence changed it: move the refresh earlier and show a visible "last updated" time. That one decision goes in the log as evidence-changed.

Pitfalls and cost

• Sampling bias: your sample leaves out some kinds of users, so it misleads you. Loud tickets are not typical, so sample randomly.
• Replays show what, not why. Pair them with tickets or a call.
• Privacy: mask personal data and follow policy.
• Time cost: under two hours a week on average, so protect it on the calendar or it vanishes.
• Ritual drift: reading without a log produces warm feelings and no change.

Direct answer

Retiring a feature is also called sunsetting it. I decide using four checks: who uses it and how much it matters to them, what it costs to keep, whether the job it does can be done another way, and whether the removal can be staged and reversed. I handle the people it hurts with early notice, a migration path and personal contact for the most affected, because a retirement is a customer-experience event, not just a cleanup.

1. Evidence before the decision (thresholds set first)

Agree thresholds before looking at the data: for example, fewer than 5% of monthly active users touching the feature in 90 days, and no top-revenue account depending on it. The 5% is a team choice to be agreed, not a universal rule. Then check:

• Reach and depth: how many use it, how often, for how long.
• Who the loud minority are: revenue share, strategic accounts, renewal dates. Loud is not the same as important, and quiet is not the same as unimportant.
• Job to be done (the progress a customer is trying to make, in their circumstances, when they use it): what progress the feature serves and whether another route exists.
• Cost: maintenance, security patches, support, and what the team cannot build while carrying it.

2. Handling the people it hurts

1. Announce early with a dated timeline, the deprecation window (the period between announcement and removal during which the feature still works), for example three months. Sample wording: "On 1 March we will retire the CSV export. Until then it works as today. Scheduled file delivery already does the same job; here is a two-minute guide, and we will move your existing schedules for you if you reply to this message."
2. Offer a migration path: alternative feature, export, or script.
3. Contact the highest-impact accounts personally.
4. Keep a grace period (extra time after the deadline) or an exception process for real blockers.
5. Track usage during the sunset to find who still depends on it.

Staging and reversing the removal

The fourth check decides how the retirement is rolled out. Do not delete in one step. First hide the feature from new accounts while existing users keep it. Then, if usage tracking shows no one blocked, switch it off for existing users behind a feature flag (a switch that turns it off without a new release) while the code and data stay in place for the whole deprecation window, so the flag can be flipped back if a real blocker appears. Delete the code only after the window ends and migration tracking shows nobody still depends on it.

Worked example (illustrative numbers)

40,000 monthly active users; 1,200 used the feature in 90 days, which is 3% (1,200 divided by 40,000). Under the 5% threshold. Check: 150 of those users belong to 9 accounts on the top plan, and two renew next quarter. The decision: the second rule ('no top-revenue account depending on it') is not yet met, so retirement is conditional. Hold the announcement until the account team has spoken with the 9 accounts. If they can move to the documented alternative, retire as planned with those accounts on the personal-contact list and a data export. If any of them depends on the feature in a way the alternative cannot cover, either extend their grace period or keep a limited version for them, and decide with the two renewals next quarter in view.

A heavily used feature: removing an export to save cost

Suppose the same 40,000 monthly active users, but 14,000 used a spreadsheet export in 90 days, which is 35% (14,000 divided by 40,000). That is far above the 5% threshold, so the usage test fails. Retirement is then only viable with a replacement: scheduled file delivery or an API route, a migration guide, a long deprecation window, and tracking of who has not migrated. Cost saving alone is a weak reason if customers lose their workflow. If keeping the export costs roughly one engineer-week a month (illustrative), that is small beside 14,000 people losing a routine, so I would keep it unless the replacement lets them move with almost no effort.

Pitfalls

• Deciding from usage alone, without checking who the users are.
• Short notice: it turns a fixable change into churn.
• Retiring without a measure of success: track migration completion and cancellations.

Direct answer

Tell a story where evidence about customers (not a loud opinion) made you move or delay something on the roadmap (the team's ordered plan of what to build and roughly when), and where you won people over by showing the evidence, the options and the cost of each, instead of announcing a verdict. Always name what you gave up by changing the plan.

Story shape

• Situation: the planned item, who wanted it, and the commitment attached.
• What changed your mind: at least two kinds of evidence that agree (triangulation means checking a claim against independent sources). Typical mix: customer calls, product usage data, support ticket themes.
• Bringing others along:
  1. Share the raw evidence early, including what does not fit your conclusion.
  2. Present two or three options with cost of delay (what waiting costs us) and cost of being wrong.
  3. Talk to the engineering lead and the sales or customer-facing lead (the person who owns the relationship with those customers) one-on-one before the group meeting.
  4. Agree a checkpoint: "If the new item does not move X by date Y, we revisit."
• Result and reflection.

Worked example (illustrative skeleton)

A technical product manager has "scheduled data exports" planned for next quarter because several large customers asked for it. Eleven discovery calls (customer conversations to learn about their problem and what they do today, not to sell) show the real job: customers copy numbers into a finance spreadsheet every Monday to reconcile figures. Usage data shows a "download CSV" button pressed by the same accounts on almost the same weekday pattern, and tickets repeat "numbers differ from our finance report".

The conclusion: exports automate a workaround; the root problem is the figures not matching finance's definitions. The PM proposes delaying exports by one quarter and building a reconciliation view first. For the engineering lead, the argument is smaller scope. In the one-on-one: "I want to defer scheduled exports by a quarter. A reconciliation view removes the cause instead of automating the workaround, and I think it is a smaller build. Here are the 11 call notes and the Monday CSV pattern. What am I missing?" For the account team, a promise to tell the three requesting customers directly, with a date: "I will call each of them by Friday, explain what we heard, give a date for the first reconciliation view, and offer a manual weekly export until then."

Before the group meeting the PM also lays out the options with cost of delay and cost of being wrong, and shows the evidence that does not fit. Option A: build scheduled exports as planned. Cost of delay: none to the three requesting customers. Cost of being wrong: the Monday spreadsheet work and the mismatched numbers remain, and exports may be built for a workaround. Option B: reconciliation view first (recommended). Cost of delay: exports slip one quarter and the three requesters wait. Cost of being wrong: if the real need turns out to be scheduling, a quarter is lost, which the manual weekly export partly covers. Option C: a small slice of both. Cost of delay: both arrive later and the team splits focus. Cost of being wrong: a half-built version of each. The evidence that does not fit: two of the eleven customers only wanted to email a file to a colleague, which exports would serve, so the PM states that openly instead of leaving it out.

The checkpoint metric (the number you agree to watch to judge the decision) is reconciliation-related support tickets and weekly CSV downloads from those accounts, reviewed six weeks after the first reconciliation view ships (the date promised to the three requesting customers), not six weeks from today. Illustrative numbers: tickets at 40 a month should fall to 24 or fewer (a 40% drop), and CSV downloads at 120 a week should fall to 60 or fewer. If neither moves, we revisit and build exports.

Trade-offs and pitfalls

• Do not overreact to one account or one call. A change of plan needs convergent evidence.
• Delay has a price: promised customers, engineering morale, planning credibility. Name it and plan the communication.
• Avoid "I told them they were wrong". The goal is a shared view of the evidence.
• Data analyst or engineer angle: you can be the person who brings the evidence (the pattern in the usage logs), not only the PM who decides.