Product and Engineering Collaboration Questions

Direct answer

I would not negotiate the 30% until the number means something. First I establish what the metric is and what it measures today, then I agree what success means (the exact definition, the date, and what counts as a win), and only then I check whether 30% is realistic and, if it is not, trade it for intermediate deliverables the PM can show leadership.

Structured approach

1. Pin the metric definition (day 1). Ask: relative or absolute? "30% better" on an 18% rate means 23.4% (relative) or 48% (30 percentage points). These differ by a factor of two, so the first conversation settles it in writing. Also fix the numerator (the count of successes), the denominator (the count of everyone eligible), the population and the time window.
2. Establish the baseline (week 1). Pull at least 4 to 6 weeks of history, not one number. Record the average and the week-to-week spread, because the spread is the noise any improvement has to rise above. If the data does not exist, say so and instrument it (add tracking so the events are recorded); the first deliverable becomes "we can measure this".
3. Agree what success means. Target value, deadline, how it will be read (same query, same dashboard, same owner), and a guardrail metric (a second measure that must not get worse while you improve the first, for example support tickets or latency).
4. Test the target against the work. Ask engineering which levers move the metric (a lever is a specific change that could raise it, such as a shorter setup form or a reminder email) and what each plausibly delivers. Treat these as ranges, not promises. Compare the sum of the plausible levers to the gap.
5. Negotiate intermediate deliverables if the target is unrealistic. Offer dated checkpoints: baseline published, first lever shipped and measured, a revised forecast. Leadership gets visible progress and an honest range; the team avoids a hidden miss.

Worked example (illustrative numbers)

Metric: share of new signups who finish setup in week 1. Six weeks of data (activated / signups): 700/4000, 760/4100, 690/3900, 740/4000, 810/4200, 720/3900. Weekly rates range from 17.5% to 19.3%. Pooled: 4,420 / 24,100 = 18.3%.

• Relative 30% target: 18.3% x 1.3 = 23.8%.
• Absolute 30 points: 48.3%, which is not a credible target for a setup funnel.
• The weekly spread is about 1.8 points (the weekly rates have a standard deviation of about 0.65 points), so a lift of one point in a single week cannot be distinguished from noise. Averaging three weeks cuts that noise to roughly 0.4 points, which is why the later reforecast quotes a three-week lift. Checkpoints should compare several weeks, not one.

Proposed plan: week 2 publish the baseline and agreed definition; end of month ship the highest-impact lever and read three weeks of results; reforecast. If the levers engineering names add up to roughly 2 to 3 points, I say plainly that 23.8% is a stretch for the quarter and propose a lower committed target with 23.8% as the stretch goal.

Written definition excerpt: "Setup completion rate = accounts that complete all 4 setup steps within 7 days of signup, divided by all accounts created that week (excluding internal test accounts). Source: the weekly activation query, read each Monday for the cohort created two Mondays earlier, because a signup from the last day of a week needs a further 7 days to complete setup, so the most recent cohort is always still immature and is never compared with the baseline. Guardrail: setup support tickets per 100 signups must not rise. Target: 23.8% (relative 30%) by 30 June, owner: PM."

Reforecast message to the PM after the first lever: "Baseline is 18.3%. The reminder email lifted week 1 completion to 20.1% over three weeks, about 1.8 points. Remaining levers add an estimated 2 to 3 points. Forecast for 30 June: 22 to 23%. 23.8% needs a further lever; here are two options and their costs."

Trade-offs and pitfalls

• Anchoring on the 30% before the baseline exists makes every later number a negotiation about face.
• Do not quietly redefine the metric to make the target reachable; changes to the definition are agreed with the PM and recorded.
• Pushing back with "that is impossible" loses the room. Bring the baseline, the noise band and the lever estimates instead.
• What would change my call: a lever with strong prior evidence (a past experiment on the same funnel) would justify keeping the full target.

Direct answer

I give engineers the problem, the evidence and the success measure, not just a task list, and I check understanding by asking them to say it back and to propose alternatives. The sign it is working: engineers challenge the plan with cheaper ways to reach the goal, and can explain what a feature is for without looking at the ticket.

What I actually do

1. A short problem brief instead of a solution ticket: who has the problem, what we saw (evidence), the outcome we want, how we will measure it, constraints, and what we are not doing.
2. Let engineers hear it first-hand: attend customer calls, read support tickets, watch a session recording.
3. Open sprint planning with the goal. (Sprint planning is the meeting where the team picks the work for the next one to two weeks.) Spend the first minutes stating the goal and the number it should move; every ticket links back to the brief.
4. Ask for a play-back at handoff (the moment a piece of work passes from the person who defined it to the engineer who will build it). In the play-back the engineer restates the problem and what they would do differently. This is where estimates become realistic, because they now know which parts matter.
5. Translate the business goal into technical acceptance criteria (the list of conditions the finished work must meet to count as done). Example: goal, "fewer customers abandon checkout after a payment error". Success metric: payment-error abandonment rate on the dashboard. Technical criteria: a repeated payment request cannot charge twice (idempotent: safe to repeat), failure messages are specific to the cause, and the payment step responds within an agreed time at the 95th percentile (the value that 95% of requests are faster than). With illustrative numbers: today 12 of every 100 customers who hit a payment error abandon checkout, the goal is 8 or fewer, and the agreed time is 2 seconds at the 95th percentile. Engineers can estimate and design against that.
6. Carry engineering concerns upward. Scalability, reliability and cost concerns go into the brief as stated constraints with a cost. I raise them in leadership reviews in business terms ("this approach cannot exceed X load without extra spend or rework"), so they are decisions, not complaints.

How I know it is working

• Questions in refinement (the regular meeting where upcoming work is discussed and clarified before it is scheduled) shift from "what do you want?" to "what are we trying to achieve?".
• Engineers propose simpler routes, and some proposals I accept.
• Fewer "that is not what I meant" reworks late in the sprint.
• A spot check: ask two or three engineers in review what the feature is for. A sample that small only shows whether something is badly wrong, so I combine it with the other signs.

Pitfalls

• A long brief nobody reads; keep to one page.
• Explaining the why once at kickoff and never again.
• Treating the why as persuasion: if engineers still disagree, there may be a real flaw to hear.

Direct answer

I would stop talking about "quality" and price the request: how often the shortcut is likely to cause a problem, how much each problem costs, and how fast we would find out. Then I would offer ways to launch sooner that keep the exposure small. A lower bar is a loan: the launch gets earlier, and someone pays interest later.

Structured elaboration

Translate each shortcut into a business effect

• Lower accuracy (accuracy is the share of results a model gets right): turn the percentage into a count of wrong results. A drop of one percentage point on 1,000,000 weekly predictions is 1,000,000 x 0.01 = 10,000 more wrong results every week. Then ask what a wrong result costs: a support contact, a refund, a lost customer.
• Lower test coverage (the share of the code that automated tests exercise): untested paths are where defects reach customers undetected, and they also make every later change slower and riskier, because nobody can tell what a change broke.

Ask three questions to price it: how likely is a problem, what does one cost, and how quickly would we notice? A problem we see in minutes costs far less than one customers report a week later.

Worked example (illustrative numbers)

Expected value means the average outcome if you could repeat the decision many times: probability times cost. Suppose launching one week earlier is worth about $20,000 of revenue pulled forward. Skipping tests on the payment-retry path (the code that retries a customer's payment after it fails) carries, say, a 25% chance of a defect (a bug) reaching customers, costing about $50,000 in refunds and engineering time. In practice you would estimate the 25% from history (for example, 1 of the last 4 launches that skipped tests on a money path had a customer-visible bug; four launches is a thin sample, so treat 25% as a placeholder to be checked, not a measurement) and the $50,000 from the last comparable incident (refunds issued plus engineer hours at their loaded rate).

• Expected cost of the shortcut: 0.25 x 50,000 = $12,500, against a $20,000 gain. That is close enough to be a real decision, not an obvious yes.
• Now offer an alternative: release to 10% of traffic first. If cost scales with exposure, the exposed loss is $5,000 and the expected cost is 0.25 x 5,000 = $1,250. The benefit scales with exposure too: 10% of traffic captures only about 10% of the $20,000 gain, 0.10 x 20,000 = $2,000 in that week, so the fair comparison is a $2,000 gain against a $1,250 expected cost. That is still positive but thin, so the staged rollout is a cheap way to learn whether the 25% is real, with the rest of the revenue arriving as exposure widens (for example to 50% and then 100% once a clean week has passed), not a way to keep the full $20,000 at a tenth of the risk.
  Expected value is a blunt tool because it hides rare large losses, so I use it to open the conversation, not to win it.

Alternatives I would offer

1. Narrow the launch, not the quality bar: ship the core flow first and defer the optional parts.
2. Staged rollout behind a feature flag (a switch that turns the feature on for some users and off for others), with a clear way to switch it off.
3. Spend the lower bar only on low-risk paths: relax checks where failure is cheap and reversible, and keep full testing on money, security and data.
4. A dated follow-up: if the shortcut happens, agree in writing what gets fixed, by when, and who owns it.

Trade-offs and pitfalls

• Not every shortcut is wrong; the goal is a conscious decision with a price on it.
• Refuse a lower bar when failure is irreversible, regulated or hard to detect.
• Avoid scary but vague warnings ("it might break"); numbers and a clear ask make the PM an ally.

Direct answer

Trust returns through fast, honest, specific communication followed by visible follow-through. I would speak first with facts and a time for the next update, stop the harm for the affected users, then fix the release process and report back on what changed. A regression is a feature or behaviour that worked before and broke after a release. Here it hit a subset of users, so one early job is to identify exactly who.

What I tell them now (first hours)

Product and customer-facing teams need five things: what is broken, who is affected (the cohort: the group of users sharing a trait, here the annual-plan accounts, with a way to check a given customer), what customers can do meanwhile, when the next update comes, and who owns it. Do not guess at a cause or a fix time you cannot defend. Example update:

"14:10. Since the 13:00 release, customers on the annual plan cannot export reports; about a quarter of accounts are affected, list attached. Cause not confirmed. Workaround: use the monthly view. We are preparing a rollback and will update at 15:00 whatever the status. Owner: the on-call engineering lead, reachable in the incident channel."

Reducing customer impact in the short term

1. Roll back (return to the previous release), or switch the change off with a feature flag (a configuration switch that turns one feature on or off without redeploying), if that is safe. Prefer the fastest reversible action.
2. If rollback is not possible, ship the narrowest fix, with a second engineer reviewing it.
3. Give support a macro (a ready-made reply) and the affected-account list for proactive outreach.
4. Repair damaged data for the affected users and confirm it.

What changes so it does not recur

Hold a blameless review (focused on how the process allowed it, not who caused it), and agree a small number of changes, each with an owner and a date (for example, "regression test: owned by the engineer who wrote the change, due Friday"):

• A regression test (an automated test that fails if this exact failure comes back) that reproduces this failure.
• Staged rollout: release to a small share of users first and watch error rates by cohort before widening. For example, 5% of accounts for one day, then 25% for one day, then 100%, moving on only if the error rate stays at or below the old release's rate. Comparing by cohort matters: a 1% error rate overall can hide 4% among annual-plan accounts if they are a quarter of traffic and the rest have none (0.25 x 4% = 1%).
• Automatic rollback criteria agreed beforehand.
• Release notes sent to product and support before release, listing what could affect customers.
• For a model or data change: evaluate performance on segments, not just the overall average, because an average can hide a subset getting worse. For example, a model with 94% accuracy for 90% of users and 64% for the other 10% averages 0.9 x 94 + 0.1 x 64 = 91%, which looks fine while one group is badly served.

Owners and dates for those changes (illustrative): regression test, the engineer who wrote the change, due Friday; staged rollout, the release engineer, live within two weeks; automatic rollback criteria, the on-call lead with product agreeing the threshold, due within two weeks; release notes to support, the product manager, from the next release; segment evaluation for model or data changes, the owner of that model, before its next release.

Rebuilding the relationship

Trust is rebuilt by keeping small commitments. Send the updates at the times you promised, share the review conclusions within a week, and report back two weeks later on whether each change was done. Invite product into the release-readiness check (the short pre-release review of risks, tests and rollback plan) so they see the safeguards and share ownership. A report-back message, two weeks later, might read: "Of the four actions from the review, three are done: regression test added, staged rollout live at 5%, 25%, 100%, release notes now go to support a day ahead. The automatic-rollback threshold is still open and due Friday. Affected accounts: all 14 outreach calls made, no open tickets."

Pitfalls

Over-promising ("this will never happen again"); hiding the affected cohort; blaming a person or team; adding heavy approvals that slow every release when a staged rollout would catch the problem more cheaply.

Direct answer

I would not answer yes or no. I would show leadership a capacity-based plan: commit firmly to one feature plus a thin slice of the second, present ranges instead of single-point dates, and ask them to decide which feature leads. The arithmetic shows two full features do not fit even in the best case, so the negotiation is about which one and how much of the other.

Structured elaboration

1. Assess feasibility with the people doing the work. Start from capacity, then ask the engineers for estimates as ranges. A worked example with an illustrative team of 6 engineers over a 12-week quarter:

• Gross capacity: 6 x 12 = 72 engineer-weeks.
• Subtract about 25% for on-call (engineers taking turns to respond to production problems), support, code review, leave and unplanned work: 72 x 0.75 = 54 plannable engineer-weeks (the weeks genuinely available for planned feature work).
• To check the 25%, look at the last two quarters. For example, if last quarter the team had 72 engineer-weeks and the tickets show 17 went on incidents, reviews, support and leave, then 17 / 72, about 24%, supports the assumption. If the real figure were 40%, the plan would shrink accordingly.
• Estimates from the engineers: feature A 30 to 42 (midpoint 36), feature B 30 to 42 (midpoint 36).
• Midpoint total is 72 against 54 plannable, so 54 / 36 = 1.5 features, which is the gap leadership senses. Even the low ends add to 30 + 30 = 60, still 6 over.

2. What I offer instead

I recommend option 1: it gives leadership one finished, high-impact outcome and a visible start on the second. Option 3 only works if 18 engineer-weeks of other commitments are named and dropped. Adding engineers mid-quarter does not close that gap quickly, because new people need time to ramp up and take attention from the existing team.

3. Present risk and expected impact on one page

• For each option: the impact leadership expects (in their own metric, with the assumption behind it stated), my confidence, and the biggest risk.
• A checkpoint at week 6, the midpoint of the quarter: compare real progress with the plan, and decide then whether the second feature's slice grows or stays.
• What we cut first if A runs toward 42: the extras in B's slice, so A's date holds.
• Leadership chooses which feature leads. Which has higher impact is a product judgement; my role is to say what each choice costs.

What would change my call

Narrower estimate ranges (after a short technical spike, an experiment limited to a fixed time, say three days, to remove an unknown) could move B's slice larger. A contractual deadline on one feature settles which leads. If leadership wants both in full, I ask which 18 engineer-weeks of other work stop.

Closing the loop

Send the plan in writing, tell the team and the owner of the feature that got the slice that the decision was made with the numbers, and report against the week-6 checkpoint whether or not it is good news.

Trade-offs and pitfalls

• A plan with no spare capacity looks efficient and fails on the first surprise.
• Padding estimates quietly erodes trust; show the buffer openly as its own line.
• Saying "we can try" commits the team without leadership having accepted the risk.