Process Analysis and Improvement Questions

How I’d read the pilot

A faster cycle time with more defects and escalations usually means the change improved speed by pushing work downstream or reducing controls too aggressively. I would not roll it out broadly yet, because the pilot may be trading one problem for another.

Decision

I’d ask whether the quality impact is within an acceptable threshold for the business. If defects or escalations cross a critical limit, I’d pause rollout and redesign the change. If the quality drop is small and clearly tied to a temporary learning curve, I might extend the pilot with safeguards.

How I’d redesign the experiment

• Define success criteria up front for both speed and quality.
• Measure a control group or pre/post baseline over the same volume mix.
• Segment results by case type, because the change may help simple cases and hurt complex ones.
• Add leading indicators like rework rate, defect severity, and escalation reason.
• Run the pilot long enough to account for ramp-up effects.

Worked example

Take a concrete instance of this pattern: a support-ticket triage pilot that auto-routes tickets to a fast lane using a complexity classifier. Baseline (the 3 weeks before the pilot, 1,000 tickets): average cycle time 4.2 days, 20 of 1,000 tickets escalated (2.0%). The team's pre-agreed pause threshold: if the overall escalation rate crosses 4%, pause rollout regardless of the cycle-time gain.

Pilot (3 weeks, 1,000 tickets through the new routing): average cycle time drops to 3.1 days, roughly a 26% improvement ((4.2-3.1)/4.2 = 0.262). But 60 of 1,000 tickets escalate (6.0%), crossing the 4% threshold.

Segmenting by case type, as the redesign calls for, shows the problem is not evenly spread. Complex cases (25% of volume, 250 tickets): escalations rose from 8 (3.2%) at baseline to 40 (16.0%) in the pilot, a five-fold jump. Simple cases (75% of volume, 750 tickets): escalations rose only from 12 (1.6%) to 20 (2.67%), a mild increase consistent with normal rollout variance. Check the total: (40+20)/(250+750) = 60/1,000 = 6.0%, matching the headline rate.

That segmentation is the actual decision, not the headline 6% number: it blends a complex-case failure (16.0%, well above threshold) with a simple-case result (2.67%, under threshold). The redesigned experiment therefore doesn't pause everything or roll out everything. It extends the pilot for simple-case routing only, reverts complex cases to the old manual triage while the classifier's complex-case accuracy is investigated, and adds escalation-reason coding as a leading indicator so the next read distinguishes complexity misclassification from a genuine quality regression instead of averaging the two into one number.

Goal

I’d want a conclusion I can trust: either the process is genuinely better end to end, or the apparent gain was created by shifting cost and risk elsewhere.

How I’d build the map

I would start by splitting onboarding into customer segments, because “the same process” usually hides branches. Then I’d collect actual cases from sales, operations, and finance and build a swimlane view showing each handoff, decision point, and exception.

Evidence over assumptions

• Review completed onboarding files, emails, and system timestamps.
• Shadow a few live cases from start to finish.
• Compare simple vs. complex customers to see where the path changes.
• Maintain an assumptions log for anything not directly observed.

I’d pay close attention to where information changes hands, where approval is required, and where work pauses waiting for clarification. Those are often the real sources of delay.

A worked finding that disproves “one path” (illustrative)

Splitting by segment and pulling real timestamps is what actually surfaces the branches. For a small-business self-serve deal, the sales-to-operations handoff, sales marking the deal closed-won to operations starting provisioning, shows an average gap of about 4 hours in the CRM timestamps. For an enterprise deal with custom contract terms, the same handoff shows an average gap of 6 business days, because legal review and a custom statement of work sit in between that “it’s basically the same process” never mentions. Comparing a sample of 20 recent enterprise closes against 20 recent self-serve closes this way is what disproves the team’s claim: the self-serve path has one handoff and one owner, the enterprise path has three additional handoffs (legal, finance for custom billing terms, and a dedicated onboarding manager) that never show up if you only interview the team lead about “the” onboarding process.

Validation

Once the draft map is ready, I’d walk it back to the people who do the work and ask them to challenge it with recent examples. If the team says it is all one path, the data usually reveals the exceptions, exactly the 4-hour-versus-6-day gap above. The goal is a current-state map that reflects how work actually moves, not how people think it should move.

Situation: In a previous role, our onboarding workflow had too many back-and-forths between sales, operations, and finance, so customers were waiting longer than expected to get fully activated, averaging around 12 business days from signed contract to full account activation.

Task: I was asked to map the process end to end, find the main sources of delay, and make the improvement stick.

Action: I shadowed 12 real cases over about three weeks, reviewed ticket timestamps, and built a swimlane map to separate work by team. That showed two main issues: unclear handoffs and repeated manual checks on the same customer data. Specifically, the map showed sales handing off requests in three different formats depending on which rep filled them out, which forced finance to manually reconcile missing fields on roughly a third of accounts before it could bill them, adding an estimated 2-3 days to the timeline on its own. I worked with each team to standardize the intake form and define ownership at each step. We also added a simple checklist before the handoff so incomplete requests were caught earlier.

Result: The process became much easier to follow, rework dropped, and stakeholders had a shared view of where delays were happening. Cycle time from signed contract to full activation fell from about 12 days to roughly 5 days over the following two months, and the share of accounts needing manual finance reconciliation dropped from about a third to under 10%. To check whether it stuck, I tracked cycle time, rework count, and exceptions weekly for the following quarter, not just the first couple of weeks, since a new process often looks good briefly before old habits creep back, and reviewed the trend in a recurring ops meeting.

This taught me that process improvement only lasts when the new workflow is simple enough to use and has a clear measurement loop.

How I’d diagnose it

I would separate the problem into three questions: where work is waiting, where work is actually being done, and where work is being handed off.

• First, I’d map the queue at each stage: backlog size, aging, and SLA breach points.
• Then I’d compare wait time to active processing time. Long waits with short touch time usually mean a bottleneck in queueing, not execution.
• I’d look for one stage with a growing queue while others report being “busy.” That often means the constraint is upstream or at a handoff.

Worked example. Say the flow runs Team A -> Team B -> Team C, with a 3-business-day SLA end to end. Pulling two weeks of timestamped status history for 50 cases shows: Team A's queue holds steady around 15 items with active handling time flat at 20 minutes/item; Team B's queue grows from 20 items to 80 items over the same two weeks while its active handling time stays flat at 15 minutes/item; Team C's queue holds steady around 10 items with handling time flat at 25 minutes/item. Team B says it is waiting on Team A, but Team A's own queue and handling time are both flat, so nothing in Team A's data supports that claim. What the data actually shows is Team B's own queue growing while Team B's per-case work hasn't gotten any slower, which means cases are arriving at Team B faster than Team B can clear them: a capacity constraint at Team B itself, not an upstream dependency. Of the 50 sampled cases, the ones that breach the 3-day SLA are the ones that sat in Team B's growing queue for 2 or more of those 3 days, confirming Team B, not Team A or Team C, as where the fix belongs.

Data I need first

The first data I’d ask for is timestamped status history for a sample of cases: created, started, paused, waiting on another team, completed. That tells me whether the delay is due to capacity, approvals, dependencies, or rework.

How I’d decide

If the queue is growing before a step, the bottleneck is likely there even if that team says it is waiting on someone else. If processing time is high but queue time is low, the issue is execution efficiency or quality. I’d use both views together so I don’t confuse busy teams with constrained systems.

Using the metrics together

I would track all four because each answers a different question.

• Cycle time shows how long active work takes once it starts.
• Lead time shows the customer’s total wait from request to completion.
• Throughput shows how much work the system finishes in a period.
• First-pass yield shows how often work is completed without rework.

Used together, they tell a much better story than any single metric. For example, if throughput is stable but lead time is rising, the problem is usually queueing or batching. If cycle time is low but first-pass yield is poor, the team may be moving quickly but creating rework. If throughput rises while lead time stays flat, demand may also be rising.

Worked example

Take a support-ticket queue with these two snapshots:

Cycle time, throughput, and first-pass yield are unchanged between the two weeks: agents are working just as fast, closing the same volume, and getting it right the first time just as often. The only number that moved is lead time, which nearly tripled from 6 to 14 hours. That gap between lead time and cycle time (12 hours of pure waiting in week 5, versus 4 hours in week 1) is the signature of a queueing problem: tickets are piling up before anyone starts them, most likely because intake volume briefly exceeded agent capacity or a triage step is batching tickets before routing them. The fix is adding intake capacity or shrinking the batch size, not asking agents to work faster or auditing their quality, since neither cycle time nor first-pass yield shows a problem.

Contrast that with a second scenario: cycle time and lead time both stay at 2 and 6 hours, throughput holds at 40 tickets/day, but first-pass yield drops from 90% to 70%. Nothing about speed changed, agents are still starting and finishing tickets on the same clock, but three in ten tickets now get reopened. That is a quality problem, not a queueing or capacity problem, and it points at inadequate review or unclear resolution criteria rather than staffing.

What a healthy process looks like

A healthy process has predictable throughput, short and stable cycle times, low waiting time, and high first-pass yield. I’d also watch the trend over time, because a one-week improvement can hide a broader capacity or quality problem.