Technical Leadership and Influence Questions

Direct answer

I led the redesign of an image-processing pipeline that was the main bottleneck for our mobile app under load. I chose a hybrid design (object storage plus a queue plus a pool of containerized workers) over both a pure serverless rebuild and a bigger version of the existing monolith, because it hit the latency target without taking on the operational risk of either extreme, and I got the team behind it with a working prototype and real numbers instead of a design doc alone.

Worked example: the problem and the options on the table

Peak uploads were causing multi-second delays and timeouts, and the team needed thumbnails and metadata back within a couple of seconds for most uploads, including during seasonal traffic spikes many times normal volume. I considered four real options, not two:

1. Scale the existing approach harder: more virtual machines behind the monolith, autoscaled.
2. A queue plus a fleet of workers pulling tasks, running on virtual machines.
3. A fully serverless pipeline: cloud functions triggered directly by the upload, for everything.
4. A hybrid: object storage triggers a queue, and a pool of containerized workers (I used AWS Fargate, a serverless container runtime, so the team didn't have to manage the underlying machines) consumes from it.

I chose the hybrid. Cloud functions are billed and constrained per invocation and struggled with the larger images and native image-processing libraries the heavier tasks needed, with inconsistent startup latency on cold instances. A bigger monolith fleet fixed nothing structurally; it just moved the same bottleneck to more machines and kept the cost scaling linearly with peak, not average, load. Containerized workers behind a queue gave native binaries room to run, decoupled the upload path from processing so a slow burst degraded queue depth rather than user-facing latency, and let the team test and version the processing logic like normal application code instead of a pile of small functions.

flowchart LR
  U[Upload] --> S3["Object storage (S3)"]
  S3 --> Q["Queue (SQS)"]
  Q --> W["Worker pool (Fargate containers)"]
  W --> R["Processed thumbnail + metadata"]
  W -.retry on failure.-> DLQ["Dead-letter queue"]

How I got the team and organization behind it

A design doc alone wasn't going to settle it, because the disagreement was really about risk tolerance, not taste. I built a small prototype: a containerized worker doing the real image processing, wired to a test queue with synthetic burst traffic, and brought latency and cost numbers to the review rather than an opinion. I walked the plan past the people who had to live with the consequences: the product owner for the latency requirement, two backend engineers who would build and operate it, an SRE for on-call and cost implications, and a security reviewer for the data-access footprint of the new storage and queue paths. I rolled it out behind a flag starting with a small slice of traffic and watched tail latency, error rate, and cost against the baseline before expanding, so the decision was reversible if the prototype's numbers didn't hold in production.

Trade-offs I accepted, and where this pattern shows up elsewhere

I accepted more operational surface than a pure serverless design (a queue and a worker fleet to run and monitor) in exchange for predictable cost and headroom on heavy tasks, and I accepted more upfront build cost than "just add more machines" in exchange for a design that actually removed the bottleneck instead of relocating it.

The same reasoning shows up under different names across a lot of architecture decisions: centralized platform team vs. distributed feature-team ownership of infrastructure; microservice isolation vs. a service mesh handling cross-cutting concerns; REST vs. event sourcing (writes recorded as an append-only log) paired with CQRS, command query responsibility segregation, which splits the write model from a separately optimized read model, for an event-driven system; even a tooling pick like choosing one BI dashboard product over another can produce the exact same stakeholder-disagreement shape this story did. What stays constant is the method: name the real options (not two extremes), get numbers before opinions harden, and pick the option whose downside you can live with if you're wrong.

Trade-offs and pitfalls

• Presenting only two options. Real decisions usually have three or four live candidates; framing it as a binary hides the option that might actually be right.
• Skipping the prototype because the design "obviously" works. The prototype is what turns a design review into a data review; without it, disagreement stays at the level of opinion.
• Not planning the rollback path. A hybrid architecture like this is only safe to ship gradually if there's a real flag and a way to watch it fail small before it fails big.
• Letting the stakes change the story, not the reasoning. If the same decision had a customer commitment or a sales opportunity riding on the deadline, the criteria don't change, but the acceptable risk margin does; that's worth naming explicitly rather than quietly cutting corners on validation under pressure.

Direct answer

In the first ninety days, spend the first third mostly listening and mapping who actually owns what and what they're stuck on, use the middle third to ship one or two small, real wins that are visibly useful to the people you need trust from, and use the last third to put a concrete, evidence-backed proposal in front of the stakeholders whose buy-in matters most. Credibility at staff level is not built by announcing expertise; it's built by being visibly useful on something that mattered to someone else before you ask them to trust your judgment on something bigger.

Structured elaboration

• Days 0-30: map the actual decision-making landscape, not just the org chart. One-on-ones with the people whose work will intersect with yours, engineering peers, product, and the executives you'll eventually need buy-in from, asking open questions about what's blocking them rather than pitching your own agenda. The output is a short, honest written map: who owns what, what's actually broken, and what a real win would look like to each of them. This is also when you do a fast technical audit (architecture, known pain points, recent incidents) so your later recommendations are grounded in the system as it actually is, not as it was described to you.
• Days 30-60: ship something small, real, and visible. A fixed production bug, a canary deployment that measurably improved something the team already cared about, or a piece of documentation that unblocks a recurring question. The size matters less than that it's real and that the people who needed it notice. This is also when a lightweight recurring cadence (a short weekly sync, office hours) starts, so people have a reliable channel to bring you problems instead of only meeting you in a crisis.
• Days 60-90: bring the first real proposal. By now you have earned enough trust and gathered enough evidence to put forward something with actual stakes, an architecture recommendation, a resourcing ask, a process change, backed by what you learned in the first sixty days rather than by outside experience alone. Frame it with the trade-offs and the evidence, not just the recommendation, since the goal here is showing your reasoning is trustworthy, not just that you have opinions.
• Track the signals that credibility is actually building, rather than assuming it: are people bringing you problems before you ask, are your recommendations getting adopted without you having to push, are peers referencing something you shipped when explaining a decision to someone else. Absence of pushback is not the same as trust; look for people actively building on what you did.

This applies across a wide range of specific contexts: building technical credibility with VPs and directors without formal authority, concrete tactics for building credibility and trust with product, engineering, and executive stakeholders together, the signals that indicate a staff-level engineer or data scientist has genuinely built credibility with executives, twenty minutes with an executive during a sales engagement where you need to build credibility and alignment fast, an architectural recommendation that directly affected a sales opportunity, onboarding a newly-appointed executive to what you actually do in your first ninety days working with them, a playbook for influencing up when your recommendation conflicts with an executive's stated priorities, advising an executive on an underperforming KPI, persuading a skeptical product manager or executive to adopt a new architectural pattern, a C-level executive insisting on a roadmap direction that conflicts with the architects' recommendation, an influence strategy for removing a shared dependency that teams resist replacing, influencing product priorities by quantifying the business impact of a technical improvement, a very short pitch to convince a director you can lead a cross-team reliability initiative, convincing product or design leadership to adopt an API-first approach with no formal authority, and convincing leadership to prioritize API quality over feature velocity: the sequence (listen and map, ship something small and real, then bring a backed proposal) is the same regardless of who the specific audience is.

Worked example

Joining as a staff engineer on a team with an established product organization, the first thirty days were mostly one-on-ones with the product leads, the SRE team, and the two senior engineers who had been there longest, asking what was actually slowing them down day to day. A recurring theme surfaced: a flaky, poorly documented deployment pipeline that everyone worked around rather than fixed, because nobody had time to own it.

Rather than proposing a redesign immediately, the first real deliverable in weeks four through six was fixing the specific, recurring flakiness in that pipeline and writing a short runbook for the failure modes the team kept hitting. It was a small, unglamorous fix, not an architectural statement, but it was something three different engineers had personally been blocked by, and it was visible within a week because deploys got measurably more reliable for everyone using that pipeline.

By day seventy, with that credibility and a clearer picture of the actual pain points from the first month of listening, the proposal for the following quarter's architecture work landed differently than it would have on day one: engineers who had been skeptical of an unfamiliar new hire's opinion were now willing to review a real design doc, because the deployment fix had already demonstrated the judgment behind it was sound.

Trade-offs and pitfalls

• Trying to ship something impressive in the first thirty days, before you actually understand the system, risks shipping something that looks good and breaks something you didn't know depended on it; the listening phase is not optional even when it feels slow.
• Picking a "quick win" that nobody outside your own head actually cared about doesn't build credibility, it just consumes your first sixty days; validate that the win matters to the people whose trust you need before committing to it.
• Front-loading a big architectural proposal before you've earned any track record reads as an outsider telling insiders what to do, even when the technical reasoning is sound; sequence matters as much as substance here.
• Treating the ninety-day plan as a checklist to complete rather than a genuine effort to understand and help means the "wins" can feel performative to the people watching; the goal is real usefulness, not a self-narrated success story.

Direct answer

Do not frame it as centralize versus federate in the abstract; frame it as which failure mode the organization can tolerate less right now, duplicated effort and inconsistent controls, or a platform bottleneck that slows every team down. Get the groups to agree on that trade-off explicitly, propose a hybrid that keeps a thin, mandatory core centralized (security, shared contracts, a catalog) while leaving team-specific logic federated, and put governance guardrails in place so the "consistency" side does not silently become a queue.

Structured elaboration

• Separate what genuinely needs one source of truth from what does not. Identity, access controls, data contracts, and anything with real compliance exposure benefit from a single owner because inconsistency there creates real risk. Team-specific pipeline logic, feature engineering, and anything iterating fast usually does not need central ownership, and forcing it through one team just adds latency without reducing risk.
• Make the competing incentives explicit instead of pretending they don't exist. The team that wants centralization is usually optimizing for consistency, audit-ability, and lower duplicated cost. The team that wants federation is usually optimizing for speed and not being blocked by another team's backlog. Naming both incentives out loud, rather than letting one side argue "best practice" and the other argue "we need to move fast," is what makes the eventual guardrails feel earned rather than imposed.
• Propose a hybrid with a genuinely thin mandatory core: a managed core (identity and access management, standardized CI/CD templates, a shared schema or data-contract standard, policy-as-code checks that run automatically) plus federated ownership of everything downstream of that core. The core has to be thin enough that owning it does not become its own bottleneck, or you have just renamed the centralization problem.
• Put decision-quality guardrails on both sides: for the centralized core, explicit service-level agreements and a self-service path so teams are not waiting on a human for routine requests; for the federated side, mandatory data contracts between teams and automated compliance checks at the CI/CD gate, rather than a manual review that becomes the new bottleneck.
• Get agreement through a governance body with real teeth, not just a meeting. A recurring cross-functional review (platform lead, a rotating team representative, security) that owns exceptions and revisits the split as the organization changes gives both sides a channel that does not require re-litigating the whole model every time a new team joins.

Worked example

A growing data organization had eight product teams each building and maintaining their own ingestion pipelines into a shared warehouse. The platform team wanted to centralize ingestion entirely, citing three separate incidents where inconsistent schemas broke downstream dashboards. Product teams wanted to keep full ownership, citing a real case where waiting on the platform team's backlog delayed a launch by six weeks.

Rather than pick a side, I proposed splitting the decision along the line that actually mattered: schema and data-contract enforcement (where inconsistency was the demonstrated root cause of the incidents) moved to a thin, centrally-owned layer with automated validation at write time, while the ingestion logic itself, the code that actually pulls and transforms each team's data, stayed with each product team. The platform team got what it needed (consistency where inconsistency had caused real incidents) without becoming a bottleneck on ingestion code changes, and product teams kept the speed they needed on the part that was actually slowing them down before. We set up a quarterly review with one rotating product-team representative and the platform lead to handle exceptions and revisit which layer new capabilities belonged in as the platform matured.

The test that made the decision durable, and that both sides could check independently afterward, was simple: did the incident rate related to schema drift go to roughly zero (it did, because that was now enforced automatically rather than by convention), and did any team's shipping cadence slow down because of the centralized layer (it did not, because the mandatory core was thin enough to validate in CI without a human in the loop).

Trade-offs and pitfalls

• A "hybrid" proposed as a political compromise rather than derived from where inconsistency actually caused harm just recreates both failure modes at smaller scale; ground the split in evidence (the actual incidents, the actual delays), not in splitting the difference.
• A thin core has a strong pull to grow: every team's edge case becomes a request to add "just one more thing" to the mandatory layer, and left unchecked the core re-becomes the bottleneck it was designed to avoid. Someone has to own saying no to scope creep on the core.
• Federated ownership without enforced contracts degrades quietly; the incidents that justified centralizing schema enforcement in the first place don't announce themselves until something downstream breaks.
• A governance body without real authority to resolve disputes becomes theater; if the quarterly review can only make recommendations that either side can ignore, the underlying tension never actually resolves, it just gets a recurring meeting.

Direct answer

Treat it as a decision-design problem, not a debate to referee. Before any joint meeting, separate "who is right" from "how will we decide": name a single decision-maker (it can be you, facilitating), agree with both teams on what evidence would actually settle the question, and get that agreement BEFORE anyone sees how the criteria cut in their favor. Then run one or two time-boxed sessions, not an open-ended argument, and close with a written decision record both teams sign off on.

Structured elaboration

• Split the ownership question from the technical question. "Which team owns the canonical customer-data model" is really two decisions: who is accountable for maintaining the thing going forward, and what the thing technically looks like. Conflating them is why these disputes drag on: people defend the technical shape because they are actually worried about losing ownership, not because the shape itself is wrong.
• Pre-commit to decision criteria before scoring anything. Typical criteria: blast radius if the choice is wrong, migration cost for existing downstream consumers, which team's domain the concept most naturally sits in, and how reversible the choice is. Circulate the criteria list and get both sides to agree it is the right list before applying it to their options. That single step converts a status fight into a shared exercise, because nobody can argue the referee is biased once they picked the rules.
• Structure the session itself. Require a short written pre-read from each side: what they want, why, and the cost of NOT deciding. Open the session by inventorying where the two teams already agree (usually more than either side realizes) before touching the contested part; it resets the room from adversarial to collaborative.
• Use a time-boxed spike when the merits are genuinely close. If the argument is a real coin flip, e.g. batch versus streaming ingestion ownership, or which of two forecasting models to standardize on, run a short trial: both approaches against a shared test set or a two-week side-by-side, rather than arguing priors indefinitely.
• Close with a written decision record, not meeting notes: the decision, the criteria used, who owns follow-through, and a revisit date. A decision that exists only as memory gets re-litigated within a month.

This same mechanism generalizes across a wide range of ownership disputes: two engineering teams unable to agree on a canonical data model (including the specific case of two teams' conflicting canonical customer-data models), finance versus sales disagreeing on the canonical source for "revenue," engineering and product disagreeing on a metric's definition, two teams reconciling conflicting forecasting models used for strategic planning, multiple senior stakeholders converging on one set of model fairness metrics, a cross-team workshop aligning on AI model evaluation metrics, two product teams disagreeing on how to interpret an A/B test, a normalize-for-efficiency versus preserve-raw-fidelity disagreement, moderating a session to finalize SLOs when metrics are noisy and opinions conflict, a strong disagreement with a PM or engineering lead over an architecture decision, securing alignment between product, security, and operations on a ship-now-versus-delay trade-off, two business units with conflicting platform priorities, aligning engineering leads and product on a fast-but-lower-quality versus slower-but-more-maintainable path, a roadmap conflict where an engineering manager insists on one sequencing and product insists on another, a technical disagreement between research favoring complexity and product favoring earlier delivery, building consensus among five teams resistant to a new architecture pattern due to migration cost, a data platform charter that engineering and product VPs must both agree to, mediating a product-wants-speed versus compliance-wants-stability schema-change conflict, facilitating a cross-team choice between batch and streaming ingestion, and two teams sharing a datastore disagreeing over a zero-downtime schema migration. The domain changes; the mechanism (agreed criteria before facts, a time-boxed session, a written record) does not.

Worked example

Two teams shared ownership of a fraud-scoring pipeline and disagreed on whether the canonical scoring path should be the existing hourly batch model (cheaper, simpler to operate) or a new low-latency online model one team had already prototyped (better user experience, higher infrastructure cost). The debate had stalled for weeks because each side kept re-litigating the other's numbers.

I proposed, and both leads agreed to, five weighted criteria before either side presented anything: detection latency, precision and recall on high-risk traffic, incremental infra cost, operational complexity, and regulatory risk. We scored the two options against those criteria in a single 45-minute session, and the score gaps clustered on two axes: online scoring clearly won on latency and precision for high-risk traffic, batch clearly won on cost and operational simplicity. That made the real shape of the trade-off visible instead of an all-or-nothing fight: rather than pick one architecture for all traffic, we scoped a two-week trial of online scoring on just the highest-risk 15% of traffic, with an explicit metric (true positive rate at fixed false positive rate) and a rollback trigger (cost overrun or no measurable lift) agreed in advance. The trial gave a directional answer (online scoring lifted true positives on that segment; batch was operationally cheaper and good enough elsewhere), and we wrote up a decision record that kept batch as the default and online scoring for the high-risk bucket, with the infra lead as owner of the online path and a revisit at the next quarterly planning cycle.

The concrete number that mattered here was not a single precision figure but the trial's simple back-of-envelope framing before we ran it: if a 15% traffic slice costs $c$ extra per unit time to run online and catches even one additional true fraud case worth more than $c$, the trial pays for itself. Stating that threshold up front is what let both sides agree the trial was worth running, independent of what it would show.

Trade-offs and pitfalls

• A facilitator who is also a stakeholder looks partisan even when they are not; if you have a real stake in the outcome, say so explicitly and hand the criteria-scoring pen to someone else.
• Over-processing a low-stakes disagreement burns goodwill; reserve the full session-plus-decision-record treatment for genuinely contested, high-blast-radius calls like this one, not every disagreement between two teams.
• A criteria list built unilaterally by one side quietly becomes an ambush disguised as objectivity; both sides must ratify the list before it is used.
• Treating the written decision record as a formality rather than a real commitment is exactly why re-litigation happens later; route any re-litigation attempt to the named decision-maker rather than reopening the room from scratch.

Direct answer

A spike earns its cost by being falsifiable: define upfront the smallest slice that answers the specific unknown blocking the migration decision, and write down, before starting, what result would make you recommend not doing the migration, because a spike that can only confirm the plan you already wanted is not actually reducing risk.

Structured elaboration

• Scope to the riskiest unknown, not the easiest slice. Pick the piece of the migration where you genuinely do not know the answer, can this hit the latency target, is the data-consistency approach viable, does the team have the skill to operate this, rather than the piece that is easiest to demo. A spike that proves something you were already confident about wastes the time-box.
• Fix the time-box and the decision criteria before starting, not after seeing results. Pick a hard ceiling, days to a few weeks, scaled to how big the eventual migration is, and write the pass or fail thresholds down in advance: a target latency or error-rate ceiling, a maximum acceptable operational overhead, a rough cost bound. Deciding the bar after seeing the numbers is how spikes turn into confirmation exercises.
• Build a real vertical slice, not a mockup. It should exercise the actual mechanism you are worried about end to end, one real data path, one real deploy, one real failure injected, rather than a simplified version that avoids the hard part.
• Name the kill criteria explicitly, not just the success criteria. What specifically, seen in the spike, would make the full migration a bad idea: cost that scales worse than expected, a consistency problem that cannot be papered over, a team skill gap a few weeks will not close. If you cannot articulate a result that would stop you, the spike is not actually testing the decision.

Worked example

Say the question is whether to extract a single service, payments, out of a monolith as the first step of a larger migration. The riskiest unknown is not whether the code can be written, it is whether the new path can hit the existing latency budget and whether isolating payment failures actually prevents them from cascading into checkout, so the spike targets exactly that: one real payment endpoint, backed by a real partitioned data store, deployed with a working rollback path, and one real checkout flow routed through it. The time-box is four weeks, and the criteria are fixed beforehand: latency has to stay within a defined margin of the current monolith path, a deliberately injected failure in the new service has to stay isolated rather than take down checkout, and the team operating it has to be comfortable running it on-call. As a rough sanity check on scope before committing, if one service takes a small team four weeks and the eventual migration covers on the order of ten comparable services, a naive linear estimate puts the full effort around ten team-months, shown below, which is the number that actually gets the migration greenlit or shelved, not a vibe. If the injected-failure test shows cascading impact into checkout despite the isolation work, or the operating team clearly is not ready, that is a legitimate reason to pause the migration and fix the isolation approach first rather than proceed on schedule.

Trade-offs and pitfalls

A spike built in an artificially clean environment, synthetic load, no real production interference, can pass cleanly and still not predict production behavior, so the environment has to be as close to real as the time-box allows. The bigger pitfall is sunk-cost pressure: once a team has spent weeks and built something that mostly works, there is real pressure to call it a success even against the pre-agreed criteria, which is exactly why the criteria need to be fixed and written down before the spike starts, not negotiated afterward.