Role Responsibilities and Success Criteria Questions

Direct answer

Ask to see the artifacts a team actually uses to run itself, not just a description of its service-level agreements (SLAs, formal commitments about uptime, latency, or accuracy) and service-level objectives (SLOs, the internal targets a team holds itself to, which may be stricter than the external SLA). A team with real operational maturity can produce a dashboard and a runbook on request; a team that's only aspirationally following its targets will describe numbers it can't actually show you.

Dashboards to request

"Could you show me, or describe, the dashboard you look at to decide if the service is healthy right now?" A specific answer naming actual metrics, like 99th-percentile (p99, the value below which 99% of measurements fall) latency or error rate, is a good sign; a vague "we just check if things are broken" is a warning sign.

Runbooks to request

"What does the runbook for your most common alert look like?" Ask to see an actual on-call runbook, or at least hear a specific example of a step-by-step response to a real past incident.

On-call interviews

Ask to speak with, or ask about, whoever is currently on-call: "What's the last page you got, and what did resolving it involve?" "How often does the team get paged outside business hours, and how has that trended?"

Signals the targets are realistic, monitored, and owned

Realistic: the target is specific and has a rationale that traces to something outside the team, like "our client SLA is 300 milliseconds, so we hold an internal SLO of 250 to leave ourselves room to react before we breach the contract," not a round number nobody can justify. Listen specifically for that gap between the two numbers. A team whose internal objective is set exactly at its external commitment learns it has a problem at the same moment its customer does, which defeats the reason the objective and the agreement are two separate numbers in the first place, so hearing an SLO quoted as identical to the SLA is a sign the distinction is nominal. Monitored: the team can show you a live number against the target right now, not just describe it from memory. Owned: there's a named person or rotation responsible when the target is breached, and the team can describe what happens next, who gets paged and what the escalation path is, rather than "someone probably notices eventually."

Worked example

If I ask what the current p99 latency target is and where the team stands against it, and the interviewer immediately pulls up a number and says "target's 250 milliseconds, we've averaged 230 this month, breached it twice after a deploy last week and rolled both back," that's strong evidence of a real, owned SLO. If instead they say "I think we have a target somewhere, let me check with someone," the target exists on paper but isn't operationally alive.

Trade-offs and pitfalls

Pushing too hard on this in an early screening interview can feel adversarial, so calibrate depth to the interview stage and to whether you're talking to the hiring manager or a future teammate. The bigger pitfall is taking a confident verbal answer at face value without asking for the artifact behind it, confidence isn't the same as evidence.

Owning a component means acting like its long-term steward, not just its most recent contributor: you know its weak points, you're the first call when it breaks, and you're driving what happens to it next rather than waiting to be told.

What ownership looks like across five areas

• Code quality: maintain a testing bar for the component, actively drive down a known recurring source of bugs, and keep its architecture documented well enough that someone else could reason about a change without asking you first.
• Documentation: maintain a runbook (a step-by-step guide for operating and debugging the component) and an architecture decision record, or ADR (a short document capturing why a significant design choice was made), so knowledge doesn't live only in your head.
• Incident response: be the primary escalation point when it breaks, write a blameless postmortem (an incident review focused on fixing the system rather than blaming the person) after any real incident, and track whether the same class of incident recurs afterward.
• Roadmap planning: proactively flag technical debt or scaling risk before it becomes an emergency, and propose, not just execute, the next quarter's improvements to the component.
• Mentoring: bring at least one other contributor up to speed on the component so it isn't a bus-factor-of-one situation, where only one person understands it well enough to keep it running.

Success metrics to track the impact

A downward trend in repeat incidents for that specific component over time, how long it takes to onboard a second contributor to real changes, PR review turnaround time within the component, and a qualitative signal like fewer escalations bypassing you straight to your manager.

Worked example

Owning a checkout service in a given quarter: write the runbook that didn't exist, add an alert for the exact failure mode that caused the last outage, mentor a teammate through their first on-call shift on the service, and get a scaling proposal you authored onto next quarter's roadmap instead of waiting for someone else to notice the risk.

Trade-offs and pitfalls

Ownership without documentation or mentoring creates a single point of failure (SPOF): if you're the only one who understands the component, you haven't reduced organizational risk, you've just centralized it in yourself. There's also a trade-off against over-scoping ownership into empire-building; a component is still one part of a larger system, and treating every adjacent decision as yours to control slows the team down rather than protecting it.

Before I join a new team, I want answers in four areas: what I'll be measured on, what data and tooling I'll inherit, how the team actually operates day to day, and what's already on fire. Below are twelve questions: ten across those four areas, plus two on team structure, each with why it matters.

Metrics and KPIs (key performance indicators, the specific numbers used to judge whether the work is succeeding)

1. "What are the two or three metrics you'll actually judge my first six months against?" Aligns my effort with what's measured, rather than what merely looks good.
2. "Is there an existing dashboard or report showing the current baseline for those metrics?" Grounds any target I commit to in reality instead of a guess.
3. "Which of those metrics are actually inside my control, and which can I only influence?" A metric I am judged on but cannot move without three other teams agreeing is a target I should renegotiate now, not at the review.

Data and tooling

4. "What's the primary stack, and where does data quality or reliability break down today?" Surfaces known landmines before I inherit them silently.
5. "Which tools are the actual source of truth versus legacy systems people still half-use?" Avoids wasting early weeks mastering a tool the team is already abandoning.

Process and rhythms

6. "What's the meeting cadence, standups, planning, one-on-ones, and who actually attends?" Sets realistic expectations for calendar load and where decisions get made.
7. "How do decisions get made when the team disagrees, and who has the final call?" Clarifies decision rights (who is actually authorized to decide) before I need that answer under pressure.

Immediate problems

8. "What's the one thing that's been broken or blocked the longest?" Reveals the real priority behind the polished job description.
9. "What did the person previously in this role do well, and where did they struggle?" Sets a realistic bar and flags landmines without me having to rediscover them the hard way.
10. "What has been tried on that problem already, and why did it not stick?" Stops me from proposing in week two the exact fix that failed last year for reasons nobody wrote down.

Team and reporting

11. "Who do I report to day-to-day versus who formally evaluates my performance?" These can differ on matrixed teams (a structure where you have both a functional manager and a separate day-to-day project or product lead, each with real influence over your work), and the gap matters.
12. "How many people are on the team, and how is work currently split among them?" Shapes whether I'm stepping into an ownership role or a support role.

Trade-offs and a closing practice

Ask the hiring manager and a team lead separately where you can; their answers to "what's broken" often diverge, and the gap is informative. Don't ask questions the job posting already answered, and prioritize the ones most likely to change your decision or your first 90 days. A strong habit once you land the role: turn the answers into a short written role-expectations document you both sign off on, so priorities don't quietly drift a month later.

Direct answer

Before the call, do the homework a genuinely curious, well-prepared candidate would do: read what the team has published publicly, read what others have said about working there, and bring specific, current signals to the conversation instead of questions you could have answered yourself with 30 minutes of research.

Public sources to consult

The company's engineering blog, since its topics and frequency reveal what the team is proud of or currently building; public repositories if any exist, for commit cadence and recent activity; the job posting itself, read closely for named tools and responsibilities rather than just the title; review sites like Glassdoor or Blind for team-specific commentary, treated as directional hypotheses rather than fact; and the product itself, used firsthand for rough edges or recent changes.

Harder-to-reach sources

LinkedIn profiles of current team members, where tenure patterns (a burst of very recent joiners, or a lot of very long tenure) both tell you something different; any mutual connections who might give an informal read; and the interviewer's own public talks or writing, if they have any.

Signals to look for

Commit cadence and recency on public repositories, to judge whether something is actively maintained or legacy; recurring themes in open issues, which hint at known pain points; incident history if publicly documented, like a status page's past record; product metrics you can observe externally, like app store ratings and review themes; the pattern of recent job postings, since a burst of hiring for one specialty suggests a recent gap or new initiative; and engineering blog posts, since a post about a migration tells you exactly what's currently being rebuilt.

How to prioritize what to bring up

Favor things that are recent (shows fresh research, not a five-year-old post), specific to this team rather than generic to the company, and genuinely uncertain to you. Save your best, most specific question for something public information alone couldn't resolve, that's what the live conversation is actually for, rather than asking something the job posting already answered.

Worked example

If the engineering blog has a post from two months ago about migrating to a new queueing system, and the job posting separately mentions "own reliability of the payments pipeline," I'd connect the two and ask something like "how has the queueing migration affected the reliability work this role owns, is that mostly finished or still in progress." That shows I read both artifacts and connected them, rather than asking a generic "what's a typical day like" question.

Trade-offs and pitfalls

Over-researching to the point of asking overly narrow or gotcha-sounding questions can come across as trying to catch the team out rather than being genuinely curious. The other pitfall is treating review-site commentary as fully reliable, it's useful for hypotheses to verify live, not conclusions to state as fact in the interview.

Five 90-day metrics should be concrete, cheap to measure with data you already have or can get quickly, and tied to something the team actually cares about, not vanity numbers. I use the SMART framework (Specific, Measurable, Achievable, Relevant, Time-bound) as the filter for whether a proposed metric is real or just an aspiration. Below is the set I would commit to, each with why it matters, the data source and method, and a target range. The ranges are opening positions stated on the assumption of an average existing baseline; I would confirm or renegotiate each one in week two or three against the team's real numbers, and I say that out loud rather than pretending day-one precision I do not have.

1. Shipped, reviewed deliverables tied to the team's top objective. Why it matters: it proves I can navigate the codebase, the review culture, and the release process, which is the precondition for everything else. Source and method: count merged changes in version control, filtered to the objective's tracking label, excluding trivial changes. Target: at least two substantive changes merged by day 45 and six to eight by day 90, and every one of them reviewed by someone other than my onboarding buddy.

2. One validated impact result, with its unit cost. Why it matters: output that nobody can attribute is indistinguishable from busywork. Source and method: an A/B test (a controlled comparison against a held-out group that does not get the change) read from the experimentation platform, paired with the serving or query cost per unit pulled from the cloud billing export. Target: one experiment designed, shipped and read out inside 90 days, with the result reported honestly whether it won or lost, and unit cost held at or below the agreed ceiling rather than traded away for the win.

3. A reliability improvement on something a downstream consumer depends on. Why it matters: the pipeline or service running is not the point, the consumer trusting it is. Source and method: the scheduler or monitoring tool's run history for failure and late-delivery rate over a rolling 30 days, confirmed with the two or three teams that consume the output. Target: cut the failure-plus-late rate on one named dependency by roughly half against its trailing 30-day baseline, and get that confirmed by a consumer in their own words, not just on the dashboard.

4. Finished work with a quality bar attached. Why it matters: counting work in progress is how a quarter disappears with nothing landed. Source and method: the team's tracker, counting only items that reached production and carry a documented accuracy, adoption, or acceptance check. Source of the quality number is whatever the team already records; if it records nothing, establishing that record is itself part of the metric. Target: two to three finished items by day 90, none of them still labelled in review.

5. Adoption of one thing I built, by a named group. Why it matters: a dashboard or tool nobody opens is an output metric pretending to be an impact metric. Source and method: the tool's own usage log for weekly active viewers in the target group, plus a short qualitative check with three of those users on whether it changed a decision. Target: a majority of the named target group using it weekly by day 90, and at least one concrete decision someone can point to. I would avoid leaning on a Net Promoter Score here: it is a loyalty measure built from a single likelihood-to-recommend question scored zero to ten, it needs far more respondents than a ten-person stakeholder group to be stable, and weekly active use plus three honest conversations tells you more at this size.

How the same five slots look in different roles

The slots are deliberately generic because they travel. Slot 1 is a Software Engineer's "did specific, verifiable work land." Slot 2 is the AI Engineer flavor: impact plus inference cost per prediction, not accuracy alone. Slot 3 is the Data Engineer's consumer-benefit framing. Slot 4 is the Data Scientist's way of counting finished models or analyses rather than work in progress. Slot 5 is the BI Analyst's dashboard-adoption framing. A Solutions Architect's version blends slots 3 and 5: pick a specific customer or internal team consuming your architecture recommendations, and track whether that team actually adopted the recommended design, not just whether you delivered a document, confirmed directly with them.

Trade-offs and pitfalls

Committing to a hard numeric target before you have seen the team's real baseline is a common mistake, which is why every range above is explicitly an opening position to be reset in week two or three. The opposite mistake is worse in an interview: refusing to name any number at all reads as hedging, so state the range and state what would move it. A subtler pitfall is picking five metrics that are all about your own output (things you did) with none about downstream impact (things that changed because of what you did); slots 1 and 4 are output, slots 2, 3 and 5 are impact, and a strong set always has at least one of each.