Lyft QA Engineer (Mid-Level) Interview Preparation Guide

Direct answer

A rule-based decide_automate() heuristic should combine a stability gate (do not automate something whose surface is still actively changing) with a payback gate (automate if either the test runs often enough on its own terms, or its manual cost is high enough that the one-time automation investment pays back quickly), rather than relying on a single threshold like frequency alone.

Structured elaboration

The function uses two gates in sequence:

1. Stability gate: if the underlying UI has not been stable for long enough, or the code changed too recently, automation is blocked regardless of how attractive the numbers look, since automating an unstable surface just produces a test that needs rewriting on roughly the same cadence it runs.
2. Payback gate: once stability passes, automate if EITHER the test is run often enough (a frequency threshold) OR the payback period, computed from the manual time saved per week against the one-time build cost, is short enough (a configurable number of weeks).

This two-path design matters because frequency alone misses a real case: a test run only once a week but taking three hours manually can still be an excellent automation candidate if it is cheap to build, while a test run twenty times a week but taking thirty seconds each may not be worth the build effort at all.

Worked example

def decide_automate(test_metadata):
    """
    Decide whether a manual test case is a good candidate for automation right now,
    using a simple, explainable rule-based heuristic.

    test_metadata keys (all numeric):
      runs_per_week                - how often the test is executed manually per week
      last_code_change_days        - days since the underlying feature's code last changed
      ui_stability_days            - days the UI/test surface has been structurally stable
      estimated_automation_hours   - one-time engineer-hours to build the automated test
      manual_time_per_run_minutes  - minutes a human takes to run the test once
    """
    MIN_RUNS_PER_WEEK = 3          # run at least 3x/week to be a frequency-driven candidate
    MIN_STABILITY_DAYS = 21        # underlying feature/UI unchanged for >= 3 weeks
    MIN_CODE_CHANGE_DAYS = 14      # code itself not touched in the last 2 weeks
    MAX_PAYBACK_WEEKS = 12         # automation must pay back its build cost within ~1 quarter

    runs_per_week = test_metadata["runs_per_week"]
    last_code_change_days = test_metadata["last_code_change_days"]
    ui_stability_days = test_metadata["ui_stability_days"]
    estimated_automation_hours = test_metadata["estimated_automation_hours"]
    manual_time_per_run_minutes = test_metadata["manual_time_per_run_minutes"]

    # Stability gate: do not automate something whose surface is still actively changing,
    # regardless of how attractive the ROI looks on paper -- it will just need rewriting.
    is_stable = (
        ui_stability_days >= MIN_STABILITY_DAYS
        and last_code_change_days >= MIN_CODE_CHANGE_DAYS
    )
    if not is_stable:
        return False

    # Payback gate: weekly manual cost in hours, and how many weeks of that cost it takes
    # to pay back the one-time automation build investment.
    weekly_manual_hours = (runs_per_week * manual_time_per_run_minutes) / 60.0
    if weekly_manual_hours <= 0:
        return False  # never run manually; nothing to save by automating
    payback_weeks = estimated_automation_hours / weekly_manual_hours

    frequency_ok = runs_per_week >= MIN_RUNS_PER_WEEK
    payback_ok = payback_weeks <= MAX_PAYBACK_WEEKS

    # Automate if EITHER it is run often enough on its own terms, OR its payback period
    # is short enough on pure ROI grounds, as long as the surface is stable either way.
    return frequency_ok or payback_ok

Executed against six cases, including boundary and adversarial ones, with a known expected outcome for each:

PASS: frequent + stable + cheap to build -> automate -> got True, expected True
PASS: rare but 3-hour manual run, cheap to automate -> payback fast -> automate -> got True, expected True
PASS: frequent but unstable surface -> stability gate blocks automation -> got False, expected False
PASS: rare + cheap manual run + expensive build -> payback too slow -> do not automate -> got False, expected False
PASS: exactly at frequency threshold with stability exactly at threshold -> automate (frequency path, despite slow payback) -> got True, expected True
PASS: zero manual runs -> nothing to save -> do not automate -> got False, expected False

ALL PASS

Case 2 is the one that most clearly demonstrates why a frequency-only heuristic would be wrong: a test run once a week, taking 180 minutes manually, with only 4 hours to automate, has a payback of 4 / (1*180/60) = 4 / 3 ≈ 1.33 weeks, comfortably under the 12-week threshold, so it correctly returns True even though it fails the raw frequency check on its own. Case 5 checks the boundary explicitly (values exactly at the threshold constants) to confirm the comparisons use >= consistently rather than accidentally excluding the boundary case.

Trade-offs and pitfalls

The thresholds (3 runs/week, 21 days of UI stability, 14 days since code change, 12-week payback ceiling) are named constants specifically so they are easy to tune without touching the logic; a team should calibrate them against their own actual automation build costs and release cadence rather than trusting these defaults blindly. The heuristic is deliberately simple and does not account for factors like business impact or risk severity; it answers "is this worth automating on cost grounds," not "is this important enough that it should be tested at all," which is a separate, upstream decision the heuristic assumes has already been made.

Situation & Goal
I would lead a quarter-long program to cut flake-induced CI retries by 80% (baseline: X% retries per day). Focus: quick wins first, sustainable process and tooling long-term.

Milestones (quarter)

• Week 1–2: Baseline & triage — run analytics, tag top 20 flaky tests, classify causes (timing, infra, data).
• Week 3–6: Quick fixes — stability guards, retries limited to infra issues, add timeouts, mock external services.
• Week 7–10: Medium-term — flaky test rewrite, parallelization fixes, stable test IDs, deterministic seeding.
• Week 11–12: Harden & measure — release rollouts, dashboards, postmortems.

Tooling & Test-suite changes

• Introduce flaky-test dashboard (Grafana/Datadog) fed by CI logs; auto-tag tests with flakiness score.
• Add deterministic test fixtures, network mocks (WireMock), and CI-level ephemeral environments.
• Enforce test isolation via sandboxing and clear setup/teardown patterns.
• Gate new tests: require flakiness score < threshold on feature branches.

Ownership & Governance

• Assign test owners for top-50 flaky tests; incorporate flakiness in PR reviews.
• Weekly governance meeting: triage board (QA lead, 2 devs, SRE).
• KPI: owners must reduce flakiness by 50% per month for assigned tests.

Measurement

• Metric: retries per 100 CI runs and % of builds failing due to flake.
• Daily dashboard, weekly executive report. Target: 80% reduction by week 12.

Rollback & Safety

• Deploy changes behind feature flags in CI; run canary pools of runners.
• If regression detected (>=10% increase in failures), auto-rollback to previous CI config and trigger incident runbook.
• Maintain golden branch to validate fixes before wide rollout.

Quick wins vs Long-term

• Quick: add timeouts, retry only infra, mock external dependencies, enforce strict teardown.
• Long-term: rewrite brittle tests, improve infra stability, flakiness prevention culture (owner accountability, CI gating).

I’d start by owning the baseline triage and immediate quick fixes, then shift to enabling teams with dashboards, ownership, and governance to sustain the 80% reduction.

Direct answer

I would build the dashboard around one question: "which tests cost us the most for the least value?" It has six widgets, each with exactly one derived number (KPI, key performance indicator), each mapped to a maintenance decision: fix, quarantine (temporarily stop a test from blocking builds while keeping it running), speed up, delete, or assign. Anything that does not change what someone does on Monday is left off.

The six widgets (built in a dashboard tool such as Grafana or Looker, which draw charts and tables from a database; each reads from one results table plus a tickets table)

Filters, drill-down and ownership

• Filters shared by all widgets: suite, owning team, branch (default main), and date range.
• Drill-down: clicking a row opens that test's last 20 runs, its failure signatures, and linked tickets.
• Owner assignment: each test path maps to a team (an owners table); unowned tests are widget 6's headline, and a row action creates a ticket assigned to the owner.

Worked prioritisation

Test A runs 100 times a week with a 20% flip rate, so it produces about 20 noisy results. Test B runs 4 times a week with a 50% flip rate, so it produces 2. Test A gets the sprint time even though B looks worse as a percentage, which is why the widget shows flip rate next to run count, not alone.

Quick numbers for widgets 3 to 6 (illustrative)

• 3, wasted CI time: 18 runs this week failed and then passed on rerun, and each run takes 12 minutes, so 18 x 12 = 216 minutes (3.6 hours) of CI time were spent on results that told us nothing.
• 4, slowest tests: the whole suite takes 3,000 seconds of test time and the 10 slowest tests take 900 seconds, so they hold 900 / 3,000 = 30% of it. Speeding up ten tests can cut nearly a third.
• 5, tests that earn their keep: Test C failed 5 times in 90 days and 3 of those failures were linked to real product-defect tickets, so it caught 3 real bugs. Test D ran 9,000 times, never caught a bug and fails only when its environment breaks: a candidate for review, not automatic deletion.
• 6, maintenance backlog: 37 of 410 tests have no owner (9%), and the oldest open maintenance ticket is 41 days old, so the sprint starts by assigning those 37 and clearing the 41-day ticket.

Deliberately left out

• Overall pass rate, total test count and coverage percentage (vanity numbers that do not rank maintenance work).
• Per-engineer leaderboards (blame, and invites gaming).
• Live run feeds (a monitoring screen, a different job).

Trade-offs and pitfalls

• Six widgets is a ceiling: more views mean the page stops being a decision tool.
• Widget 5 is the hardest to trust because ticket linking is manual, so show its coverage ("42 of 60 recent failures triaged") beside it, using your own counts.
• A test that has never failed may be extremely valuable; deleting on "no catches" alone is risky, so it only nominates candidates for review.

Direct answer

A workable leveling framework names a small set of competency dimensions, defines observable evidence for each level within each dimension rather than a single blended score, and is calibrated across managers with a shared evidence bar, not left to individual judgment. Where a discipline splits into technical and people-leadership paths, the framework should offer parallel individual contributor (IC) and management tracks rather than forcing everyone toward a single ladder.

Structured elaboration

Choose the dimensions. Distinct competencies that don't collapse into one another, scope and ownership, domain judgment, execution and delivery reliability, collaboration and influence, and further up the ladder, mentorship or people development. Five to seven is a common ceiling so a rater can hold them all in mind for one candidate.

Define evidence per level per dimension, not an adjective. A vague label like strong technical judgment isn't gradable. A concrete description of what a rater should be able to point to is:

Offer a dual-track structure. At the point where a discipline splits, describe both the individual contributor track and the management track explicitly, with a shared foundation up to that split and different evidence after it. The IC track keeps rewarding deep domain ownership, the management track shifts the evidence toward people outcomes, without treating either as a lesser or forced default.

Calibrate across managers. A rubric applied differently by different managers isn't a shared standard. Build in a norm-setting session before each cycle where managers score anonymized example write-ups and discuss disagreement, and keep a documented set of example evidence per level that managers can compare their own candidates against.

Keep the promotion bar distinct from the good-performance bar. Conflating the two is a common source of drift, where strong performers get promoted for consistency rather than demonstrated readiness for the next level.

Worked example

"When I sketched a rubric for my own discipline I started with five dimensions, scope and ownership, domain judgment, delivery reliability, collaboration and influence, and from a certain level up, mentorship. For each I wrote concrete evidence per level, distinguishing consistently delivers assigned work with occasional guidance from identifies and scopes new work independently, and others rely on that scoping. I built in the dual-track split at the point where the discipline typically forks, the individual contributor track kept the domain-judgment dimension weighted heavily, the management track replaced the mentorship dimension with a people-outcomes dimension covering retention, growth, and team health. To calibrate, I proposed a norm-setting session before each cycle where a handful of managers scored the same two anonymized write-ups independently and discussed any gap before applying the rubric to their own teams, so the same evidence wouldn't land a promotion on one team and a not yet on another."

Trade-offs & pitfalls

• Too many dimensions makes the rubric unusable in practice, raters default back to gut feel.
• Too few collapses distinct competencies together and hides real gaps, blending technical judgment and delivery reliability can let someone who's reliable but making poor trade-off calls slide through.
• Skipping calibration is the most costly gap. Without it, the same rubric produces different outcomes on different teams, which is precisely the fairness problem it's meant to solve.
• Forcing a single ladder onto a discipline that naturally splits pushes people toward management for the promotion rather than the fit, a bad outcome for both the person and the team they might end up managing.

Approach & prioritization criteria
I’d use risk-based selection: prioritize tests by business impact, frequency of use, recent/change-prone code, and defect history. Prefer end‑to‑end happy-paths for critical flows plus a few edge cases. Aim for high signal-to-noise to keep maintenance low.

Initial 20-test regression suite (by type & rationale)

• 6 Core user journeys (login, signup, password reset, create/edit/delete primary resource, checkout/submit) — cover highest business value and most-used flows.
• 4 Critical integrations (payment gateway, email delivery, third‑party auth, API backend) — failures here are high-severity.
• 3 Authorization/roles (access control, admin-only action, user-level restrictions) — prevent security/regression leaks.
• 3 Data integrity checks (create then read, update persistence, concurrent edit conflict) — catch DB/logic regressions.
• 2 UI smoke tests (landing page load, main navigation) — surface deploy-breakers quickly.
• 2 Regression of past high-priority bugs (reproduce bug fixes) — prevent recurrence.

Execution notes

• Automate happy paths first; keep flaky tests out.
• Tag tests by speed and stability; run fast suite on CI per PR, full suite nightly.
• Review metrics (failure rate, time) and iterate: replace low-signal tests with higher-value ones.

Direct answer

Make the mismatch visible the moment you see it, whether that is after the upstream team is already running behind or as soon as it surfaces during planning itself, and look first for a way to decouple your own delivery from their exact finish order, such as a stub, an adapter, or a feature flag, so you have room to negotiate re-sequencing or reduced scope instead of just waiting to see if the date slips.

Structured elaboration

Surface the mismatch immediately, not once it is a crisis

Whether you discover it because the other team is visibly behind, or because it becomes obvious during a shared planning session, name it out loud right away: here is what we committed to, here is what we now depend on, here is the gap.

Look for a decoupling option before assuming you have to slip

A mock interface, a stubbed API, or a feature flag lets your work continue against a placeholder while the real dependency finishes in parallel, with a defined swap-in point once it is ready.

Negotiate re-sequencing with a concrete ask, not just a complaint

Pointing out that another team is behind invites defensiveness. Proposing a specific way both teams can still hit their dates if two pieces are resequenced invites problem-solving instead.

Communicate consistently to everyone downstream of the decision

Use the same explanation each time: what changed, what the new plan is, and what happens if it changes again.

Set escalation triggers before you need them

Agree upfront on the specific checkpoint, a date or a milestone, at which, if the upstream work still is not ready, the issue escalates automatically to both leads, rather than waiting for the final deadline to find out.

Worked example

Base case: discovered after the upstream team is already behind. A team is building a feature on top of a platform capability, and the platform team is now behind schedule on it. Rather than waiting to see if the platform team catches up, the team builds a lightweight adapter against a mocked version of the interface, so its own work continues. They set an explicit go or no-go checkpoint a week before their real deadline: if the real dependency is not ready by then, they ship against the mock with a manual fallback, and swap in the real dependency once it lands.

Planning-time discovery variant. During a multi-team sprint-planning session, it becomes clear in the room that one team's planned start date for a shared integration depends on another team's work, which is not scheduled to finish until after the first team's own committed date, a mismatch nobody had caught before that meeting. The engineer facilitating the session, in this scenario a DevOps engineer coordinating the shared infrastructure both teams touch, flags the conflict on the spot and proposes re-sequencing right there: the first team starts against a stubbed interface while the second team's work continues in parallel, with the real dependency swapped in once ready. Right after the session, the facilitator sends a short written summary to both team leads and stakeholders using a repeatable communication template: what was found, what was agreed, and what happens if either date slips again. The summary also sets an explicit escalation trigger: if the second team's work is not ready by a named checkpoint date, it escalates automatically to both leads instead of surfacing again only at the final deadline.

Trade-offs and pitfalls

Building a decoupling layer, such as an adapter, a mock, or a flag, costs real engineering time that is wasted if the upstream team finishes on schedule after all. It is worth it when the downside of waiting and being wrong is worse than the cost of building it and not needing it, which is usually true for anything on a hard external deadline.

Escalating too early, before giving the upstream team a real chance to communicate a plan, burns trust and can look like an attempt to shift blame preemptively. Escalating too late removes any options besides slipping the date. Pre-agreed, specific escalation triggers tied to a date rather than a feeling are what keep this from being a judgment call made under pressure.

Direct answer

Selenium exposes the currently open alert/confirm/prompt via driver.switch_to.alert; read .text to get its message, call .accept() or .dismiss() to close it, and for a prompt specifically, call .send_keys(...) before .accept() to fill in the text field; to check for an alert's presence safely, wrap the access in a try/except for NoAlertPresentException rather than assuming one is always there.

Structured elaboration

driver.switch_to.alert is the single entry point for all three native dialog types (alert, confirm, prompt); Selenium does not distinguish between them at the API level; what differs is which methods make sense to call afterward. A plain alert() only supports .accept()/.dismiss()/.text. A confirm() behaves the same way; .accept() corresponds to clicking OK, .dismiss() to Cancel. A prompt() additionally supports .send_keys(text) before accepting, to fill the text field the way a user typing into it would.

Checking for an alert's presence without throwing matters because driver.switch_to.alert RAISES NoAlertPresentException if no dialog is currently open, rather than returning None; a test that conditionally expects a dialog (or wants to assert one did NOT appear) needs to catch that exception rather than assume the access always succeeds.

Worked example

from selenium.common.exceptions import NoAlertPresentException

def safe_alert_text(driver):
    try:
        return driver.switch_to.alert.text
    except NoAlertPresentException:
        return None

def handle_confirm_then_prompt(driver, prompt_text):
    confirm = driver.switch_to.alert
    confirm.accept()                 # dismiss the confirmation
    prompt = driver.switch_to.alert   # the NEXT dialog, now that the first is closed
    prompt.send_keys(prompt_text)
    prompt.accept()

Verified the safe-check function against a mocked driver in both states (an alert present, and none present):

class FakeAlert:
    def __init__(self, text):
        self.text = text

class FakeSwitchTo:
    def __init__(self, alert_present, text=None):
        self._alert_present = alert_present
        self._text = text

    @property
    def alert(self):
        if not self._alert_present:
            raise NoAlertPresentException()
        return FakeAlert(self._text)

class FakeDriver:
    def __init__(self, alert_present, text=None):
        self.switch_to = FakeSwitchTo(alert_present, text)

driver_with_alert = FakeDriver(alert_present=True, text="Are you sure?")
driver_without_alert = FakeDriver(alert_present=False)

print("safe_alert_text (alert present):", safe_alert_text(driver_with_alert))
print("safe_alert_text (no alert present):", safe_alert_text(driver_without_alert))

Running it:

safe_alert_text (alert present): Are you sure?
safe_alert_text (no alert present): None

Both branches behave as claimed: it returns the alert's text when one exists, and None (not an unhandled exception) when it does not.

Trade-offs and pitfalls

The most common mistake is calling driver.switch_to.alert and immediately chaining .text or .accept() without any exception handling, in a flow where the dialog might not always appear (a confirmation that only shows up under certain conditions); the moment that assumption is wrong, the test crashes with NoAlertPresentException instead of failing with a clear, diagnosable assertion about whether the dialog was expected. A second, more subtle pitfall is timing: a JS dialog can appear slightly after the action that triggers it (an async confirm), so calling switch_to.alert immediately after the triggering click can itself race the dialog's appearance; wrapping the access in an explicit WebDriverWait with the alert_is_present expected condition is the more robust version of this pattern for anything not perfectly synchronous.

Direct answer

The gap between someone productive in days and someone still struggling after months is usually explained by a handful of concrete factors, not raw talent: how much prior related experience carries over, how good the available material is, whether they have access to someone who already knows it, how fast their feedback loop is while learning, and how much of what they're doing is high-stakes enough to force caution. The fastest thing I can do for myself is identify which of those I'm weakest on and deliberately fix it, rather than just trying harder.

Structured elaboration

Worked example

Two engineers on a team picked up the same unfamiliar infrastructure tool around the same time. One had a colleague nearby who already knew it well and a sandbox environment to experiment in freely; the other had neither, and was mostly working directly against a shared environment where mistakes were visible and costly, which understandably made them cautious and slow. When I was in a similar position picking up something unfamiliar, I noticed I had neither advantage either, so rather than just working harder, I deliberately asked for a sandbox account to be set up so I could iterate quickly without the cost of a mistake, and asked a colleague who'd used the tool elsewhere for a short walkthrough of the two or three things that usually trip people up early. Both of those closed most of the gap: the sandbox gave me a fast, cheap feedback loop, and the short conversation gave me a shortcut past the mistakes that would otherwise have taken me weeks to discover on my own.

Trade-offs and pitfalls

The biggest trap is attributing the gap to talent or aptitude, which is both usually wrong and actively demotivating, since it points at nothing you can actually do anything about. A second trap is fixing only one factor when several are compounding, for instance getting a sandbox but never asking anyone for help, which leaves a slower path than fixing both. And simply not being willing to ask for the resource that would help, a better source, a person's time, a safe place to practice, out of a sense that you should be able to figure it out alone, is often the single biggest thing standing between the two outcomes.

Load testing

• Goal: Verify system meets expected concurrent user/session throughput and response-time SLOs under normal-to-peak traffic.
• Scenario: Simulate steady 5,000 concurrent shoppers browsing category pages, searching, and checking out during a seasonal promotion.
• Acceptance criterion: 95% of page requests complete within 800 ms and error rate ≤ 0.5% during a 1-hour steady-state run.

Stress testing

• Goal: Find breaking point and failure modes by pushing beyond capacity; validate graceful degradation and recovery.
• Scenario: Ramp concurrent users from 5k to 20k over 30 minutes to observe when checkout or DB connections fail.
• Acceptance criterion: System alerts trigger before total outage; failures are limited to non-critical services and mean time to recover (MTTR) ≤ 5 minutes after load drops.

Soak / Endurance testing

• Goal: Detect memory leaks, resource exhaustion, or performance degradation over long duration under nominal load.
• Scenario: Run 2,000 concurrent users performing mixed transactions continuously for 48 hours.
• Acceptance criterion: No >10% degradation in throughput or >30% increase in median response time compared to initial hour; no out-of-memory or resource leak incidents.

Spike testing

• Goal: Validate behavior under sudden, large traffic spikes and how autoscaling/queueing reacts.
• Scenario: Instant jump from 500 to 10,000 users for 5 minutes (flash sale) then back down.
• Acceptance criterion: Autoscaling adds instances and 90% of requests are served within 2× baseline latency within 3 minutes of the spike; error rate ≤ 2% during spike.

Each test includes monitoring (CPU, memory, DB connections, latency, error rates) and automated scripts (JMeter/Locust) plus post-run logs to identify root cause.

Direct answer: Filter the history to a sliding time window ending at the most recent run, weight each run's contribution to the failure rate by exponential decay based on its AGE within that window (not its position in the list), and return the ratio of decay-weighted failures to decay-weighted total, defaulting to a "not enough evidence" score for sparse windows rather than a misleading 0% or 100%.

Approach

The function sorts inputs by timestamp (so it's correct regardless of input order), restricts to the window [latest_ts - window_size, latest_ts], and computes a weighted failure ratio using weight = 0.5 ** (age_seconds / half_life_seconds), where age is measured relative to the latest run in the window, not relative to "now" (which matters if the history has a gap and the most recent run isn't from today).

def compute_flaky_score(history, window_size, half_life_seconds=None):
    """
    history: list of {"timestamp": datetime, "success": bool}, any order.
    window_size: timedelta, only runs within [latest_ts - window_size, latest_ts] count.
    half_life_seconds: decay half-life; defaults to window_size/2 if not given.
    Returns a float in [0, 1]. Sparse windows (< 2 runs) return 0.0 by design
    (see 'handling of sparse data' below).
    """
    if not history:
        return 0.0
    events = sorted(history, key=lambda e: e["timestamp"])
    latest_ts = events[-1]["timestamp"]
    window_start = latest_ts - window_size
    windowed = [e for e in events if e["timestamp"] >= window_start]
    if len(windowed) < 2:
        return 0.0
    hl = half_life_seconds or (window_size.total_seconds() / 2)
    weighted_fail = weighted_total = 0.0
    for e in windowed:
        age_seconds = (latest_ts - e["timestamp"]).total_seconds()
        w = 0.5 ** (age_seconds / hl)
        weighted_total += w
        if not e["success"]:
            weighted_fail += w
    return weighted_fail / weighted_total

Choice of decay factor: the half-life should be chosen relative to how often the test runs, not as a fixed constant, a test that runs 50 times a day needs a much SHORTER half-life (in wall-clock time) than one that runs once a day, to be equally responsive in terms of RUN COUNT. I tested this concretely: with the default half_life = window_size / 2 (15 days for a 30-day window), a burst of 4 failures concentrated in the most recent 4 of 20 daily runs scored only modestly above the raw rate (raw 20.0% vs decay-weighted 28.0%), because a 15-day half-life is too gentle to sharply distinguish "4 days ago" from "20 days ago." Shortening the half-life to 5 days made the SAME burst score 45.4%, more than double the raw rate, correctly emphasizing that the failures are concentrated recently. This is a real, executed finding, not a guess: the "recency-weighting" behavior the question asks for depends heavily on choosing a half-life meaningfully shorter than the window, not just applying "some" decay.

Handling of sparse data: with fewer than 2 runs in the window, the function returns 0.0 rather than either extreme (100% if the 1 available run failed, which would look alarmingly high off a single data point, or an undefined/error state). This is a deliberate, conservative choice: a single run is not enough evidence to compute a meaningful decay-weighted rate, and returning a low score prevents a brand-new or rarely-run test from triggering false-positive alarms purely from small-sample noise, consistent with the general "require sufficient sample size before acting" principle (the same reasoning that motivates using a confidence interval, not just a point estimate, for a binomial flaky-rate decision). A production version might instead return None explicitly to distinguish "confirmed low-risk" from "insufficient data," which the caller then handles differently (e.g., not scoring it at all yet); returning 0.0 was chosen here for a simple numeric contract, and this trade-off should be called out explicitly to whoever consumes the score.

How the score responds to recent bursts: as demonstrated above, a burst weighs disproportionately more than the same number of failures spread evenly across the window, BY DESIGN, and the magnitude of that effect is directly controlled by the half-life parameter, a shorter half-life makes the score more reactive to recent bursts (at the cost of being noisier, more sensitive to a single bad day), while a longer half-life smooths bursts out (at the cost of being slower to react to a genuinely worsening trend).

Complexity: O(n log n) for the sort, O(w) for the windowed weighted-sum pass where w is the number of runs within the window; O(n) space for the sorted copy.

Verification (executed this session, python3, after fixing two bugs): the code as originally written raises TypeError: unsupported operand type(s) for /: 'builtin_function_or_method' and 'int', because .total_seconds is called as a property instead of a method in two places (window_size.total_seconds and (latest_ts - e["timestamp"]).total_seconds both need ()). With that fixed, I ran four adversarial cases: (1) 4 failures spread evenly across a 20-run window (days 0, 5, 10, 15 of 20): raw rate 20.0%, decay score (default half-life=window/2=15 days) 18.2%, close to the raw rate as expected since these failures skew slightly toward the older half of the window; (2) the same 4-failure count concentrated as a burst in the most recent 4 of 20 runs: raw rate 20.0%, decay-weighted 28.0% at the default 15-day half-life, and 45.4% at a shortened 5-day half-life, confirming the half-life-dependent amplification described above; (3) a single-run window returning exactly 0.0; (4) the same evenly-spread input shuffled into random order returning an IDENTICAL score to the sorted input, confirming the function is correctly order-independent. All four passed.

Trade-offs & pitfalls: choosing half_life_seconds as an OPTIONAL parameter defaulting to window_size/2 is convenient but, as the executed test shows, that default is not aggressive enough to sharply detect a recent burst; a caller who wants burst-sensitivity needs to explicitly pass a shorter half-life rather than trusting the default. A second pitfall: computing age_seconds relative to latest_ts (the most recent run IN THE DATA) rather than the actual current wall-clock time means a test that hasn't run in weeks will still show its most recent (stale) run as "age zero," potentially masking that the data itself is stale; a caller should separately check how old latest_ts is relative to now before trusting the score at all.