Lyft Solutions Architect (Entry Level) Interview Preparation Guide

Direct answer

A centralized lock caps throughput at whatever one coordinator can serialize, so the fix is to stop needing a single point of serialization: use optimistic concurrency where conflicts are rare, partition ownership so most operations never contend at all, replace "acquire before act" with a monotonically increasing sequence that orders operations after the fact, or push conflict resolution into the application where domain knowledge can merge concurrent updates without blocking either writer.

Structured elaboration

Optimistic concurrency control. Instead of acquiring a lock before reading and writing, read a version (a row version number or timestamp), do the work locally, then write conditionally: "update only if the version is still what I read." If another writer got there first, the write is rejected and the operation retries. This preserves correctness (exactly one of the racing writers wins, the loser sees a definitive failure rather than silent overwrite) without ever blocking a reader or holding a lock across the work. It is a good fit when conflicts are the exception, not the rule; under high contention the retry rate itself becomes the bottleneck.

Partitioned ownership. Instead of one lock guarding the whole resource, split the resource (by key range, by tenant, by shard) so each partition has its own independent point of coordination, or no coordination need at all if each partition is owned by exactly one writer. Correctness comes from the partitioning key guaranteeing that concurrent operations on different partitions never touch the same data; cross-partition operations are the residual hard case and usually need one of the other three approaches or a higher-level transaction pattern.

Sequence-based approaches. Instead of gating access before the fact, assign every operation a monotonically increasing sequence number (from a counter, a log offset, or a timestamp with a tie-breaker) and let operations apply in sequence order, detecting and resolving out-of-order arrivals after the fact rather than blocking to prevent them. This is the pattern behind append-only logs and event sourcing: writers never wait for each other, and correctness comes from every reader processing the sequence in the same order.

Application-level conflict resolution. When two writers legitimately modify the same data concurrently and neither should simply lose, encode a merge rule in the application: last-write-wins with a well-defined tiebreak, a domain-specific merge (like summing two concurrent counter increments instead of picking one), or a conflict-free replicated data type whose merge function is mathematically guaranteed to converge regardless of arrival order. Correctness here means "any order of application produces the same final result," which is a stronger and harder-to-verify property than the other three approaches, but it is the only one of the four that never rejects or retries a write.

Worked example

Assume, as a planning input illustrating the shape of the trade-off rather than a measured fact: a single mutex-protected counter, serialized through one lock, sustains roughly 500 operations/sec before lock contention and context-switch overhead dominate. Partitioning it into 16 independent shard counters, each capable of the same per-shard throughput since they no longer share a lock, gives:

The cost of that 16x gain is that no single read sees an exact real-time global total; a caller who needs the true total sums all 16 shards, which is eventually consistent with in-flight writes rather than instantaneously exact. That is the general shape of the trade-off across all four techniques: throughput is bought by relaxing some property (blocking-free correctness for retries, or a single serialization point for a scan-and-sum read) that the centralized lock previously gave you for free.

Trade-offs & pitfalls

Optimistic concurrency degrades under high contention (retries compound instead of resolving), so it is the wrong default for a hot key with many concurrent writers, exactly the case a centralized lock was protecting. Partitioning shifts, rather than removes, the hard problem to any operation that spans partitions. Sequence-based approaches move complexity from "prevent the race" to "make every consumer correctly handle re-ordering and idempotent replay," which is a real engineering cost, not a free lunch. Application-level merge functions are the hardest to get right and the easiest to get subtly wrong (a merge rule that is not truly associative and commutative will produce different results depending on arrival order, defeating the whole point); they deserve the most scrutiny and the smallest blast radius when first deployed. The senior-level judgment call is matching contention pattern to technique, not defaulting to one everywhere.

Fault tolerance, high availability, and resilience are related but answer different questions. Fault tolerance means a failure is masked entirely: the system keeps producing correct output with no visible interruption. High availability (HA) means downtime is minimized and recovery is fast, but a brief, bounded interruption is expected and acceptable. Resilience is the broadest term: the system's ability to keep delivering acceptable service under any kind of stress, not just component failure, including load spikes, bad inputs, or a slow dependency, usually by degrading gracefully rather than failing outright.

How each shows up architecturally

Fault tolerance and HA are usually about infrastructure failing; resilience is about the system's response to any kind of stress, including ones where nothing has technically "failed" yet (a slow but technically-up dependency, for instance).

Effect on MTTR and MTBF

MTTR (mean time to repair/recover) and MTBF (mean time between failures) combine into steady-state availability:

Fault tolerance mainly extends effective MTBF from the user's point of view: individual component failures still happen at whatever rate they happen, but they don't count as user-visible failures because they're masked, so the failure interval a customer would notice grows. High availability mainly drives MTTR down: the goal isn't to prevent the primary from ever failing, it's to make detection and recovery fast and automatic. Resilience patterns move both numbers in the same direction from a different angle: a circuit breaker doesn't prevent a dependency from failing (MTBF of the dependency is unchanged) but it prevents that dependency's failure from becoming your incident at all, which is a third way to improve the user-visible number.

Worked example

Take a service with a component MTBF of 720 hours and an MTTR of 30 minutes (0.5 hours) once a failure is detected and recovered:

That converts to annual downtime using $525{,}600$ minutes per year (365 days $\times$ 24 hours $\times$ 60 minutes):

Now compare two improvements starting from that baseline, holding the other variable fixed:

• Cut MTTR to 5 minutes (better HA: faster automated failover) with MTBF unchanged at 720h: $A = 720/720.083 \approx 0.999884$, about 61.0 minutes/year of downtime, a ~6x reduction driven entirely by faster recovery.
• Double MTBF to 1440 hours (better fault tolerance: the failure that used to happen now gets masked half as often) with MTTR unchanged at 30 min: $A = 1440/1440.5 \approx 0.999653$, about 182.4 minutes/year, a 2x reduction.

Neither number is "the metrics." They're two independent levers on the same availability formula, and which one is cheaper to pull depends on the system: automating failover (MTTR) is often cheaper than adding redundant hardware paths everywhere (MTBF).

Trade-offs and pitfalls

The common mix-up in interviews is treating "high availability" as if it means "never goes down," which is what fault tolerance actually promises, and at a much higher engineering cost (consensus protocols, lockstep replication) than HA's health-check-and-failover pattern. A resilient system is not automatically fault-tolerant or highly available either: a service with excellent circuit breakers and graceful degradation for its dependencies can still have a single database with no HA story of its own. These three properties are complementary, not substitutes, and a system typically needs different amounts of each depending on the blast radius of the component: mask failures (fault tolerance) for the smallest, cheapest, most critical primitives; fail over fast (HA) for stateful tiers where full masking is expensive; and degrade gracefully (resilience) at the edges where "reduced functionality" beats "hard failure." The same reasoning applies outside a classic web-service stack too: a GPU training job gets fault tolerance from checkpointing plus redundant nodes (a crashed worker resumes from the last checkpoint instead of restarting the whole job), and a data pipeline gets resilience from feature-store (a system that serves precomputed inputs to a machine-learning model) fallback values or a stale-but-served cache when an upstream API is delayed rather than hard-failing the request.

Direct answer

I start from the assumption that the thing I pushed for is the prime suspect, not a bystander, and I root-cause it the same way I would anyone else's mistake, quickly and without defensiveness, because the instinct to protect a decision I championed is exactly the bias most likely to slow down finding the real cause. Once I have a real answer, I tell stakeholders before they find out on their own, and I choose between reconfiguring, slowing the rollout, and reversing based on evidence, not on how much I want to have been right.

Structured elaboration

• Treat the adoption decision itself as a hypothesis to test, not a given: actively look for evidence it caused the regression rather than looking for reasons it is innocent.
• Separate the possible root causes explicitly: was the tool's actual behavior different from what I understood, incomplete learning on my part, was an assumption about how it would behave simply wrong, or was the integration into the existing system the real problem, since the fix and the lesson differ for each.
• Get ahead of the disclosure: tell the people affected what is known as soon as it is known, rather than letting them discover the regression and connect it back to the decision themselves.
• Choose the remediation on evidence: reconfigure if the root cause is a fixable misconfiguration, slow or pause the rollout if the evidence is still incomplete, reverse if the tool genuinely cannot do what was needed.
• Afterward, name what evidence, available before the decision, would have caught this earlier, so the gap closes in the decision process itself, not just in this one instance.

Worked example

I had championed adopting a new caching layer to reduce database load, argued for it, and it was rolled out. Within a couple of weeks, a subset of reads started returning stale data intermittently, something nobody had seen before the change. I assumed from the start that the cache was the prime suspect rather than looking elsewhere first, and worked backward: I confirmed the caching layer's default invalidation behavior did not cover one specific write path used by a background job, meaning that path's writes were not invalidating the cache the way I had assumed. That was incomplete learning about the tool's actual invalidation model, not a wrong assumption about the business logic and not a pure integration bug. I told the team and the affected downstream stakeholder the same day the root cause was confirmed, including that it had been happening for roughly two weeks before it was caught, rather than only mentioning it once a fix was ready. Since the specific gap was fixable, adding the missing invalidation call, I chose to reconfigure and add monitoring on cache staleness rather than reverting the whole adoption, since the underlying idea was sound and the miss was narrow and now understood. Afterward, the concrete change was adding a checklist item to any future tool-adoption proposal: name every write path the new component touches and confirm invalidation or consistency behavior for each one, since that is precisely the evidence that would have caught this before rollout.

Trade-offs and pitfalls

• Defending the decision instead of investigating it as a suspect is the single most common way a champion misses or delays finding their own root cause.
• Delaying disclosure until a fix is ready, instead of disclosing as soon as the cause is known, damages trust more than the original mistake does.
• Reverting reflexively, without separating whether the cause was incomplete learning, a wrong assumption, or an integration bug, can throw away a genuinely good decision over a narrow, fixable gap, or conversely keep a genuinely bad decision alive by treating it as fixable when it is not.

Read-your-writes, monotonic reads, and monotonic writes are per-client session guarantees layered on top of a weaker (often eventually consistent) store, without paying for full linearizability across the whole system. Read-your-writes (RYW) promises a client that any read after its own write reflects that write. Monotonic reads promises a client's successive reads never go backwards in time. Monotonic writes promises a client's own writes are applied in the order it issued them. Each is enforced with a small piece of per-session state, not by coordinating the whole cluster.

What each guarantee promises, and how it breaks

Worked example: a monotonic-reads violation and its fix

Consider a likes counter on a post, replicated across replica X and replica Y with asynchronous replication.

Without the guarantee:

1. Client issues Read1, routed to replica X, which has applied all writes up to likes = 42. Client sees 42.
2. Client issues Read2 moments later. This time the request lands on replica Y, which has only applied writes up to likes = 40 (Y is lagging behind X). Client sees 40, a value older than what it already observed.

With the mechanism:
3. The client library remembers the highest value it has seen, last_seen_version = 42.
4. Read2 is sent with that version attached. Replica Y checks its own applied version (40) against the required version (42), sees it has not caught up, and either forwards the read to a replica that has (X, or any replica at version 42 or later) or holds the request briefly until its own replication catches up. The client never observes a regression.

Trade-offs and pitfalls

Sticky routing is the cheapest fix but weakens load balancing and complicates failover: if the pinned replica dies, the client loses its anchor, and if the guarantee is tied to a session cookie rather than the account, two devices logged into the same account (a phone and a laptop) are different sessions from the server's point of view and do not automatically share the guarantee with each other. Version tokens are more portable across devices and do not concentrate load onto one replica, but they add a token to every request and response, and the server still has to decide what a replica does when it cannot yet satisfy the token: wait, redirect, or refuse, which is itself a small latency-versus-freshness decision. Finally, these guarantees compose per client but do not add up to global consistency: a system can offer all three to every client individually and still allow two different clients to observe each other's writes in different orders. They make one user's own experience feel correct; they are not a substitute for linearizability when multiple clients must agree on a single order of events.

Direct answer

I would not sign a flat $2.5M for three years on day one. I would structure it as a staged commitment: a short pilot against written success criteria, a smaller firm commitment, milestone-based releases of the rest, and exit terms that let us leave without being trapped. The terms I rank highest are, in order: the pilot and its exit rights, payment tied to milestones, price caps and usage floors that limit what we can be forced to pay, service-level agreements (SLAs) with real remedies, and data portability. Together these make the vendor carry part of the risk.

Terms: committed-usage discount is a lower unit price in exchange for promising to buy a minimum volume. A usage floor is the minimum we must pay for. A pilot and a PoC are the same thing here: a short, limited trial before the main commitment. A tranche is one instalment of the total payment. Non-cancellable means we cannot cancel and must pay regardless. An anchor is the first, ambitious position that the final deal is pulled toward. A price cap limits how much prices can rise at renewal. A PoC (proof of concept) is a limited trial to test that the product works for our case. SLA means service-level agreement, with service credits (refunds) if missed. TCO means total cost of ownership (the full cost of buying and running the product over its life). I assume the business case already produced the three value scenarios below, and here I use only those numbers.

Step 1: Know our walk-away (the point at which we leave the deal) and their incentives

Before negotiating I work out my BATNA (best alternative to a negotiated agreement): the next-best vendor, building in-house, or staying with what we have. I also work out what the vendor wants: a headline contract value, committed revenue, a reference logo (a named customer they can show to other buyers), and a good quarter-end (closing deals before their sales period ends). Those are things I can trade (multi-year commitment, a public case study) in exchange for risk-sharing terms.

Step 2: The structure

1. Paid pilot (60 to 90 days) with written success criteria (for example: query latency, data from three source systems loaded, 20 named users active, accuracy checks agreed). If the vendor demands non-cancellable annual fees for a PoC, I push back: pilot fee capped, credited in full against the later commitment if we proceed, refundable or terminable if criteria are missed.
2. Staged commitment. Phase 1 firm, phases 2 and 3 released by milestones.
3. Milestone-based payments: each tranche paid only when its milestone is accepted in writing.
4. Price caps and floors: annual price increase capped (for example 5% a year); usage floor set at a level we are confident to reach (say 70% of forecast), with a rollover of unused volume; a true-up (a later adjustment for actual use) only upward at the agreed unit price. Worked example (the $2.5M is the total over the three-year term): forecast 100,000 units over the whole term at $25 is $2.5M, about 33,000 units a year; a 70% floor means we pay for at least 70,000 units over the term ($1.75M) even if we use only 50,000, which protects us from paying for the full forecast if adoption disappoints. With a rollover, the 20,000 unused units are credited to next year. If we use 120,000, the true-up charges the extra 20,000 at $25, with no penalty rate.
5. Performance SLAs: availability target, support response times, with service credits and a right to terminate for repeated misses.
6. Data portability and exit: our data remains ours, exportable in an open format (one that other software can read, not a vendor-only file type) at any time, with transition help for a fixed number of days at agreed rates, and a termination-for-convenience right (the right to end the contract without having to show a fault) after year 1 with a declining exit fee (a charge for leaving early that falls each year).
7. Outcome-linked incentives: part of the fee depends on a measured result we both define.
8. Vendor KPIs and compliance: key performance indicators (measures such as ticket resolution time) reviewed quarterly, with reporting duties, audit rights for security and data ownership, and milestones written into the contract, not into an email.

Step 3: Compare fixed, success-based and hybrid pricing by downside exposure

Assumptions (illustrative): a three-year, $2.5M proposal. Value we realise over three years under three scenarios: worst $1.0M, expected $4.0M, best $6.0M. "Net" means value realised minus what we pay (internal implementation cost left out).

Arithmetic: success-based worst is 35% x $1.0M = $0.35M, raised to the $0.5M floor; expected 35% x $4.0M = $1.4M; best 35% x $6.0M = $2.1M (under the $3.0M cap). Net = value minus payment: $1.0M - $0.5M = +$0.5M; $4.0M - $1.4M = $2.6M; $6.0M - $2.1M = $3.9M.

Reading it: fixed fee has the largest downside (we lose $1.5M if the product disappoints). Hybrid cuts the worst case from -$1.5M to -$0.5M and costs nothing in the expected and best cases. Pure success-based looks best for us on every line, which is why a vendor will usually refuse it or want a high floor and a clear measurement rule. My recommendation: ask for the hybrid and open with success-based as the anchor (the ambitious first position) to pull the final deal toward the hybrid. The three scenario values come from the pilot results, reference customers' reported results and our own adoption forecast; I would test them with the vendor's own case studies.

Step 4: Present best, expected and worst commercial scenarios to win concessions

I show the vendor three consumption scenarios for the commitment. "If we reach the best case, you earn $X and we expand; if worst, we still pay the floor." This helps win a committed-usage discount and better payment terms (net 60, meaning invoices are due 60 days after receipt, or milestone-based instead of annual in advance), because the vendor can see the revenue it gets in exchange.

Step 5: Terms that matter most to me, ranked

Items 1 to 5 protect us most; item 6 matters less and is the first to trade away.

1. Pilot with written success criteria and exit rights.
2. Milestone-based payment.
3. Usage floor and rollover, and price caps.
4. SLAs with service credits and termination rights.
5. Data ownership, portability and exit assistance.
6. Outcome-linked incentives and KPI reporting.

Pitfalls

Defining success loosely ("improved insight"); signing the pilot's annual fee as non-cancellable and then being locked in; a usage floor above realistic demand; forgetting data export costs; negotiating price while ignoring exit. What would change my ranking: a regulated workload would move data portability and security terms to the top.

Direct answer

I would separate a rule's identity from its versions: a pricing_rule row says what the rule is (type, region), and every edit creates a new immutable pricing_rule_version row with its parameters, schedule, author and approver. Versions are published together as a numbered rule set snapshot per region, and a one-row pointer says which snapshot is live, so rollback is a single pointer change and every price can name the exact snapshot it came from. Times are stored as local wall-clock time plus an IANA time zone (the standard zone names such as America/New_York), never as UTC offsets, so a "1 a.m. to 3 a.m." rule stays correct across daylight-saving changes. At runtime the engine does not query these tables: it loads the live snapshot into memory, grouped by rule type in precedence order, and evaluates a quote in a single pass over a handful of rules.

The five requirements and how they combine

The precedence order is: emergency override, then time-of-day multiplier on top of live surge, then region caps, then promotion. Caps run after multipliers so nothing can push a price above them; promotions run last because they can only lower the fare.

Schema

Relational tables (PostgreSQL 16; this DDL, the data definition language that creates the tables, runs unchanged on an empty database):

CREATE TYPE rule_type AS ENUM ('EMERGENCY_OVERRIDE', 'TIME_OF_DAY', 'PROMOTION', 'REGION_CAP');
CREATE TYPE rule_status AS ENUM ('DRAFT', 'APPROVED', 'ACTIVE', 'RETIRED');

CREATE TABLE region (
  region_id  text PRIMARY KEY,
  tz_name    text NOT NULL            -- IANA zone, e.g. 'America/New_York'
);

CREATE TABLE holiday_calendar (
  region_id     text REFERENCES region,
  local_date    date NOT NULL,
  holiday_code  text NOT NULL,        -- 'NYE', 'THANKSGIVING'
  PRIMARY KEY (region_id, local_date)
);

-- Stable identity of a rule; never edited except to retire.
CREATE TABLE pricing_rule (
  rule_id     text PRIMARY KEY,
  rule_type   rule_type NOT NULL,
  region_id   text NOT NULL REFERENCES region,
  created_by  text NOT NULL,
  created_at  timestamptz NOT NULL DEFAULT now()
);

-- Every change is a new immutable version row.
CREATE TABLE pricing_rule_version (
  rule_id        text NOT NULL REFERENCES pricing_rule,
  version        int  NOT NULL,
  status         rule_status NOT NULL,
  priority       int  NOT NULL,
  params         jsonb NOT NULL,          -- validated per rule_type (see JSON Schema)
  days_of_week   smallint[] NOT NULL DEFAULT '{0,1,2,3,4,5,6}',   -- local, Mon=0
  local_start    time NOT NULL DEFAULT '00:00',
  local_end      time NOT NULL DEFAULT '00:00',                   -- start=end means all day
  holiday_code   text,                    -- only on this holiday (NULL = any day)
  effective_from timestamptz NOT NULL,
  effective_to   timestamptz,             -- NULL = open-ended
  author         text NOT NULL,
  approver       text,
  change_reason  text NOT NULL,
  PRIMARY KEY (rule_id, version),
  CHECK (approver IS NULL OR approver <> author),
  CHECK (effective_to IS NULL OR effective_to > effective_from)
);

-- A published, immutable snapshot: the unit the engine loads and the unit you roll back to.
CREATE TABLE ruleset (
  ruleset_version int PRIMARY KEY,
  region_id       text NOT NULL REFERENCES region,
  published_at    timestamptz NOT NULL,
  published_by    text NOT NULL,
  rolled_back_from int                    -- set when this snapshot restores an older one
);

CREATE TABLE ruleset_member (
  ruleset_version int  NOT NULL REFERENCES ruleset,
  rule_id         text NOT NULL,
  version         int  NOT NULL,
  PRIMARY KEY (ruleset_version, rule_id),
  FOREIGN KEY (rule_id, version) REFERENCES pricing_rule_version
);

-- Which region points at which snapshot right now (one row per region; rollback = one UPDATE).
CREATE TABLE active_ruleset (
  region_id       text PRIMARY KEY REFERENCES region,
  ruleset_version int NOT NULL REFERENCES ruleset,
  switched_at     timestamptz NOT NULL,
  switched_by     text NOT NULL
);

-- User-specific promotions: the promo is a PROMOTION rule; the grant says who may use it.
CREATE TABLE promo_grant (
  rule_id           text NOT NULL REFERENCES pricing_rule,
  user_id           text NOT NULL,
  redemptions_left  int  NOT NULL CHECK (redemptions_left >= 0),
  expires_at        timestamptz NOT NULL,
  PRIMARY KEY (rule_id, user_id)
);

Design points in the schema:

• Immutability. pricing_rule_version rows are never updated. A correction is version N+1. Rollback publishes a new ruleset whose members are the old versions and records rolled_back_from, so the history shows the rollback as its own event rather than rewriting the past.
• Four-eyes check in the database. CHECK (approver IS NULL OR approver <> author) means nobody can approve their own rule, enforced even if the application has a bug.
• Rollback is one statement: UPDATE active_ruleset SET ruleset_version = 41, switched_at = now(), switched_by = 'oncall/kpatel' WHERE region_id = 'nyc';
• Promotions are split in two: the promotion's terms live in a rule version like any other; promo_grant records which users may use it and how many uses remain. Decrementing redemptions_left happens when a trip is booked, in the same transaction as the booking, so two simultaneous bookings cannot both spend the last use.

Per-type parameters as JSON

params is jsonb (PostgreSQL's binary JSON column type: it stores a JSON document but stays queryable and indexable, unlike a plain text column) so new rule types do not need migrations, but each type has a JSON Schema (a separate specification describing what shape a valid JSON document must have, not the example JSON shown below) checked at write time. For example, a promotion:

{
  "type": "object",
  "required": ["pct_off", "max_off", "currency"],
  "properties": {
    "pct_off":  {"type": "number", "exclusiveMinimum": 0, "maximum": 0.5},
    "max_off":  {"type": "number", "exclusiveMinimum": 0},
    "currency": {"type": "string", "pattern": "^[A-Z]{3}$"},
    "stackable": {"type": "boolean", "default": false}
  },
  "additionalProperties": false
}

Emergency override: {"max_multiplier": 1.0}. Region cap: {"max_multiplier": 2.5}. Time-of-day: {"multiplier": 1.2}. additionalProperties: false rejects a typo such as "pct_of" instead of silently ignoring it.

Time zones, holidays and daylight saving

A time rule stores local_start, local_end, days_of_week and optionally holiday_code; the region stores its IANA zone; holidays live in holiday_calendar keyed by region and local date. At evaluation the engine converts the request's UTC instant to local time with the zone database and compares wall-clock times. Three consequences, all visible in the run below:

• Fall back (clocks go back an hour): 01:30 happens twice. A 01:00 to 03:00 rule matches both, which is what the business meant ("late night"), and it lasts three real hours that night.
• Spring forward: 02:00 to 03:00 local does not exist, so the same rule runs only one real hour that night. If the business wants a fixed duration instead, the rule must be expressed in hours from a start time, and that is a different rule type.
• Windows that cross midnight: "20:00 to 02:00 on New Year's Eve" must still apply at 01:30 on 1 January. The engine decides which calendar day the window opened on (yesterday, for the morning part) and checks the day and holiday against that day, not against today.

Runtime evaluation

When a snapshot is published, the engine compiles it once per region into lists grouped by rule type and sorted by (-priority, rule_id): Python sorts ascending, so negating priority puts the highest-priority rule first within its type, and the rule_id tie-break means that if two rules of the same type ever share a priority, evaluation order is still the same every time rather than depending on how the rows happened to load. A region typically has tens of live rules, so a quote scans them once: O(k) for k rules (the work grows in direct proportion to the number of rules, not by looking anything up in an index), no database round trip, and the sort key makes the result independent of load order.

The key logic for time windows lives in in_window, below. For an all-day rule (start == end) the window is always treated as opening today. For a same-day window (start < end) it just checks whether the local time falls inside it, and the window also opened today. For a window that crosses midnight (start > end, such as 20:00 to 02:00), it splits into two cases by comparing the local time t against start: if t >= start the request is in the evening part and the window is treated as having opened today (anchor = today); if t < end the request is in the small-hours part and the window is treated as having opened yesterday (anchor = date.fromordinal(today.toordinal() - 1), i.e. yesterday's date). That anchor date is what gets checked against the day-of-week and holiday filters, which is why a 20:00-to-02:00 New Year's Eve rule still matches at 01:30 on 1 January: the window is considered to have opened on 31 December.

A reference evaluator (Python 3.10 or later; zoneinfo, the standard-library module for IANA time zones, needs the system time-zone database, the operating system's own copy of the IANA rules mapping zone names to UTC offsets and daylight-saving transitions, or pip install tzdata on Windows):

from dataclasses import dataclass
from datetime import datetime, time, date, timezone
from zoneinfo import ZoneInfo

@dataclass(frozen=True)
class Rule:
    rule_id: str
    version: int
    rule_type: str          # EMERGENCY_OVERRIDE | TIME_OF_DAY | PROMOTION | REGION_CAP
    priority: int           # higher wins inside a type
    params: dict
    days: frozenset = frozenset(range(7))   # local weekday, Mon=0
    start: time = time(0, 0)
    end: time = time(0, 0)                  # start == end means all day
    holiday: str | None = None              # rule applies only on this holiday code

def in_window(rule, local_dt, holidays):
    t = local_dt.time().replace(tzinfo=None)
    today = local_dt.date()
    if rule.start == rule.end:                       # all-day rule
        anchor = today
    elif rule.start < rule.end:                      # same-day window
        if not (rule.start <= t < rule.end):
            return False
        anchor = today
    elif t >= rule.start:                            # crosses midnight, evening part
        anchor = today
    elif t < rule.end:                               # crosses midnight, morning part
        anchor = date.fromordinal(today.toordinal() - 1)   # window opened yesterday
    else:
        return False
    if anchor.weekday() not in rule.days:
        return False
    return rule.holiday is None or holidays.get(anchor) == rule.holiday

def evaluate(ruleset, tz_name, holidays, instant_utc, surge, user_promos):
    local = instant_utc.astimezone(ZoneInfo(tz_name))
    active = [r for r in ruleset if in_window(r, local, holidays)]
    by_type = {}
    for r in sorted(active, key=lambda r: (-r.priority, r.rule_id)):   # deterministic order
        by_type.setdefault(r.rule_type, []).append(r)
    trace, mult = [], surge
    if "EMERGENCY_OVERRIDE" in by_type:                  # 1. short-circuits everything
        r = by_type["EMERGENCY_OVERRIDE"][0]
        mult = min(mult, r.params["max_multiplier"])
        trace.append(f"{r.rule_id}v{r.version} emergency cap -> {mult:.2f}")
        return mult, 0.0, local, trace
    if "TIME_OF_DAY" in by_type:                          # 2. one time multiplier, top priority
        r = by_type["TIME_OF_DAY"][0]
        mult *= r.params["multiplier"]
        trace.append(f"{r.rule_id}v{r.version} x{r.params['multiplier']} -> {mult:.2f}")
    if "REGION_CAP" in by_type:                           # 3. most restrictive cap wins
        cap = min(r.params["max_multiplier"] for r in by_type["REGION_CAP"])
        if mult > cap:
            mult = cap
            trace.append(f"region cap {cap} -> {mult:.2f}")
    promos = [r for r in by_type.get("PROMOTION", []) if r.rule_id in user_promos]
    discount = max((r.params["pct_off"] for r in promos), default=0.0)   # 4. best single promo
    if discount:
        trace.append(f"promo {discount:.0%} off fare")
    return mult, discount, local, trace

RULES = [
    Rule("tod_late_night", 3, "TIME_OF_DAY", 10, {"multiplier": 1.2}, start=time(1), end=time(3)),
    Rule("tod_nye", 1, "TIME_OF_DAY", 50, {"multiplier": 1.5}, start=time(20), end=time(2), holiday="NYE"),
    Rule("cap_city", 7, "REGION_CAP", 0, {"max_multiplier": 2.5}),
    Rule("cap_regulator", 2, "REGION_CAP", 0, {"max_multiplier": 2.0}),
    Rule("promo_welcome", 1, "PROMOTION", 5, {"pct_off": 0.20}),
]
HOLIDAYS = {date(2026, 12, 31): "NYE"}
TZ = "America/New_York"

cases = [
    ("fall-back 1:30 EDT", datetime(2026, 11, 1, 5, 30, tzinfo=timezone.utc), 1.0, set()),
    ("fall-back 1:30 EST", datetime(2026, 11, 1, 6, 30, tzinfo=timezone.utc), 1.0, set()),
    ("spring-fwd 03:30",   datetime(2026, 3, 8, 7, 30, tzinfo=timezone.utc), 1.0, set()),
    ("NYE 23:00 surge 1.8", datetime(2027, 1, 1, 4, 0, tzinfo=timezone.utc), 1.8, {"promo_welcome"}),
    ("Jan 1 01:30",        datetime(2027, 1, 1, 6, 30, tzinfo=timezone.utc), 1.0, set()),
]
for label, ts, surge, promos in cases:
    mult, disc, local, trace = evaluate(RULES, TZ, HOLIDAYS, ts, surge, promos)
    print(f"{label:<20} local={local:%Y-%m-%d %H:%M %Z} mult={mult:.2f} promo={disc:.0%}")
    for line in trace:
        print("    " + line)

emergency = RULES + [Rule("storm_2026", 1, "EMERGENCY_OVERRIDE", 100, {"max_multiplier": 1.0})]
mult, disc, local, trace = evaluate(emergency, TZ, HOLIDAYS, cases[3][1], 1.8, {"promo_welcome"})
print(f"{'NYE + emergency':<20} local={local:%Y-%m-%d %H:%M %Z} mult={mult:.2f} promo={disc:.0%}")
for line in trace:
    print("    " + line)

Output:

fall-back 1:30 EDT   local=2026-11-01 01:30 EDT mult=1.20 promo=0%
    tod_late_nightv3 x1.2 -> 1.20
fall-back 1:30 EST   local=2026-11-01 01:30 EST mult=1.20 promo=0%
    tod_late_nightv3 x1.2 -> 1.20
spring-fwd 03:30     local=2026-03-08 03:30 EDT mult=1.00 promo=0%
NYE 23:00 surge 1.8  local=2026-12-31 23:00 EST mult=2.00 promo=20%
    tod_nyev1 x1.5 -> 2.70
    region cap 2.0 -> 2.00
    promo 20% off fare
Jan 1 01:30          local=2027-01-01 01:30 EST mult=1.50 promo=0%
    tod_nyev1 x1.5 -> 1.50
NYE + emergency      local=2026-12-31 23:00 EST mult=1.00 promo=0%
    storm_2026v1 emergency cap -> 1.00

Reading it: both 01:30 instants on 1 November get the late-night x1.2; 03:30 on the spring-forward night is outside the window; on New Year's Eve the higher-priority holiday rule beats the late-night rule, and the x2.7 result (1.8 surge × 1.5) is clamped by the stricter of two caps (x2.0, not x2.5); at 01:30 on 1 January the midnight-crossing holiday window still applies; the emergency override wins over everything and also suppresses the promotion, because evaluate returns as soon as it sees an emergency rule, before the promotion block ever runs. That is deliberate, not an accidental side effect: during an emergency the override needs to guarantee one exact, auditable price with nothing else touching it, and letting a promotion also apply would mean the effective price during a declared emergency depends on which riders happen to hold a promo code, which defeats the point of a single, reviewable emergency number.

Trade-offs and pitfalls

• Storing UTC times for a local-time rule is the classic bug: "18:00 to 20:00" stored as 23:00 to 01:00 UTC shifts by an hour twice a year.
• Evaluating rules with SQL on every quote works at low volume but puts the pricing database on the hot path (the code path executed on every single request, where added latency or an extra round trip multiplies across the full request volume). Compile the snapshot into memory and subscribe to publish events instead.
• Whether promotions should stack is a business decision; the schema supports it with a stackable flag, but the default must be no, or overlapping campaigns multiply discounts.
• Holiday calendars are data, not code. Regional holidays move (Easter, lunar new year), so the calendar table is loaded a year ahead and reviewed.
• Where this would change: if rules needed to match on many dimensions at once (product, zone, rider segment, time), the in-memory lists become an index keyed by region and product, but the evaluation order stays the same.

Direct answer

I would not try to resolve "infra wants cheap, data science wants fast" as a debate about whose priority wins. I would convert it into a shared decision process: get both teams to agree, in writing, on what "cost" and "speed" mean as actual numbers for this workload before evaluating a single technology, replace the argument with a short, time-boxed pilot measured against those agreed numbers, make the call transparently against that pre-agreed criteria, and treat communication and rollback as part of the decision itself, not an afterthought once a winner is picked.

The decision process, step by step

1. Frame the trade-off quantitatively, before anyone proposes an option. Get both teams in the same room and turn each side's position into a number: infra states a hard monthly budget ceiling, data science states a target such as p95 (95th percentile, meaning 95 percent of requests are faster than this) query latency. Written down as joint constraints on the same decision, the two positions stop being a values argument ("cost matters more" versus "speed matters more") and become a trade-off surface you can actually evaluate options against. If, once written down, nothing on the market satisfies both constraints, that itself is the finding, and it needs to be escalated honestly rather than resolved by whoever argues loudest.

2. Shortlist and score on paper first. Narrow the field to two to four realistic candidates, and build a lightweight cost model (compute, storage, and operational overhead) and a rough latency or throughput estimate from public benchmarks and vendor documentation for each, before spending any pilot time. Discard anything that clearly fails a hard constraint from step 1.

3. Run a pilot to make the trade-off empirical, not rhetorical. Load a representative (not full) slice of production data and the workload's actual hardest queries into the top two remaining candidates, and measure the same metrics for both: dollars per unit of workload, p95/p99 (95th and 99th percentile) latency, and a proxy for operational burden such as the number of manual interventions or alerts fired during the pilot window. Time-box it, roughly one to two weeks, so it cannot turn into an open-ended bake-off where each side just waits for numbers that favor it.

4. Align the two stakeholder groups on the results. Bring the pilot's numbers back to both teams together, not separately, and score them against the criteria they both signed off on in step 1, not criteria introduced after seeing who is losing (a common failure mode is quietly re-weighting the scorecard once the result is known). Where the winning option under-serves one team's original number, name that gap explicitly and get an actual acknowledgment, not silence, that the trade-off is acceptable, or escalate it.

5. Make the final call and minimize disruption. Decide, then document a short decision record (what was chosen, the alternatives considered, and the specific, measurable condition that would trigger revisiting it, for example "if write volume exceeds twice the pilot's projection" or "if p95 breaches the agreed target for N consecutive days"). Sequence the migration to touch the smallest reasonable blast radius first, a low-traffic table or service before the primary one, because that sequencing choice, not just which technology won, is what actually minimizes disruption.

6. Communicate the decision. Announce the decision with the reasoning, not just the outcome: the agreed criteria, the pilot's numbers, and where each team's original ask landed, delivered to both teams and any downstream consumers of the data. Set an explicit checkpoint, for example 90 days after cutover, to revisit with real production numbers instead of pilot numbers.

7. Define the rollback plan before cutover, not after something breaks. Concretely define what rollback means for this specific choice: run a dual-write or shadow-read period so the old store stays live and in sync for a fixed window, write a scripted cutback runbook, and set an explicit, numeric rollback trigger agreed in advance, for example "if the error rate exceeds X percent or latency regresses more than Y percent within the first week, we revert." A rollback decided under production pressure should be a checklist someone already agreed to, not a negotiation happening in an incident channel.

Worked example

Two options came out of the shortlist: a cheaper, slower one that infra prefers, and a pricier, faster one that data science prefers. Both teams agreed in step 1 that cost and speed matter equally (weight 0.4 each) and operational burden matters somewhat less (weight 0.2).

options = {
    "Option A (infra-favored: cheaper, slower)": {
        "compute_usd_per_month": 1_200,
        "storage_usd_per_gb_month": 0.023,
        "data_gb": 20_000,
        "p95_query_ms": 850,
    },
    "Option B (DS-favored: pricier, faster)": {
        "compute_usd_per_month": 4_800,
        "storage_usd_per_gb_month": 0.10,
        "data_gb": 20_000,
        "p95_query_ms": 120,
    },
}

weights = {"cost": 0.4, "speed": 0.4, "ops_burden": 0.2}  # agreed by both teams in step 1
ops_burden_score = {"Option A (infra-favored: cheaper, slower)": 0.8,   # 0-1, 1 = lowest burden
                     "Option B (DS-favored: pricier, faster)": 0.5}

results = {}
for name, o in options.items():
    monthly_cost = o["compute_usd_per_month"] + o["storage_usd_per_gb_month"] * o["data_gb"]
    results[name] = {"monthly_cost_usd": monthly_cost, "p95_query_ms": o["p95_query_ms"]}
    print(f"{name}: ${monthly_cost:,.0f}/month, p95 = {o['p95_query_ms']} ms")

costs = [r["monthly_cost_usd"] for r in results.values()]
lats = [r["p95_query_ms"] for r in results.values()]
cmin, cmax = min(costs), max(costs)
lmin, lmax = min(lats), max(lats)

print()
for name, r in results.items():
    cost_score = 1 - (r["monthly_cost_usd"] - cmin) / (cmax - cmin) if cmax != cmin else 1.0
    speed_score = 1 - (r["p95_query_ms"] - lmin) / (lmax - lmin) if lmax != lmin else 1.0
    burden = ops_burden_score[name]
    total = weights["cost"] * cost_score + weights["speed"] * speed_score + weights["ops_burden"] * burden
    print(f"{name}: cost_score={cost_score:.2f} speed_score={speed_score:.2f} "
          f"ops_score={burden:.2f} -> weighted_total={total:.3f}")

Output:

Option A (infra-favored: cheaper, slower): $1,660/month, p95 = 850 ms
Option B (DS-favored: pricier, faster): $6,800/month, p95 = 120 ms

Option A (infra-favored: cheaper, slower): cost_score=1.00 speed_score=0.00 ops_score=0.80 -> weighted_total=0.560
Option B (DS-favored: pricier, faster): cost_score=0.00 speed_score=1.00 ops_score=0.50 -> weighted_total=0.500

$\text{score}(o) = w_{cost} \cdot s_{cost}(o) + w_{speed} \cdot s_{speed}(o) + w_{ops} \cdot s_{ops}(o)$

Option A wins, 0.560 to 0.500, a margin of 0.06 on a 0-1 scale. That narrowness is itself a piece of information worth communicating: this was not a landslide, so the write-up to both teams should say the call was close, the data science team's speed concern was real and only barely outweighed, and the rollback plan and 90-day review checkpoint matter more here than they would for a lopsided result.

Trade-offs and pitfalls

• Common wrong turn: presenting a comparison deck and asking each team to vote. That proceduralizes a popularity contest instead of resolving anything, and it tends to produce a worse decision than either team's original instinct, because a deck omits which parts of each option would need a workaround after launch.
• Common wrong turn: running the pilot without agreeing on criteria first. Without that, both sides re-litigate afterward by pointing to whichever sub-metric favors them, and the pilot bought nothing.
• Pitfall: choosing a "split the difference" hybrid to avoid conflict when the pilot data clearly favors one option. A negotiated compromise that ignores the numbers you just measured throws away the value of running the pilot at all.
• Pitfall: skipping the rollback plan because everyone feels confident about the call. That confidence is exactly when it is easiest to skip, and exactly when skipping it is most expensive if the decision turns out wrong.
• Escalation path: if the pilot genuinely ties on the agreed criteria, escalate with that same evidence to a joint decision-owner, an architecture review or the teams' shared manager, rather than let the disagreement continue indefinitely. A tie is itself a valid, informative pilot outcome, not a failure of the process.

Proving that a technology change improved reliability, cost, or developer velocity requires the same discipline as any causal claim: a baseline captured before the change, a comparison that isolates the change from everything else happening at the same time, and a report that states the uncertainty honestly instead of presenting a single before-and-after number as fact.

Baseline, control, and rigor

Capture the baseline before the change ships, not after. Record the metric over a period long enough to see its natural variation (commonly 4 to 8 weeks, or long enough to span at least one full business cycle such as a weekly or monthly peak), because a single pre-change data point cannot tell you whether post-change movement is the change or ordinary noise.

Use a control where one is available, and be explicit when one is not. The strongest design is a staged or canary rollout, where part of the traffic or part of the team migrates first while the rest continues on the old system under identical external conditions during the same window; this isolates the change from confounders like a holiday traffic dip or an unrelated incident elsewhere in the stack. When a true control is impossible (for example a single monolith-wide platform migration with no way to run half the org on the old system), say so explicitly and rely instead on an interrupted-time-series comparison: the same metric, same team, same season, immediately before and after, with any known confounding event during the window called out by name rather than absorbed silently into the result.

Pick metrics that match the claimed benefit, and use more than one. For reliability: incident count and mean time to recovery, plus the error-budget burn rate against the service's service-level objective, or SLO (the reliability target you've committed to, for example 99.9% uptime per month). An error budget is the amount of allowed failure that target implies (0.1% of the month, in this example), and the burn rate is how fast you're consuming it: a rate of 2x means you're on pace to use up a month's entire allowed failure in half a month, which is the signal most teams use to slow down risky changes. For cost: total infrastructure spend normalized per unit of traffic or per customer, not raw spend, since raw spend conflates cost efficiency with growth. For developer velocity: deployment frequency, lead time for changes (time from commit to production), and change failure rate, three of the four metrics the DevOps Research and Assessment (DORA) research program tracks (the fourth, time to restore service, is the same failure-recovery signal already captured above as mean time to recovery), plus a qualitative measure such as a developer survey on perceived friction, since a purely quantitative velocity number can improve while developer experience quietly worsens in a way the metric does not capture.

State statistical rigor honestly. Report the baseline period's own variance (for example "incident count varied between 3 and 9 per month in the six months before the change") alongside the post-change number, so a reviewer can judge whether the observed shift is larger than the system's normal noise, rather than reporting only two point estimates and letting the reader assume any difference is meaningful.

Worked example

A team migrates a service from self-managed virtual machines to a managed container orchestration platform, claiming it will improve deployment frequency. Baseline: over the 8 weeks before migration, the team shipped a median of 3 deployments per week, ranging from 1 to 6. They run a staged migration, moving one of three service instances to the new platform while the other two stay on the old one for the same 6-week window, so external factors (a product launch, a holiday freeze) affect both groups equally. Post-migration, the migrated instance's owning team ships a median of 9 deployments per week over that window, versus a median of 4 for the two teams still on the old platform during the identical window. Because both groups experienced the same external conditions, the roughly 2x differential (not the raw 3-to-9 jump, which could partly be a general uptick) is the number attributable to the platform change, and it is reported to stakeholders as "deployment frequency increased by approximately 2x relative to a matched control group over the same 6-week window," with the baseline range disclosed alongside it.

Trade-offs and pitfalls

The honest version of this analysis takes longer and produces a less dramatic headline than "deployments went from 3 to 9 a week," so there is real organizational pressure to skip the control and report the bigger, uncontrolled number; resist it, because a claim that does not survive a skeptical rereading damages the credibility of every future measurement from the same team. A common pitfall is picking the metric after seeing which one moved favorably, which turns measurement into storytelling; commit to the metric and the comparison design before the change ships. Reporting to stakeholders should lead with the plain-language claim and its confidence level, then the supporting numbers, since a wall of statistics with no headline is as ineffective as a headline with no statistics behind it.

Direct answer

"They really want this" is a second-hand opinion about a solution, not a need. First, ask Sales for the customer's exact words and for access to the customer (a call, or joining the next one). Then, with the customer, open with: "Walk me through the last time this came up. What were you trying to get done?" That reveals the business objective (the outcome they are after) and what they do today. Close with a success question: "Six months from now, if this is working well, what is different and how would you know?"

Step 1: what to ask Sales (a few minutes)

• What did the customer actually say, word for word? A request often arrives rephrased as a feature.
• What is riding on it: a renewal, a new deal, a complaint? That sets urgency, not priority.
• Who at the customer should I talk to, the person who feels the pain or only the buyer?
• Has anything been promised? Ask early so nothing is committed before the need is understood.

Step 2: what to ask the customer, and why

1. "Walk me through the last time this came up." A real recent event gives facts. Contrast that with "would you use an export button?", which is a leading question (one that suggests the answer you hope for, or asks about your solution instead of their situation) and invites a polite yes. A neutral version asks about the situation: "How do you get these figures out of the system today?"
2. "What do you do today, and what does that cost you?" The workaround and its cost tell you how serious it is.
3. "What happens if you do not get this?" This separates a need from a nice-to-have.
4. "Who will judge whether this worked, and what will they look at?" This yields the success measure.

Worked example

Request from Sales: "Customer X wants Excel export on the reports page." In the call, the customer says their finance team builds a monthly board pack, and it takes two days because figures are copied from screens. Success, in their words: "pack assembled in half a day, and no figure that differs from the system". The numbers are the customer's illustrative estimates to be confirmed. The real requirement is "get a month's figures into the board pack, accurate, in half a day", which an export might satisfy, or a scheduled report might do better.

What to write down

The customer's objective, today's workaround and cost, the success measure and who judges it, and the assumptions you still need to check.

Pitfalls

• Treating Sales' enthusiasm as demand. It may be one voice.
• Asking leading questions about the feature, such as "Wouldn't an export button save you time?" It plants the solution and the answer. Ask what they do today instead.
• Promising before understanding. Say instead: "I would like to understand this properly first."

Direct answer
A prescriptive requirement locks in a specific implementation the engineer might have good reasons to change; the fix is separating the actual constraint (a real safety, performance, or compliance need) from the implementation detail that was really just a guess at how to satisfy it.

Structured elaboration
The rewrite process:

1. Ask why the specific technology was named. Usually it's because the requirement author had a mental model of the solution and wrote it down as if it were the requirement, when the real requirement is the outcome that mental model was trying to achieve.
2. State the outcome the choice was meant to guarantee. If a specific database was named because of a durability need, the real requirement is the durability guarantee, not that database.
3. Preserve genuinely essential constraints as explicit constraints, not as an implementation choice. If there's a real reason a specific technology is required (an existing integration, a compliance certification only that vendor has), state that reason explicitly as a constraint rather than leaving it implicit in a prescriptive line.

Worked example
Original prescriptive text: "Store transaction records in PostgreSQL with synchronous replication to a secondary in the same region, using a Kafka topic for downstream event distribution."

Rewritten, outcome-focused: "Transaction records must survive a single-node failure with zero data loss (durability requirement), be queryable with strong consistency for the primary write path, and be available as a stream of events to downstream consumers within 5 seconds of the write (event distribution requirement)." This preserves the real safety requirement (zero data loss on node failure) and the real performance requirement (5-second event propagation) while giving engineering the freedom to choose the specific database and messaging technology, potentially discovering a better-fitting option than the one originally assumed.

Trade-offs and pitfalls
The main risk in over-correcting toward "never name a technology" is that some prescriptive constraints ARE genuinely essential (an existing system only integrates with a specific technology, or a compliance requirement is certified against a specific vendor), and stripping those out in the name of flexibility creates rework when engineering picks an incompatible alternative. The discipline is not "never name a technology," but "never name a technology without stating the real reason, so engineering can tell whether the constraint is negotiable."