Marketplace, Dispatch, and Logistics System Design Questions

Requirements:

• Functional: assign nearest/optimal driver per request in ≤100 ms P95
• Scale: 10k TPS, regional failover
• Non-functional: high availability, graceful degradation

High-level architecture:

1. Ingress/load-balancer: Regional API Gateway + global DNS (Route53/anycast) → regional ELBs.
2. Streaming/event layer: Kafka (regional clusters, cross-region replication via MirrorMaker2) for asynchronous updates (driver status, telemetry).
3. State/store & feature serving: Redis Cluster (sharded by region + geohash) for real-time driver availability and precomputed features (ETA, surge multiplier, recent acceptance rate). Durable store: CockroachDB or Spanner for authoritative records and cross-region replication.
4. Feature updater: Lightweight stream processors (Flink/ksql) consuming Kafka to compute streaming features and write to Redis with TTLs.
5. Model inference: Two-tier inference:
   • Fast local model: optimized lightweight model (TensorRT/ONNX Runtime) deployed per region (K8s + GPU/CPU pods) serving via gRPC with batching micro-latency.
   • Heavy global model: larger model for offline training and periodic updates.
6. Router/Scoring service (stateless): Receives assignment request, queries Redis for candidate drivers (spatial index via geohash + Redis geo), fetches features, calls local inference (parallel RPCs with timeout), applies business rules and returns assignment.
7. Observability & autoscaling: Prometheus/Grafana, SLO-based autoscaling, circuit breakers.

Data partitioning:

• Primary partition by region (legal/latency boundary).
• Within region, shard by geohash prefix (hot spots get more shards).
• Kafka topics keyed by driver_id; Redis slots per shard.
• Cross-region replication only for critical state; prefer eventual consistency.

Latency and throughput strategies:

• Pre-filter candidates in Redis (top N by distance); limit model calls to N (e.g., 5–10).
• Use async pipelining and bounded concurrent RPCs.
• Model batching small (micro-batches) to keep P95 <100ms.
• Reserve capacity for 10k TPS per region; autoscale inference pods and Redis.

Fallback logic if model unavailable:

1. Short-circuit: If local model call fails or exceeds timeout (e.g., 30 ms), fallback to:
   • Heuristic scorer: deterministic rule-based score (distance, driver acceptance rate, idle time, vehicle type).
   • Cached model outputs: last-known model scores from Redis (TTL short).
2. Circuit breaker: If model error rate > threshold, flip to heuristic mode and trigger warm-up of new model pods.
3. Degraded mode: Increase candidate count and use simple ranking to avoid starvation.
4. Post-facto correction: Log assignments for offline re-ranking and learning; generate alerts.

Trade-offs:

• Strong regional partitioning optimizes latency but complicates cross-region rebalancing.
• Heuristics ensure availability at slight quality loss; periodic A/B test to measure impact.

This design balances sub-100 ms P95 latency using in-memory feature serving, lightweight regional inference, sharded spatial indexing, and robust fallbacks for availability.

Requirements & constraints:

• Single-city grid/graph with realistic zones, road network, travel times, demand distribution; objective: maximize driver utility (earnings minus costs) and service-level (wait time, cancellations).
• Support episodic RL (shifts), stochastic rider arrivals, time-varying traffic, and route-level decisions.

State:

• Driver-centric: current location (node/zone), time-of-day, remaining shift time, current passenger status (idle/en-route), local supply/demand features (estimated nearby idle drivers, queue lengths), smoothed traffic speeds on adjacent edges, recent earnings, battery/fuel if applicable.
• Global context: city-wide time features and historical demand heatmap embedding.

Action:

• Discrete: {accept/reject request, move to neighboring zone i, reposition to zone j from a shortlist, wait}.
• Optionally hierarchical: high-level target zone + low-level path planner.

Reward:

• Dense per-step: r = fare_received - fuel_cost - idle_cost - penalty * (long_pickup_time) + bonus * completing high-utilization trips.
• Include shaping for service metrics: negative for cancellations, large terminal reward for shift earnings.

Transition dynamics:

• Travel time on an edge sampled from a time-dependent distribution conditioned on traffic state (e.g., Gaussian with mean = baseline_speed(zone,time) and variance).
• Rider arrivals: Poisson processes per zone with lambda(z,t) from historical data; request destinations sampled from OD distribution.
• Matching: when idle and in zone, probabilistic match with arriving rider depending on surge, distance, and queue.
• Stochasticity: random rider cancellations, variable fare multipliers.

Incorporating stochastic rider arrivals & traffic:

• Fit zone-time Poisson rates and OD matrices from data; incorporate correlated fluctuations (events) via a latent demand factor.
• Traffic modeled as non-stationary stochastic process with temporal correlations (AR(1) or learned generative model); add noise and rare-peak events.
• Domain randomization: vary demand intensity, traffic noise, and matching delays during training.

Validation & sim-to-real transfer:

• Calibrate simulator to real traces using parameter fitting (travel-time distributions, arrival rates, match delays). Use metrics: trip time CDF, wait-time distribution, spatial demand heatmaps.
• Offline policy evaluation using historical logs (counterfactual policy evaluation / importance sampling) and A/B testing in shadow mode.
• Robustness checks: train on randomized ensembles; adversarial perturbations for worst-case.
• Small-scale real-world rollout: start with a constrained cohort, monitor key metrics (accept rate, pickup time, earnings), use iterative online fine-tuning / off-policy RL with conservative constraints.
• Safety nets: guardrails in policy (max reposition distance), human-in-loop override, and continual recalibration with live telemetry.