Public API (examples)
- RequestScheduler.request(opts: RequestOptions): Promise<Response>
- RequestScheduler.cancel(id: string): void
- RequestScheduler.configure(config: SchedulerConfig): void
- RequestScheduler.metrics(): SchedulerMetrics
- Event hooks: on('queued'|'sent'|'retry'|'done'|'error', cb)
RequestOptions: { id?, url, method, body?, priority: number, domain?: string, dedupeKey?: string, timeoutMs?, retries?, backoffBaseMs? }
Core data structures
- Priority queue per global scheduler (binary heap) keyed by priority + enqueueTime.
- Per-domain TokenBucket { capacity, tokens, refillRate, lastRefill } for rate-limiting.
- Concurrency Semaphore global and optional per-domain counters.
- Dedupe map: Map<dedupeKey, { promise, count, subscribers[] }> to coalesce identical requests.
- Retry policy store per-request with exponential backoff + jitter.
- Metrics registry: counters and histograms (queueTime, success/fail, retries).
Algorithms
- Enqueue: compute domain, dedupeKey. If dedupeKey active -> attach subscriber to existing promise. Else push into heap.
- Scheduler loop: while globalConcurrency < limit:
- peek highest priority; check domain token bucket (refill lazily). If tokens available and domainConcurrency < domainLimit, pop and dispatch.
- Otherwise, sleep until next refill or token available.
- Dispatch: increment concurrency, consume token, start fetch with AbortController and timeout. On transient error and retries left -> schedule retry with backoffMs = base * 2^attempt + random(-jitter, +jitter).
- Cancellation: if queued remove from heap; if in-flight use AbortController; update dedupe subscribers.
Dedupe details
- Build dedupeKey from (method,url,bodyHash,headersWhitelist). First request stores promise and response; subsequent attach subscriber that resolves/rejects when first completes. Time-to-live configurable to avoid stale caching.
Correctness & Performance Validation
- Unit tests for token bucket, priority ordering, dedupe semantics, cancellation.
- Integration tests in headless Chrome (Puppeteer) simulating network conditions (latency, throttling) and failures.
- Load tests: spawn many parallel requests in the page, assert concurrency limits, per-domain rates, and SLA percentiles (p50/p95/p99 queue+response times).
- Chaos testing: inject random network errors, long delays, and verify retry/backoff behavior and that dedupe avoids duplicate outbound calls.
- Metrics-based assertions: monitor counters, ensure token buckets refill rates match expectations, and no memory leaks (ensure dedupe map clears).
- Use browser performance APIs and DevTools protocol to collect timing and heap snapshots.
This design prioritizes predictable client-side behavior, composability with frameworks, and observability for debugging under heavy load.
