Real-Time and Streaming System Design Questions

Direct answer

The hard part of comments is not the thread, it is the anchor: a comment points at "these words", and those words move, change, or disappear as people edit. So I would first pin down requirements with product, then keep comment threads as ordinary server-owned records pushed in real time, and store each comment's anchor as a position inside the document's own collaborative model (a pointer to specific characters that moves with them) rather than as a character offset. For real-time sync I would compare (A) a server-authoritative comment service with pub/sub push (publish/subscribe: publishers send events to a channel and every subscriber to that channel receives them, without publisher and subscriber knowing about each other directly) and (B) comments stored inside the document's replicated data model, pick A for most products, and support offline-first comments with a local outbox and client-generated IDs.

Clarifying requirements with product

Assumptions I would state

• Comments must be live for anyone with the document open (within about 1 second).
• Roles: owner, editor, commenter, viewer. Commenters and above can comment; authors can edit their own comments; editors can resolve.
• Edit history per comment is kept; deletion is a soft delete visible as "comment deleted".
• Typical document: up to 500 threads, up to 50 concurrent viewers.
• The document editor already syncs text in real time with a collaborative model (a sequence CRDT, conflict-free replicated data type, or OT, operational transformation, with a central server).

The anchor problem, concretely

User A comments on "quarterly revenue" at characters 120 to 137. User B inserts 30 characters at position 50. If the anchor is stored as offsets 120..137, it now points at the wrong words. The fix is to anchor to stable identities: a CRDT gives every character a permanent ID, and an OT system can transform stored ranges through each operation, just as it transforms cursors. Store {start_id, end_id} (or a range the server keeps transformed), plus a copy of the quoted text for display if the range is later deleted.

Approach A: server-authoritative comment service with pub/sub push

• Comments are rows in a comments table: thread_id, comment_id, doc_id, anchor, author, body, version, created_at, resolved.
• Create and edit go through a REST (or RPC, remote procedure call: calling a function on another server as if it were a local one) API that checks permissions, writes the row, then publishes comment.created on the document's channel.
• Everyone with the document open is subscribed to that channel over the WebSocket (a persistent, two-way connection between browser and server that lets the server push events without the client asking again) they already have for editing, and receives the event within the push latency.
• Ordering within a thread is by server-assigned sequence per thread, so every client shows the same order.
• Concurrent edits to the same comment use optimistic concurrency (proceed as if there is no conflict, but detect and reject the write if one occurred instead of locking up front): the client sends the version it edited; if the server has moved on, it returns a conflict and the client shows "this comment changed, review before saving".

Approach B: comments inside the document's collaborative model

• Threads are stored as part of the replicated document state (for example a map of thread IDs to comment lists inside the same CRDT), and sync over exactly the same channel and merge logic as text.
• Anchors and comments share one consistency model: a thread and the text it points at can never disagree about ordering, and undo and version history of the document include comments for free.
• But: permission checks become harder (the document model usually lets any editor change anything, and here a commenter must be able to add comments without editing text), notifications and search need to extract comments from the model, and the document grows with every comment ever written.

Offline-first comments

• The client creates the comment with a client-generated UUID (universally unique ID) and stores it in a local outbox (IndexedDB, the browser's local database) along with the anchor as character IDs and the quoted text.
• On reconnect, it sends each outbox item; the server treats the UUID as an idempotency key (a request identifier that makes a retry safe, because a repeat of the same key is recognised and not applied twice).
• The server then re-resolves the anchor against the current document: if the anchored characters still exist, the comment attaches normally; if they were deleted while the user was offline, the comment is saved as detached and shown with its quoted text in the sidebar.
• If the user lost permission while offline, the server rejects with a clear error and the client keeps the text so the user can copy it.

stateDiagram-v2
    [*] --> Draft
    Draft --> Pending: saved offline
    Draft --> Open: saved online
    Pending --> Open: synced, anchor found
    Pending --> Detached: synced, anchored text deleted
    Pending --> Rejected: synced, no permission
    Open --> Detached: anchored text deleted later
    Open --> Resolved: editor resolves
    Resolved --> Open: reopened

Trade-offs and my choice

I would choose A, with anchors stored as CRDT character IDs (or OT-transformed ranges) so they move with the text. It keeps access control and integrations simple, and comments are not keystroke-rate data that needs the document's merge machinery. I would switch to B if the product is offline-first as a core promise (for example a field app used mostly without signal), because then native merging outweighs the permission complexity.

Worked example: fan-out load

Fan-out is copying one event out to every subscriber who should receive it. A popular document with 50 viewers and a burst of 20 comments in a minute produces 20 x 50 = 1,000 push events in that minute, about 17 per second: trivial for a WebSocket gateway. Load becomes interesting only for large all-hands documents (for example 2,000 viewers), where 20 comments per minute is 40,000 pushes per minute. There I would send a small "thread X changed" event and let clients fetch the thread lazily only if it is on screen.

Communicating the design to product and QA

• To product: a one-page decision record: the requirements as understood, the assumptions (numbered, so they can be confirmed or corrected one by one), the chosen approach and the one it was chosen over, and the visible behaviours they must sign off, especially the detached comment and conflicting edit experiences, with mock-ups.
• To QA: the state diagram above plus a test matrix derived from it: comment on text another user deletes at the same time; comment offline, text deleted before reconnect; two users edit the same comment; permission revoked while offline; the same offline comment sent twice (retry); 500-thread document load time; reconnect after a server restart. Each row names the expected state and message.
• Agree on acceptance criteria as measurable statements: "a comment appears to other viewers within 1 second at p95" (the 95th percentile), "no duplicate comments after retries".

Pitfalls

• Offset anchors are the classic bug: comments drift onto the wrong text after any edit above them.
• Deleting comments whose text was deleted silently loses reviewer work; detach instead.
• Server-generated IDs for offline comments make retries create duplicates.
• Treating comments as document edits for permissions lets commenters change text, or blocks them from commenting.

Direct answer

I would build one pipeline with two entry paths. VOD (video on demand: uploaded files) goes through a chunked, parallel batch encode: split the source into pieces, encode every piece of every quality level in parallel, and stitch. Live goes through always-on real-time encoders, one set per stream, running redundantly. Both paths produce the same output: CMAF fragments (Common Media Application Format, a fragmented-MP4 segment format that both HLS, HTTP Live Streaming, and DASH, Dynamic Adaptive Streaming over HTTP, players can use), with keyframes aligned across all qualities, packaged into HLS and DASH manifests. Codec choice follows audience and economics: H.264 for every title (it plays everywhere), H.265/HEVC for high resolutions on devices that decode it, and AV1 only where the bandwidth savings repay its much higher encode cost, which in practice means popular VOD first.

Some terms: transcoding is decoding a source video and re-encoding it at different settings; an ABR ladder (adaptive bitrate ladder) is the set of qualities (for example 1080p at 5 Mbps down to 240p at 300 kbps) a player switches between as bandwidth changes, with each individual quality level called a rung of that ladder; packaging wraps encoded video into segments plus manifests (HLS .m3u8 playlists, DASH .mpd files) that tell the player what exists.

Pipeline

flowchart LR
    U[VOD upload] --> SPL[Split at scene or GOP, group of pictures, boundaries]
    SPL --> Q[Chunk encode queue]
    Q --> W[Encoder workers per codec and rung]
    W --> ST[Stitch and verify]
    L[Live ingest RTMP or SRT] --> LE[Real-time encoders, redundant pair]
    ST --> MEZ[MEZ: mezzanine, the intermediate CMAF segments in object storage]
    LE --> MEZ
    MEZ --> PKG[Packager: HLS and DASH manifests, DRM]
    PKG --> CDN[CDN]

Live ingest typically arrives over RTMP (an older but ubiquitous streaming push protocol) or SRT (Secure Reliable Transport, a UDP-based protocol that tolerates lossy networks better).

Segment duration

• VOD: 6-second segments with a keyframe every 2 seconds. Apple's HLS authoring specification says target durations should be 6 seconds and key frames (IDR frames, which a decoder can start from without earlier data) should be present every two seconds. Longer segments mean fewer requests and better compression; shorter ones let the player switch quality sooner.
• Live, standard latency: 2 to 4 second segments. Players typically buffer about three segments before starting, so latency behind real time is roughly three segment durations plus encode and CDN time.
• Live, low latency: keep ordinary-length segments (a few seconds) but publish each one in sub-second partial chunks (Low-Latency HLS partial segments, or chunked CMAF for DASH), so a player can fetch the start of a segment while it is still being encoded. This keeps request counts and compression close to normal while cutting latency to a few seconds.
• The keyframe interval (GOP, group of pictures) must divide the segment duration and be identical across every rung and codec. A player can only start decoding a segment at a keyframe (a full frame that stands alone; every other frame is stored as a difference from earlier frames). If segments are 6 s and the keyframe interval is 2 s, every rung has a keyframe exactly on every segment boundary, so the player can switch from the 1080p rung to the 480p rung right there. If one rung instead used a 2.5 s keyframe interval, its keyframes would drift away from the 6-second segment lines, so the frame sitting at that boundary in the new rung would not be a keyframe: the player would either show a broken frame for an instant or stall until that rung's next real keyframe, which is the glitch.

Codecs and tracks

Audio: encode once per language (AAC stereo for compatibility, optionally a 5.1 surround track), independent of video. Audio renditions are listed as alternates (the specific manifest syntax is implementation detail, not something to memorize here: HLS calls them EXT-X-MEDIA TYPE=AUDIO groups, DASH calls them audio adaptation sets), so adding a language never re-encodes video.

Subtitles and captions: WebVTT for HLS and DASH (a plain-text subtitle format; IMSC1 text profile where richer styling is needed), delivered as separate segmented tracks; live closed captions (CEA-608/708, the broadcast-TV caption standards, one older and one newer) are carried in the video stream from the encoder rather than as a separate track. None of these format names change the design; they only matter when you implement packaging. Subtitles are tiny and can be added after publication.

Parallelization

• VOD, chunked encoding: split the source into chunks (say 30 to 60 seconds) at keyframe or scene boundaries, and encode each (chunk x rung x codec) as an independent job. A 2-hour film is 120 to 240 chunks per rendition, so wall-clock encode time is bounded by the slowest chunk, not the film's length. The stitcher verifies timestamps and keyframe alignment at chunk boundaries.
• Live: you cannot chunk the future. Parallelize across rungs (each rung or codec on its own encoder process or hardware encoder) and across streams. Run two encoders per important stream in different availability zones (physically separate data centers within one cloud region, each with independent power and networking, so a failure in one does not take out the other), producing identical segment numbers and timestamps, so the packager can fail over without the player noticing.
• Just-in-time packaging: store CMAF segments once and generate HLS or DASH manifests (and apply DRM, digital rights management, encryption) at request time, cached by the CDN. That avoids storing two copies of every segment.

Pre-warming encoders

• Scheduled live events: start encoders (and fill the GPU or hardware-encoder pool) 15 to 30 minutes early, run a test signal through the full chain, and have the ingest endpoint ready before the broadcaster connects.
• Unscheduled live: keep a warm pool of idle encoder instances with images already pulled and codecs initialised, sized to the peak rate of new streams per minute; a cold virtual machine plus container start is far slower than a user will wait at "Go live".
• VOD: pre-warming matters less; queue depth and a priority lane for fresh uploads keep time-to-publish predictable.

Worked example: when does AV1 pay for itself?

All inputs below are assumptions for illustration; substitute your measured bitrates and prices.

Assume a 1080p stream averages 4 Mbit/s in H.264, AV1 needs 30% fewer bits for the same quality, CDN delivery costs 0.02 dollars per GB, and AV1 encoding of a one-hour title costs 20 dollars more than H.264 alone.

Only AV1-capable viewers produce savings, so if half the audience can decode AV1 the break-even doubles to about 3,700 viewing hours. The rule this produces: encode H.264 for everything at upload, then re-encode to AV1 when a title's views cross a threshold. The same logic explains why AV1 for live usually waits for hardware encoders: a live stream cannot amortize a slow encode over future views.

Trade-offs: latency, quality, cost

Three encoder knobs behind the table below: an encoder preset trades encode speed for compression efficiency (a faster preset finishes sooner but needs slightly more bits for the same quality); lookahead frames are future frames the encoder examines before compressing the current one, so it can spend bits more wisely at the cost of extra buffering delay; multi-pass encodes the content twice, once to analyze it and once to compress using what it learned, which is why it is VOD-only (a live stream has no "later" frames to look ahead into).

Pitfalls: misaligned keyframes across rungs (quality switches stutter); re-encoding the audio per video rung (wasted compute and duplicate storage); one fixed ladder for all content (a cartoon needs far fewer bits than a sports match, so per-title ladders save bandwidth); and live encoders without redundancy, where one crashed virtual machine takes the stream down.

Direct answer

Instrument each tier (WebSocket gateways, message broker, consumers) with aggregate metrics for health and SLOs (service-level objectives), structured logs for per-connection events such as connects, disconnects and auth failures, and distributed traces for a sample of messages end to end. Never label metrics per connection (a label is a key-value tag attached to a metric, for example connection_id=abc123, and every distinct combination of label values becomes its own stored time series, a separately tracked history of one metric over time): with a million connections that creates millions of time series. Define two user-facing SLOs (connection success and end-to-end delivery latency), alert on error-budget burn rate rather than raw thresholds, back each common failure mode with a runbook, and lay out dashboards top-down from SLO status to per-tier detail.

Terms

• WebSocket: a long-lived two-way connection between browser or app and server, upgraded from an HTTP request.
• Backpressure: a slow receiver forcing the sender to buffer, slow down or drop. In this system it shows up as growing send buffers on gateways and growing lag in the broker.
• Consumer lag / queue depth: how many messages are waiting in the broker for a consumer that has not processed them yet.
• SLI / SLO: an SLI (service-level indicator) is the measured quantity (for example "share of messages delivered within 1 s"); the SLO is the target for it over a window (99% over 30 days). The error budget is what the SLO allows to fail (1%).
• p50/p95/p99: 50th, 95th and 99th percentiles.

Pipeline and where to measure

flowchart LR
  PUB[Publisher service] -->|stamp publish time| BRK[(Message broker)]
  BRK --> CON[Consumers: route and fan-out]
  CON --> GW[WebSocket gateways]
  GW --> CL[Clients: sampled delivery acks]
  GW -.-> MET[Metrics, logs, traces]
  CON -.-> MET
  BRK -.-> MET

What to instrument

Gateways: per-connection metrics, aggregated

Record per-connection facts into histograms and counters per node (a histogram buckets the observed values so percentiles like p99 can be computed from the distribution later; a counter only ever increases, such as a running total; a gauge, used just below, is a value that can go up or down, such as how many connections are open right now), not per-connection series:

• open connections (gauge) and connection age (histogram);
• connect attempts, successes, failures by reason (handshake error, auth failure, rate-limited);
• reconnection rate: connects from a client that disconnected within the last N minutes (the client sends a reconnect flag and attempt count);
• disconnects by close code: 1000 normal, 1001 going away, 1006 abnormal (no close frame, the WebSocket message that cleanly announces "this connection is ending"; its absence usually means a network drop), 1008 policy violation, 1011 server error;
• auth failures by reason: expired token, bad signature, revoked;
• backpressure indicators: per-connection send-buffer bytes (histogram), messages dropped or coalesced for slow clients, slow-client disconnects;
• node resources: file descriptors (OS-level handles a process holds open, one per socket; running out blocks new connections), memory, event-loop lag (how far behind a single-threaded server's task queue is running, a sign the process cannot keep up) or scheduler delay, CPU.

Broker and consumers

• queue depth / consumer lag per topic (a named stream of messages in the broker) and partition (one ordered, independently consumed slice of a topic), and its rate of change;
• publish rate, consume rate, publish errors;
• consumer processing time (histogram), retries, dead-letter count (a DLQ, dead-letter queue, holds messages that repeatedly failed).

Message latency percentiles

The publisher stamps a publish timestamp into each message. Gateways record publish-to-send latency in a histogram. For the true end-to-end figure, a sample of clients (say 1%) acknowledges receipt and the gateway records publish-to-ack. Report p50, p95 and p99 from the histograms, never averages of percentiles across nodes.

Logs and traces

• Logs: one structured line per connect and disconnect (connection id, user id hash, node, close code, duration, reason), and per auth failure. This is where per-connection detail lives, queryable when needed without costing a time series each.
• Traces: propagate trace context in message headers from publisher through broker, consumer and gateway, sampled (for example 0.1% plus every message slower than the p99 threshold), so a slow message can be followed hop by hop.

Why not per-connection metrics: the arithmetic

1,000,000 connections × 3 metrics each = 3,000,000 time series, most of them churning as clients reconnect. A time-series database would struggle, and the dashboards would be unreadable anyway. Histograms per node give the same distribution at a few hundred series.

SLOs and alerting

Burn-rate alerting pages when the error budget is being consumed too fast, instead of on any spike. Burn rate is how many times faster than "exactly on budget" you are failing:

The thresholds below are not arbitrary: they are solved backward from a policy decision. Decide how much of the 30-day budget a page is allowed to cost (2% within an hour) and how long you are willing to wait to detect that rate, then burn rate = target share ÷ (window ÷ total budget window). Over a 30-day window (720 hours):

• Page when burn rate exceeds 14.4 over both the last 1 hour and the last 5 minutes: solving 2% ÷ (1 h ÷ 720 h) gives 14.4, and 14.4 × 1 h ÷ 720 h = 2% of the monthly budget already gone in an hour.
• Ticket when it exceeds 6 over 6 hours: solving 5% ÷ (6 h ÷ 720 h) gives 6, and 6 × 6 ÷ 720 = 5% of the budget.

The page rule needs both windows to breach together. The long window (1 hour) confirms the failure rate is high enough to matter; the short window (5 minutes) confirms the problem is still happening right now. A long window alone would keep paging for hours after a fix ships, because the bad period is still being averaged in; a short window alone would page on a brief blip that self-resolves before anyone looks. Requiring both is what stops the page from firing long after recovery. The ticket rule uses a single 6-hour window instead of a paired one, because a ticket is not urgent enough to need that same speed of recovery detection. Add cause alerts that do not page but warn: consumer lag growing for 10 minutes, a node above 80% of its file-descriptor limit, send-buffer p99 rising.

Runbooks for the common failure modes

Dashboards for on-call

1. Top row: SLO status. Current SLIs, error budget remaining, burn rate, with deploy annotations.
2. Traffic: open connections, connects/s, reconnect rate, disconnects by close code, messages/s in and out.
3. Latency: delivery p50/p95/p99, by region.
4. Backpressure and queues: consumer lag by topic, send-buffer distribution, drops.
5. Per-node table (sortable): connections, CPU, memory, fds, event-loop lag, to find the one bad node.
6. Auth: failures by reason.

Every panel links to the matching logs query and runbook, so on-call moves from "what is red" to "what do I do" in one click.

Pitfalls

• Averaging p99 across nodes (mathematically meaningless: a percentile is a position in a sorted set of values, not a quantity that averages linearly, so the mean of several nodes' p99s is not the p99 of their combined traffic, and can be badly wrong when nodes carry uneven load); aggregate histograms, then compute one percentile over the combined data.
• Measuring latency only up to the broker: most delay in real incidents hides in fan-out (one broker message being delivered out to many gateway connections at once) and slow clients.
• Alerting on open-connection count dropping, which also happens at night and during graceful deploys.

Direct answer

I would build this as low-latency HTTP streaming over a multi-CDN, not as WebRTC: redundant venue encoders send contribution feeds over SRT to two cloud regions, a cloud transcoder produces a five-rung bitrate ladder with 2-second closed GOPs (a GOP, group of pictures, is the run of frames between two keyframes; closed means no frame in it references a frame in a neighboring GOP, so every GOP boundary is safe to switch bitrate rung or join the stream at), a packager writes CMAF chunks once and exposes them as both LL-HLS and LL-DASH (the low-latency modes of HLS, HTTP Live Streaming, and DASH, Dynamic Adaptive Streaming over HTTP), with 2-second segments split into 0.5-second parts. Two or three CDNs sit behind an origin shield, and the player targets about 1.5 seconds of buffer behind the live edge (the most recently published moment of the stream) and speeds up slightly to catch up when it drifts. That lands a glass-to-glass latency budget of about 3.7 seconds against the 5-second requirement, and it still rides on ordinary HTTP caching, which is the only way I know to serve roughly 8 Tbps affordably.

Requirements I am designing to

• 2M concurrent viewers, one event, a few hours long, audience concentrated in one or two continents.
• End-to-end (glass-to-glass) latency under 5 s: from the camera sensor to the pixel on the viewer's screen. I treat 5 s as a ceiling and design for about 3.5 to 4 s so that network jitter does not break the service-level objective (SLO, the target we promise to hit).
• Web, iOS, Android, and smart-TV players; DRM (digital rights management, the encryption and licensing that stops a stream from being copied) is required because it is licensed sport.
• No single failure (one encoder, one region, one CDN) may take the stream down.

Terms used below: ABR (adaptive bitrate) means the player picks among several encodings of the same video and switches as bandwidth changes; a GOP (group of pictures) is the run of frames between two keyframes, and a player can only start decoding at a keyframe (a GOP is closed when no frame inside it references a frame in a neighboring GOP, so every GOP boundary decodes on its own); a CDN (content delivery network) is a fleet of edge caches close to viewers.

Architecture

flowchart LR
  CAM[Venue cameras] --> ENC1[Venue encoder A]
  CAM --> ENC2[Venue encoder B]
  ENC1 -->|SRT| R1[Region 1 transcode]
  ENC2 -->|SRT| R2[Region 2 transcode]
  R1 --> PK1[CMAF packager 1]
  R2 --> PK2[CMAF packager 2]
  PK1 --> OS[Origin shield]
  PK2 --> OS
  OS --> CDNA[CDN A]
  OS --> CDNB[CDN B]
  CDNA --> P[Players]
  CDNB --> P

1. Ingest topology

• Two independent contribution encoders at the venue, fed from the same camera router (the on-site switcher that feeds both encoders from the same cameras), on two different network paths (fibre plus a bonded cellular, several mobile-data connections combined into one higher-throughput link, or satellite backup).
• Each encoder sends a high-quality mezzanine feed (a lightly compressed, near-source-quality copy meant for further processing downstream, not for a viewer to watch directly; for example 1080p60 at 20 to 30 Mbps) over SRT (Secure Reliable Transport, a UDP-based contribution protocol that retransmits lost packets inside a fixed latency window you configure). I prefer SRT over RTMP (Real-Time Messaging Protocol, the older TCP-based ingest protocol) because it tolerates packet loss on long-haul links without TCP's stall-and-recover behaviour and it has encryption built in.
• Feed A goes to cloud region 1, feed B to region 2. Both carry the same embedded timecode (a running clock value encoded directly into the video signal itself, not just a file timestamp), so both transcoders cut keyframes on the same frame and both packagers produce segments with identical numbers and boundaries. That is what makes failover seamless: a player can take part 1234.2 from region 1 and part 1234.3 from region 2.

2. Encoder and packager choices

• Codec: H.264 (AVC, Advanced Video Coding, the most widely supported video compression standard) for the full ladder because every device decodes it; add an HEVC (H.265, High Efficiency Video Coding, a newer codec that needs roughly half the bitrate of H.264 for the same quality but is not decoded by every device) ladder only if the device mix justifies doubling transcode and cache footprint. For a single event I would ship H.264 only.
• Low-latency encoder settings: no long look-ahead (frames the encoder studies ahead of the one it is currently compressing, to choose better settings, at the cost of delay), few or no B-frames (frames that reference future frames force the encoder to wait), and a closed GOP of exactly 2 s aligned across every rung, so any rung switch lands on a keyframe.
• Ladder (my assumed viewer mix, used in the arithmetic below):

• Packager: write CMAF (Common Media Application Format, fragmented MP4 that both HLS and DASH can reference) exactly once, encrypted with CENC in cbcs mode (CENC, Common Encryption, is a shared encryption standard; cbcs is one of its two encryption patterns, the one that FairPlay (Apple's DRM), Widevine (Google's) and PlayReady (Microsoft's) can all decrypt on current devices). The HLS playlist and the DASH manifest are two thin text views over the same media objects, so the CDN caches one copy of each chunk instead of two.

3. Segment and part durations

The protocol rules decide this, not taste. HLS requires a player to stay at least three target durations (a target duration is the nominal segment length the playlist advertises) behind the live edge (the HOLD-BACK rule). With the traditional 6 s segments that is 18 s of mandatory delay before anything else is counted, so classic HLS cannot meet 5 s at all.

LL-HLS (Low-Latency HLS) splits each segment into partial segments ("parts") that are published as soon as they are encoded, and relaxes the rule to PART-HOLD-BACK, which must be at least twice the part target duration and should be at least three times it. I choose:

• Segment = 2 s (one GOP), so a player joining or switching rung gets a keyframe within 2 s.
• Part = 0.5 s, so PART-HOLD-BACK = 3 x 0.5 = 1.5 s.
• Blocking playlist reload (the server holds a playlist request until the requested part exists, instead of the player polling) and preload hints (the playlist names the next part before it exists so the player can request it early). LL-DASH achieves the same with CMAF chunked transfer encoding (an HTTP response mode with no fixed length announced up front, so bytes can be sent piece by piece as they are produced): the segment URL is fetched once and bytes stream in as each chunk is encoded.

Why not simply 1 s segments with no parts, relying on the classic 3x hold-back rule instead of LL-HLS parts? It would not reach the same latency: the classic rule needs 3 x 1 s = 3 s of hold-back, twice the 1.5 s this design gets from 0.5 s parts, so it still misses the budget. It is not cheaper on requests either: without parts, a player fetches one 1 s segment and polls the ordinary (non-blocking) playlist about once a second to see what is new, roughly 2 objects/s per viewer, against about 4 objects/s (2 parts plus 2 blocking playlist reloads) for the parts design below. Parts cost more requests per viewer than plain 1 s segments would; that cost buys a hold-back of 1.5 s instead of 3 s, without halving the GOP on every rung the way shrinking the segment itself would.

4. CDN selection

• Multi-CDN, two or three providers, with capacity reserved in advance for the event window and a steering layer (DNS or a client-side switch in the player) that moves traffic based on real-user error rate and throughput. Two reasons: the peak is a large, sudden, geographically concentrated load that I would not want riding on one vendor's spare capacity, and one CDN outage during a final would be the whole incident.
• Selection criteria I would test in a rehearsal, not assume: support for LL-HLS blocking requests and chunked transfer end to end (some edges buffer the whole object before forwarding it, which silently adds a part duration or more), request-collapsing behaviour (many viewers asking for the same not-yet-complete part must turn into one upstream request instead of many), and per-region capacity.
• An origin shield (a mid-tier cache in front of the packagers) means the origin sees, per rung, roughly one request per part per shield, not one per viewer.

5. Edge packaging

"Edge packaging" can mean running the packager at hundreds of CDN points of presence (PoPs: the many edge data centers a CDN operates close to viewers). I would not do that for this event: it multiplies compute by the number of PoPs, and it risks two PoPs producing slightly different manifests for the same moment. I would package centrally at origin (CMAF once) and use edge compute only for thin, per-request work: validating the signed playback token, rewriting manifest URLs to steer to a CDN, and optionally inserting ad markers. That keeps the cacheable media identical everywhere.

6. Player buffer strategy

• Target latency about 3.7 s (matching the glass-to-glass budget below), buffer target about 1.5 s behind the live edge (three parts).
• Catch-up: when the player drifts more than about 0.5 s past target (after a stall), play at up to about 1.05x until back on target; below target, slow slightly instead of stalling. Sports audiences notice a stall far more than a 5% speed change, and much less than a 20 s lag behind the TV broadcast.
• Conservative ABR at the live edge: with only 1.5 s of buffer there is no room for a bad guess, so switch down fast and up slowly, and estimate bandwidth per part rather than per segment (a part that arrives at encoding speed tells you nothing about spare bandwidth, a known low-latency ABR trap).
• Start-up: join at the most recent independent part (a part that starts exactly on a keyframe, so playback can begin there without needing any earlier part), not the start of the segment, so the first frame shows quickly.

Worked example: capacity and the latency budget

Egress. Weighted average bitrate from the ladder above:

Over a 3-hour event at that level: 7.836 Tbps / 8 = 0.9795 TB/s, times 10,800 s, is about 10,580 TB (about 10.6 PB) delivered. Egress is the dominant cost line, which is why the design protects cacheability.

Requests. With 0.5 s parts, each viewer fetches 2 media objects per second and makes about 2 blocking playlist requests per second: 2M x 4 = 8M requests/s at the edge. Behind the shield, with request collapsing working, the origin sees about 5 rungs x 2 parts/s = 10 media requests per second per shield node, plus a similar number of playlist requests, instead of millions. The CDN carries the audience; the origin does not.

Glass-to-glass latency budget (allocations I design to and then measure in rehearsal with a clock burned into the video):

That leaves 1.3 s of headroom under 5 s for a slow last mile (the final network hop from the CDN edge to the viewer's device) or a single rebuffer-and-catch-up.

Trade-offs and pitfalls

• Why not WebRTC? It gives sub-second latency, but it is delivered through media servers (SFUs, selective forwarding units) rather than HTTP caches, so 2M viewers means a very large, stateful server fleet and no standard studio DRM. Sub-second latency is only worth that cost for interactive formats (betting inside the stream, watch-parties with talkback). For a broadcast with a 5 s target it is the wrong tool.
• Latency vs rebuffering. Every second removed from the player buffer is a second less protection against a bandwidth dip. If rehearsal shows high rebuffer rates on mobile, I would move to 4 s target latency before shrinking parts further.
• Parts smaller than 0.5 s multiply request rate (0.25 s parts means 8 requests/s per viewer, 16M/s total) and push CDNs harder for little perceived gain.
• Common wrong turns: quoting a segment duration without the hold-back rule; forgetting that an intermediate cache which buffers whole objects breaks chunked delivery; single-CDN "because they said they have capacity"; timecode not aligned across redundant chains, which makes failover a visible jump.
• What would change the design: a hard requirement below about 2 s (then WebRTC or a WebRTC-based distribution tier for the interactive subset), or a global audience with no dominant region (then more CDN capacity reservation per region and region-local packagers).

Direct answer

Do not rely on purging caches to enforce a takedown. Purging removes copies; it does not stop a player that already holds a manifest and a valid token from requesting the next segment. Instead, make availability a policy check that runs at the CDN edge on every request, driven by a small, versioned rule set (a numbered list of block/allow decisions, stored as tiny key-value entries like movie-123: blocked in DE since v42, so shipping an update means pushing a few hundred bytes, not purging gigabytes of cached video) that the control plane (the internal system that configures and coordinates the CDN itself, separate from the "data plane" that actually serves video) pushes to all edges in well under two minutes. Purge only the small, mutable objects (manifests and the title's metadata), use short-lived tokens plus a deny rule to cut off in-flight sessions, and let the origin return a cacheable "unavailable" answer as the backstop. Every step writes to an append-only audit log, and a set of probes in each region proves, with timestamps, that the content is actually blocked.

Terms

• CDN (content delivery network): edge caches that serve video close to viewers. Edge means those cache servers; origin is your own storage and packaging behind them.
• Purge / invalidation: telling the CDN to drop cached copies so the next request goes to origin. Purge by tag (also called surrogate key or cache tag) drops every object labelled with a tag in one call, instead of listing URLs.
• Cache-busting: changing the URL (for example adding a version) so old cached copies are simply never requested again.
• Rights window: the dates a title may be shown in a territory. A geo-restriction limits a title to certain countries.
• Geo-IP: mapping a client's IP address to a location using a commercial database.
• HTTP 451: the status code "Unavailable For Legal Reasons", meant for exactly this kind of block.

Architecture

flowchart LR
  L[Legal or rights system] -->|change request| R[Rights service<br/>source of truth]
  R -->|rule v42| Q[Control-plane bus]
  Q --> E[Edge policy store<br/>all PoPs]
  Q --> A[Playback API and<br/>licence server]
  Q --> P[Purge worker<br/>by tag, rate-limited]
  E -->|ack v42| AU[(Append-only audit log)]
  P --> AU
  V[Regional probes] --> AU

(PoP: point of presence, a CDN site.)

Enforcement points, and why each exists

My choice: enforce at the edge, backstop at origin. Origin-only enforcement means correctness depends on every cached copy expiring or being purged within two minutes, which puts you at the mercy of purge rate limits and cache TTLs (time-to-live). Edge enforcement makes the purge merely a clean-up step. The origin still checks, so a misconfigured edge fails safe.

The two-minute budget

Only four rows sit on the critical path: 10 + 5 + 45 + 30 = 90 s. The playback-API/licence step and the purge step run in parallel and finish well inside that 90 s, so they cost nothing extra. Add the 30 s margin and the budget totals the full 120 s. The budget closes because the only thing on the critical path is pushing a few hundred bytes of policy to the edge, not removing gigabytes of cached video.

Worked example: why purge-by-URL does not fit

A 2-hour title in 6 renditions with 4 s segments has 7,200 / 4 = 1,800 segments per rendition, so 6 x 1,800 = 10,800 segment URLs, plus 6 rendition playlists and 1 master manifest: 10,807 URLs. Purge APIs are rate-limited per account, so purging ten thousand URLs per title (and possibly many titles in one rights change) can blow the budget and crowd out other purges. Tagging every object for the title with title:movie-123 turns this into one purge call. And with edge enforcement you do not even need to purge segments for a takedown: a blocked request never reaches the cached segment.

Handling each kind of change

• Immediate takedown: edge rule "blocked everywhere" plus content deny for tokens, licence server refuses renewals, purge manifests by tag. In-flight viewers stop at their next segment request, within seconds of the rule landing.
• Rights-window change: schedule the rule with an effective time (blocked in FR from 2026-10-01T00:00Z) pushed hours ahead, so edges switch on their own clock at the exact moment instead of racing a push. For a window opening, the same mechanism makes content available; cache-busting the manifest URL avoids viewers receiving a stale cached "unavailable" answer.
• Geo-restriction: the edge rule is keyed by country. The decision uses the country the edge computes for the request, and the same rule is applied at token issuance, so a token is never issued for a blocked territory in the first place.

Geo-IP reliability caveat

Geo-IP is a best-effort signal, not proof of location:

• VPNs and proxies make a viewer appear to be in another country.
• Mobile carriers often route traffic through gateways in another region or country, so a legitimate viewer can be mislocated.
• Corporate networks and satellite links exit the internet far from the user.
• Databases are updated periodically, so reassigned IP ranges are wrong for a while.

Consequences for the design: combine geo-IP with the account's billing country and, on mobile, optional device location for high-value restrictions; block known VPN and hosting-provider ranges where contracts require it; accept that a small error rate in both directions exists and say so. In the audit trail, record which geo signal was used and the database version, so a compliance report states "blocked for requests geolocated to DE by database version X" rather than claiming every viewer in Germany was physically blocked, which no IP-based system can prove.

The compliance audit trail

An append-only log (write-once storage with object lock, so entries cannot be edited or deleted during the retention period) records, for each change:

1. Request: who requested it, the legal reference (for example the takedown notice id), scope (titles, territories), time received.
2. Decision: who approved it, the rule version committed, commit timestamp.
3. Propagation: each edge PoP's acknowledgement of the rule version with its timestamp; the playback API and licence server acknowledgements.
4. Purge: the purge request ids and the CDN's responses.
5. Verification: probe results from each region (URL, status 451, timestamp, geolocated country).
6. Completion: time the last acknowledgement arrived, which gives the measured time to effect for this change.

That record is what you hand over when asked "prove this was unavailable from time T". Alert when any PoP has not acknowledged within 60 s, and treat an unacknowledged PoP as a breach until proved otherwise.

Trade-offs: origin load and rate limits

• Mass purges cause origin stampedes: if a huge purge empties caches for content that remains available elsewhere, every edge refetches from origin at once. Purge narrowly by tag, protect origin with a shield tier (a mid-layer cache that collapses duplicate misses), and do not purge segments at all for takedowns.
• Cache the "no": origin's 451 responses should be cacheable for a short TTL (for example 30 s) so blocked titles do not hammer origin with misses.
• Purge API rate limits are a hard ceiling; queue purges, prioritise legal takedowns over routine updates, and batch by tag.
• Offline downloads cannot be removed within two minutes: a device with no network never hears about the change. They stop at the next licence renewal or check-in, so titles at risk of takedown should carry short offline licence durations. State this limit in the compliance policy instead of implying two-minute coverage for offline copies.
• Edge compute on every request has a per-request cost; keep the rule lookup a local key-value read.