Approach & Goals
I’d prioritize writes under 50–100ms, serve reads at high QPS with eventual consistency, and provide clear API semantics so SDK users understand staleness vs freshness trade-offs. Success metrics: write latency, 99th read latency, feed freshness (median staleness), and cache hit rate.
Architecture (high level)
- Ingest: write API → front-tier → write queue (Kafka) for async processing.
- Storage: user timeline store (denormalized), follower index, canonical activity store.
- Read path: cache (CDN + edge cache + Redis), read API that queries cache then fallback to timeline store.
Fan-out trade-offs
- Fan-out-on-write (push): upon write, fan-out to follower timeline shards (fast reads, higher write IO). Use for users with small follower counts; heavy hitters use hybrid.
- Fan-out-on-read (pull): store canonical activity; construct feed at read time (lower write cost, higher read latency). Use for celebrities and cold followers.
- Hybrid: push to active followers; mark heavy followers for pull. Maintain per-user fanout strategy stored in metadata.
Caching & Denormalization
- Denormalize feed entries (activity + actor metadata) at write to avoid joins.
- Multi-layer caching: CDN for public posts, edge caches for user feed snapshots, Redis for hot users. Use cache invalidation via asynchronous events (invalidate or update on write).
- TTL and background refresh to bound staleness.
Ordering & Consistency
- Provide weak ordering: causal ordering for a single user’s posts by timestamp + sequence number; across users ordering is best-effort (server timestamps + optional global monotonic token).
- Expose fields: event_timestamp, source_sequence, ingested_at.
API Contract for Clients
- Endpoints:
- POST /activities — returns 202 + activity_id and ingestion_token.
- GET /feeds/{user}?limit=&cursor=&consistency=strong|eventual|freshness_ms
- consistency param:
- strong (blocks until activity is visible or times out),
- eventual (default, low latency),
- freshness_ms (serve data no older than X ms if possible).
- Webhooks/notifications for push updates and optional long-poll for near-real-time.
- Response: items[], next_cursor, served_from (cache|store), server_time, max_staleness_ms.
Documenting Consistency for SDK Users
- Provide an “API Consistency” section in docs with:
- Clear definitions (eventual vs strong vs freshness).
- Typical latency numbers and SLOs (e.g., 95% of writes visible in <2s for eventual).
- Examples & code snippets using consistency param.
- Best practices: optimistic UI (show local pending activity), subscribe to webhook/WS for confirmation, fallbacks for strong reads.
- SDK helpers: optimistic updates, re-fetch strategies, and metrics hooks for monitoring staleness.
Product trade-offs
- Push-heavy wins UX (fast reads) but higher infra cost; pull-heavy saves cost but increases client complexity.
- Offer hybrid as managed feature and surface cost/latency implications in pricing/tiers.
This balances product UX, developer experience, and operational cost while making consistency explicit and controllable for SDK users.
