Caching Strategies & In-Memory Optimization Questions

Five metrics

1. Time to First Byte (TTFB), machine-centric: how long the browser waits for the first byte of the response after making the request. Directly reflects whether the response was served from a cache, CDN (content delivery network) or browser, versus computed fresh at origin; a big drop in TTFB after adding caching is the clearest proof the cache is working.
2. First Contentful Paint (FCP), user-centric: when the first visible content appears on screen. Matters because it's the user's first evidence the page is loading at all, and is improved by caching the assets, CSS, fonts, hero image, that block that first paint.
3. Largest Contentful Paint (LCP), user-centric: when the largest, usually most meaningful, content element finishes rendering. This is a Core Web Vital that Google uses as a ranking and UX signal, so it matters to business outcomes like search ranking and perceived load speed, not just engineering pride.
4. Cache hit ratio, machine-centric: the fraction of requests served from cache versus origin. The direct internal measure of whether the caching configuration is effective, explaining WHY the other three numbers moved.
5. Repeat-visit load time, user-centric: how fast the page loads for a user who has already visited, and so already has browser-cached assets, versus a first-time visitor. This is specifically where caching's payoff should show up most; if repeat visits aren't meaningfully faster than first visits, the caching strategy likely isn't working as intended.

Why they matter to business/UX outcomes

The user-centric metrics (FCP, LCP, repeat-visit load time) are what a real person actually experiences and what correlates with bounce rate and conversion. The machine-centric ones (TTFB, hit ratio) are the diagnostic instruments that explain why the user-centric numbers move, so a caching change should be judged by both: did the internal metric improve, and did that translate into a user-facing improvement? A high hit ratio that doesn't move LCP means the wrong things are being cached.

Comparison

Choosing between them

The deciding questions are: does the SERVER need to read this automatically, only cookies do that; does it need to survive the tab closing, localStorage and IndexedDB yes, sessionStorage no; and how much data, a few KB fits anywhere, hundreds of KB to MB pushes you toward IndexedDB. Avoid putting an auth token in localStorage/sessionStorage when possible: an HttpOnly cookie is the only one of the four immune to a JavaScript-based XSS read.

Why this happens

Hashed assets, a filename like app.a1b2c3.js that changes whenever the content changes, let each version be cached forever at the edge, since the URL itself uniquely identifies the content and no invalidation is ever needed for that specific file. The failure mode in this question happens at the SEAM between deploys: a new HTML page referencing the new hashed asset URLs can go live before the CDN has actually cached those files, so the first requests are cache misses reaching the origin, and if the origin isn't ready yet, still finishing its own deploy, or overwhelmed by a burst of simultaneous first-requests from many users loading the new page at once, those requests 503 or 404.

Warm-up strategy

1. Deploy assets to the origin BEFORE flipping the HTML. The new hashed files must exist and be servable from the origin before any page references them; assets first, then the page that points at them. This ordering rule is the single most important step, since nothing downstream matters if the origin isn't ready.
2. Pre-populate the CDN cache with an explicit warm-up pass, hitting each new asset URL through the CDN from the deploy pipeline itself, from multiple regions if the edge network is geographically distributed, BEFORE flipping traffic to the new page, so the cache is already populated by the time real users arrive rather than every region's first real user paying a cold-cache miss.
3. Use origin shielding, a mid-tier cache layer between edge points of presence and the true origin, so that even if many edge nodes each experience their own first miss for a new asset, they collapse into ONE request to the shield instead of every edge node independently hammering the origin for the same new file, the CDN-layer version of preventing a stampede.
4. Serve with a short grace period during the warm-up window: configure a couple of retries with brief backoff on a failed origin fetch before returning an error, so a request arriving in the brief window while the shield's cache is still populating gets a successful, if slightly delayed, response instead of a hard failure.

Avoiding cache fragmentation

Fragmentation here means the same logical asset ending up cached inconsistently across edge nodes because of how it was requested, differing query strings, or an inconsistent Vary header causing the CDN to store multiple copies keyed differently for what should be the same response. Serve hashed assets from a single, consistent path with no query strings, the hash already lives in the filename, so a separate cache-busting query parameter is unnecessary and fragments caching if mixed with hashed filenames, and ensure the origin sends a consistent, cacheable Cache-Control header, a long max-age plus immutable, for every response to that path, so the CDN treats every request for that exact URL identically regardless of which edge node it came through.

Gracefully serving during the warm-up window

Keep the PREVIOUS deploy's assets servable rather than deleting them immediately, only reclaiming them once client lag has plausibly passed, so a user whose browser still has a stale HTML page cached client-side, and requests an OLD asset hash during the transition, still resolves successfully instead of 404ing. For the NEW assets specifically, the origin-shield retry and backoff above covers the brief real warm-up gap.

Trade-off

Running an explicit warm-up pass from the deploy pipeline adds a step, and a small amount of deploy-time latency, before traffic is allowed to see the new deploy, versus flipping traffic immediately and accepting a burst of cold-cache misses. The warm-up pass is worth it because this failure mode is customer-visible and happens at the worst possible time, right after you just shipped something.

Framing

Multi-tenant means many customer organizations share the same application infrastructure, but each tenant's data must stay logically isolated, and different tenants can reasonably want different trade-offs: a tenant on a real-time collaboration plan might need near-immediate invalidation, a tenant on a reporting-only plan might be fine with minutes of staleness. The caching policy has to support that variation without becoming a different codebase per tenant.

Storage layout

Namespaced keys in a shared cache, tenant:<tenant_id>:<entity>:<id>, is the right default: it gets the operational simplicity of one cluster, one thing to monitor, scale, and patch, while keeping tenants logically separated by key prefix. Reserve a fully separate cache instance per tenant for the small number of tenants whose scale or isolation needs justify the extra operational cost, a very large enterprise tenant whose traffic could otherwise crowd out smaller tenants sharing the cluster, rather than making that the default for everyone.

Invalidation flows, per tenant

Attach a per-tenant CONFIGURATION, not per-tenant code, controlling default TTL and whether writes trigger explicit invalidation or rely on TTL expiry alone. A tenant needing near-real-time consistency gets a short TTL plus explicit delete-on-write invalidation; a tenant fine with staleness gets a longer TTL and no explicit invalidation at all, cheaper, since explicit invalidation is itself a nontrivial code path to keep correct, a bug there can leak or corrupt another tenant's cached data if the logic is sloppy. The invalidation LOGIC stays shared code; only the CONFIGURATION varies per tenant.

Cost controls

Set a per-tenant memory quota, measured via that tenant's key-prefix footprint, and ENFORCE it, not just monitor it, so one tenant's unusually large or hot dataset can't push a smaller tenant's data out of the shared cache under eviction pressure. Track cache cost per tenant, roughly memory used times time, if the product needs to attribute infrastructure cost back to specific accounts, which the namespaced key layout makes straightforward to sample and measure by prefix.

Tenant-scoped purge, done efficiently

Purging everything for one tenant by scanning and deleting every key matching that tenant's prefix works but is proportional to that tenant's key count, and can be slow for a large tenant. The efficient approach: include a per-tenant VERSION number as part of the key, tenant:<tenant_id>:v<version>:<entity>:<id>, so a full-tenant purge becomes incrementing that tenant's version counter, a single, cheap write, after which every OLD key for that tenant is orphaned, no code path reads a stale version, and can be reclaimed lazily via TTL expiry rather than an immediate bulk delete. It's the same idea as reclaiming old release-versioned cache keys: bump a version rather than delete each key individually.

Trade-off

Per-tenant configuration, rather than one global TTL and invalidation policy for everyone, costs real complexity: more configuration surface, more test cases, a bug might only show up for one tenant's specific TTL and invalidation combination, and the version-counter purge trades immediate key reclamation for lazy TTL-based cleanup. That complexity is worth it specifically because tenants genuinely have DIFFERENT consistency requirements, so a single global policy would either be too aggressive for tenants who don't need it or too loose for tenants who do.

How normalization works

A non-normalized GraphQL client would cache each query's response as one opaque blob, keyed by the query and its variables. A normalized cache (Apollo's normalized cache is the common example) instead flattens every object returned by ANY query into one flat store keyed by a stable identity, typically __typename:id (for example Product:42), and each query's cached result becomes a set of REFERENCES into that shared store rather than a duplicated copy of the object.

Worked example: why this raises the hit rate

Query A, getProduct(id: 42), returns { __typename: "Product", id: 42, name: "Widget", price: 9.99 }. Query B, searchProducts(...), returns a list that happens to also include product 42 among its results. Without normalization these are two separate cached blobs; if the price changes, only whichever query re-runs picks up the new value, and the OTHER screen keeps showing the old price until it also happens to refetch. With normalization, both query results point at the SAME Product:42 entry in the flat store, so updating that one entry, whether from a mutation's response, a subscription push, or a manual cache write, updates every screen currently displaying product 42, even ones reached through an entirely different query. This also means a query whose entities are ALL already present in the store can sometimes be served straight from cache even if that exact query shape has never run before, and a mutation response can be written directly into the store (for example, bumping a cached post's like count after likePost(id)) without a network refetch just to reflect an obvious, already-known update.

Additional steps for offline sync

Persist the normalized store to on-device storage (for example SQLite or the platform's key-value storage via a persistence link) so the app has real data to show on a cold launch with no network at all. Queue mutations made while offline in a durable local outbox, and replay them in order once connectivity returns, rather than losing them or silently dropping them.

Additional steps for conflict resolution

A queued offline mutation can end up applying to data that's now stale, someone else (or another of the same user's devices) changed the same entity while this device was offline. Options, in increasing complexity: last-write-wins (simplest, accepts the risk of silently overwriting a concurrent change); a server-side version or ETag check that rejects a mutation built on stale data and forces the client to refetch and let the user redo the edit against current data; or field-level merging for cases where two concurrent edits touch different fields of the same entity and both can be safely kept.

Additional steps for cache eviction

A normalized store can grow without bound as a user browses, every product ever viewed stays in the flat store forever by default, since nothing automatically knows it's no longer needed. Mobile specifically needs an explicit eviction policy: garbage-collect entities no longer referenced by any currently-active query result (Apollo's cache exposes a gc() call for exactly this), and cap the total persisted cache size so browsing doesn't quietly consume an unbounded amount of the device's limited storage over time.