Caching Strategies & In-Memory Optimization Questions

Which query params belong in the key

Only parameters that change the response body belong in the cache key. For a search endpoint with ?q= and ?page=, both change what's returned, so both belong. A ?utm_source= tracking parameter never changes the response, so it must be stripped before key construction; otherwise every unique marketing link becomes a guaranteed cache miss for content that is actually identical.

Canonicalizing and normalizing

• Sort query parameters into a fixed order before using them as a key, so ?page=2&q=shoes and ?q=shoes&page=2 land on the same cache entry.
• Normalize encoding and case where the parameter is genuinely case-insensitive, %20 versus +, uppercase versus lowercase.
• Strip a known allowlist of non-affecting params rather than trying to guess. An allowlist of "these params matter" is safer than a blocklist of "these don't," since a blocklist silently breaks the moment a new response-affecting param is added and someone forgets to exempt it.

Cookies and personalization leakage

Cookies, session ids, auth tokens, personalization state, generally must NOT be part of the cache key for a shared edge cache, since that explodes the key space to roughly one entry per user, defeating the point of caching.

• If a response genuinely varies by cookie, such as a personalized homepage, mark it uncacheable at the shared edge and let it cache client-side (a private, per-browser cache) at most, or cache a cookie-free "shell" and personalize it client-side after load.
• If a cookie is present but doesn't affect this specific response, such as an analytics cookie, it must be excluded from the key even though it's technically there.
• The classic bug is a proxy or CDN (content delivery network) respecting a Vary: Cookie response header literally, which key-splits by the raw cookie header, nearly always unique per user, and silently turns the cache into a per-user pass-through with an effective 0% hit rate. Use Vary only on the specific header that actually changes the response, such as Accept-Language, and handle auth/personalization by not caching at the shared layer at all rather than via Vary: Cookie.

Definition

TTL (time-to-live) is the duration a cached entry is considered valid before it is treated as expired and either removed or refreshed. Most caches implement this lazily: the entry stays in memory past its TTL until something actually reads it, at which point the read notices the timestamp has passed and treats it as a miss, rather than a background process constantly scanning for expired entries.

Short vs long TTL trade-off

• Short TTL: data stays fresher, lower risk of serving stale values, but the cache hit rate drops and backend/database load rises, since entries expire and get re-fetched more often.
• Long TTL: higher hit rate and lower backend load, but a write to the source of truth can take up to the full TTL to be reflected to cache readers, worse staleness.

Worked example

A product-price endpoint gets 1,000 requests per minute for the same 50 hot products. With a 60-second TTL, each hot key gets re-fetched from the database roughly once per minute, about 50 database reads per minute total, no matter how many of the 1,000 requests hit it (the other roughly 950 are cache hits). Drop the TTL to 5 seconds and the same key gets re-fetched roughly 12 times more often, about 600 database reads per minute, for freshness that in most product-price use cases nobody needed.

Sliding vs fixed TTL

• Fixed TTL: the expiry time is set once, at write time, and does not change no matter how often the entry is read. Simple and predictable, but a very hot key still expires and re-fetches on schedule even though it's clearly still wanted.
• Sliding TTL (refresh on access): every read extends the expiry, so a hot key effectively never expires as long as it keeps being read, while a key nobody reads anymore ages out and frees memory on its own. The risk is that a key read often enough can now live forever, so sliding TTL is usually paired with a hard cap.

Redis/Memcached examples

Redis: SET key value EX 60 sets a fixed 60-second TTL; calling EXPIRE key 60 again on every read is how you'd implement sliding TTL manually, since plain GET doesn't refresh it (GETEX in newer Redis versions can set a new TTL as part of the read). Memcached: the expire parameter on set works the same as Redis's fixed TTL, and it also has no native sliding-TTL primitive, so it has to be done by re-issuing set/touch on read.

When to use which

Use fixed TTL by default, simpler, predictable memory pressure. Reach for sliding TTL for session-like data, such as an authenticated session token, where "still being used" should mean "still cached," and pair it with a hard maximum lifetime so an endlessly-active session can't cache forever.

Split shared data from personalized data first

The 100:1 read-to-write ratio makes this a strong caching candidate, but the personalization field is the part that needs the most care: don't cache the whole response, including is_in_user_wishlist, under one key. Cache the parts that are the same for every user separately from the part that's specific to one user.

Cache key design

Cache the product detail (name, price, description, the joined data that currently takes ~300ms) under a key like product:{id}:v{schema_version}, containing no user-specific fields at all. The version segment lets you change the shape of what's cached (add a field, change a join) without colliding with the old cached shape; bump it and old entries simply age out. Fetch is_in_user_wishlist through a separate, small, fast lookup, either its own tiny cache entry (wishlist:{user_id}:{product_id}) or a targeted call against the much smaller wishlist table, and merge the two server-side before responding. Baking the wishlist flag into the shared product cache would force you to invalidate that shared, expensive-to-rebuild entry every time any user toggles their wishlist, which defeats the point of caching it, or worse, risks serving one user's personalization to another if a key isn't scoped correctly.

TTL and invalidation

Given the 100:1 ratio, writes to a product are relatively rare, so a TTL (time-to-live: how long the cached entry is kept before it's treated as expired) in the minutes range (5-15 minutes) is a reasonable safety net. But the primary correctness mechanism should be explicit invalidation: when a product update happens (price, description, etc.), delete or refresh product:{id}:v{schema_version} synchronously as part of that write path. That way a customer never sees a stale price for up to the full TTL window, TTL is just a backstop against invalidation bugs, not the thing you rely on for correctness.

Expected effect on p99

Caching only helps the shared, expensive part of the response: on a cache hit, the ~300ms database join collapses to a cache lookup (typically low single-digit milliseconds), plus the small added cost of the separate wishlist lookup. With a high hit ratio (most traffic reusing the same popular products), the large majority of requests should see something close to that low-single-digit-millisecond total instead of 300ms, which is what actually moves p99 (the 99th-percentile response time, the value below which 99% of requests complete). Treat that as a directional expectation to validate with real hit-ratio and latency dashboards once it's live, not a guaranteed number, since the actual mix of popular versus long-tail products in real traffic determines the real hit ratio.

Why GraphQL breaks normal HTTP caching

Most GraphQL traffic is a POST to a single /graphql endpoint with a query body, so you lose the free caching that comes with GET requests and distinct URLs (a CDN or browser cache normally keys on the URL). Two clients asking for different fields of the same object also get different response shapes, so caching "the response" as one blob has almost no reuse across users.

1. Persisted queries

A persisted query replaces the full query text with a short hash. The client first sends only sha256(query). If the server already has that hash mapped to the full query (from an earlier request, or pre-registered at build time), it executes immediately; if not, it returns a "not found" error and the client resends the full query once, which the server then stores. Two wins: the request body shrinks to a hash plus variables, and because the hash is small and deterministic you can send it as a GET query parameter, which makes the response cacheable by an ordinary CDN or browser cache the same way a REST GET is.

2. Cache keys: root-level vs field-level

Root-level means the cache key is hash(operation) + variables + caller-identity, and the ENTIRE response is stored under that one key. Simple, but reuse is poor: add one field to the query, or change one variable, and you get a full cache miss even though most of the underlying data is unchanged.

Field-level (normalized) caching keys each resolved object independently, typically Type:id:field(args), e.g. User:42:posts(first:10). A gateway resolves a query by composing these smaller cached entries, so a query that reuses User:42 from a different operation is a hit even though the two operations look nothing alike as documents. This is what production GraphQL gateways actually do via cache-control directives per field or type, each declaring a max age and a scope (PUBLIC or PRIVATE).

3. Response fingerprinting

Fingerprinting means deriving the cache key deterministically from the operation, not from the raw request text, so two requests that mean the same thing hash to the same key. In practice: normalize variable ordering (sort keys before hashing), strip formatting differences in the query text (or better, always operate on the persisted-query hash, which already normalizes this), and include only inputs that actually affect the resolved data.

4. Partial-field caching

A single query commonly mixes cheap, cacheable fields (product name, price) with expensive, personalized ones (recommended-for-you). Because the cache operates at the field level, the gateway fills the cacheable fields from cache and only invokes the resolver for the personalized field, then merges both into one response. This is the real payoff of field-level over root-level caching: partial hits instead of all-or-nothing.

5. Query whitelisting

For a public-facing API, combine persisted queries with an allowlist: the server only executes operation hashes registered ahead of time, and rejects anything else. This bounds the set of distinct cache keys you'll ever see and closes off a denial-of-service vector where an attacker crafts a maximally expensive nested query on the fly.

6. Authorization and Vary behavior

A GraphQL response is shaped per caller, so caching by operation plus variables alone can leak one user's data to another if two users happen to ask the same question. Treat identity the same way HTTP treats Vary: Authorization or Vary: Cookie: any field marked PRIVATE must fold the caller's identity (or a normalized permission-scope hash) into its cache key, and PRIVATE fields should never sit in a shared edge cache, only in a per-caller layer such as an application Redis keyed by user id. PUBLIC-scoped fields carry no identity in their key and can sit safely at the edge.

Trade-off

Field-level caching gets real reuse but costs real complexity: invalidation has to fan out per changed entity (a mutation touching Post:99 must bust every cached field keyed by Post:99, not one blob), and you need a normalized store plus a gateway that composes partial hits. Root-level caching is simpler but only pays off for queries that are already nearly-static and shared across many callers.

Step-by-step checklist

1. Scope it first. Is this one instance or the whole fleet/all replicas? Is it correlated with a specific time of day, or random? That narrows the search dramatically before you touch a single command.

2. Check Redis's own built-in latency monitor. LATENCY HISTORY and LATENCY DOCTOR (or redis-cli --latency) bucket recent spikes by event type (a slow command, a fork, an expire cycle), which is usually the fastest way to see what kind of event caused the spike, before digging further.

3. Check the slow-command log. SLOWLOG GET lists recently executed commands that exceeded slowlog-log-slower-than (10ms by default). Look for known offenders: KEYS * (an O(N) full-keyspace scan that blocks the single command-processing thread, SCAN is the non-blocking alternative), or SORT/SMEMBERS/HGETALL on an unusually large key.

4. Pull INFO and check its sections:

   • Clients: connected_clients approaching maxclients, or a nonzero blocked_clients.
   • Memory: used_memory versus maxmemory, and mem_fragmentation_ratio well above roughly 1.5, which suggests fragmentation is slowing allocations.
   • Persistence: rdb_bgsave_in_progress, aof_rewrite_in_progress, and latest_fork_usec. A background save (RDB, a point-in-time snapshot) or AOF rewrite (append-only log, Redis's other persistence format) forks the process, and on a large dataset that fork's copy-on-write setup (the OS defers actually duplicating a memory page until either the parent or the child process writes to it, so a burst of page copies can land right when real traffic is writing) itself briefly pauses things, a classic cause of periodic, recurring spikes.
   • Stats: sudden jumps in expired_keys or evicted_keys, a burst of either costs real CPU.

5. Look for oversized keys. redis-cli --bigkeys (or MEMORY USAGE <key> on a suspect) finds an unusually large hash, set, or list; since Redis runs most commands on a single thread, one giant key makes any command touching it slow, and stalls everything queued behind it.

6. Check the OS level. CPU and steal time (top; steal time is the percentage of time this VM wanted to run but the hypervisor gave the CPU to another tenant instead, especially relevant on a shared/virtualized host), memory pressure and whether the host has started swapping (catastrophic for a latency-sensitive in-memory store), and disk I/O stats if persistence is enabled, since a slow disk can stall the fsync (forcing the OS to actually flush buffered writes to disk instead of leaving them in memory) or fork path.

7. Review config flags. maxmemory-policy (a wrong policy can cause eviction storms), appendfsync (always is much slower and safer than everysec), and the slowlog/latency-monitor thresholds themselves, to make sure they're actually catching what you'd expect.

8. Shortlist the likely cause from what the above turned up: a large or growing key, an O(N) command run by some script, a background save/rewrite fork on a big dataset, memory pressure triggering eviction, or noisy-neighbor contention at the OS level.