Caching Strategies & In-Memory Optimization Questions

Framing

GDPR (General Data Protection Regulation) caching constraints boil down to two things a naive cache design gets wrong: caches routinely outlive the legal basis for holding data (a TTL (time to live, how long a cached entry is kept before it's treated as expired) that's "whatever felt reasonable for performance" instead of tied to data-retention policy), and a "delete my data" request has to actually reach every copy, not just the primary database, including copies sitting in application caches, CDN edges, and any derived/aggregate caches built from that data.

Tokenization and encryption

• Don't cache raw PII (personally identifiable information) where a pseudonymous token will do. Cache by an internal user ID or a reversible token, and resolve the token to real PII only at the point of use, ideally never storing the mapping in the cache layer itself, only in the system of record with its own access controls.
• Where PII must be cached (e.g. an address for shipping-estimate lookups), encrypt it at rest in the cache using an application-managed key, not just relying on the cache infrastructure's disk encryption, so a cache dump or misconfigured cache instance doesn't expose plaintext PII.
• Field-level encryption lets you cache the non-PII parts of a record (product preferences, non-identifying settings) freely while treating the PII fields as a separate, more tightly controlled, shorter-TTL cache entry.

Cache scoping and TTLs

• Scope caches by data sensitivity, not just by convenience of the code path: a "user profile" cache that mixes a public display name with a private phone number forces the whole entry to inherit the stricter handling requirements of its most sensitive field. Split it: public/non-PII fields in one cache entry with a generous TTL, PII fields in a separate entry with a short TTL and encryption.
• Tie TTLs to actual retention policy, not just performance tuning. If the data-retention policy says an inactive user's data should be gone within 30 days of a deletion request, no cache entry for that data should be able to outlive that regardless of TTL jitter or an unlucky refresh timing; keep PII-bearing TTLs well under the compliance deadline, with margin.

Handling deletion requests across distributed caches and CDNs

This is the part naive designs miss entirely. A "right to erasure" request has to propagate to:

1. The system of record (straightforward, it's a database delete).
2. Every application-layer cache node holding that user's data (a distributed cache like Redis/Memcached across multiple app-cache clusters or regions).
3. CDN edge caches, if any response ever included PII (rare and usually avoidable by design, but user-specific API responses proxied through a CDN with caching enabled are a real risk).
4. Any derived caches: search indexes, recommendation feature caches, analytics rollups that embed the raw PII rather than an aggregate.

Approach:

• Maintain a per-user (or per-record) cache-key registry, or a deterministic key-derivation scheme (user:{id}:profile, user:{id}:address) so a deletion job can compute exactly which cache keys to purge without a full scan.
• On a deletion request, fire an explicit invalidation (DEL) against every distributed cache cluster/region for every derived key, plus a CDN purge API call for any edge-cached PII-bearing URLs.
• For caches you can't enumerate cheaply (e.g. sharded across many nodes with no central index), rely on short TTLs as the backstop and document the maximum propagation delay (e.g. "erasure completes within 24 hours") as part of the compliance process, since GDPR itself allows a reasonable completion window, it doesn't require synchronous global deletion.
• Log every purge action with a timestamp for audit purposes; GDPR compliance requires being able to demonstrate the request was honored, not just that it probably was.

Worked example

User requests deletion. The service: (1) marks the user record deleted in the system of record, (2) computes the deterministic key set (user:42:profile, user:42:address, user:42:recommendations) and issues DEL against the Redis cluster in every region within seconds, (3) calls the CDN purge API for the handful of user-specific edge-cached URLs, (4) relies on a 15-minute TTL as the backstop for any cache node that didn't get the explicit purge (e.g. a transient network partition), (5) writes an audit log entry with all four steps' completion status.

Trade-offs

Explicit purge-on-delete is more engineering work than just setting a short TTL and waiting, but a TTL-only approach means you're implicitly claiming every PII-bearing cache always expires within the TTL, which is a much weaker and harder-to-audit compliance story than actively purging it and logging the purge, with the TTL kept only as a backstop.

Direct answer

Build a health check that pulls the same fields an operator would read from redis-cli INFO (plus a latency sample) and scores five things: memory usage against the configured limit, eviction rate, replication lag, command latency at the 95th percentile (p95), and connection saturation, then rolls them up into an overall status an orchestrator can act on.

Structured elaboration

Checks and how to source them:

• Memory usage: used_memory versus maxmemory from redis-cli INFO memory. High usage alone isn't automatically bad if the instance is configured to evict, but combined with unexpected evictions (below) it signals real pressure.
• Eviction rate: the delta in evicted_keys between two samples, from redis-cli INFO stats. Whether evictions are a problem depends entirely on intent, expected and fine for an instance explicitly configured as an LRU (least-recently-used) cache with an eviction policy, but a red flag if this Redis instance is meant to behave like a durable data store with no eviction policy configured.
• Replication lag: the gap between master_repl_offset on the primary and slave_repl_offset reported by the replica, from redis-cli INFO replication. A growing gap means the replica is falling behind and would serve stale data (or be a worse failover target) if promoted right now.
• Command latency p95: sampled via redis-cli --latency-history, or read from INFO commandstats / the LATENCY HISTORY command. A rising p95 usually means the instance is CPU-bound on a slow command, or is contending with a large maxmemory eviction pass.
• Connection saturation: connected_clients versus maxclients from redis-cli INFO clients. Approaching the limit means new client connections will start being rejected outright.

Suggested threshold starting points (common planning defaults, not a universal SLA (service level agreement, a contractual performance commitment), tune to your workload):

Worked example

def evaluate_redis_health(info):
    '''info: fields you'd parse out of `redis-cli INFO`, plus a p95 command
    latency sample from `redis-cli --latency-history` or INFO commandstats.'''
    checks = []

    mem_pct = info["used_memory"] / info["maxmemory"]
    if mem_pct >= 0.90:
        checks.append(("memory_usage", "CRITICAL", f"{mem_pct:.0%} of maxmemory"))
    elif mem_pct >= 0.80:
        checks.append(("memory_usage", "WARN", f"{mem_pct:.0%} of maxmemory"))
    else:
        checks.append(("memory_usage", "OK", f"{mem_pct:.0%} of maxmemory"))

    evictions = info["evicted_keys_delta"]
    if info["expected_evictions"]:
        checks.append(("eviction_rate", "OK", f"{evictions}/min, policy allows eviction"))
    elif evictions >= 1000:
        checks.append(("eviction_rate", "CRITICAL", f"{evictions}/min, unexpected and sustained"))
    elif evictions > 0:
        checks.append(("eviction_rate", "WARN", f"{evictions}/min, unexpected for this instance"))
    else:
        checks.append(("eviction_rate", "OK", "0/min"))

    lag_bytes = info["master_repl_offset"] - info["slave_repl_offset"]
    if lag_bytes >= 1_000_000:
        checks.append(("replication_lag", "CRITICAL", f"{lag_bytes} bytes behind"))
    elif lag_bytes >= 100_000:
        checks.append(("replication_lag", "WARN", f"{lag_bytes} bytes behind"))
    else:
        checks.append(("replication_lag", "OK", f"{lag_bytes} bytes behind"))

    p95 = info["command_latency_p95_ms"]
    if p95 >= 50:
        checks.append(("command_latency_p95", "CRITICAL", f"{p95}ms"))
    elif p95 >= 5:
        checks.append(("command_latency_p95", "WARN", f"{p95}ms"))
    else:
        checks.append(("command_latency_p95", "OK", f"{p95}ms"))

    conn_pct = info["connected_clients"] / info["maxclients"]
    if conn_pct >= 0.90:
        checks.append(("connection_saturation", "CRITICAL", f"{conn_pct:.0%} of maxclients"))
    elif conn_pct >= 0.80:
        checks.append(("connection_saturation", "WARN", f"{conn_pct:.0%} of maxclients"))
    else:
        checks.append(("connection_saturation", "OK", f"{conn_pct:.0%} of maxclients"))

    severity_rank = {"OK": 0, "WARN": 1, "CRITICAL": 2}
    overall = max(checks, key=lambda c: severity_rank[c[1]])[1]
    return checks, overall


sample_info = {
    "used_memory": 8_600_000_000, "maxmemory": 10_000_000_000,
    "evicted_keys_delta": 0, "expected_evictions": False,
    "master_repl_offset": 5_213_442, "slave_repl_offset": 5_090_112,
    "command_latency_p95_ms": 3.4,
    "connected_clients": 812, "maxclients": 1000,
}
checks, overall = evaluate_redis_health(sample_info)
for name, status, detail in checks:
    print(f"{name:22} {status:9} {detail}")
print(f"overall: {overall}")

Output:

memory_usage           WARN      86% of maxmemory
eviction_rate          OK        0/min
replication_lag        WARN      123330 bytes behind
command_latency_p95    OK        3.4ms
connection_saturation  WARN      81% of maxclients
overall: WARN

An orchestrator would treat CRITICAL as fail-the-health-check (stop routing traffic here, consider replacing the node), and WARN as a signal to page or auto-scale rather than an immediate failure, since multiple simultaneous WARNs (as in the sample above) often indicate the whole node is under real pressure even if no single check alone has crossed into CRITICAL. A sustained, unexpected eviction rate (1000/min or more, in this sketch) escalates to CRITICAL on its own, the same way the other four checks do, rather than only ever topping out at WARN regardless of how bad it gets.

Trade-offs and pitfalls

Complexity is O(1), six constant-time comparisons regardless of cluster size (eviction rate now has its own WARN/CRITICAL split like the other four checks), the real cost is the round trip to collect INFO and a latency sample, run it on an interval, not on every request. Edge cases worth handling explicitly: a brand-new node with no traffic yet (avoid false-CRITICAL on latency from too small a sample), a node correctly configured to evict as a genuine LRU cache (don't alert on expected evictions, that's precisely why the expected_evictions flag exists above), and a replica that's intentionally paused for a maintenance snapshot (exclude it from the fleet's aggregate health rather than treating its lag as a real incident).

Direct answer

Define the target as a time to first byte (TTFB, the time from when the client sends a request to when it receives the first byte of the response) at the 95th percentile (p95, the response time that 95% of requests are faster than), then break that overall budget down into a sub-budget per stage of the request path, so each stage has a concrete number to be held accountable to, and let caching absorb whatever the data-fetch stage can't hit on its own.

Structured elaboration

Why p95 and not an average: an average hides the tail, a service that's fast for 95% of users and terrible for 5% has a fine average and a broken experience for one in twenty requests, which is exactly the population caching interventions usually help most.

Budget allocation approach: split the 200ms target across the stages a request actually passes through (network and TLS handshake, load balancer / routing, application logic, data fetch), assign each a sub-budget that sums to the target, and instrument each stage separately so you know which one is actually over budget rather than guessing.

A roadmap of caching interventions to close a gap in the data-fetch budget, in the order they usually pay off fastest:

1. Instrument first: measure the current p95 broken down by stage, so you know the data-fetch stage is actually the offender before investing in a fix for it.
2. Add edge/CDN caching for cacheable GET responses, the cheapest win since it removes the request from your infrastructure entirely for a cache hit.
3. Add application-level cache-aside caching for the hottest database reads (the query pattern the endpoint's p95 keeps tripping on).
4. Add short-lived negative/computed-result caching for expensive endpoints that can't cache the full response but can cache an expensive intermediate computation.
5. Re-measure against the same per-stage breakdown, and only keep iterating on caching if the data-fetch stage is still the one missing its sub-budget.

Worked example

Say instrumentation shows the data-fetch stage at 160ms p95 against a 100ms sub-budget, entirely due to one uncached database query that averages 15ms but spikes to 160ms under load. Moving that query behind a cache-aside layer with a short time-to-live (TTL) drops the cache-hit path to roughly a 5ms in-memory lookup; even with a conservative 70% hit ratio, the p95 (driven by the worst 5% of requests, disproportionately the cache misses) still improves substantially because the majority of requests no longer touch the database at all, and the database itself faces less load, which also helps the miss path recover faster.

Trade-offs and pitfalls

A budget that isn't broken down per stage tells you that you're over target but not why, always instrument per-stage before adding caching as the fix, the bottleneck might be TLS handshake overhead or a slow downstream call that caching does nothing for. Don't compare a p95 measured server-side (TTFB) against a budget meant for full page load, those are different metrics measured at different points, mixing them produces a number that looks like a violation or a pass for the wrong reason.

Why GraphQL breaks normal HTTP caching

Most GraphQL traffic is a POST to a single /graphql endpoint with a query body, so you lose the free caching that comes with GET requests and distinct URLs (a CDN or browser cache normally keys on the URL). Two clients asking for different fields of the same object also get different response shapes, so caching "the response" as one blob has almost no reuse across users.

1. Persisted queries

A persisted query replaces the full query text with a short hash. The client first sends only sha256(query). If the server already has that hash mapped to the full query (from an earlier request, or pre-registered at build time), it executes immediately; if not, it returns a "not found" error and the client resends the full query once, which the server then stores. Two wins: the request body shrinks to a hash plus variables, and because the hash is small and deterministic you can send it as a GET query parameter, which makes the response cacheable by an ordinary CDN or browser cache the same way a REST GET is.

2. Cache keys: root-level vs field-level

Root-level means the cache key is hash(operation) + variables + caller-identity, and the ENTIRE response is stored under that one key. Simple, but reuse is poor: add one field to the query, or change one variable, and you get a full cache miss even though most of the underlying data is unchanged.

Field-level (normalized) caching keys each resolved object independently, typically Type:id:field(args), e.g. User:42:posts(first:10). A gateway resolves a query by composing these smaller cached entries, so a query that reuses User:42 from a different operation is a hit even though the two operations look nothing alike as documents. This is what production GraphQL gateways actually do via cache-control directives per field or type, each declaring a max age and a scope (PUBLIC or PRIVATE).

3. Response fingerprinting

Fingerprinting means deriving the cache key deterministically from the operation, not from the raw request text, so two requests that mean the same thing hash to the same key. In practice: normalize variable ordering (sort keys before hashing), strip formatting differences in the query text (or better, always operate on the persisted-query hash, which already normalizes this), and include only inputs that actually affect the resolved data.

4. Partial-field caching

A single query commonly mixes cheap, cacheable fields (product name, price) with expensive, personalized ones (recommended-for-you). Because the cache operates at the field level, the gateway fills the cacheable fields from cache and only invokes the resolver for the personalized field, then merges both into one response. This is the real payoff of field-level over root-level caching: partial hits instead of all-or-nothing.

5. Query whitelisting

For a public-facing API, combine persisted queries with an allowlist: the server only executes operation hashes registered ahead of time, and rejects anything else. This bounds the set of distinct cache keys you'll ever see and closes off a denial-of-service vector where an attacker crafts a maximally expensive nested query on the fly.

6. Authorization and Vary behavior

A GraphQL response is shaped per caller, so caching by operation plus variables alone can leak one user's data to another if two users happen to ask the same question. Treat identity the same way HTTP treats Vary: Authorization or Vary: Cookie: any field marked PRIVATE must fold the caller's identity (or a normalized permission-scope hash) into its cache key, and PRIVATE fields should never sit in a shared edge cache, only in a per-caller layer such as an application Redis keyed by user id. PUBLIC-scoped fields carry no identity in their key and can sit safely at the edge.

Trade-off

Field-level caching gets real reuse but costs real complexity: invalidation has to fan out per changed entity (a mutation touching Post:99 must bust every cached field keyed by Post:99, not one blob), and you need a normalized store plus a gateway that composes partial hits. Root-level caching is simpler but only pays off for queries that are already nearly-static and shared across many callers.

The two options

In-process (local, "L1") caches like Caffeine or Guava (popular Java in-memory caching libraries) live inside the application's own process. Remote/shared ("L2") caches like Redis live outside it, reached over the network and shared by every instance.

Comparison

Guidance

Use a local cache for small, very-hot, mostly-read data where a little staleness across instances is tolerable, feature flags, small config or lookup tables, things where all instances converging on the same value within a few seconds is fine. Use a remote cache when the data set is too large to duplicate across every instance, needs a consistent view across instances, or changes often enough that per-instance staleness would actually matter.

Two-level strategy

Put a small local L1 in front of a shared L2 (Redis). L1 absorbs the very highest-QPS keys at sub-microsecond latency and takes load off L2; L2 remains the shared source of truth across instances and survives individual instance restarts and deploys, which also softens a cold rolling-deployment start, since a freshly started instance only has an empty local L1 and still serves from the already-warm shared L2 instead of hitting the origin directly. The risk is that L1 can go stale relative to L2 after a write, until L1's own entry expires. Mitigate that by keeping L1 TTLs deliberately short (seconds, not minutes) or by broadcasting an invalidation (pub/sub) to every instance's L1 the moment L2 is invalidated, so writes propagate quickly instead of waiting out L1's TTL.