Privacy by Design and Default Questions

Direct answer

Start from the decisions the analytics must support, then collect only the events that inform them, and make every data design choice (what, how much, how long, who sees it) serve those questions. The balance between utility and privacy is not a single dial: most product questions are about groups of users, so aggregate, sample and strip identifiers early, and reserve user-level data for the few cases that need it and have consent.

Design steps

1. Question first: write the metric or decision for each event ("which onboarding step loses users?"). An event with no decision attached is not collected.
2. Define the schema as an allow-list: event name plus a small set of approved properties. No free-text fields, no full URLs with query strings, no raw email or IP address.
3. Identify users with a random, resettable ID (the user can reset it in settings, after which new events cannot be linked to the old ones) rather than an email or device serial. Coarsen where precision is not needed (country instead of city; app version, not full device fingerprint).
4. Sample where totals suffice: for high-volume events, record a fraction and scale up. Example: at 2,000,000 events per day, 1 percent sampling stores 20,000 events. If 1,200 of them are "completed onboarding" (6 percent), the estimate is 6 percent of 2,000,000 = 120,000 events, which is the same as multiplying the 1,200 by 100. The uncertainty is about plus or minus 0.3 percentage points at 95 percent confidence, plenty for a funnel rate, though very rare events need full capture.
5. Store in tiers: event-level data for a short window, then aggregates only. Restrict access to the raw tier to a small group, and give analysts the aggregate tables.
6. Honour consent and deletion: analytics that is not strictly necessary follows the user's consent choice, and a deletion request removes or detaches that user's ID from the event store.

Trade-offs and what is lost

Cohorts and funnels

A cohort is a group of users who share a trait or a start date, such as everyone who signed up in March. A funnel is the ordered steps users move through, such as install, sign up, first purchase, with the share lost at each step. Cohort and funnel reports usually reach analysts through BI (business intelligence) dashboards, and that analysis can run on pseudonymous IDs with consent where required. Build deletion into the pipeline: when a user deletes their account or withdraws consent, the ID is removed from event tables and from derived cohort tables on the next batch, and the schedule is documented. Report small cohorts carefully: a segment of two people can identify them, so suppress groups under a minimum size, for example 20 users (an illustrative threshold your team sets). With that threshold, a country table might show US 8,400 and Iceland 35, while Liechtenstein with 4 users is merged into "Other".

Ethical side

User research and telemetry should not surprise users. Ask "would a user be comfortable seeing this event list?" and publish a plain-language description of what is measured. Avoid measuring things only because they are easy, such as keystroke-level behaviour, or recording that a user searched for a medical condition when a count of searches would answer the question.

Pitfall

Collecting everything "for later" and then promising to anonymize. Raw events with a stable ID plus enough detail can often be tied back to a person, so minimizing at collection is stronger than cleaning afterwards.

Short answer. Keep the most identifying form the shortest and the least identifying form the longest: raw events 30 days, pseudonymized event-level data 13 months, aggregates 25 months, ML features 90 days rolling. Windows below are illustrative starting points; legal and the product's real analysis needs set the final numbers.

Key terms. Raw events carry direct identifiers (user id, device id, full IP). Pseudonymized means the identifier is replaced by a keyed hash (HMAC, a hash made with a secret key) so rows still join, yet the person is not named. It is still personal data. Aggregated means counts and averages with no per-person rows.

Retention schedule

Snapshots are the exception to the 90-day feature window: a snapshot freezes feature values for reproducing a model, so it holds per-user data for up to 180 days. Either build snapshots from pseudonymized rows with no direct identifiers, or accept that a deletion request does not reach an existing snapshot until it expires (at most 180 days), say so in the policy, and exclude deleted users when the next snapshot is built. A trained model is not edited either: retrain on the next scheduled cycle from data that already excludes the deleted user.

Why 13 and 25 months: 13 months lets a year-over-year comparison include one extra month; 25 months gives two full years of trend. If analysts can answer questions from aggregates, the event-level tiers can shrink.

One record traced. A user's page view is stored on 2026-01-05. The raw copy is dropped on 2026-02-04 (30 days). The pseudonymized copy lives until 2027-02-05 (13 months). Only its contribution to a daily count remains after that, kept until about 2028-02. If the user deletes their account on 2026-03-01, the raw copy is already gone, so the pseudonymized rows are deleted that day and their keyed id goes on the suppression list. Per-user features are recomputed without them within their 90-day window, and any snapshot that still holds them expires by 180 days from its creation.

Logs by type (forensic need versus minimization)

• Security and audit logs: 12 months. Forensics needs the long window to see slow intrusions.
• Application debug logs: 14 days, free text scrubbed of PII at ingestion.
• Access logs with IP: 30 days, then IP truncated.

Archival. Archiving personal data is still retention, so never "archive raw" to dodge a window. Only aggregates go to long-term cold storage.

Backups and replicas. Backups expire on their own rotation (for example 35 days). Replicas follow the primary. State in the policy that a restore must replay the deletion ledger (the append-only list of deletion requests, kept without readable personal data) before data is served again.

Verification

• Each expiry job writes a receipt: table, partitions dropped, row counts.
• A daily check asserts that the oldest event time in each tier is inside its window.
• A weekly test injects a synthetic user (a fake test person created for this check), deletes it, and queries every tier and backup index for it.
• Per-user deletions are checked against the suppression list so deleted users do not reappear from late events.

Trade-off. Shorter raw windows make incident debugging and model retraining harder. Keep a pseudonymized copy for that need rather than extending the raw window.

Direct answer. Treat the feature store as a governed catalog: every feature carries metadata (sensitivity class, allowed purposes, source data, retention), access policy reads that metadata at request time, and user-keyed lineage lets a deletion or consent change reach the online store, offline store and training datasets. A feature store is a shared service that computes, stores and serves model input features, with an online store for low-latency serving and an offline store for training.

1. Sensitivity tracked per feature
(An entity ID is the key a feature is stored under, here the user's ID. Lineage is the recorded chain of which source tables fed which feature and which features fed which training set. A manifest is a saved list that records exactly what went into one training set.)

• Registering a feature requires: owner, source datasets, classification (for example public, personal, sensitive), allowed purposes, retention.
• Derived features inherit the highest sensitivity of their inputs unless a reviewer records why it is lower. Example: avg_night_heart_rate_30d, derived from health data, is classed sensitive-health, allowed for sleep-coach, denied for ads-ranking.
• Lineage (which tables fed which feature) is stored, so reclassifying a source cascades to its features.

2. Access follows sensitivity

• Read policy checks the caller's declared purpose against the feature's allowed purposes, for training jobs and for online serving, and logs every read.
• A model that requests a denied feature fails at training-set build time, not in production.

3. Deletion and consent propagation

• Keys are the user (entity) ID. A consent service publishes events on withdrawal or deletion.
• Online store: consume the event and delete or invalidate the user's keys quickly, or check a consent flag at read time. Real-time enforcement suits serving. Worked example: user 8841 withdraws consent for ads-ranking. Within minutes the flag ads-ranking: withdrawn takes effect on the online path: keys for features whose only allowed purpose is ads-ranking are deleted for 8841, while keys for a feature shared with a purpose the user still allows (such as sleep-coach) stay and are simply refused to any ads-ranking read, so the next ads request gets no personalised features; offline rows are purged in the nightly job only for features with no remaining allowed purpose, and the training set built that weekend for ads-ranking carries the filter consent[ads-ranking] != withdrawn, so its manifest lists 8841 as excluded.
• Offline store: batch purge or rewrite partitioned tables on a schedule, and apply the consent flag as a join filter (when the training set is assembled by joining tables, rows for users whose flag says withdrawn are dropped) when building each training set. Batch enforcement suits training, because a training set is built at a point in time.
• Dynamic consent flags: a per-user, per-purpose flag in the store (for example user 8841, ads-ranking: withdrawn on 12 March) drives both paths, so changing a purpose does not require recomputing every feature.

4. Dataset versioning and audit trail. Every training set is saved as a versioned manifest: feature list and versions, snapshot time, consent filter applied, and the model trained on it. Auditors can see which data a model used and prove withdrawn users were excluded after a date. Deleting a user from a trained model needs a retraining policy (for example retrain within a set window); state this limit honestly.

5. Pseudonymization and noise. Keyed pseudonymous IDs (a secret key turns each real ID into a code, so analysts see codes only) reduce exposure but the data stays personal data while the key exists. Adding noise (small random changes to feature values, the idea behind differential privacy) hides any one person's contribution but costs model accuracy, and the more noise the stronger the protection; it suits aggregate statistics and sensitive features, and many feature stores never need it.

Trade-off. Strict purpose checks slow feature reuse; offset with a self-service request flow for new purposes.

Short answer. Build a connector contract and an orchestrator. Every system that holds personal data registers a connector with find, export and delete operations. An orchestrator sends a request out to all connectors at once, tracks each one's state, retries failures, and produces a completeness report. Completeness is argued from three things: an inventory that is checked for gaps, per-connector receipts, and a recurring test with a synthetic person.

1. Finding every place the person's data lives

• Data inventory. A catalog of every datastore and processor, with which columns, objects or fields hold personal data and which key links them to a person.
• Identity resolution (working out that several identifiers belong to the same person). A person has many identifiers (user id, email, device ids, hashed ids). The orchestrator expands the verified identity into the full identifier set before searching.
• Gap detection. A scheduled scan of cloud accounts flags any new bucket, table or topic with identifier-like columns that has no connector. CI blocks new datastores without registration.
• Processors. Systems you do not run (analytics vendor, email provider) get an API connector or a tracked manual task with a due date.
• Multi-tenant SaaS. If you are the processor for business customers, the request routes to that tenant, and your system acts on their instruction.

Worked run (illustrative). Request R-1042 arrives 2026-03-02 for ana@example.com, identity verified 2026-03-03. Identity resolution expands it to user id 7, two device ids and a hashed email. States on 2026-03-05: main database succeeded (3 rows exported), search index succeeded, email vendor running (API call pending), analytics vendor failed (retry 2 of 5, then dead-letter queue), support tool not_applicable (no record found, receipt logged). R-1042 stays open; the day-20 alert on 2026-03-22 pages the owner of the analytics connector.

2. Intake, identity and rate limiting. Verify identity before any search runs. Rate-limit and throttle requests per account and per source to stop enumeration (guessing many identifiers to learn who has an account) or flooding. The orchestrator never exposes whether data exists to an unverified requester.

3. Partial failure. Each request has a state per connector: pending, running, succeeded, failed, not_applicable. Rules:

• Operations are idempotent (running one twice has the same effect as once), keyed by request id plus connector.
• Failures retry with backoff (waiting longer between attempts), then go to a dead-letter queue (a holding area for jobs that kept failing) and a human.
• The request cannot complete while any connector is failed or pending.
• Alerts fire at day 20 of a 30-day target, and the extension decision (whether to use the allowed extension for a complex request and tell the requester why) is made before the month ends.

4. Export and redaction. Each connector returns data in a common schema. A review step removes other people's data. Delivery is by expiring link after verification. Files are exported as JSON or CSV, and objects as the original files in an archive.

5. Delete as well as export. The same fan-out handles erasure with the same states, plus a legal-hold check (is this data frozen by a legal duty or dispute, so it must not be deleted yet).

6. Proving completeness

• Coverage: every inventory entry has a connector or a documented exemption.
• Receipts: each connector logs query hash, row and object counts, and timestamps.
• Canary: a synthetic person (a fake test person) is seeded in every store weekly and must appear in every export and vanish on delete.
• Residual honesty: data that cannot be linked to a person without more information is out of reach, and the controller is not required to keep extra data just to find it (GDPR Art. 11). Example: web logs keyed only by a random cookie id that you cannot tie to an account need not be linked to people just to answer a request, unless the person supplies extra information, such as that cookie id, that lets you identify their records.

Short answer. Split each log event at ingestion into two streams that share a correlation id (a random identifier stamped on every event from one request, so the two copies can be matched later). The analytics stream has PII removed or replaced by a keyed pseudonym and is broadly readable. The raw sensitive fields go to a small, restricted store that developers can open only through time-limited, logged, approved access.

Technical implementation

1. Schema and allowlist. Services log structured events with declared fields tagged sensitive or not. A continuous integration (CI) check fails a build that logs an untagged field.

2. Ingestion processor. Reads the tag: drops free text with PII, replaces user ids with an HMAC (keyed hash) so analysts can still count and join per user, coarsens the IP address (the /24 form zeroes the last of its four number groups, so 203.0.113.42 becomes 203.0.113.0, which points at a network rather than a device; or keep only the country), and writes the cleaned event to the analytics store.

3. Restricted store. The original sensitive fields go to a separate store, encrypted, with a short retention such as 14 days (illustrative), keyed by the correlation id and request id.

   Example (illustrative values). One raw event: {corr: "c-71f3", user: "ana.ruiz@example.com", ip: "203.0.113.42", path: "/checkout", status: 500, note: "card declined for Ana Ruiz"}. The analytics stream receives {corr: "c-71f3", user: "u_9be2d4", ip: "203.0.113.0", path: "/checkout", status: 500}: the email is replaced by an HMAC token, the IP is coarsened, and the free-text note is dropped. The restricted store receives {corr: "c-71f3", email: "ana.ruiz@example.com", ip: "203.0.113.42", note: "card declined for Ana Ruiz"}. An analyst can count 500s per user from the first copy; only an approved developer holding c-71f3 can open the second.

4. Debug path. For most incidents, sanitized logs, stack traces and request ids are enough. When not, the developer requests raw context for one service and one time window.

Authorization and friction

• Just-in-time access (permission granted only when needed and removed automatically afterwards): a ticket with a reason, approved by an on-call manager or privacy contact, expiring in 1 to 4 hours, scoped to the service and window.
• Fast path, also called break-glass access (like breaking the glass on a fire alarm), for a severity-one outage (a major, customer-impacting incident): on-call engineer self-approves, with mandatory review within 24 hours. This is the trade-off: speed during an outage in return for after-the-fact scrutiny.
• Raw fields are masked in the viewer and revealed per field, and bulk export is disabled.

Personnel controls

• Short access list; training before access; the approver is never the requester on the normal path (the break-glass path above is the one exception, which is why it gets a mandatory review within 24 hours).
• Quarterly recertification: each manager re-confirms that every person on the access list still needs it, and unconfirmed access is removed.

Auditing

• An append-only audit log records who, when, which window, and which fields were revealed.
• Alerts for unusual volume (for example, a person revealing far more records than peers) and weekly sampled reviews by privacy or security.
• Reports to leadership: number of break-glass accesses and review outcomes.

Keeping detection capability. Security detection needs actor, action, outcome, source and time. Keep those, in pseudonymous form, in a security stream readable by the security team, where only that team holds the key, so that only it can recompute the keyed hash for a user id under investigation and match it (an HMAC cannot be decoded; recomputing for a known candidate, or a separate lookup vault, is how it is resolved). Drop what detection does not use, such as message bodies and full names.

Trade-offs. Pseudonymized data is still personal data, so retention and access rules still apply. Over-masking can make analytics useless, so the schema tags are reviewed with analysts. The key for the HMAC lives in a key management service (a hardened system that stores and hands out encryption keys so they never sit in code or config files) and is rotated on a schedule, which breaks joins across the rotation unless planned.