Privacy by Design and Default Questions
Designing privacy into products, architectures and the software development lifecycle instead of bolting it on afterward. Covers the privacy-by-design principles and privacy-protective defaults, data minimization and purpose limitation at design time, personal-data flow mapping and PII inventory for a feature, consent and preference design including consent storage, enforcement across web and mobile and region-specific consent flows, retention and deletion design including legal holds, deletion across backups and immutable logs, provable deletion, and data-subject access, export and erasure flows, PII-safe logging, telemetry and session replay, on-device processing to limit data exposure, privacy-versus-personalization trade-offs, privacy in ML training data, feature stores and erasure from trained models, and privacy review gates, checklists, CI/CD checks, release readiness and requirements traceability inside engineering workflows. Includes designing privacy-first features, reference architectures, remediating privacy design flaws and rollout programs across product teams, and working with product, legal and security partners. Cryptographic and statistical privacy techniques, formal impact assessments, data-protection law, cross-border transfer rules, vendor risk, and encryption or access-control mechanics are covered elsewhere.
You lead a product that trains ML models on user data. Define a product strategy to manage training data lifecycle: labeling, retention, deletion on user request, auditing datasets for leaked PII, and versioning models so deletion requests are honored. Include trade-offs.
Sample Answer
Direct answer. Treat training data as a governed asset with a lifecycle: collect only what the model needs, label it with provenance, retain it for a defined period, honor deletion by removing it from datasets and scheduling retraining, and keep manifests so you can show which model was trained on which data. Be honest that removing a person's data from a trained model is hard: a model stores what it learned as numbers (weights) spread across the whole model, not as rows you can look up, so deleting the training row does not remove its influence. There is no cheap, reliable way yet to subtract one person from the weights, so the strategy relies on retraining cadence and on avoiding personal data in training where possible.
Lifecycle stages.
- Collection and consent: record the purpose and lawful basis (the legal ground for using the data; a question for counsel) at ingestion. Exclude users who opted out of training use. Collect fewer fields: drop or tokenize direct identifiers before they reach the training store.
- Labeling: labelers see the minimum, through access-controlled tools; vendors are bound by contract; labels link to records by pseudonymous ID, not names.
- Storage and retention: the raw store, the processed features and the training snapshots each have a retention class. Raw data expires sooner than the derived snapshots only if the derived ones no longer identify anyone.
- Dataset manifests: every training run records a manifest: the list of record IDs (pseudonymous), schema and filter version, and code commit, plus a hash of the list. This gives auditable reproducibility and lets you answer "was this user's data in model v7?" by looking the ID up in the stored list, without keeping the underlying data forever. The hash alone cannot answer membership; it only proves the stored list has not been altered since training. An illustrative entry:
{model: "v7", trained: "2026-04-12", snapshot: "snap-0412", id_list_hash: "5b1e...", filter_version: "f19", code_commit: "a3c91e7"}, where the hash is a short fingerprint that changes if any listed ID changes (the full ID list sits in the manifest store undersnap-0412). - PII audit: scan datasets before training for emails, phone numbers, ID numbers and secrets using detectors plus sampled human review; block runs that fail; scan model outputs for memorized personal data in evaluation (a model can sometimes reproduce rare training text, such as an email address, word for word).
- Deletion on request: delete the user's rows from raw data, features and snapshots using the manifest index. Persisted artifacts such as embeddings (lists of numbers that represent a text or user so models can compare them), cached features and fine-tuned weights (the model's adjustable numbers after extra training on your data) are tracked as derived data too: delete derived features directly; for weights, record the request and remove the user in the next scheduled retrain.
- Model versioning: each model version links to its manifest; versions older than a defined age are retired so no stale model keeps data in it indefinitely.
Deletion-aware retraining cadence. Choose a cadence (for example, retrain quarterly and always before a model is relaunched) where each retrain excludes everyone who has deleted or opted out. In between, apply an output-side suppression filter (a check that stops the model from returning data about people on the pending-deletion list) and keep that list. Whether an interim lag is acceptable is a legal and risk decision, and you state the maximum lag to users.
| Option | Benefit | Cost |
|---|---|---|
| Frequent full retrain | Strongest deletion story | Compute cost, regression risk |
| Periodic retrain plus suppression | Lower cost | Deleted data influences the model until next retrain |
| Sharded training (train by data slice, retrain only the affected slice) | Cheaper deletions | More engineering, possible accuracy loss |
| Train on de-identified or synthetic data | Fewer deletion obligations | Utility loss; de-identification must be shown to hold |
In practice the usual default is periodic retrain plus suppression; the other rows apply when the deletion lag or the cost is unacceptable. Sharded training example: split users into 10 slices, each with its own sub-model, combined when serving; a deletion from slice 4 retrains only slice 4, about one tenth of the compute of a full retrain.
Worked example. A model is trained on 10 million user records. Over a quarter, 4,000 users request deletion (0.04%). The manifest finds their IDs, the next scheduled retrain excludes them, and the old model version is retired 30 days after rollout, so no model trained with them stays in production beyond about one retrain cycle plus 30 days. Numbers are illustrative.
Trade-offs. Faster deletion costs compute and may hurt model quality; stronger minimization may lower accuracy. As product lead, I would set the cadence with legal, publish the lag, and measure it.
Your org has a privacy checklist and review gate, yet teams still ship privacy bugs because compliance is something done at the end. How would you embed privacy into how an ML or product team works day to day so it stops being a separate checklist?
Sample Answer
Direct answer. The checklist fails because it is a late, separate, binary step owned by someone else, so teams optimize for passing it. I would move privacy into the existing workflow: design docs, tickets, code review, pipelines and retrospectives, and let the review gate become a small escalation path for genuinely new risk rather than the main control.
What is wrong with the current setup (the critique)
- Review at the end finds problems after the architecture and data model are fixed, so fixes are expensive and get waived.
- A checklist asks "did you do privacy?" instead of "which personal data does this feature touch and why?".
- Nobody on the team owns the answer, so privacy bugs are the review team's problem.
What I would change, in order
- Start from the bugs. Classify the last few quarters of shipped privacy bugs (for example PII in logs, features trained on data without a purpose check, missing deletion path). Build the first changes around the top classes, so the effort answers a real pain.
- Put the questions in the design doc template. A short section: what personal data, what purpose, retention, who can access, what happens on user deletion or consent withdrawal. A design is not review-ready with it blank. For ML, add: training data sources, whether labels or outputs are personal data.
- Add to the definition of done. Stories touching personal data carry acceptance criteria (for example "event schema tagged with data classification", "deletion path tested"), reviewed like any other criterion.
- Automate the repeatable. Pipeline checks for PII in logs and untagged schema changes catch the common mistakes at pull request time, so humans spend time on judgement.
- Name a privacy champion per team (an engineer with a few hours a week, not a new role), trained by the privacy team, who is the first reviewer and escalates only new patterns.
- Close the loop. Every privacy bug gets a blameless retrospective and a new automated check or template question.
Reasoning on ownership. The engineering manager owns the outcome, the privacy team owns the standard and the tooling, and the gate owns only escalations such as a new sensitive data category.
Measures. Privacy bugs found before merge versus after release, time from first design doc to privacy sign-off, and share of designs that arrive with the section filled in.
What would change my call. If most bugs come from one shared platform, fix it there instead of training every team.
Your ML platform has a shared feature store, and some features are derived from sensitive personal data. How would you build privacy controls into the store itself so that sensitivity is tracked per feature, access follows it, and a user's deletion or consent change propagates?
Sample Answer
Direct answer. Treat the feature store as a governed catalog: every feature carries metadata (sensitivity class, allowed purposes, source data, retention), access policy reads that metadata at request time, and user-keyed lineage lets a deletion or consent change reach the online store, offline store and training datasets. A feature store is a shared service that computes, stores and serves model input features, with an online store for low-latency serving and an offline store for training.
1. Sensitivity tracked per feature
(An entity ID is the key a feature is stored under, here the user's ID. Lineage is the recorded chain of which source tables fed which feature and which features fed which training set. A manifest is a saved list that records exactly what went into one training set.)
- Registering a feature requires: owner, source datasets, classification (for example public, personal, sensitive), allowed purposes, retention.
- Derived features inherit the highest sensitivity of their inputs unless a reviewer records why it is lower. Example:
avg_night_heart_rate_30d, derived from health data, is classed sensitive-health, allowed forsleep-coach, denied forads-ranking. - Lineage (which tables fed which feature) is stored, so reclassifying a source cascades to its features.
2. Access follows sensitivity
- Read policy checks the caller's declared purpose against the feature's allowed purposes, for training jobs and for online serving, and logs every read.
- A model that requests a denied feature fails at training-set build time, not in production.
3. Deletion and consent propagation
- Keys are the user (entity) ID. A consent service publishes events on withdrawal or deletion.
- Online store: consume the event and delete or invalidate the user's keys quickly, or check a consent flag at read time. Real-time enforcement suits serving. Worked example: user 8841 withdraws consent for
ads-ranking. Within minutes the flagads-ranking: withdrawntakes effect on the online path: keys for features whose only allowed purpose isads-rankingare deleted for 8841, while keys for a feature shared with a purpose the user still allows (such assleep-coach) stay and are simply refused to anyads-rankingread, so the next ads request gets no personalised features; offline rows are purged in the nightly job only for features with no remaining allowed purpose, and the training set built that weekend forads-rankingcarries the filterconsent[ads-ranking] != withdrawn, so its manifest lists 8841 as excluded. - Offline store: batch purge or rewrite partitioned tables on a schedule, and apply the consent flag as a join filter (when the training set is assembled by joining tables, rows for users whose flag says withdrawn are dropped) when building each training set. Batch enforcement suits training, because a training set is built at a point in time.
- Dynamic consent flags: a per-user, per-purpose flag in the store (for example
user 8841, ads-ranking: withdrawn on 12 March) drives both paths, so changing a purpose does not require recomputing every feature.
4. Dataset versioning and audit trail. Every training set is saved as a versioned manifest: feature list and versions, snapshot time, consent filter applied, and the model trained on it. Auditors can see which data a model used and prove withdrawn users were excluded after a date. Deleting a user from a trained model needs a retraining policy (for example retrain within a set window); state this limit honestly.
5. Pseudonymization and noise. Keyed pseudonymous IDs (a secret key turns each real ID into a code, so analysts see codes only) reduce exposure but the data stays personal data while the key exists. Adding noise (small random changes to feature values, the idea behind differential privacy) hides any one person's contribution but costs model accuracy, and the more noise the stronger the protection; it suits aggregate statistics and sensitive features, and many feature stores never need it.
Trade-off. Strict purpose checks slow feature reuse; offset with a self-service request flow for new purposes.
You must decide whether a heavy personalization model runs on the device or in the cloud. How would you frame the recommendation, what privacy and security benefits would you weigh, and what would you give up?
Sample Answer
Direct answer
I would frame it as a decision about where the data and the model should live, not a slogan. My default recommendation is a split: run a smaller personalization model on the device so raw behavioural data never leaves it, and keep heavy training and any non-personal model work in the cloud. I would put the heavy model fully in the cloud only if the device genuinely cannot run it at acceptable quality, and then minimize what is sent.
How I frame the recommendation
- What data does the model need (clicks, location, health, messages), and how sensitive is it?
- Who benefits from keeping it local, and who bears the cost (battery, storage, app size, device fragmentation)?
- What quality does the product need, and does a compressed model deliver it?
- Who owns the decision: product and the ML lead decide quality and cost, security and privacy sign off on data flows.
Privacy and security benefits of on-device
- Raw personal data stays on the phone, so there is less to breach, subpoena (a legal order forcing a company to hand over data it holds), over-retain or misuse for a new purpose later (data minimization).
- Smaller attack surface (fewer places an attacker can reach) on the server side, no per-user profile store to protect.
- Works offline and with lower latency.
- Simpler deletion: clearing app data clears the personalization state.
Benefits of cloud
- Larger, better models and quick updates, with central monitoring.
- Easier evaluation and abuse detection.
- Model and weights are protected from extraction by users, and keys are handled centrally.
What you give up with on-device
| Cost | Detail |
|---|---|
| Quality | A compressed model (shrunk by storing its numbers with fewer bits, or by training a small model to imitate a big one) is usually weaker than a full-size one |
| Device cost | Battery, memory, thermal limits, and older phones that cannot run it (device fragmentation: many phone models with different hardware) |
| Model security | The weights (the learned numbers that are the model) ship inside the app. An attacker can copy them out of the app package and reuse or resell them (extraction), edit them so the model behaves differently, for example always ranking one seller first (tampering), or send many inputs and study the outputs to rebuild the model or infer what it was trained on (probing) |
| Visibility | You cannot easily see failures or abuse |
| Update speed | Model rollouts depend on app and OS update cycles |
Worked example
A shopping app wants personalized rankings from browsing history. Option A sends the full browse history to the server each session. Option B keeps history on the device, scores 200 candidate items locally using a small model, and sends only the IDs of the items the user tapped, not the browsing history (the server can then count taps per item across users, but each session still uploads a short list). With illustrative numbers: Option A uploads about 300 browse events of roughly 100 bytes each, 30 KB per session, and ranks at a 6.0% click-through rate. Option B uploads 5 tapped item IDs, well under 1 KB, and a smaller model might reach 5.4%, a 10% relative drop (0.6 / 6.0), while scoring 200 items at about 5 ms each costs roughly 1 second of processor time per session (200 x 5 ms = 1,000 ms), which is the battery cost to measure. Option B loses some ranking quality and some insight into long-tail behaviour, but the server never holds the browse history. I would recommend B, with a cloud fallback for devices below a hardware threshold that sends only the minimum features and is opt-in.
Decision flips
If the model needs data from many users to work (collaborative signals) or needs frequent updates, I would move that part to the cloud (collaborative signals, such as "people who bought this also bought that", only exist when many users' data is combined) with pseudonymous, minimized inputs. If the data is highly sensitive (health), I lean on-device. If model theft is the bigger business risk, cloud wins.
A user exercises their right to be forgotten, but their data has already shaped trained models and derived features. How would you handle this, when is retraining actually necessary, and what would you tell the user and the regulator?
Sample Answer
Direct answer
Treat the request in two layers. First, delete the person's data everywhere it is stored as data: source tables, feature stores, training snapshots, caches and exports. Second, decide model by model whether the trained artifact still holds the person's information. Retraining is necessary only when it does, or when you cannot show that it does not. I would tell the user exactly what was deleted and what happens to models by a stated date, and the regulator only what a complaint or inquiry asks for, with counsel (the company's lawyers) deciding the legal position.
Step 1: Map what the person touched
| Layer | Example | Action |
|---|---|---|
| Source records | orders, chat logs | delete (GDPR Art. 17, the right to erasure) |
| Derived features (stored in a feature store, the shared service that holds model inputs) | "avg basket size", embeddings | delete or recompute without the person |
| Training snapshots | frozen parquet files (a columnar file format common for training data) | delete the rows, keep a deletion ledger |
| Trained models | churn model v7 | assess, then retrain, retire or keep |
| Backups | nightly copy | let expire on schedule, block restores from reintroducing the data |
A deletion ledger is a list of person IDs that every training job reads and excludes from its input. It stops the data coming back at the next run. Store the IDs hashed (run through a one-way scramble, with a secret key added so common IDs cannot be guessed) so the ledger itself does not become a list of people to leak; hashed IDs are still pseudonymous, which is acceptable for a list that exists only to enforce exclusion.
Pseudonymizing (swapping names for IDs while a key exists elsewhere) is not erasure: the data is still personal data. Only truly irreversible anonymization removes it, and that is hard to prove.
Step 2: When is retraining actually necessary?
Ask whether the model could reveal or reproduce this person's data.
- Low risk, no immediate retrain: a model trained on millions of rows with simple features (for example a gradient-boosted churn model, which combines many small decision trees and learns general patterns rather than storing rows) where one person has a negligible influence and the model outputs only a score. Exclude them via the ledger at the next scheduled retrain and record the date.
- High risk, retrain or retire promptly: a model that stores records (a nearest-neighbour lookup keeps the training records and returns the closest ones; an embedding index holds numeric fingerprints of text or items that can be mapped back to them), a fine-tuned language model that can regurgitate text, or a small model where the person is a large share of the data. Test with a membership-inference check (can an attacker tell whether a record was in the training set? Example: show the model 1,000 records it trained on and 1,000 it never saw, and let the attacker guess which is which. Guessing right 1,030 times out of 2,000 is 51.5%, close to the 50% of a coin flip, so no meaningful leakage; 90% would mean the model gives records away) and a probe for memorized strings. If the test cannot clear it, retrain without the person or roll back to a model trained before their data.
- Cost trade-off: a retrain costing days of compute can be batched into the normal cadence if risk is low. Machine-unlearning research methods (techniques that try to remove one person's influence from a trained model without retraining it) exist but are not settled enough to be your only defence.
Step 3: What to tell whom
- User: within one month of the request (Art. 12(3) sets one month, extendable by two further months for complex or numerous requests, with the person told of the extension and the reason within the first month): confirm deletion from systems, say that models will exclude them from the next training run by a date, and name any retained data with the legal reason (Art. 17(3) lists the exceptions to erasure, such as a legal obligation to keep invoices or the need to defend legal claims).
- Regulator: erasure requests are not themselves reported. If the user complains, show the record: ledger entry, model risk assessment, dates. Do not claim "the model forgot you" unless you tested it. Wording and any contested position go through counsel and the data protection officer (DPO, the person some organisations must appoint to advise on and monitor GDPR compliance).
- Recipients: tell processors (vendors that handle data on your behalf) and partners who received the data unless impossible or disproportionate effort (Art. 19).
Worked example
A support-ticket classifier is trained monthly on 2 million rows. A user asks for erasure on day 3. You delete their 14 tickets and features by day 5, add them to the ledger, and the day-30 training run excludes them. A membership check on the live model shows no leakage above chance, so you keep it until then and record that. If the same data had fine-tuned a model that quotes tickets, you retire it now.
Unlock Full Question Bank
Get access to all 12 Privacy by Design and Default interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.