Privacy-Preserving Experiment Design Questions

Techniques and considerations for designing experiments and data collection strategies that protect privacy. Covers methods such as differential privacy, secure aggregation, federated learning, synthetic data, data minimization, consent management, de-identification, and privacy risk assessment, with emphasis on maintaining data utility and regulatory compliance while enabling robust experimentation.

MediumSystem Design

61 practiced

Design a logging and auditing scheme for experiments that allows detection of privacy incidents while minimizing exposure of sensitive information. What events should be logged, how should logs be anonymized, what retention policies would you set, and what alert thresholds should trigger investigation?

Sample Answer

Requirements & constraints:- Detect privacy incidents (unauthorized access, unexpected joins/exfiltration) while minimizing sensitive data in logs.- Support forensic investigation, compliance, and operational alerts.- Low impact on analyst productivity.

High-level design:- Centralized, tamper-evident audit logs ingested into SIEM (e.g., Splunk, Elastic + SIEM).- Logs are write-once, encrypted at rest, signed for integrity, and access-controlled.- Schema includes event metadata (actor, action, resource, timestamp, context, experiment_id, risk_score).

What to log (granularity & examples):- Authentication/authorization: successful/failed logins, token refresh, privilege elevation.- Data access: reads/writes to PII-sensitive tables, dataset exports, downloads, notebook cell executions that touch protected datasets.- Joins and queries: queries that join experiment data with identity tables or external sources.- Model actions: training runs using identifiable features, model export/serve events, dataset snapshots used for training.- Administrative actions: ACL changes, dataset deletions, retention changes, key rotations.- Anomalous behaviors: large bulk queries, high-rate API calls, out-of-hours access.

Anonymization & minimization strategy:- Never store raw PII in logs. Replace direct identifiers with irreversible pseudonyms: - HMAC-SHA256(id, key) with per-environment key stored in KMS; keys rotated and only available to a small ops team. - Store partial hashes (prefix) only when necessary to detect duplicates; otherwise aggregate counts.- Use differential-privacy-style noise for aggregated metrics exposed to dashboards.- Redact sensitive query text; store query fingerprint (normalized SQL) + metadata rather than raw literals.- Role-based access to de-anonymization: a documented, audited process (legal/privilege) to map hashes back to identifiers only during approved investigations.

Retention & storage:- Short-term detailed logs (including pseudonyms) retained 90 days for operational forensics.- Medium-term aggregated/anonymized summaries retained 1–3 years for compliance/analytics.- Archive encrypted raw logs (write-once) for 7 years if required by regulation, with strict access controls.- Auto-delete or purge mechanism with immutable audit trail for deletion actions.

Alert thresholds & detection rules (examples):- Immediate high-priority alerts: - Access to >10 distinct PII records by single actor within 1 hour. - Query that joins experiment table with identity table without an approved purpose flag. - Export of dataset labeled "sensitive" to external storage.- Medium-priority: - >3 failed privilege elevation attempts in 10 minutes. - Unusual access outside normal hours for that user (statistical anomaly). - Sudden spike: actor’s data access rate >5x baseline for last 30 days.- Low-priority / informational: - New dataset access patterns, model training using new sensitive features.

Scoring & triage:- Attach risk_score to each event (based on sensitivity of resource, action type, volume, user risk profile).- SIEM correlates events to produce incidents; incidents with risk_score > threshold (e.g., 80/100) create paging alert to privacy/ops.

Investigation workflow:- Automate initial triage: collect event stream, fingerprints, aggregated metrics; block automated risky actions (e.g., quarantine dataset exports).- If de-anonymization needed: require multi-person approval, log request, decrypt via KMS audit trail.- Post-incident: preserve logs, root-cause analysis, revoke credentials or change keys, notify stakeholders per policy.

Best practices & trade-offs:- Balance forensic usefulness vs exposure by using pseudonyms + guarded de-anonymization.- Prefer storing fingerprints and metadata over raw queries/PII.- Tune thresholds iteratively to reduce false positives; use ML anomaly detection for behavioral baselines.- Regular audits, key rotation, and red-team exercises to validate detection coverage.

HardSystem Design

123 practiced

Architect a production-scale privacy-preserving experimentation platform supporting 50M monthly users, concurrent A/B tests, and ML training pipelines while ensuring a per-user annual epsilon ≤ 2. Describe components (ingestion, privacy ledger, accountant, secure aggregation), dataflows, storage and retention policies, monitoring, and operational controls to enforce and audit guarantees.

Sample Answer

Requirements & constraints:- 50M monthly active users, many concurrent A/B/ML experiments, per-user annual ε ≤ 2 (DP budget).- Must support analytics, feature/label collection for ML pipelines, and auditability.

High-level architecture:- Client SDKs / edge collectors → Ingestion layer → Privacy Proxy / Secure Aggregator → Privacy Ledger + Accountant → DP-safe data stores → Downstream analytics/ML pipelines (with DP mechanisms) → Monitoring & Audit UI.

Components and responsibilities:1. Ingestion- Lightweight clients sample/report events, include stable user IDs hashed with salt.- Rate-limits and per-endpoint sampling to reduce per-user contribution.- TLS, auth, signed requests.

2. Privacy Proxy & Secure Aggregation- Edge proxy enforces local pre-processing (clipping, quantization) and optional local randomization (e.g., generalized randomized response) where applicable.- For high-granularity statistics and ML training, use secure aggregation (MPC or VDAF) so raw contributions are never visible unaggregated. Aggregators run multi-party protocol; only aggregates released to analyst pipelines.

3. Privacy Ledger- Immutable append-only ledger (write-once storage like cloud object store + signed Merkle roots) recording: user hashed-id, event meta (type, timestamp bucket), contribution weight, mechanism used, ε charge, experiment id, sampling flags.- Ledger partitions by time window and experiment; retention for audit window.

4. Privacy Accountant- Global centralized accountant service consumes ledger; computes per-user cumulative ε (using composition rules: advanced composition, moments accountant for DP-SGD, amplification by subsampling).- Enforces per-user, per-year cap: if an incoming event would cause exceedance, block or downgrade to zero-contribution channel.- Exposes APIs for experimenters to query remaining budget and simulate projected spend.

5. DP Mechanisms- For ML training: DP-SGD with moments accountant; clip gradients, add calibrated Gaussian noise; use subsampling and secure aggregation to compute noisy gradients.- For A/B metrics: apply sample-and-aggregate with calibrated noise (Laplace/Gaussian), or use aggregation over cohorts via secure aggregation then additive DP noise.

Dataflows:- Events → Privacy Proxy (clip, sample) → Optional local randomization → Secure Aggregator (MPC) → Aggregates → DP noise added (if not already private) → Store in DP dataset.- Ledger entries written at ingestion and finalized post-aggregation; Accountant processes ledger to update budget.

Storage & retention:- Raw raw-level unhashed PII never stored. Hashed IDs only in ledger and short-lived caches.- Raw encrypted buffers for MPC participants only until aggregation; zeroize after aggregation.- DP datasets stored long-term for analytics; raw/detailed intermediate artifacts retained only for minimal audit window (e.g., 90 days) then purged.- Ledger retention for audit: keep signed ledger for 3–7 years (regulatory dependent) but store only hashed identifiers and ε charges.

Monitoring & operational controls:- Real-time telemetry: per-user ε burn rate distribution, active experiments, per-experiment expected spend, MPC health, aggregator liveness, noise scale metrics.- Alerts: unusual surge in ε consumption, ledger write failures, MPC party dropouts, budget exceed attempts.- Quotas & circuit breakers: global and per-experiment budget planners; auto-throttling experiments when global per-user burn risk high.- CI/CD gated by privacy checks: experiment manifests must declare expected touchpoints, sampling rates, DP mechanism; automated static checks estimate projected ε.

Auditing & compliance:- Reproducible auditor pipeline reads ledger and recomputes per-user composition; verifies signatures and Merkle root.- Periodic third-party audit capability by exporting signed aggregates and ledger proofs.- Experiment-level provenance: who launched, config, expected ε, approval workflow.

Key calculations & best practices:- Use amplification by subsampling (Poisson or uniform) and secure aggregation to reduce noise needed.- Prefer Gaussian noise with moments accountant for ML; use analytical composition to track cumulative ε.- Conservative defaults: cap per-experiment ε substantially lower than annual cap; require approval for higher.- Run simulations to convert sampling/noise parameters into expected utility and ε.

Example: DP-SGD pipeline- Each user contributes at most k examples per epoch (clipping to L2 norm C).- Subsample ratio q = batch_size / population → amplification factor.- Noise σ chosen so that per-step (q, σ) yields ε_step via moments accountant; accountant composes steps across epochs to ensure annual ε ≤ 2.

Why this design:- Secure aggregation + ledger preserves privacy while retaining auditability.- Centralized accountant enforces guarantees proactively rather than retroactively.- Sampling, clipping, and DP mechanisms provide tunable trade-offs between utility and privacy.- Operational controls and immutable ledger support compliance and reproducibility.

Practical rollout steps:1. Start with analytics pipelines using secure aggregation + additive DP for metrics.2. Add ledger+accountant; enforce experiment approval gates.3. Gradually onboard DP-SGD training with MPC-based gradient aggregation.4. Monitor, tune noise/sampling, and publish utility/privacy trade-offs for stakeholders.

EasyTechnical

76 practiced

Describe a consent management strategy for experiments and analytics. Which consent metadata should be stored (purpose, timestamp, version, scope), how do you support revocation, and how should consent status affect sampling and data retention for A/B tests and model training?

Sample Answer

Situation: Designing a consent-management strategy to ensure experiments and analytics respect user choices (GDPR/CCPA).

Consent metadata to store (per user, per consent action):- user_id (hashed/pseudonymized) - purpose (enum: analytics, personalization, A/B-testing, model-training) - scope (channels/data sources: web, mobile, server-logs) - status (granted / revoked) - timestamp (ISO8601) - version (policy/version id) - granularity (global, product-level, experiment-level) - provenance (consent UI id, IP, locale) - expiry (optional TTL)

Why: purpose+scope allow precise enforcement; version/timestamp support audits and rollback.

Supporting revocation:- Immediate enforcement for future actions: check consent store at exposure/training time; block sampling if no consent.- For already collected data: apply policy by purpose/version — either delete, anonymize (irreversibly), or mark “no-longer-processable” (tombstone) depending on legal/regulatory requirements.- Maintain an immutable audit log of consent changes for compliance.- Provide an async workflow: revoke → stop future exposures → queue job to purge/anonymize historical data from analytics stores, model training datasets, and metrics pipelines; track progress and notify stakeholders.

How consent affects sampling and training:- Sampling: - Only include users with explicit consent for the experiment purpose. Use consent-aware sampler that checks consent store at assignment time. - For experiments where consent changes mid-exposure, do not retroactively reassign users; instead stop showing experiment variants and record exposure termination. For metrics, exclude post-revocation events.- Data retention and model training: - Exclude data without model-training consent from training datasets. If revocation occurs and policy requires deletion, remove affected records from training data and retrain if necessary (or flag model lineage to exclude impacted training runs). - For experiments metrics, maintain two datasets: full metrics (for internal debugging, with legal clearance) and consented metrics (official results). Use consented metrics for business decisions. - Enforce retention TTL per purpose (e.g., analytics: 13 months; training: minimal necessary period) and implement automated purge jobs that respect consent scope/version.

Operational practices:- Consent-aware instrumentation: centralized library that attaches consent metadata to events and enforces checks before logging or exposing payloads.- Monitoring & auditing: dashboards for consent status distribution, pending purge jobs, and experiment coverage by consent.- Minimal data principle: only collect fields required for the purpose; pseudonymize identifiers.- Documentation: map legal requirements to technical actions for each purpose/version.

Example enforcement pseudocode:

python

if consent_store.has(user_id, purpose='A/B-testing'):
    assign_to_experiment(user_id)
else:
    assign_to_control_or_exclude(user_id)

This approach ensures legal compliance, reproducible experiments (consented metrics), and defensible model training pipelines while minimizing data scope and supporting user rights.

HardTechnical

56 practiced

Propose a design for a privacy-preserving multi-armed bandit system where arms are adapted over time and users may receive multiple treatments. Explain how to bound each user's cumulative privacy loss, allocate epsilon between exploration and exploitation, and implement private reward reporting.

Sample Answer

Requirements & goals:- Arms adapt over time; users may receive multiple treatments across rounds.- Provide strong per-user differential privacy (DP) guarantee: each user's cumulative interactions have ε_user (and δ) bound.- Maintain learning utility (low regret).

High-level design:1) Privacy accounting per user- Treat a user's entire sequence of reports as a single DP subject. Use composition theorems (advanced composition or Rényi DP / Moments Accountant) to convert per-interaction noise budgets ε_t into an overall ε_user: ε_user ≈ sqrt(2 log(1/δ) Σ_t ε_t^2 ) + Σ_t ε_t (advanced) or use RDP for tighter sums. Set a target ε_user (policy decision) and enforce Σ_t via the chosen accountant.

2) Epsilon allocation between exploration & exploitation- Split total per-user budget ε_user between exploration and exploitation phases: ε_explore = α ε_user, ε_exploit = (1−α) ε_user. Choose α by simulation (α~0.4–0.6 often balances learning vs final personalization). More principled: allocate per-round ε_t proportional to 1/√t (i.e., ε_t = C / √t) so Σ_t ε_t = O(√T) and variance per-round decays slowly—this limits cumulative privacy while focusing budget early when exploration yields most information. Solve C so Σ_t ε_t ≤ ε_user via targeted T or tail truncation.

3) Private reward reporting (per-interaction mechanics)- Clip rewards to a known bounded range [0,1] to control sensitivity.- Choose mechanism: - Central model with secure aggregation: users add Gaussian noise calibrated for per-report (σ = sqrt(2 log(1.25/δ')) / ε_t ) then server aggregates—secure aggregation reduces required noise if many users report same arm. - Local DP if no trusted aggregator: randomized response or Laplace/Gaussian per-report. Use Gaussian/Laplace noise calibrated to ε_t, sensitivity 1/T_clip.- Use sub-sampling or shuffling when available to amplify privacy (reduces effective ε).

4) Private bandit algorithm- Use DP-UCB or private Thompson Sampling: - DP-UCB: maintain noisy counts and rewards with DP additives; compute UCB with noise-aware confidence intervals that include privacy noise variance. - Private Thompson Sampling: maintain posterior with noisy sufficient statistics (private counts and sums).- Account for adaptive selection bias: use bootstrap or importance-weighted estimators when reporting comes from adaptive sampling.

5) Bounding cumulative privacy loss & practical settings- For each user i, choose ε_user,i and per-round sequence ε_{i,t} (e.g., geometric or 1/√t schedule). Use RDP accountant to produce tight ε_user,i at time horizon or stop when budget exhausted (deny further personalization).- Example schedule: for T_max rounds, set ε_{t} = ε_user / (2 √T_max) * (1/√t) normalized so Σ_t ε_t ≤ ε_user. Alternatively, allocate fixed ε for first K exploratory exposures, then small ε for exploitation logs.

6) Implementation details & safeguards- Always clip rewards and features before noise.- Use secure aggregation + central DP when possible to dramatically reduce per-user noise.- Track per-user cumulative ε spent and enforce limits (graceful degradation: fall back to public model).- Validate via simulations to quantify regret vs ε trade-off.

Trade-offs:- Local DP is strongest trust model but higher noise (worse utility).- Secure aggregation + central DP gives better utility if trust/secure compute is available.- Allocation schedule (front-loaded vs decaying) affects early learning speed vs long-term personalization.

Takeaway: pick a target per-user ε, use a principled per-round schedule (e.g., ε_t ∝ 1/√t) with RDP accountant, clip rewards, use Gaussian/Laplace noise or secure aggregation, and plug noisy sufficient statistics into DP-UCB/TS with noise-aware confidence intervals. Tune α and schedules by simulation to balance regret and privacy.

EasyTechnical

119 practiced

Compare and contrast de-identification, pseudonymization, and anonymization. For each term define it, describe common techniques (masking, hashing, generalization), and explain situations where de-identification may still allow re-identification risk.

Sample Answer

De-identification, pseudonymization, and anonymization are related but distinct goals for protecting personal data.

Definitions- De-identification: Any process that removes or transforms identifiers so records cannot be readily attributed to an individual. It’s an umbrella term that includes pseudonymization and anonymization.- Pseudonymization: Replaces direct identifiers (name, SSN) with pseudonyms/tokens (random IDs, hashes). A reversible mapping or key exists (held separately) so re-identification is possible if keys are accessible.- Anonymization: Irreversibly transforms data so individuals cannot be re-identified by any reasonable means. No feasible re-linking mechanism remains.

Common techniques- Masking: Redact or partially show fields (e.g., show last 4 digits of phone). Used in de-identification and pseudonymization.- Hashing / Tokenization: Replace identifiers with hashes or tokens. If salted and keys are managed, this is pseudonymization; unsalted hashes of known values can be vulnerable.- Generalization: Replace exact values with bins or ranges (age 37 → 30–40) or remove location precision. Used to reduce uniqueness.- Suppression: Remove high-risk attributes or rare values.- Noise addition / Differential privacy: Add calibrated noise to outputs or use DP mechanisms to achieve stronger anonymity guarantees.

When de-identification still permits re-identification- Quasi-identifiers: Combinations of non-PII (age, ZIP, gender) can uniquely identify people when linked with external datasets (linkage attack).- Small subpopulations or rare attributes make records unique despite removed direct identifiers.- Weak hashing: Unsalted hashes of emails/IDs are reversible via rainbow tables or guessed inputs.- Linkability via longitudinal data: Time-stamped events can reveal identity when correlated with external logs (e.g., hospital visits).- Inadequate aggregation: Overly granular generalization leaves high re-identification risk.

Practical guidance for a data scientist- Treat pseudonymized data as personal data and protect keys separately.- Quantify re-identification risk (k-anonymity, l-diversity, t-closeness) and consider differential privacy for analytics.- Combine techniques: generalize quasi-identifiers, suppress outliers, use tokenization with access controls, and audit linkage risks before sharing.

Unlock Full Question Bank

Get access to hundreds of Privacy-Preserving Experiment Design interview questions and detailed answers.

Join thousands of developers preparing for their dream job.