Requirements clarification: support EU and CA residents, enforce data residency (stay in-region), meet GDPR/CCPA obligations (consent, DSARs, erasure), and ensure deletions propagate to live data, backups, and replicas.
Design summary (architecture + operational controls):
- Data residency
- Region-tagged tenants and sharded storage: separate EU and US clusters (AWS eu-west-1, us-west-2) with IAM boundaries and VPC controls.
- Enforce routing at API gateway using geo-lookup + tenant metadata; deny cross-region export unless explicit legal basis.
- Data transfer logs + continuous compliance checks (AWS Config, Azure Policy).
- Pseudonymization
- Store identifiers in two-tier model: primary key = pseudonymous ID; mapping table (PII) encrypted with KMS keys scoped per-region.
- Use format-preserving tokens or HSM-backed encryption for re-identification; access only via audited microservice with strict RBAC and just-in-time privileges.
- Consent management
- Central consent service storing granular consents (purpose, channel, timestamp, version) in append-only store.
- Policy engine evaluates consent before processing; SDKs propagate consent tokens to downstream services.
- UI for users to view/withdraw consent; periodic consent re-validation flows; audit trail retained.
- Right-to-be-forgotten workflow
- Expose DSAR endpoint triggering orchestrated workflow:
- Mark user record as "erase_pending" in primary DB (soft-delete).
- Kick async fan-out to dependent services via reliable queue (Kafka/SQS) with idempotent handlers to remove or pseudonymize copies.
- Re-identification mapping destroyed only after verifying legal holds; irreversible pseudonymization if required.
- Provide verification to user and regulator with timestamps and proof of propagation.
- Backups and replicas
- Backups are region-scoped and encrypted; retain retention policy matching legal requirements but support expedited purge on valid erasure requests.
- Maintain metadata index of backup segments containing subject IDs to allow targeted deletion where feasible.
- If technical constraints prevent immediate backup purge, document legal basis and schedule; apply cryptographic erasure: rotate/delete KMS keys used to encrypt affected backup segments to render data unrecoverable, and record actions in audit logs.
Operational controls
- Audit logging, SIEM, and tamper-evident logs for all re-identification and deletion operations.
- Regular privacy impact assessments, pen tests, and DR drills for deletion propagation.
- SOPs for legal hold exceptions, cross-border transfer reviews, and incident response.
- SLA for DSAR/erasure timelines aligned to GDPR/CCPA, with monitoring and reporting.
Trade-offs
- Key trade-off between immediate irreversibility vs. ability to restore for legal/operational needs—use soft-delete + escrowed re-id keys with strict governance where needed.
This design ensures region enforcement, minimizes exposure of PII, provides auditable consent and erasure workflows, and handles backups/replicas using targeted deletion and cryptographic erasure when practical.
