Overview: Goal is full observability for debugging while preventing PII leakage. Deliver a pragmatic, cross-functional policy that engineers can implement, legal/compliance can sign off, and product/ops can monitor.
- Data classification (must-have)
- PII-sensitive: full name, SSN, government IDs, credit card numbers, full email, phone, payment tokens.
- PII-low-risk: hashed user IDs, geo to city level, obfuscated email domain.
- Non-PII: telemetry, timings, error types, stack traces (without PII).
- Logging policy & formats
- Use structured logs (JSON) with well-known fields (user_id, session_id, error, payload).
- Never log raw request bodies; parse and sanitize before emitting.
- Masking / redaction rules
- Deterministic rules: regex-based redaction for card, SSN, emails — replace with REDACTED:TYPE.
- Partial masking for debugging: show last 4 digits of card, first letter + masked rest of email.
- Contextual logs: avoid concatenating multiple fields into single free-text messages.
- Reversible vs irreversible
- Irreversible hashing (SHA-256 with per-service salt) for analytics IDs used in alerts — preserves correlation without revealing PII.
- Reversible tokenization only when necessary for support workflows: store token in logs and keep mapping in a secure vault (KMS/ HSM) accessible via strict RBAC and ephemeral keys; require approval and audit trail.
- Access controls & auditing
- Role-based access to log stores; separate privileges: developer (redacted logs), SRE (redacted + broader telemetry), support/admin (tokenized access with justification).
- Use IAM + single-sign-on + time-limited access; require just-in-time elevated access with approval workflow.
- All access and any de-tokenization requests are logged, signed, and retained.
- Retention & lifecycle
- Short retention (30 days) for raw structured logs containing any tokenized identifiers.
- Long-term aggregated/irreversibly-hashed telemetry: 365 days.
- Automated purge jobs and legal hold overrides, with alerts on failures.
- Operational governance & testing
- Integrate static checks and unit tests to ensure no PII fields are emitted.
- CI gating: linting for log statements, sandboxed leakage scanner on PRs.
- Periodic audit (quarterly) and breach drills.
- KPIs & rollout
- Measure false-positive/negative masking rate, number of elevated-access requests, mean time to redact, and observability “debug-success” (time to resolve errors post-mask).
- Roll out with a pilot on one service, monitor impact on debug time, iterate.
Example: On 500-error, log:
{
"error":"PaymentFailure",
"user_id_hashed":"sha256(salt+12345)",
"card_last4":"4242",
"amount":"29.99",
"request_id":"r-abc"
}
Full card number never logged. De-tokenization requires support approval and is recorded.
This balances debugging needs with privacy, enforceable through automation, RBAC, and monitoring.
