Data Classification and Sensitivity Handling Questions

Classifying data by sensitivity and applying controls proportionate to that classification: identifying personal, sensitive, and special-category data and tagging it through its lifecycle. Covers classification schemes, labeling, and how classification drives access, encryption, and retention decisions. Includes assessing the impact of a given data type on privacy and security risk.

MediumTechnical

39 practiced

A team requests a new ML feature that requires PII (email, phone). Explain the data privacy and security considerations you must address before collecting and using PII in model training and serving. Include at least four concrete controls or practices.

HardTechnical

40 practiced

Problem solving (hard): Design an automated compliance check system that detects whether datasets or model inputs contain PII before allowing training in Azure ML. Include how you'd use Azure Purview for data classification, metadata tagging, blocking policies (policy-as-code), and operator workflows for remediation and approval.

Sample Answer

Requirements & constraints:- Prevent training when inputs/datasets contain PII (false positives tolerated lower than false negatives)- Integrate with existing Azure data stores (Blob, ADLS Gen2, SQL), Databricks, Azure ML- Automatable, auditable, operator-approved remediation path, policy-as-code, low latency for CI/CD pipelines

High-level architecture:Data sources → Azure Purview (scan & classifier) → Purview catalog / metadata tags → Event Grid → Compliance Engine (Policy-as-Code) → Azure ML gating + Quarantine area → Operator Remediation & Approval UI -> Git-backed policy repo

Core components & flow:1. Discovery & classification- Schedule Purview scans on registered assets (Blob/ADLS/SQL, Databricks tables).- Use built-in & custom classifiers (regex, ML-based) to surface PII types (SSN, email, PHI). Purview produces sensitivity labels & glossary terms and stores lineage.

2. Metadata tagging & propagation- Purview writes tags to the catalog and uses Microsoft Information Protection labels where supported.- On new dataset registration (or ML pipeline start), pipeline queries Purview API to fetch sensitivity tags and lineage.

3. Policy-as-code & enforcement- Implement policies in a Git repo using Azure Policy + custom Rego (OPA) or Azure Policy definitions that evaluate Purview metadata.- Compliance Engine (serverless function / Azure Container App) subscribes to Event Grid; on events or pipeline calls, it runs policies (Rego/Azure Policy) against asset metadata.- If PII absent → gate passes and training continues. If PII detected → action depends on policy risk level: - Block (hard fail) for "sensitive" data - Quarantine for "possible PII" (move snapshot to secure container with restricted ACLs) - Allow with approval for "low risk" with enforced differential privacy / masking

4. Integration with Azure ML- Azure ML training jobs call Compliance Engine pre-start (via REST). The engine returns allow/block and an audit token. Azure ML SDK enforces that only jobs with allow token can access training data paths (managed identity + Azure RBAC).

5. Operator workflows (remediation & approval)- Automated remediation: recommended masking scripts (Databricks notebooks) are auto-generated templates based on detected PII types (hashing, tokenization, redaction, synthetic data synthesis).- Operators see findings in an Approval UI (Power Apps / Logic Apps + Azure AD SSO) with context: dataset preview (redacted), Purview classifier confidence, lineage, suggested remediation, estimated utility impact.- Approvals performed via signed Git PRs to policy repo or via UI action that issues an allow token scoped to the job and TTL. All actions produce audit logs in Azure Monitor / Log Analytics & Purview.

6. Observability & metrics- Track: % datasets scanned, % blocked trainings, classifier precision/recall (via sampling), time-to-approve, number of policy overrides.- Alerts for spikes in blocked attempts.

Security & governance trade-offs:- Relying solely on pattern matching yields false positives/negatives — mitigate with ML classifiers and human-in-the-loop.- Blocking can impede ML velocity; provide clear SLAs for approvals and automated masking templates to reduce friction.

Implementation choices & technologies:- Classification and catalog: Azure Purview- Policy engine: Azure Policy + OPA/Rego in a Compliance Engine microservice- Events & orchestration: Event Grid + Azure Functions / Logic Apps- Storage quarantine: ADLS with strict ACLs, managed identity- Approval UI: Power Apps/React + Azure AD; Git-backed policy-as-code in GitHub/Azure Repos- Audit: Azure Monitor, Purview activity logs, immutable audit storage in Log Analytics / Storage Account

Example operator flow:- Databricks job writes dataset → Purview scan flags “email” with 92% confidence → Event Grid notifies Compliance Engine → policy marks as “quarantine” → Azure ML training blocked; dataset moved to quarantine container and masking notebook created → operator receives notification, runs masking notebook in dev environment, re-registers masked dataset → Purview rescans, passes policy → operator approves training; Azure ML starts using allow token.

Why this works:- Purview centralizes classification & lineage so policies act on trusted metadata.- Policy-as-code ensures reproducible enforcement and auditability.- Human-in-loop remediation balances risk and ML productivity.

Unlock Full Question Bank

Get access to hundreds of Data Classification and Sensitivity Handling interview questions and detailed answers.

Join thousands of developers preparing for their dream job.