Audit Readiness, Evidence and Inspection Management Questions

Direct answer. Run it as a joint evidence program with one accountable owner. Define the audit trail up front (what must be provable about each decision), have each function own specific artifacts, version everything (data, features, model, code, config), and build a replay capability in a controlled environment so an auditor can reproduce a decision with protected data masked or accessed under supervision.

Coordination (who owns what)

• Legal and compliance: which regulations and what evidence they require, retention periods, what may be disclosed. Counsel decides on legal interpretation.
• Security and privacy: access control, encryption, masking, who can see what in the replay.
• Data engineering: training data lineage and snapshots.
• ML engineering: model versions, feature transformations, reproducible training.
• Model risk or a second-line reviewer (an independent oversight team, separate from the people who build the model, that challenges and approves it): validation reports and approvals.
  Start with a short kickoff to agree the evidence list, owners and dates, with a RACI (who is responsible, accountable, consulted, informed).

Evidence to preserve

• Training data lineage: sources, extraction queries, dataset snapshot IDs and checksums, filters, labels.
• Feature transformations (features are the input values the model scores, such as income or account age, calculated from raw data): code under version control, feature definitions and versions, so the same raw input produces the same features.
• Experiment record: parameters, random seeds (the fixed starting numbers for the training run's random choices, so rerunning training gives the same model), metrics, and the choice of the final model.
• Approvals: validation results and sign-offs, stored with the model version.
• Production decision log: for each decision, the model version, input features, output score, decision and time.

Replay without exposing sensitive data. Store inputs for each decision in a protected store. For an audit, the auditor chooses a decision ID; replay runs in an isolated environment with the exact model and feature versions and gives the same score. Identifiers are pseudonymized (replaced with tokens, with the mapping held separately) and access is time-limited and logged. Where raw data must be viewed, supervised access in a secure room or environment.

Worked example. Decision ID 88123 (illustrative): model v2.4, feature set f-17, inputs logged. Replay reproduces the score 0.31 and the decline, with the applicant's name replaced by a token (pseudonymized). If replay differs, the difference is itself a finding to investigate (changed feature code, a non-pinned dependency).

Retention versus privacy. Keeping everything forever conflicts with data minimisation. Set retention per artifact class: model versions, validation reports, approvals and decision logs for the longest applicable regulatory period (legal confirms the figure); raw training snapshots only as long as needed to reproduce or defend the model, then keep hashes, schemas and aggregate statistics instead of personal rows; delete or archive under legal hold rules when a dispute is open. State which classes are kept as pseudonymized records and who holds the token mapping.

What the replay must also show. Besides the score, record the decision threshold or policy rule that turned 0.31 into a decline (for example, decline below a cut-off set in the approved policy version), and the reason codes given to the applicant, so the auditor can follow score to decision to notice.

Pitfalls. Features computed differently in training and serving; training data refreshed in place; logs without model version; sharing raw personal data for convenience.

Direct answer. A foundation model is a large general-purpose model trained by a vendor (for example, a large language model) that you build on. I would tell the regulator exactly what we can evidence (our use case, inputs, wrapper logic, tests, monitoring and oversight), what the vendor can evidence (documentation and assurances), and what neither of us can show (the training data itself). Then I would put in compensating measures that test the model's behavior from the outside, and share sensitive material through controlled review rather than handing it over or pretending it does not exist.

1. What we provide

• Our system: purpose, inputs and prompts, decision logic around the model (the wrapper logic: our own code that prepares inputs, calls the model and handles its outputs), output handling, human review points, and logs.
• Our own evaluation of the model on our data: accuracy, error patterns and fairness checks, with versions and reproducible runs.
• Vendor material: model or system card (the vendor's document describing what the model is for, how it was tested and its limits), published training-data summary or policy if one exists, security and assurance reports (independent audit reports about the vendor, such as a SOC 2 report), contract terms on data use.
• A gap register listing each regulator question we cannot answer from first-hand evidence and why. Example row (illustrative): Question: "Was the training data screened for personal data?" | Evidence we hold: vendor's published data summary | Missing: the data itself | Measure: output testing for personal-data leakage | Owner: security lead | Review date: next model version.

2. Compensating measures
A compensating control is a different safeguard that reduces the same risk when the ideal control is unavailable.

3. Protect IP and stay honest
Share through NDAs, a supervised review room, and summaries or hashes in place of source, weights or prompts. A hash is a fingerprint of a file: it reveals nothing practical about a large, high-entropy file such as model weights (a short or guessable text such as a one-line prompt could be confirmed by someone who guesses it, so do not rely on a hash to hide short secrets), but it lets the regulator confirm later that the file they review in the supervised room is the same one we fingerprinted today. Review of the contents themselves happens in the supervised room. Never answer a question with a statement we cannot back, such as saying the data was checked for bias when we only tested outputs. Say "we cannot verify this, here is what we do instead". Counsel decides what legal obligations the vendor has to the regulator and how to word disclosures.

What changes my call. If the use is high-impact and no compensating measure gives adequate confidence, the honest recommendation is to switch vendors or add a human decision step before shipping.

Direct answer. I would answer the regulator with a short evidence pack that proves what the model was given and what happens to the data, separate from a clear statement of what I can only show indirectly. "Special category" data is the set of sensitive types in GDPR Article 9 (racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, genetic data, biometric data used to identify a person, health data, sex life or sexual orientation). Article 5(2) makes the controller (the organisation that decides why and how personal data is processed) responsible for being able to demonstrate compliance, so the burden is on us to show it, not on the regulator to find it. Legal counsel decides what we claim; the engineering job is making each claim traceable to an artifact.

1. Evidence that the model does not use special category data (input provenance)
Terms: the model registry is the system that stores each trained model version with its metadata; the serving signature is the declared list of inputs and outputs the deployed model accepts.

• Feature manifest: the exact list of input columns the deployed version consumes, exported from the model registry or the serving signature, with the model version and training-run ID.
• Data dictionary with classification tags: each source column labelled (special category, personal, non-personal) by a named owner. Show that no column tagged special category appears in the manifest.
• Lineage: a record (data lineage) tracing each training and serving input back to its source system, so the regulator can see where a column came from. Lineage matters because a "neutral" column can be derived from a sensitive one.
• Schema enforcement: the training and serving pipelines reject unexpected columns (a schema check in the pipeline code and its recent run logs).

2. Evidence that it is trustworthy and reproducible

• Reproducibility record: code commit hash, data snapshot identifier plus a content hash (a fingerprint computed from the file's bytes, so any edit changes it), library versions, random seeds, and the config. Then actually re-run training in a clean environment and show it yields the same feature manifest and comparable metrics (identical weights are not always achievable, so say what is and is not bit-exact, meaning byte-for-byte identical. For example, a rerun giving AUC 0.912 against the original 0.910 is comparable and explainable by random seeds or hardware; a rerun with a different feature list is not).
• Tamper evidence: artifacts stored in write-once or versioned storage, with hashes recorded in a ledger kept by someone outside the model team (for example internal audit's log). If anyone later edits an artifact, its hash no longer matches the ledger entry written on the day the request arrived, so the edit is detectable.
• Independent check: someone outside the model team (a second engineer or internal audit) re-computes the manifest from the deployed artifact and signs off.

3. Evidence for retention rules (storage limitation)
Article 5(1)(e) (the storage limitation principle) says data may be kept in identifiable form no longer than necessary for the purpose. Show: the written retention schedule per dataset, the deletion job configuration, and deletion logs or lifecycle-policy records for each store (raw data, training sets, feature store, logs, backups). Worked example: schedule says raw training extracts are kept 12 months; the pack shows the lifecycle rule and a sample of deletion records with dates, plus the oldest remaining object date in each store, which should be inside 12 months.

4. Being candid about what you cannot fully prove

• Proxies: absence of an Article 9 column does not prove the model does not infer sensitive traits (a proxy is a harmless-looking field that correlates with a sensitive trait; postcode can proxy ethnicity). Say so, and show the proxy testing you did (for example, checking whether outputs differ across groups using a held-out labelled sample: if approval rates were 61% for one group and 58% for another, that gap is evidence to investigate rather than proof of use of a sensitive attribute. The sample is itself special category processing and needs its own legal footing: an Article 6 lawful basis (the legal ground GDPR requires for processing any personal data) AND a separate Article 9(2) condition (one of the listed exceptions to the general ban on processing special category data, such as explicit consent or substantial public interest on a legal basis), plus safeguards such as tight access and deletion of the sample after the test).
• Backups and copies: deletion in primary stores is evidenced; say which backups expire later and when.
• History before logging existed: state the date from which evidence is complete and what is reconstructed.
• Negative claims: "never used" can only be shown for the versions and period we have evidence for.

Trade-offs and pitfalls. Over-claiming is the real risk: a confident "no sensitive data anywhere" that a later finding contradicts damages credibility more than a bounded statement. Also do not copy sensitive data into the evidence pack; share the manifest and tags, not rows.

Direct answer. I would treat audit readiness as a delivery capability: build the evidence into the normal ML workflow, designate one audit owner who absorbs the audit load, and set up a findings process before the first finding arrives. Engineers keep shipping because the audit team pulls evidence from systems rather than asking engineers to assemble it.

1. Prepare the organization (before the audit)

• Inventory every ML system and decide which are in scope under the regulation (here, the EU AI Act, which sets extra duties for systems it calls high-risk, such as AI used to screen job applicants or score creditworthiness; counsel confirms classification and which assessment route applies, internal or third-party, and the current application dates).
• Give each in-scope system an owner and a standard evidence set: intended purpose, data sources and versions, evaluation results, risk assessment, human oversight design, change history.
• Generate that evidence automatically in the pipeline (model registry entries, which record each model version with its data and settings; versioned data; saved evaluation reports, including a model card, a short standard document describing what the model is for, how it was tested and its limits), not by hand at audit time.
• Run an internal mock audit (a practice audit run by your own staff) on one system to find gaps while stakes are low.

2. Handle the audit itself

• One liaison (the single point of contact between auditors and staff) and a request tracker (request, owner, due date, status). Engineers answer through the liaison.
• Time-box engineer involvement (illustratively two scheduled interview slots per team per week) so delivery planning stays predictable.
• Protect confidential material: NDAs, a read-only data room (a controlled space holding only the documents the auditor may see), redacted or summarized views, supervised demonstrations in place of handing over model weights or raw data, and a log of what the auditor saw.

3. Respond to adverse findings
An adverse finding is an auditor's conclusion that something falls short. Example tracker and finding (illustrative):

Triage by severity, correct any factual misreading with evidence, then give each accepted finding an owner, a root cause, a dated remediation plan and a retest. Report progress to leadership until closure. Where a fix is slow, propose a compensating control (an alternative safeguard that reduces the same risk) with an expiry date.

Trade-off. Up-front automation costs engineering time now, and I would fund it for the highest-risk system first. If most findings would turn out to be documentation gaps rather than behavior problems, I would invest in templates and model-card standards before tooling.

Direct answer. I would hand over a self-contained evidence bundle: the test plan, the data and model versions tested, the results and decisions taken, and a signed manifest of hashes (the manifest itself carries a digital signature from a named release owner). Reproducibility comes from pinning every input so a third party can rerun and get the same numbers. Tamper evidence comes from hashing each artifact, recording the hashes in write-once storage with timestamps, and showing the regulator a verification run.

1. What evidence to give
Two terms first. A fairness measure compares how a model treats different groups (for example, approval rates by group). A group label is the attribute that says which group a record belongs to.

• Test plan written before testing: which groups were compared and why, which outcome measures, and the thresholds that count as acceptable, with who approved them.
• Inputs: model version, the evaluation dataset version (with how it was built and how group labels were obtained), and code commit.
• Results: raw per-group outputs and the summary report, including failures and what was done about them (retrain, mitigate, accept with sign-off).
• Decision record: who reviewed, what was decided, any residual risk accepted.

Worked fairness test (illustrative numbers, written into the plan before testing): groups A and B, 2,000 evaluation cases each; outcome measure is approval rate; acceptable means a gap of at most 5 percentage points and a ratio (lower rate divided by higher rate) of at least 0.80. Result: A approved 1,240 of 2,000 (62%), B approved 1,100 of 2,000 (55%). The gap is 7 points and the ratio is 55/62 = 0.89. With 2,000 cases per group a 7-point gap is far outside chance variation (a two-proportion z statistic of about 4.5), so it cannot be dismissed as noise. The ratio passes, the gap fails, so the result is a finding: the decision record says what was done (retrain, mitigate, or accept with a named sign-off).

2. Show the testing is reproducible
Pin the code commit, the container image or lockfile (the packaged runtime, or the file listing exact library versions), the random seed, the exact data snapshot (a frozen copy of the evaluation data, identified by hash), and the model artifact hash (the saved model file is the artifact). Rerun in a clean environment and compare the output hash and headline numbers to the original. Offer the regulator a supervised rerun (they watch you run it in your environment) if the data is sensitive.

3. Show the records were not altered

manifest.json (excerpt, illustrative)
model_sha256:   <hash of model file>
eval_data_sha256: <hash of dataset snapshot>
code_commit:    <git commit id>
report_sha256:  <hash of results report>
recorded_at:    <timestamp from the storage service>
signed_by:      <release owner>  signature: <digital signature over the lines above>

A hash is a fingerprint of a file: change one byte and the fingerprint changes completely. Trace: the string eval_set v3: 4000 rows has SHA-256 starting 445a4c739b219d64; change it to 4001 rows and it starts 36d7a54e249da90d. Verification is recomputing the hash today and comparing it with the recorded one. Compute each hash with a standard tool such as sha256sum, store the manifest in object storage with write-once retention (WORM, write once read many), and restrict who can change retention. A later hash mismatch proves a change. Be precise about the limit: this proves the files are unchanged since recorded, not that the test was well designed.

Trade-offs. Group labels used for fairness testing are often sensitive personal data, so involve privacy and counsel on how they were collected and who can see them. Which fairness measure a regulator accepts is the regulator's and counsel's call, so record why you chose yours.