Model registries are now standard components of mature ML platforms. A model is trained, evaluated, registered with a version number, and that version is what gets deployed. If a decision needs to be reviewed, the team can retrieve the model version that was active at the time. This is progress. It is also incomplete.
The model version is one input to an output. The other inputs are the features that were fed to the model at the time of inference — and features are not static. Feature pipelines update. Data sources change. Schema migrations occur. A feature value computed on 14 March may be different from the value computed on 15 March from nominally the same data source, because something upstream changed. The model version is identical. The input is not. The output is different. And without a decision log that captures the feature state at the time of each inference, that difference is invisible.
The Dutch Belastingdienst case — the same governance failure that underlies the Toeslagenaffaire — illustrates the downstream cost of this invisible difference with precision. The Donner Report found that the tax authority's system produced outcomes that could not be reconstructed or audited after the fact, not because the model was undocumented, but because the input state at the time of each decision had not been logged in a way that allowed individual outcomes to be examined.
What the Donner Report established about decision auditability
The Donner Report's analysis of the Belastingdienst's systems found a consistent pattern: individual decisions — the assignment of a risk score to a specific benefit application — could not be reconstructed after the fact because the inputs to those decisions were not logged at the granularity required to reproduce the output. The model had documentation. The data had lineage. The individual decision — the combination of model state and input state that produced a specific score for a specific family — had neither.
This made it impossible to challenge individual decisions through any administrative review process. A family that believed their score was incorrect could not be told what inputs had driven it, because the inputs were not preserved. A reviewer investigating systematic bias could not examine whether the model had behaved differently for different demographic groups, because the individual decision records did not contain the feature values that would reveal that difference.
"The system was not designed to be auditable at the level of individual decisions. The inputs to the risk model were not systematically logged alongside the outputs. This made it impossible to reconstruct why a specific application had received a specific score, to identify whether the model had applied consistent criteria, or to determine whether particular characteristics of applicants had systematically influenced outcomes. The absence of decision-level logging was not an oversight — it reflected an assumption that the system's outputs would not be individually challenged."
The EU AI Act's Article 12 makes decision-level logging a structural requirement for high-risk AI systems. The requirement is specific: AI systems must be designed with logging capabilities that allow for the monitoring of their operation and enable post-hoc verification of outputs. For systems that affect individuals, the log must contain sufficient information to reconstruct the output — which means the feature state at the time of inference, not just the model version.
The four components of a decision record
A decision record is not the same as a prediction log. A prediction log records the model's output. A decision record records everything that would be needed to reconstruct the output and defend it to an auditor, a regulator, or an individual who believes it was incorrect. The distinction is the difference between logging what the system said and logging why it said it.
The storage and retention question
The objection most teams raise to decision-level logging is storage cost. Storing feature snapshots for every inference at production scale is expensive. The objection is valid and the resolution is proportionate: not every inference requires a full decision record. The logging requirement is risk-stratified.
For inferences that produce outputs with low impact on individuals — a product recommendation, an internal content ranking — a prediction log is sufficient. For inferences that produce outputs that are acted on against individuals — benefit decisions, credit decisions, medical alerts, fraud flags — a decision record is required. The distinction is not technical. It is the same distinction the EU AI Act draws between high-risk and non-high-risk AI systems: the severity and reversibility of the impact on the individuals affected.
The decision record retention period is also risk-stratified: it must cover at least the period during which a decision can be legally challenged. For benefit decisions in many jurisdictions, this is five to ten years. Designing a logging system that stores full decision records only for high-impact inferences, with a defined retention period, is a tractable engineering problem. It is also a design decision that must be made before deployment, not after a regulatory inquiry asks why the records do not exist.
For any AI system whose outputs are acted on against individuals, add four fields to the inference log — not the model registry, the inference log — for every output that triggers a downstream action.
Feature snapshot: the actual feature values fed to the model at inference time, stored as a versioned JSON record associated with the entity ID and timestamp.
Model state reference: a pointer to the exact model artefact — not just the version number, but the specific registered artefact hash — that produced the output.
Output with confidence: the prediction label and the associated probability or confidence score, not just the label.
Downstream action: a structured record of what action was taken as a result of this output — not the system's recommendation, the action that was actually executed.
These four fields, associated with the entity ID and timestamp, constitute a decision record. A decision record is what the Belastingdienst did not have. It is what Article 12 requires. And it is what transforms a model registry from a versioning tool into an audit trail.