An AI impact assessment is a structured, documented evaluation of an AI system's risks to individuals, groups, and organizations, completed before deployment and revisited across the system's lifecycle. For high-risk systems under frameworks like the EU AI Act, this documentation is not optional: it is an obligation, and the organization must be able to produce it on demand.

Most organizations building or buying AI systems have never written one. They have a data protection impact assessment template somewhere, and the instinct is to adapt it. That instinct is half right. An AI impact assessment shares DNA with a DPIA, both force a pause before deployment and both require documented reasoning about harm, but an AI system introduces failure modes a DPIA was never built to catch: model drift, proxy discrimination, automation bias in human reviewers, and outputs that are wrong in ways nobody explicitly programmed.

This article lays out what an AI impact assessment needs to contain, how it differs from a DPIA, and a template you can adapt for a real system going into production.

What Is an AI Impact Assessment, Exactly?

An AI impact assessment is a governance artifact, not a one-time checkbox. It documents what the system does, who it affects, what could go wrong, how likely and severe that is, and what controls reduce the risk to an acceptable level before the system goes live.

It answers five questions an auditor, regulator, or affected individual is entitled to ask:

  • What is this system for, and what decisions does it make or influence?
  • Whose data does it use, and on what basis?
  • What could it get wrong, and who bears the cost when it does?
  • What has been done to reduce that risk?
  • Who approved the residual risk, and when will this be reviewed again?

If the assessment cannot answer all five with specifics, it is a template with blanks, not a completed impact assessment.

How Is This Different From a DPIA?

A Data Protection Impact Assessment focuses on one axis: personal data processing risk. An AI impact assessment is broader by necessity, because an AI system can cause harm without touching personal data at all, and can cause harm that has nothing to do with privacy.

Three differences matter in practice.

First, scope of harm. A DPIA asks whether personal data is adequately protected. An AI impact assessment must also ask whether the system's outputs are accurate, whether they are consistent across demographic groups, and whether an error compounds because the system is making thousands of similar decisions per day rather than one.

First, scope of harm, continued: this is why a hiring-screening tool with no personal data leakage can still fail an AI impact assessment on fairness grounds alone.

Second, the object of assessment. A DPIA assesses a processing activity. An AI impact assessment assesses a model's behavior, which changes. Retraining, fine-tuning, or a shift in the input data distribution can alter what the system does without anyone touching the code. The assessment has to account for drift, not just initial design.

Third, who is accountable for the finding. A DPIA typically sits with a data protection officer. An AI impact assessment usually needs technical sign-off (does the model perform as claimed, on what test set, with what error bounds), legal sign-off (does this satisfy the applicable regulatory basis), and business sign-off (is the residual risk acceptable given what the system is for). One signature is not enough for a high-risk system.

Why Does This Matter Under the EU AI Act?

High-risk AI systems under the EU AI Act carry documented obligations before market placement: a risk management system across the lifecycle, technical documentation, data governance for training and validation data, human oversight design, and post-market monitoring. An impact assessment is the practical mechanism that produces and threads together that evidence.

The obligation is not satisfied by a narrative memo asserting the system is safe. It is satisfied by a document trail: what was tested, against what criteria, with what results, and what was changed as a result. Regulators and auditors are not evaluating intent. They are evaluating evidence.

Organizations outside the EU are not exempt from this logic even where the AI Act does not directly apply. A system deployed against SG PDPA obligations, or evaluated against the Singapore Model AI Governance Framework, faces the same underlying question: can you show your reasoning, not just state your conclusion. The specific citations differ by jurisdiction. The discipline of documented, pre-deployment risk evaluation does not.

When Should an Impact Assessment Be Done?

Before deployment, and again at defined trigger points. Waiting until a system is live to document its risks defeats the purpose: the assessment is meant to change design decisions while they are still cheap to change.

Trigger points that should prompt a new or updated assessment:

  • Initial deployment of any system classified as high-risk or that makes or materially influences decisions about individuals.
  • Material retraining or fine-tuning on a new or expanded dataset.
  • A change in the system's purpose or the population it is applied to.
  • A confirmed incident (a wrong decision, a discrimination complaint, a security finding) that suggests the original risk analysis missed something.
  • A fixed review cycle, commonly annual, even with no known changes, because drift can happen silently.

An assessment written once at launch and never revisited is a compliance artifact for a system that no longer exists in that form.

What Should the Document Contain? A Working Template

The sections below form a working AI impact assessment template. Each should be a substantive entry, not a placeholder, before the system is approved for deployment.

  • System description and purpose. What the system does, what decision or output it produces, and what business or operational problem it solves. Include what happens if the system is wrong: does a human review the output before action, or does the system act autonomously.
  • Classification rationale. Why this system is or is not classified as high-risk under the applicable framework, with reference to the specific criteria used (sector, decision type, population affected, degree of autonomy).
  • Stakeholder and affected-party mapping. Who is subject to the system's decisions, who operates it, and who is accountable for its outputs. Distinguish between the deployer, the developer, and any third party whose data feeds the system.
  • Data governance summary. Source, lawful basis, and quality controls for training, validation, and (where applicable) live input data. Note known gaps or biases in the training data and how they were addressed or accepted.
  • Risk identification. Enumerate specific failure modes: inaccurate outputs, biased outputs across defined subgroups, security vulnerabilities (prompt injection, data leakage, adversarial inputs), overreliance by human reviewers, and lifecycle drift.
  • Risk scoring. Likelihood and severity for each identified risk, using a defined and consistent scale, with the resulting risk level (for example, low, medium, high, unacceptable).
  • Mitigation measures. For each risk scored above the acceptance threshold, the specific control applied: technical (thresholding, human-in-the-loop review, monitoring), procedural (escalation paths, appeal mechanism), or contractual (vendor obligations).
  • Human oversight design. Where and how a human can intervene, override, or halt the system, and what training or authority that human has to do so meaningfully rather than as a formality.
  • Testing and validation evidence. What the system was tested against, including performance across relevant subgroups, and the results, with dates and version references.
  • Residual risk statement and sign-off. The risk that remains after mitigation, an explicit statement of who accepted it, their role, and the date. This is the accountability record an auditor will look for first.
  • Monitoring and review plan. What is monitored post-deployment, how often, by whom, and the trigger conditions that require a re-assessment.
  • Incident and complaint log reference. Where issues raised after deployment are recorded and how they feed back into the next review cycle.

A completed assessment using this structure should be defensible to someone who was not in the room when the system was built. That is the actual test of whether it is adequate.

What Goes Wrong When This Is Skipped or Done Poorly?

The most common failure is not omission, it is theater: a document exists, but the risk section is generic boilerplate copied from a template, with no evidence that anyone tested the system against the specific risks named.

A second common failure is treating the assessment as a one-time gate rather than a living record. The system passes review, ships, and the document is filed. Six months later the model has been retrained twice, the input population has shifted, and nobody has revisited the risk scoring. The paper trail says "assessed," but the assessment no longer describes the system in production.

A third failure is diffused accountability. When no named individual signs off on residual risk, the organization cannot answer the most basic audit question: who decided this was acceptable, and on what basis. A high-risk AI impact assessment without a named, dated sign-off is incomplete regardless of how thorough the rest of the document is.

Who Should Own This Inside an Organization?

Ownership works best as a shared function with one accountable lead, not a committee with no single owner. The lead is typically someone with both governance literacy and enough technical fluency to read a model card or a validation report and ask the right follow-up question, rather than accepting a vendor's summary at face value.

That person needs to work across three groups: the technical team that built or configured the system, legal and compliance for regulatory mapping, and the business owner who understands what the system is actually for and what a wrong decision costs. The assessment is only as good as the weakest of those three inputs.

This is precisely the skill set formalized in AI governance professional training: reading model risk documentation critically, mapping it to regulatory obligation, and managing the evidence trail an audit will eventually test.

Key Takeaways

  • An AI impact assessment documents purpose, data, risk, mitigation, and sign-off for a system before it goes live, and it must be revisited on defined triggers, not written once and filed.
  • It is broader than a DPIA: it covers accuracy, fairness across subgroups, and lifecycle drift, not only personal data processing.
  • Under frameworks like the EU AI Act, high-risk systems carry documented risk management and monitoring obligations; the impact assessment is the mechanism that produces the evidence.
  • A defensible assessment names specific risks with likelihood and severity, ties each to a mitigation, and records who accepted the residual risk.
  • The most common failure is a generic document with no system-specific evidence and no named accountability for the sign-off.

Structured, defensible AI risk documentation of this kind is the core discipline behind AICA's Certified AI Governance Professional (CAIGP) credential, which covers AI policy implementation, model risk documentation and impact assessments, regulatory mapping and compliance workflows, AI inventory and lifecycle controls, audit preparation and evidence management, and incident response for AI systems.