An AI incident response plan is the documented sequence an organization follows the moment a deployed model produces harmful, biased, or materially wrong output: detect, contain, assess, notify, and remediate, in that order. Most organizations discover they don't have one at the worst possible moment, mid-incident, with legal, engineering, and communications teams improvising in parallel. This is the playbook for the first 24 hours, and the governance structure that has to exist before hour one.

Why AI Incidents Are Different From Software Incidents

A traditional software incident has a fix that is usually deterministic: roll back a deployment, patch a config, restart a service. An AI incident often does not.

A model that produces a harmful output today may produce a correct output on the identical input tomorrow, because inference is probabilistic and context-dependent. Root cause can sit in the training data, the fine-tuning process, a prompt injection, a retrieval source, or an interaction between the model and a downstream system, none of which are fixed by a restart.

This changes incident response in three concrete ways. First, containment often means disabling a capability or a use case, not rolling back a version number. Second, root cause analysis requires tracing a decision through a non-deterministic system, which demands logging that most software incident playbooks never anticipated. Third, disclosure obligations attach to AI incidents specifically under regimes that do not apply to a generic outage, most notably the EU AI Act's serious incident reporting duty for high-risk systems.

What Counts as an AI Incident?

An AI incident is any instance where a model's output or behavior causes, or could plausibly cause, harm to a person, a right, an organization, or a system, whether the cause is a model defect, a data problem, misuse, or an unanticipated interaction with its environment.

That definition is deliberately broader than "the model was wrong." A model can be statistically accurate and still cause an incident, for example a credit scoring model that is accurate on average but systematically disadvantages a protected group. A model can also cause an incident through correct behavior operating outside its intended scope, for example a customer service agent that correctly answers a question it was never authorized to answer, exposing internal pricing logic.

Under the EU AI Act, a "serious incident" has a specific legal meaning: an incident or malfunction that directly or indirectly leads to death, serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of fundamental rights, or serious harm to property or the environment. Providers of high-risk AI systems must report these to the relevant market surveillance authority, generally within 15 days of becoming aware, with a shorter window (2 days) for deaths and widespread infrastructure disruption, and an even shorter window (24 hours) in certain cases involving widespread infringement. Organizations should not wait for a legal determination of "serious" to begin the internal response. The internal trigger threshold has to be lower than the legal reporting threshold, or the clock on the legal obligation starts before anyone is looking.

How Is an AI Incident Different From a Near Miss?

A near miss is an event where the model produced a flawed or risky output that was caught before it reached a decision, a user, or a downstream system, typically by a human reviewer, a guardrail, or a monitoring threshold. Near misses do not usually trigger external disclosure, but they belong in the same log as incidents. A pattern of near misses is often the earliest reliable signal that a model is drifting toward an incident, and treating them as disposable is the single most common gap in immature AI risk programs.

The First 24 Hours: A Playbook

The sequence below assumes a moderate to high severity incident, something beyond a near miss but short of a confirmed serious incident under a specific regulatory regime. Adjust timing for scale, but do not skip steps.

  1. Freeze the state, don't fix it yet. The instinct to immediately patch or roll back is understandable and usually wrong as a first move. Preserve the model version, the prompt or input logs, the configuration, and any retrieval sources exactly as they were at the time of the incident. If the system allows it, snapshot the relevant logs before any automated retention policy can purge them. Evidence destroyed in the first hour cannot be recovered in the third.
  1. Contain exposure, not just the model. Containment means stopping the harm from compounding, which is not always the same as stopping the model. Options in order of severity: disable the specific feature or use case, route the affected traffic to a human reviewer, throttle the model's autonomy (remove write access, require approval on actions), or take the system fully offline. Choose the least disruptive option that actually stops the harm. A full shutdown that was not necessary creates its own operational and reputational cost.
  1. Assemble the response team within the hour. At minimum: an engineering owner who understands the model and pipeline, a risk or compliance owner who understands the legal exposure, and a decision-maker with authority to approve containment actions without waiting for a committee. If the incident touches personal data, safety, or a regulated sector, legal counsel joins immediately, not after initial triage.
  1. Establish a single incident log, timestamped, from minute one. Every action, decision, and piece of new information gets logged with who, what, and when. This log is the backbone of the root cause analysis, the regulatory filing if one is required, and the eventual board or audit review. Reconstructing a timeline from memory two weeks later is where most incident reports lose credibility.
  1. Scope the blast radius. How many users, decisions, or downstream systems were affected? Is the issue isolated to one input pattern, one user segment, one integration, or is it systemic to the model's behavior generally? This determines both the containment strategy and, later, the notification list.
  1. Make the internal notification, even before root cause is known. Notify the risk owner, relevant business unit leads, and, where applicable, a designated executive (commonly the Chief AI Governance Officer or equivalent role) that an incident is open. Internal notification should not wait for certainty. It should happen on reasonable suspicion, with the caveat clearly stated that facts are still being established.
  1. Begin root cause analysis in parallel with containment, not after it. Waiting for containment to fully resolve before starting root cause work wastes hours the investigation cannot get back, especially if logs are time-sensitive. Assign root cause work to someone other than the person managing containment, so both tracks move without competing for the same attention.
  1. Determine regulatory disclosure obligations early, not at hour 20. If the system is in scope for the EU AI Act's high-risk provisions, or a sector-specific regime (financial services, healthcare, critical infrastructure), the disclosure clock may already be running from the moment the organization became aware, not from the moment root cause is confirmed. Legal counsel should make a preliminary disclosure determination within the first 24 hours, even if it is provisional.
  1. Prepare a holding statement before you need one. If the incident has any chance of external visibility, customers, media, or affected individuals, have a factual, non-speculative holding statement ready. It should confirm the organization is aware, is investigating, and has taken containment action. It should not speculate on cause or assign blame before the investigation supports it.
  1. Close the 24-hour window with a documented status, not a resolution. Few incidents are genuinely resolved within a day. What should exist at hour 24 is a clear record: what happened, what was done, what is still unknown, who owns the next phase, and what the regulatory posture is. This becomes the anchor document for everything that follows.

What Does Root Cause Analysis Look Like for a Model, Not Code?

Root cause analysis for a failed AI system has to test several categories that a traditional software postmortem does not, because the failure surface is different.

Data-related causes include training data that underrepresented a scenario, a labeling error propagated through fine-tuning, or a data drift where production inputs have diverged from what the model was validated against. Model-related causes include a genuine capability gap (the task was outside what the model can reliably do), a prompt or context window that produced ambiguous instructions, or a retrieval system that surfaced outdated or incorrect source material to a RAG pipeline. Systemic causes include a missing guardrail, an over-permissioned agent that took an action it should have needed approval for, or a monitoring gap where the failure mode existed but nothing was watching for it.

A rigorous root cause analysis documents which of these categories applied, with evidence, not a single narrative cause. Most real AI incidents have two or three contributing factors, and a report that names one is usually incomplete.

Who Should Own AI Incident Response?

Ownership should sit with a named role, not a committee, backed by a cross-functional team that is pre-identified before any incident occurs. In organizations with a formal AI governance structure, this is typically the Chief AI Governance Officer or equivalent, who holds accountability for the incident response framework itself: the escalation thresholds, the notification chain, the documentation standard, and the board and regulator reporting relationship.

That does not mean the governance owner personally manages every incident. It means every incident has one accountable name attached to the outcome, with engineering, legal, and communications functioning as defined roles rather than ad hoc volunteers. Organizations that assign incident response to "whoever is online" consistently produce weaker documentation and slower containment, because no one owns the decision to escalate.

What Documentation Does an Auditor or Regulator Expect to See?

At minimum: the timestamped incident log, the containment actions taken and their timing, the root cause analysis with evidence, the internal notification record, the disclosure determination and its rationale (even if the conclusion was "not reportable"), and the remediation plan with an owner and a deadline. Under frameworks like ISO/IEC 42001 and the NIST AI Risk Management Framework, the expectation is not that incidents never happen. It is that the organization can demonstrate a functioning, evidenced response system. Absence of incidents in a young AI deployment is not proof of maturity. Absence of a documented response capability is proof of its lack.

Building the Plan Before You Need It

An incident response plan written during an incident is a symptom, not a solution. The plan needs three things in place before deployment, not after: defined severity thresholds that trigger each stage of the playbook, a named response team with clear authority to act without escalation delay, and logging infrastructure that captures enough of the model's inputs, outputs, and decision context to support a root cause analysis after the fact. A model deployed without decision logging cannot be meaningfully investigated when it fails, regardless of how well the rest of the plan is written.

Tabletop exercises are the cheapest way to find the gaps. Running a simulated incident, a biased output reaching a real decision, a prompt injection extracting confidential data, an autonomous agent taking an unauthorized action, surfaces where the plan assumes information that will not actually be available in real time. Organizations that run this exercise before their first real incident consistently respond faster when one occurs.

Key Takeaways

  • An AI incident response plan has to assume non-deterministic failure modes: containment usually means restricting a capability or use case, not rolling back a version.
  • Preserve state before you fix it. Logs, configurations, and model versions destroyed in the first hour cannot support root cause analysis or regulatory disclosure later.
  • Treat the EU AI Act's serious incident reporting clock as starting from awareness, not from confirmed root cause, for high-risk systems in scope.
  • Root cause analysis for AI systems must test data, model, and systemic categories separately. Most real incidents have multiple contributing causes, not one.
  • Incident response ownership belongs to a named accountable role, not an ad hoc team, with documentation standards that satisfy ISO/IEC 42001 and NIST AI RMF expectations before an auditor asks for them.

Organizations building this capability from scratch, rather than reconstructing it after a failure, often formalize it through AICA's CCAIGO (Certified Chief AI Governance Officer) credential, which covers AI governance frameworks and operating models, global AI regulation including the EU AI Act, NIST AI RMF and ISO/IEC 42001, AI risk management and assurance, responsible AI policy design and enforcement, AI audit readiness and documentation, and board and regulator engagement.