Bern
Lisbon
New York
How do you respond to an AI incident?
When an AI incident happens, the first job is to limit the harm, not to find someone to blame. An AI incident is any event in which an AI system causes harm or comes close to it: a wrong decision about a person, leaked data, false output published in the company’s name, or an AI agent taking an unapproved action.
Hypothetical example: a software agency uses an AI assistant connected to its shared drive. A draft update for one client includes contract details and contact data belonging to another client, and the email has already been sent.
Six steps to respond to an AI incident
- Limit or stop the system. Switch off the feature, restrict it to internal use or go back to the manual process. Decide in advance who may do this without waiting for approval.
- Preserve evidence. Save the inputs, outputs, logs, model version and settings before anything is changed.
- Assess the harm. Establish who was affected, what data was involved and whether the harm is still happening. This also shows which legal duties below apply.
- Inform people inside the company. Tell the use-case owner and management and, where relevant, the data protection officer (DPO), security and legal.
- Talk to the people affected. Explain plainly what happened, what it means for them and what the company is doing, and give a named contact.
- Find and fix the root cause. Check whether the problem lay in the model, the data, the instructions, the integration or the way the tool was used. Test the fix before restarting the system.
The person who can stop a system is usually whoever holds human oversight of AI for that use case. After the incident, add a check for the same failure to the way you monitor AI after launch.
Reporting duties that may apply
Legal requirement (GDPR, Art. 33–34). A personal data breach is not only a leak: it can affect the confidentiality, integrity or availability of personal data, for example through unauthorised disclosure, alteration, loss or destruction. If the incident involves such a breach, as in the example, the organisation responsible for the data (the controller) must already notify the competent supervisory authority without undue delay and, where feasible, within 72 hours of becoming aware of it, unless the breach is unlikely to result in a risk to individuals’ rights and freedoms (GDPR, Art. 33; EDPB guide on data breaches). A processor, such as a vendor handling the data on the controller’s behalf, must notify the controller without undue delay. Telling the affected individuals is a separate assessment under Art. 34: it is required where the breach is likely to result in a high risk, with exceptions. Bring in the DPO or whoever handles data protection at step 4, not at the end.
Legal requirement (EU AI Act, Art. 26 and 73). The provider of a high-risk AI system reports serious incidents involving it to the authorities (AI Act, Art. 73). A deployer that identifies a serious incident must immediately inform first the provider, and then the importer or distributor and the relevant market surveillance authorities (Art. 26(5)). Which duty is yours depends on whether you are a provider or a deployer. The dates differ (Digital Omnibus on AI):
| Obligation | Where | Applies from |
|---|---|---|
| Deployer incident information | Art. 26, Chapter III, Section 3 | 2 December 2027 (Annex III) / 2 August 2028 (Annex I Section A; Section B follows the sector route in Art. 2(2)) |
| Serious incident reporting (provider) | Art. 73, Chapter IX | not expressly postponed; depends on classification and the Art. 111 transitional rules, so needs a legal assessment |
The full timeline is in what the EU AI Act requires.
Legal requirement (Directive (EU) 2024/2853). The new product liability directive covers software, including AI, so harm caused by a defective AI product can give rise to liability (Directive (EU) 2024/2853). Member States must transpose it by 9 December 2026, and it covers products placed on the market or put into service after that date. Earlier products stay under the previous regime, which can also create liability.
Legal requirement (GDPR, Art. 33(5)). Controllers must already keep a record of every personal data breach, even when notification is not required: what happened, its effects and the remedial action taken, and the reasoning behind notification decisions (EDPB guide).
Recommendation. Record every AI incident, including near misses, with the date, the system, the harm, the decisions taken and the fix, and keep the record with the rest of your AI documentation.
Next step: for each AI system you use, write down who can stop it, in which situations and how to reach that person outside office hours.
Sources and further reading
- Regulation (EU) 2016/679 (GDPR) — Articles 33 and 34, including Art. 33(5)
- EDPB — Data breaches (guide for SMEs)
- Regulation (EU) 2024/1689 (AI Act) — Articles 26 and 73
- EU AI Act Service Desk — Article 26
- Regulation (EU) 2026/1744 (Digital Omnibus on AI)
- Directive (EU) 2024/2853 on liability for defective products
This article is for general information and is not legal advice.
AI Horizon Conference
The AI Horizon Conference returns to Lisbon, once again bringing together entrepreneurs, investors and industry leaders to discuss the future of AI.
