What AI documentation should you keep?

A good starting point for AI documentation in a small company is a suggested starter set of eight records: the AI inventory, assessments, approval decisions, test results, training records, vendor contracts and documentation, incident reports and a change log. This is not a legal minimum; what the law requires depends on your role and systems, as set out below.

The overall structure also covers the AI use policy, the incident procedure and the access-rights settings. These need not be separate files.

Who maintains each document

Recommendation. Each document needs a named owner and a clear moment when it is updated. Without both, records go out of date within months.

For vendor files, the answers to your questions for an AI vendor are worth keeping alongside the contract.

Keep the records in one place to look, with access control. One place does not mean company-wide access: personal data and incident material should be open only to those who need them. Set a retention and deletion rule, stating how long each type of record is kept, for example after a use case is retired, and who deletes it.

What the law requires for AI documentation

Legal requirement (GDPR, Art. 30). Controllers and processors must already keep records of their processing activities, with a limited exemption for smaller organisations (GDPR, Art. 30). Where these records apply to you, AI tools that process personal data belong in them. Linking each inventory entry to the matching record avoids keeping two versions of the same information.

Legal requirement (GDPR, Art. 33(5)). Where GDPR applies, controllers must already keep a record of every personal data breach, even when notification is not required: what happened, its effects and the remedial action taken, and the reasoning behind notification decisions (GDPR, Art. 33(5); EDPB guide on data breaches). A breach is not only a leak: it can affect the confidentiality, integrity or availability of personal data, for example through alteration or loss.

Legal requirement (EU AI Act, Art. 11, 12 and 26). High-risk AI systems carry separate documentation duties (AI Act). These apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I Section A products; Section B products follow the sector route in Art. 2(2). The full timeline is in what the EU AI Act requires. The duties are:

• the provider draws up technical documentation before the system is placed on the market or put into service, and keeps it up to date (Art. 11);
• the provider designs the system so that it automatically records events in logs (Art. 12);
• the deployer keeps the logs the system automatically generates, to the extent they are under its control, for a period appropriate to the intended purpose, of at least six months, unless other Union or national law, in particular on personal data, provides otherwise (Art. 26(6); AI Act Service Desk, Art. 26).

Following the Digital Omnibus on AI, SMEs, including start-ups, and small mid-caps may provide technical documentation in a simplified form (Art. 11; Regulation (EU) 2026/1744). Which of these duties fall on your company depends on whether you are a provider or a deployer.

The starter set does not by itself meet these obligations or make a company compliant, but it is the base that specific legal requirements build on.

Next step: gather the AI documentation you already have, such as contracts, spreadsheets and emails containing approvals, into one access-controlled folder, name the person who owns it and write down the retention rule.

Sources and further reading

• Regulation (EU) 2024/1689 (AI Act) — Articles 11, 12 and 26, and Annexes I and III
• EU AI Act Service Desk — Article 26
• Regulation (EU) 2026/1744 (Digital Omnibus on AI)
• Regulation (EU) 2016/679 (GDPR) — Articles 30 and 33(5)
• EDPB — Data breaches (guide for SMEs)

This article is for general information and is not legal advice.