What can AI do, and what are its limits?

What can AI do? More than chat: AI systems also classify, predict, recommend and optimise, for example sorting emails, forecasting demand or suggesting products. This article focuses on generative AI, such as chatbots, writing assistants and image generators, which draft text, summarise, translate, answer questions and write code. It produces likely content from patterns in its training data; it does not check facts. Tools that add search, other tools or built-in checks behave differently from bare generation, but the limits below still matter.

Where the limits come from

• Invented facts. The tool can confidently state false information, including sources or figures that do not exist. This is called hallucination.
• Different answers to the same question. Ask twice and you may get two different results; one good answer does not guarantee the next.
• Dependence on data and design. Results depend on the data, the system’s design and training, how it is integrated and how well it was tested.
• Outdated knowledge. Unless connected to current sources, a model does not know about events, prices or rule changes after its data was collected.
• Bias. Patterns in the data can produce unfair results for some groups; see how AI bias shows up in a business.

The voluntary NIST Generative AI Profile describes these risks and actions for managing them.

AI agents: from answers to actions

An AI agent does not just answer; it takes actions, such as sending emails or updating records. A wrong answer then becomes a wrong action. The more an agent can do without a person confirming, the more its security risks matter.

What can AI do without a human check?

Recommendation. A person checks AI output before it is used if any of these is true:

1. It goes outside the company, to customers, partners or the public.
2. It affects a decision about a person, such as a job applicant or a customer.
3. It states facts, figures, quotes, or legal, financial or health information that someone will rely on.
4. It triggers an action that is hard to undo, such as a payment or a deletion.

These are conservative default review rules. A separately approved, narrowly scoped automated service may use testing, monitoring and escalation instead of pre-reviewing every low-risk answer. This does not override legal safeguards for decisions about people.

If none of the four applies, for example an internal draft someone will rewrite anyway, an occasional spot check is usually enough. The check must be real: the reviewer needs the time and the knowledge to spot an error, which is the core of human oversight of AI. For a company-wide structure, see the voluntary NIST AI Risk Management Framework.

Next step: list the AI outputs your company uses and note, for each, whether a person checks it and who.

Sources and further reading

• NIST AI 600-1, Generative AI Profile
• NIST AI Risk Management Framework