AI agents and automation

From answers to actions

A chatbot gives an answer and a person decides what to do. An AI agent takes the action itself: it books, sends, deletes, buys or changes a record. That moves the risk from “a wrong answer” to “a wrong act”. Acts can be costly, hard to undo and fast. Governance for agents therefore focuses on what the agent is allowed to do, when a person must approve, and how to stop it.

The main risks

• Actions beyond scope. The agent does something nobody intended, often by pursuing a goal too literally.
• Runaway cost or loops. A repeating task burns budget or overloads systems.
• Irreversible actions. Deleted data, sent messages and executed payments cannot always be recalled.
• Prompt injection. Text in an email, web page or document the agent reads can contain instructions the agent obeys.
• Excessive access. Broad permissions turn a small mistake into a large one.
• No audit trail. Without logs, nobody can reconstruct what happened or why.

Controls

1. Least privilege. Grant only the actions and data needed for the task, per task, and review permissions regularly. Permissions granted “temporarily” tend to stay.
2. Human approval points. Require a person to approve actions that are high-impact, irreversible, external-facing or above a spend threshold.
3. Limits. Set rate, spend, volume and time limits, and test that they work.
4. Stop mechanism. A tested way to halt the agent immediately, with named people authorized to use it.
5. Logging. Record every action with what, when, on whose behalf and with which inputs. Keep logs long enough to investigate.
6. Injection defenses. Treat anything the agent reads as untrusted. Separate instructions from data, restrict what untrusted content can trigger and test with hostile inputs.
7. Data boundaries. Define and enforce what the agent can and cannot reach.
8. Fallback and monitoring. Decide what happens on failure; make sure a person can take over; alert on errors, cost and unusual actions.
9. Incident playbook. Have a plan ready, and sign-off by accountable owners before each release.

The agent deployment checklist lists these with a status and evidence note for each.

Testing an agent

Test the happy path, then the unhappy ones: ambiguous goals, missing data, conflicting instructions, a malicious document, a system that is down, and a user trying to push the agent beyond its limits. Test the stop mechanism under load. Record what you found and fixed.

Accountability

Agents are not accountable. People are. Name an owner for each agent, make sure that owner can answer for what the agent did, and keep a register entry with its permissions. Under the EU AI Act and other regimes, the usual rules apply to the use the agent is put to: an agent that screens job applicants is still a hiring tool.

Example

A finance team deployed an agent to chase overdue invoices by email. It had access to the whole contact database. A customer replied with an auto-reply containing the words “forward this to your manager,” and the agent began forwarding invoices to unrelated addresses. The fixes: restrict the agent to the open-invoice list, require approval for emails to new recipients, add a daily send limit and log every email.

What to do now

• Fill in the agent checklist for one agent, even a small one.
• List what it can do today, and remove permissions it does not need.
• Test the stop button.