Why do AI Agents Fail
John Savill looks at AI agent failures as a system-level problem, examining the model alongside the components and controls that surround it.
Failure areas covered
The discussion considers whether the model is responsible for an agent’s failure or whether the problem lies elsewhere in the system. Savill walks through these areas:
- Fuzzy goals: Unclear objectives.
- Bad context: Problems with the information supplied to the agent.
- Tool misfires: Failures involving the tools an agent uses.
- Bad reasoning: Errors in the agent’s reasoning.
- Lost state: Problems retaining state.
- Hijacking and excessive privileges: Security risks involving agent manipulation and permissions.
- Flying blind: Limited visibility into agent behaviour.
The video then covers an improvement loop for understanding and addressing failures, followed by two caveats and a closing summary.
Chapters
- 00:00 — Introduction
- 00:37 — The model and what surrounds it
- 06:16 — Fuzzy goal
- 10:58 — Bad context
- 15:57 — Tool misfires
- 19:02 — Bad reasoning
- 22:53 — Lost state
- 25:47 — Hijack and over privileged
- 30:50 — Flying blind
- 35:07 — Improvement loop
- 39:46 — Two caveats
- 41:14 — Summary
- 42:19 — Close
Whiteboard
The accompanying AI agent failure whiteboard is available as a reference.