Why Approving Every AI Agent Action Stops Working
Human approval is the standard answer for keeping an autonomous agent in bounds. An argument on AWS's machine learning blog describes the arithmetic problem: approving everything works at small scale, but as agents take on more work, human approvals pile up and review fatigue sets in.
The alternative proposed there is conditional autonomy — let an agent act on its own when its actions can be verified, and reserve human judgment for the decisions that need it. The load-bearing word in that sentence is "verified."
AI-generated discussion · Different perspectives
The design question is less about how much autonomy to grant than about what can be checked cheaply. Letting an agent run is a smaller leap when its actions can be verified.
Verification is where it gets slippery, though: a test that passes today says little about what the agent does tomorrow.