Lesson 7 of 7

Guardrails, agent safety & why agents fail in production

8 min read

In a demo the agent looks brilliant. In production it emails the wrong customer, loops forever, and burns tokens overnight. What went wrong?

Keep a human on the risky edge

The single most effective guardrail: a human in the loop on irreversible actions. Let the agent run read-only, low-risk steps on its own — but any step that spends money, sends a message, or deletes data pauses for an explicit approval. You hold the yes on the actions you can't take back.

Autonomy is a dial, not a switch. Auto-approve the reversible, gate the irreversible. A human checkpoint on risky actions prevents most expensive mistakes.

Why agents fail in production

Beyond the risky action, the classic failure modes: infinite loops (no iteration cap), runaway cost (no token or dollar budget), compounding errors (a wrong step feeds the next), and prompt injection (a tool result contains instructions the agent obeys). None show up in a happy-path demo. All show up at scale.

Agents fail on the paths a demo never hits: loops, cost, error cascades, injection. Production readiness is bounding those — caps, budgets, validation, least privilege — not a better prompt.

Give an agent the least privilege it needs, never your full access. A tool result is untrusted input — text fetched from a web page or an email can carry an injected instruction — so scope credentials tightly and validate every tool output before acting on it.