Guardrails for Agents

Agents that can act autonomously amplify both capability and risk. A single misguided action — sending an unintended email, deleting a production record, calling a paid API in a loop — can cause real harm before a human notices. Guardrails are the engineering controls that bound what an agent can do and how it escalates when uncertain.

Categories of risk

• Irreversible actions: sending messages, deleting data, making purchases — cannot be undone
• Scope creep: agent acts beyond the intended task boundary because the goal was underspecified
• Prompt injection: malicious content in tool outputs overrides the agent's instructions
• Runaway loops: the agent gets stuck calling tools without making progress toward the goal
• Data exfiltration: the agent is manipulated into leaking sensitive information via tool calls

Human-in-the-loop checkpoints

Not every action needs human approval, but irreversible and high-stakes actions should pause and ask. Design a confirmation tool the agent must invoke before executing destructive operations. The confirmation includes a plain-English summary of what will happen and requires explicit human approval before proceeding.

confirm_tool = {
    "type": "function",
    "function": {
        "name": "request_approval",
        "description": "Ask the user to approve an action before executing it. Required before any irreversible operation.",
        "parameters": {
            "type": "object",
            "properties": {
                "action_summary": {"type": "string", "description": "Plain-English description of what will happen"},
                "risk_level": {"type": "string", "enum": ["low", "medium", "high"]},
            },
            "required": ["action_summary", "risk_level"],
        },
    },
}

Input and output validation

Validate all tool arguments before execution, even if the model generated them. Check data types, ranges, path traversal in file paths, and SQL/command injection patterns. Similarly, validate tool results before inserting them into context — a compromised external service can use its response to inject instructions into the agent.

Limiting blast radius

Apply the principle of least privilege to every tool. A web browsing agent should not also have access to the production database. Scope file system tools to a sandboxed directory. Restrict API call tools to the minimum set of endpoints the task requires. The goal is to ensure that even a fully compromised agent can only do limited damage.