Layered guardrails: deterministic gates keep the model in a safe lane

Two failure modes that undermine agent safety

The OWASP Top 10 for LLM Applications (2025) identifies the two failure modes that matter most for deployed agents. The first is LLM01:2025 — Prompt Injection: untrusted content in the agent's input stream hijacks its instructions, causing it to act on the attacker's goals rather than the system's intended goals. The second is LLM06:2025 — Excessive Agency: an agent has been granted more capability, autonomy, or permissions than the task requires, so a single bad decision — whether from a confused model or injected content — can have a large blast radius.

Both failures share a root cause: the system trusted the model to enforce a boundary that should have been enforced by the surrounding architecture. Prompt injection works when there is no layer between user-supplied data and the model's instruction context. Excessive agency persists when there is no layer between the model's output and the tools it can invoke. Layered guardrails exist to provide exactly those layers.

What prompt injection looks like in practice

Prompt injection does not require a sophisticated attacker. It appears wherever an agent processes data it did not generate itself — a web page it was asked to summarise, a document it was asked to classify, a database record it was asked to transform. If that data contains instruction-like syntax — an imperative verb directed at the model, a role-override phrase, a delimiter that escapes the current context — the model may follow those instructions instead of its original ones.

The defensive response is not to make the model smarter about resisting injection; it is to build a layer that validates inputs before they reach the model. As Anthropic's guidance on prompt injection defense describes, effective defenses combine structural choices — separating instructions from data, using system prompts that establish trust hierarchies — with deterministic pre-processing that can reject or sanitise inputs without involving the model at all. Lexical pattern detection (searching for known instruction-override phrases) is not sufficient on its own, but it is fast, cheap, and catches the common cases deterministically.

The key point is that a detected injection should produce a hard block, not a model judgment call. If you ask the model whether injected content is safe, the injection may affect that judgment. The detection and rejection logic must live outside the model's reasoning path.

Excessive agency and the tool allow-list

OWASP LLM06:2025 defines excessive agency as a condition where an agent can take actions — through tools, APIs, or system capabilities — that go beyond what the current task requires. The failure mode is not that the model is malicious; it is that the agent's capability envelope is too wide. When something goes wrong — a confused model, injected instructions, an edge case in the task definition — the damage is bounded only by what the agent can do, not by what the task needed it to do.

The structural defense is an allow-list: a deterministic register of which tools the agent may call, and which argument keys each tool may receive. Every proposed tool call is checked against this register before dispatch. A call to a tool not on the list is blocked unconditionally, regardless of what the model (or injected content) requested. A call to a listed tool with an unexpected argument key is also blocked — this prevents parameter-injection attacks where the attacker's payload arrives as an extra argument rather than in the task data.

This is the principle of least privilege applied to agentic systems: the agent is granted only the specific capabilities the current task requires. An agent that can only search a knowledge base and summarise text cannot delete files, call external APIs, or exfiltrate data — even if injected content explicitly requests it. The allow-list makes this true by construction, not by model self-restraint.

Adversary simulation as hardening, not as an attack recipe

Testing your own agent's guardrails against injection patterns — running controlled inputs that carry injection syntax through your validation layer to confirm that the block fires — is a defensive practice. The goal is to measure whether your detection logic catches what it should catch, to find gaps before an adversary does, and to build regression coverage so that guardrail changes do not silently weaken the system.

This is distinct from crafting novel attack payloads or finding techniques to bypass other systems' defenses. The framing that matters: you are your own red team, running known patterns against your own system in a controlled environment, measuring detection coverage, and tightening the layer based on what you find.

Where deterministic gates end and model judgment begins

Lexical injection detection and tool allow-lists are deterministic. They run without a model, produce deterministic outputs, and cannot be argued out of a block by the model's reasoning. This is their strength: they are fast, auditable, and robust to the very thing they are defending against.

Their limit is semantic scope. A crafted input that conveys harmful intent through ordinary language — without any keyword that matches a detection pattern — will pass lexical validation. A legitimate tool call with an unusual-but-valid argument structure may be over-blocked by a strict allow-list. Both gaps are real. The complement to deterministic gates is model-based semantic classification for inputs and human review for edge cases in the allow-list policy. The layers work together; neither is sufficient alone.