Building the Control Layer: Rails, Classifiers, and Containment

A guardrail is a check that runs outside the model and can block, modify or flag. NVIDIA's NeMo Guardrails organises them into five rail types, and the taxonomy is useful regardless of which tool you use, because it names five distinct decision points.

Input rails run on what arrives before the model sees it: injection detection, PII redaction, topic restriction, length and rate limits.

Dialog rails operate on the conversation rather than a single message, which is what catches the multi-turn and crescendo attacks that single-message classifiers miss by construction.

Retrieval rails run on what comes back from a knowledge base before it enters context: permission checks, provenance checks, and scanning retrieved chunks for embedded instructions.

Execution rails sit around tool calls: is this action permitted, are the arguments in range, does it need confirmation.

Output rails run on what the model produced: policy violations, leaked secrets, hallucinated citations, and validation before anything downstream consumes it.

Most deployments implement input and output rails and stop. The three in the middle are where the serious failures from the previous lesson actually live.

Each rail can be implemented three ways, with a consistent trade between cost, coverage and predictability.

Deterministic rules: regular expressions, allow and deny lists, schema validation, length caps. Microseconds, free, perfectly predictable, and trivially evaded by paraphrase, encoding or a different language. Their real value is not catching attackers but catching the accidental cases and enforcing hard structural constraints, where predictability is exactly what you want.

Specialised classifiers: small models trained to detect a specific category. Meta's Llama Guard family is the best-known example, with Llama Guard 4 released in 2025 as a 12-billion-parameter multimodal safety classifier. Microsoft's Presidio does the same job for PII detection and redaction. Tens of milliseconds, cheap, and far more robust to paraphrase than a rule.

Model-as-judge: prompting a general model to evaluate against a written policy. The most flexible, handles nuance and novel categories, and the most expensive in both latency and money. It is also itself injectable, since the content it evaluates enters its context.

The standard arrangement layers them: rules first because they are free, classifiers for the bulk of the work, and a judge reserved for the genuinely ambiguous cases that reach it.

Following one request through the layers shows where each control sits and what it can still miss.

The input arrives and passes through cheap deterministic checks, then a classifier. Blocked here, nothing reaches the model and the cost is negligible.

Surviving input joins retrieved context, which has itself passed permission and provenance checks, and reaches the model.

If the model requests a tool call, execution rails evaluate it before anything happens: permitted action, arguments in range, confirmation required for anything irreversible.

The generated output passes output rails before reaching the user, and separately before reaching any downstream system that will act on it.

The important structural property: each layer is independent, so a bypass of one does not bypass the others. That is the entire argument for defence in depth here, since no individual layer is reliable.

Every guardrail has two error modes, and the one that kills deployments is not the one teams worry about.

A false negative lets something through. A false positive blocks something legitimate.

Teams tune aggressively against false negatives because those are the ones that appear in incident reports. The result is a system that refuses ordinary requests, and the consequences are predictable: users stop using it, or route around it to an unmonitored tool, which converts a governed system into shadow use. A guardrail that drives users to an ungoverned alternative has produced negative security.

So the false-positive rate is a first-class design constraint, not an afterthought.

Three practices keep it honest. Measure both rates on a fixed evaluation set, and treat a change that improves one while silently degrading the other as a regression rather than an improvement. Set the threshold by consequence, so a medical or financial context tolerates more false positives than a brainstorming assistant. And prefer graduated responses to binary blocking: flag for review, ask a clarifying question, degrade to a safer mode, or require confirmation, rather than refusing outright.

A refusal message that explains what was blocked and offers a route forward also converts a frustrating dead end into something a user can work with.

Guardrails sit on the critical path, and their cost compounds in a way that is easy to miss during design.

A rule costs microseconds. A classifier costs tens of milliseconds. A model-as-judge costs hundreds of milliseconds to seconds, comparable to the generation it is checking.

Run input rails, output rails and a judge on both, and you can double the perceived latency of a system whose speed was part of its value.

Four techniques recover most of it.

Run independent checks concurrently rather than in sequence, since a PII classifier and an injection classifier have no dependency on each other.

Order by cost. Cheap deterministic checks first means the expensive ones run on a smaller population.

Stream with a trailing output check, so the user sees tokens immediately while the check runs on the accumulating text, accepting that you may have to retract. Whether retraction is acceptable is a product decision and it differs sharply by context.

And run the expensive checks asynchronously where the consequence permits: block on the fast ones, log and alert on the slow ones. This is a genuine trade of coverage for speed, and it should be a recorded decision rather than an implementation detail.

The most important idea in this lesson: the strongest controls are architectural, not textual.

Filtering asks whether this text is dangerous, which is an open-ended judgement about natural language and therefore never reliable. Containment asks what can this system do at all, which is a closed question with an enforceable answer.

Apply least privilege to capabilities. An agent that can read a database does not need write access. One that summarises documents does not need to send email. Every capability removed is an entire class of injection outcome removed, permanently, without a classifier.

Break the trifecta deliberately. Where a system holds private data and processes untrusted content, remove the external communication channel, or route it through a human. That single architectural decision defeats exfiltration regardless of how clever the injection is.

Enforce permissions outside the model. Retrieval filters by the user's identity before ranking, so the model never holds material the user cannot see. The model is not a security boundary and should never be asked to be one.

Validate structure rather than intent. Requiring output to conform to a schema, and executing only parameterised, allow-listed operations, means a compromised generation cannot become an arbitrary action.

And require human confirmation for anything irreversible. Slow, and the only genuinely dependable control for high-consequence actions.

The shape of a rail definition, showing how the layers compose in practice.

input_rails:
  - length_cap: 8000              # rule: microseconds
  - pii_detect: {action: redact}  # Presidio: ~10ms
  - injection_classify:           # Llama Guard: ~40ms
      threshold: 0.7
      action: flag_and_continue    # not block: see FP budget

retrieval_rails:
  - acl_filter: {by: user_identity}   # BEFORE ranking, not after
  - scan_chunks_for_instructions: {action: strip}

execution_rails:
  - allowed_tools: [search_docs, read_ticket]
  - denied: [send_email, http_post]    # trifecta broken here
  - confirm_required: [update_record]

output_rails:
  - schema_validate: strict
  - secret_scan: {action: block}
  - grounding_check: {action: flag}     # async, logged

Read the two comments that matter. The injection classifier flags rather than blocks, because at a 0.7 threshold it will be wrong often enough that blocking would damage the false-positive budget. And denied: [send_email, http_post] is doing more security work than every classifier above it combined, because it removes the exfiltration channel rather than trying to detect its misuse.

Being clear about the limits prevents guardrails being asked to carry weight they cannot bear.

They cannot make an unsuitable model suitable. A model that performs badly on your task will keep doing so; a filter on its output does not improve the underlying judgement.

They cannot reliably detect a sufficiently novel attack. Every classifier is trained on known patterns, and the attack surface is natural language, which is unbounded. Treat detection as raising cost for the attacker, not as closing the hole.

They cannot substitute for access control. If the retrieval layer hands the model a document the user should not see, an output filter is being asked to un-see it, which is the wrong layer and an unreliable one.

They cannot resolve policy questions nobody has decided. A guardrail enforces a policy. If the organisation has not decided what the system may discuss, no configuration expresses it, and the ambiguity surfaces as inconsistent behaviour blamed on the model.

And they cannot make the output true. Blocking policy violations is a different problem from grounding, which the companion cursus on hallucinations takes up.

The honest summary: guardrails reduce the rate and the blast radius of failures. They do not make a system safe, and a design that depends on them being reliable has already failed.

A control layer is a running system with its own failure modes, and it needs the treatment any production dependency gets.

Log every decision: what was checked, what fired, what the score was, what action was taken. Without this you cannot tune thresholds, cannot investigate an incident, and cannot demonstrate that controls operated.

Make blocks reviewable. Sample refusals and read them. This is the fastest way to find an over-tuned threshold, and the population of blocked-but-legitimate requests is invisible unless someone deliberately looks.

Version the configuration and treat threshold changes as deployments, with the evaluation set run before and after. A threshold quietly adjusted to reduce complaints is a security change made without review.

Monitor for degradation. A classifier's effective performance drifts as usage patterns change, and a rule written against last year's attack strings decays silently.

Plan for the guardrail failing. If the classifier service is unavailable, does the system fail open or closed? Both are defensible and the choice must be deliberate: failing open preserves availability and removes the control, failing closed preserves the control and takes the product down.

And instrument the bypass rate. If users have found a phrasing that reliably gets around a rail, that shows up in the logs before it shows up in an incident.