Skip to content
Wave of light particles flowing through faint circuit traces on a dark background
AI Agents21 min read

What are AI guardrails?

Ignas Vaitukaitis, Founder & CEO of AlphaCorp AI

AI Agent Engineer ·

What are AI guardrails?
On this page(22)
  1. What Are AI Guardrails and What Do They Protect Against?
  2. How AI Guardrails Work at the Input, Output, and Tool-Call Layers
  3. The Main Types of Guardrails: Content Safety, Prompt Injection, Data Leakage, Hallucination, and Topic Control
  4. How Guardrails Are Built: Rules, Classifier Models, and LLM Judges
  5. Deterministic rules and validation code
  6. Purpose-trained classifier models
  7. LLM judges that read a written policy
  8. Architectural guardrails by design
  9. How Guardrails Differ From System Prompts, Model Alignment, and Evals
  10. Guardrail Tools and Frameworks Compared: Open Source Libraries vs Managed Cloud Services
  11. What Do AI Guardrails Cost in Latency and Spend?
  12. Where AI Guardrails Fail: Jailbreaks, False Positives, and Coverage Gaps
  13. Which Teams and Use Cases Need Guardrails Most
  14. What Changed for AI Guardrails This Year: Agents, Tool Use, and Regulation
  15. Frequently Asked Questions About AI Guardrails
  16. Are AI guardrails the same as content filters?
  17. Can AI guardrails be bypassed?
  18. Do AI guardrails stop prompt injection?
  19. Do guardrails slow down an LLM?
  20. Are AI guardrails required by law?
  21. What is the difference between guardrails and AI alignment?
  22. Where to Start With Guardrails in Your Own Application

AI guardrails are external controls that sit around an AI model and check its inputs, outputs, and tool calls while it runs. They reduce risk without removing it. This guide explains how they work at each checkpoint, what they cost in compute and delay, which tools exist, and where independent testing shows they fail. As of November 2026, the UK AI Security Institute and NIST CAISI have each reported a successful attack on every model they tested. So plan for layers.

Key figures, with dates:

  • Compute overhead: Anthropic’s classifiers cost 23.7% extra compute in February 2025 and about 1% in January 2026, by the company’s own figures.
  • Jailbreak success: in Anthropic’s February 2025 test of 10,000 synthetic jailbreak prompts, it fell from 86% to 4.4%.
  • Attack effort: the UK AI Security Institute’s Frontier AI Trends Report, covering models released up to October 2025, saw expert jailbreak time rise from about 10 minutes to over 7 hours between two systems released six months apart.
  • Agent hijacking: a NIST CAISI competition reported in March 2026 logged more than 250,000 attack attempts and at least one successful hijack on all 13 frontier models tested.
  • Task completion: CaMeL, a 2025 design from Google, DeepMind, and ETH, solved 77% of AgentDojo tasks with provable security, against 84% for an undefended system.

What Are AI Guardrails and What Do They Protect Against?

AI guardrails are external controls that sit around a model at runtime and monitor or constrain its inputs, its outputs, and the actions it takes. They protect against four kinds of failure: people misusing the model, outside content hijacking it, the model getting things wrong on its own, and agents taking actions they shouldn’t.

That’s the working definition. There is no official one. A 2025 survey accepted at IEEE S&P calls the field fragmented and short of a shared taxonomy, then offers its own wording.

Guardrails are “external mechanisms that monitor and control LLM interactions,” according to the 2025 SoK survey on evaluating jailbreak guardrails for LLMs.

A second framing comes from CSIRO’s Data61. Its systematic review of runtime guardrails (submitted August 2024, revised January 2025) focuses on agents and treats guardrails as mechanisms that keep an agent’s runtime behaviour “within specific boundaries”.

Both definitions agree on the part that matters most in practice: a guardrail is separate from the model. You can add one, change its policy, or swap it out without retraining anything. Safety behaviour trained into a model’s weights is a different layer, usually called alignment.

Grouped by where the failure starts, the risks look like this:

  • Misuse by the user: someone tries to talk the model into producing harmful content or ignoring its rules (a jailbreak).
  • Manipulation by outside content: a web page, document, or tool result carries hidden instructions that redirect the model (prompt injection).
  • Mistakes by the model: an invented answer, a leak of sensitive data, or a drift away from the job the business deployed it for.
  • Actions by an agent: a tool call that sends, deletes, or changes something without the authority to do so.

The fourth group is the newest, and it raises the stakes. A chatbot that says something wrong produces a bad answer. An agent that does something wrong produces a bad transaction.

Expect guardrails to reduce these risks without removing them.

How AI Guardrails Work at the Input, Output, and Tool-Call Layers

AI guardrails work by checking a request at three points: before the prompt reaches the model, after the model writes its response, and whenever an agent tries to call a tool. At each point the check can pass the content, block it, or send it to a person for approval.

Follow one request through the system:

  1. Input check: the user’s prompt, plus anything that arrives with it (retrieved documents, uploaded files), is screened before the model sees it.
  2. Model call: the model writes a response or decides to use a tool.
  3. Tool-call check: for an agent, the proposed action is screened before it runs.
  4. Output check: the final response is screened before the user sees it.

The CSIRO/Data61 taxonomy of runtime guardrails (2024, revised January 2025) arranges these checkpoints as a multi-layered architecture inspired by the Swiss Cheese Model from safety engineering. The logic is simple. Every layer has holes, so you stack several and count on the holes sitting in different places.

Placement decides what a check can see, and real frameworks show how uneven that gets. The OpenAI Agents SDK guardrails documentation, as of November 2026, describes three guardrail kinds with different reach:

SDK guardrailWhere it runsWhat falls outside it
InputFirst agent onlyLater agents in the chain
OutputFinal agent onlyIntermediate agents’ outputs
ToolAround tool callsHandoffs, hosted tools, built-in shell and computer tools

Here’s what teams tend to miss until they build a multi-agent workflow: the middle of the chain runs unchecked by default. An input guardrail on the first agent says nothing about what the third agent receives through a handoff. If your risk sits in the middle of the workflow, you have to put a check there yourself.

A vertical flow diagram of a request passing through guardrails in six steps. First, the user prompt arrives with retrieved content. Second, an input guardrail screens it before the model, which in the OpenAI Agents SDK runs on the first agent only. Third, the model call writes a response or decides to use a tool. Fourth, a tool-call guardrail screens an agent action before it runs, and SDK tool guardrails do not cover handoffs, hosted tools, or built-in shell and computer tools. Fifth, an output guardrail screens the response before the user sees it, which in the SDK runs on the final agent only. Sixth, the response reaches the user. Each guardrail can pass the content, block it, or send it to a person for approval.
In the OpenAI Agents SDK, input checks cover only the first agent and output checks only the final one, so the middle of a multi-agent chain runs unchecked by default. Source: OpenAI Agents SDK documentation, 2026.

The Main Types of Guardrails: Content Safety, Prompt Injection, Data Leakage, Hallucination, and Topic Control

The main types of guardrails are content safety, prompt injection and jailbreak defence, data leakage prevention, hallucination checks, and topic control. Each one targets a different risk. Sorting by risk is more useful than sorting by product, because a single product often covers several types at once.

Guardrail typeRisk it addressesWhere it usually sits
Content safetyHarmful or disallowed contentInput and output
Prompt injection and jailbreak defenceAttempts to override the model’s instructionsInput, retrieved content
Data leakage preventionSensitive data leaving the systemOutput, tool calls
Hallucination (groundedness) checksAnswers the source documents don’t supportOutput
Topic controlOff-topic or off-brand dialogueInput and output

A few of these need more than a table cell:

  • Content safety: “harmful” is whatever the deployer’s policy says it is. Meta’s 2023 Llama Guard paper built its classifier around a customisable risk taxonomy for exactly this reason.
  • Jailbreaks versus prompt injection: the two differ by who attacks. A jailbreak comes from the user. A prompt injection arrives inside content the model was asked to read, and OWASP files it as LLM01 in its 2025 list of risks for LLM applications.
  • Data leakage: the check runs on the way out. For an agent, that includes tool calls that send data somewhere else.
  • Topic control: NVIDIA’s October 2023 NeMo Guardrails paper describes topical rails that keep a conversation on its subject and style rails that govern how the model speaks.

Hallucination checks matter most in retrieval-augmented generation (RAG) systems, where the model answers from documents it fetched. IBM’s December 2024 Granite Guardian paper treats groundedness as a RAG-specific risk alongside harm and jailbreaks.

I’d argue “hallucination guardrail” oversells what the check does. A groundedness check tests whether an answer is supported by the retrieved documents. Whether those documents are correct is a separate question that no output check answers.

How Guardrails Are Built: Rules, Classifier Models, and LLM Judges

Guardrails are built with four techniques: deterministic rules, purpose-trained classifier models, LLM judges that read a written policy, and architectural designs that restrict what untrusted data can do. They work best in combination, because each one covers a weakness in another.

Deterministic rules and validation code

Rules are ordinary code. A rule checks that an output fits an expected format or that an input stays inside set limits, and it returns the same verdict for the same content every time. OWASP’s prompt injection guidance (LLM01) recommends validating model outputs with deterministic code and filtering both inputs and outputs.

The limit is plain. A rule only catches what you can specify exactly.

Purpose-trained classifier models

A classifier is a model trained to label content as allowed or disallowed. It judges meaning, which rules can’t. Three published designs show the range:

  • Llama Guard (Meta, December 2023): a Llama2-7B model instruction-tuned to classify both prompts and responses.
  • Granite Guardian (IBM, December 2024): a model that screens prompts and responses, with jailbreaks and RAG risks in scope.
  • Constitutional Classifiers (Anthropic, February 2025): input and output classifiers trained on synthetic data generated from a written “constitution” of allowed and disallowed content.

Anthropic’s January 2026 update splits the work into two stages. A cheap linear probe reads the model’s internal activations and screens all traffic. Only the exchanges it flags go on to a stronger classifier, which reads the input and the output together.

A vertical flow diagram of Anthropic's two-stage classifier cascade from January 2026. In stage 1, a cheap linear probe on the model's internal activations screens all traffic. Exchanges it does not flag pass. Exchanges it flags escalate to stage 2, where a stronger classifier reads the input and the output together and then allows or blocks the exchange.
A cheap probe screens every exchange, and only the ones it flags reach the stronger classifier that reads input and output together. Source: Anthropic, 2026.

LLM judges that read a written policy

An LLM judge reads a plain-language policy at the moment it classifies. OpenAI’s gpt-oss-safeguard, a research preview released on 29 October 2025, is an open-weight reasoning classifier built this way. The developer supplies the policy at inference time instead of training it in, so changing the policy means editing text.

OpenAI states the trade-off itself: a classifier trained for one specific risk can still beat it on complex, high-stakes cases.

Architectural guardrails by design

Architectural approaches change the system’s design so that injected instructions have no path to control the agent. CaMeL, published in March 2025 by researchers at Google, DeepMind, and ETH (revised June 2025), separates trusted control flow from untrusted data and attaches capabilities to each piece of data. The agent can read untrusted content, but that content can’t decide what the agent does next.

Wireframe cubes of circuitry linked by glowing strands above a dark circuit-board floor
Built for production

What could a custom AI agent take off your plate?

We build production-grade AI systems that quietly handle the busywork, so your team can focus on the work that actually matters.

View Services

The payoff is a security property the authors can prove, instead of a detection rate someone has to measure.

How Guardrails Differ From System Prompts, Model Alignment, and Evals

Guardrails differ from system prompts, model alignment, and evals in where they sit and when they act. A guardrail is a separate check that runs on live traffic. A system prompt is an instruction the model reads, alignment is behaviour trained into the weights, and evals are tests run offline.

NVIDIA’s October 2023 NeMo Guardrails paper draws the line between rails built in during training and rails a developer defines at runtime. It credits runtime rails with two properties: they’re independent of the underlying LLM, and they’re interpretable. In practice, that means you can read the rule that fired, and you can keep it when you switch models.

Each of the three neighbours does a different job:

  • System prompt: text that tells the model how to behave, read by the same model that reads the user’s message and any retrieved content.
  • Model alignment: safety behaviour trained in, as in Anthropic’s 2022 Constitutional AI paper, which used self-critique and reinforcement learning from AI feedback.
  • Evals and red-teaming: offline tests that measure how a system behaves under attack and block nothing in production.

Here’s the mistake I see most often: treating a long system prompt as a guardrail. It’s the cheapest control to add and the easiest one to attack, because a prompt injection arrives as more text for the same model to weigh against your instructions. A check that runs outside the model doesn’t have that problem.

None of the four replaces another. Alignment sets the model’s default behaviour, and the system prompt scopes it to your use case. Guardrails catch what gets past both. Evals tell you how often that happens.

Guardrail Tools and Frameworks Compared: Open Source Libraries vs Managed Cloud Services

Guardrail tools come in three forms: guard models you host yourself, frameworks that wire checks into an application, and detectors a cloud vendor runs for you. The choice mostly comes down to who operates the check and how much of the policy you control.

ToolMaintainerFormWhat it covers
Llama Guard (December 2023)MetaGuard model, weights releasedPrompts and responses, against a customisable risk taxonomy
Granite Guardian (December 2024)IBMGuard modelPrompts, responses, jailbreaks, RAG groundedness
gpt-oss-safeguard (October 2025, research preview)OpenAIOpen-weight reasoning classifierAny written policy the developer supplies
NeMo Guardrails (October 2023)NVIDIAProgrammable frameworkTopical, safety, and execution rails
LlamaFirewall (May 2025)MetaAgent security frameworkJailbreaks, agent reasoning, generated code
Agents SDK guardrailsOpenAISDK featureAgent inputs, outputs, and tool calls
Azure Prompt ShieldMicrosoftHosted detectorPrompt injection and jailbreaks

The rows behave differently once you try to adopt them:

  • Self-hosted guard models: Llama Guard and gpt-oss-safeguard both ship with open weights, so you set the policy and you own the hosting.
  • Frameworks: NeMo Guardrails pairs a runtime engine with a language called Colang for writing rails, and the OpenAI Agents SDK’s guardrails only help if you build on that SDK.
  • Managed detectors: with Azure Prompt Shield, Microsoft operates the check and you call it as a service.

One version detail matters for anyone benchmarking. The Llama Guard described in Meta’s December 2023 paper is built on Llama2-7B, so confirm which release you’re pulling before you compare numbers.

The names mislead, too. Llama Guard and LlamaFirewall share a maker and half a name, and they do different jobs. Llama Guard labels content. LlamaFirewall, which Meta bills as a “final layer of defense” for agents, bundles three parts: PromptGuard 2 for jailbreak detection, Agent Alignment Checks (an experimental auditor of the agent’s chain of thought), and CodeShield, which runs static analysis on generated code.

That difference matters if you’re building AI agents with tool access. A content classifier won’t inspect the code an agent writes, and a code scanner won’t catch a harmful reply. Most agent projects end up needing one of each.

What Do AI Guardrails Cost in Latency and Spend?

AI guardrails cost extra compute, added response delay, and some lost usefulness, and Anthropic’s January 2026 classifiers put the compute share at about 1%. In February 2025, the first generation of the same system cost 23.7%.

The published figures, with their dates:

Cost measuredFigureSource and date
Compute overhead, first-generation Constitutional Classifiers23.7%Anthropic, February 2025
Compute overhead, next-generation classifiersAbout 1%Anthropic, January 2026
Added over-refusal, first generation0.38%Anthropic, February 2025
Refusal rate on harmless queries, next generation0.05%Anthropic, January 2026
Task completion with CaMeL (undefended: 84%)77%Google, DeepMind, ETH, 2025

The fall in compute came from a design change, described in Anthropic’s January 2026 write-up of its next-generation Constitutional Classifiers. A cheap first stage screens all traffic, and the expensive classifier reads only what gets flagged. Anthropic measured the 0.05% refusal rate over one month of Claude Sonnet 4.5 traffic. The earlier 0.38% rise in over-refusal was not statistically significant.

Delay and lost usefulness depend on where the check sits and how it’s built:

  • Output checks: the 2025 SoK survey on jailbreak guardrails found that post-processing guardrails add more delay than pre-processing ones, even when both use identical detectors.
  • Reasoning classifiers: OpenAI said in October 2025 that gpt-oss-safeguard can be slow and compute-heavy.
  • Architectural designs: CaMeL solved 77% of AgentDojo tasks with provable security in 2025, against 84% for an undefended system. That’s about 7 points of task completion given up.

Two caveats before any of this goes into a budget. Anthropic’s figures are self-reported. And every number here is a share of compute or a benchmark score, so dollar prices will depend on your own traffic and on which checks run on every request.

The pattern I’d copy is the cascade: run something cheap on everything and escalate only the suspicious cases.

A horizontal bar chart of compute overhead for Anthropic's Constitutional Classifiers, as reported by Anthropic. The first generation, from February 2025, added 23.7% extra compute. The next generation, from January 2026, added about 1%.
Anthropic’s next-generation classifiers add about 1% compute overhead, down from 23.7% for the first generation in February 2025. Source: Anthropic, 2026.

Where AI Guardrails Fail: Jailbreaks, False Positives, and Coverage Gaps

AI guardrails fail in three ways: determined attackers get past them, they block harmless requests, and they leave parts of a system unchecked. Vendor tests show large gains. Independent tests keep finding a way through.

The UK AI Security Institute has found “universal jailbreaks for every single system we’ve tested to date,” according to its Frontier AI Trends Report on models released up to October 2025.

The same report also carries independent evidence that safeguards are improving. AISI compared two leading systems released six months apart. Expert red-teamers jailbroke the first in about 10 minutes and needed over 7 hours for the second, roughly 40 times the effort. A more capable model doesn’t get this for free: AISI found almost no link between a model’s general capability and how well its safeguards hold (R² = 0.097).

EvidenceYearWho ran itResult
Constitutional Classifiers on 10,000 synthetic jailbreak prompts2025Anthropic (vendor)Jailbreak success fell from 86% to 4.4%
Six prompt-injection and jailbreak detectors, including Azure Prompt Shield and Meta’s Prompt Guard2025Hackett et al., ACL LLMSEC workshopUp to 100% evasion in some cases
12 published defences2025Authors of The Attacker Moves SecondAttack success above 90% for most
13 frontier models, 400+ participants, 250,000+ attack attempts2026NIST CAISI competitionAt least one successful hijack on all 13

Both sets of results can be true. Anthropic’s 4.4% comes from a fixed set of synthetic prompts, and the authors of The Attacker Moves Second argue that fixed attack sets are the wrong test: most of the 12 defences they broke had originally reported near-zero attack success. Anthropic’s own record fits. In 2025, 183 red-teamers spent more than 3,000 hours on its prototype without finding a universal jailbreak, and then a participant in the public demo found one. I read any near-zero figure as a claim about the test set until adaptive attackers have had a turn.

The other two failure modes get less attention:

  • False positives: the 2025 SoK survey names over-refusal of benign requests as the main utility cost, and an October 2025 preprint, RAG Makes Guardrails Unsafe?, reports that the guardrails making the fewest errors also had the highest false-positive rates.
  • Messages judged alone: some guardrails score each message in isolation, which multi-turn and split-request attacks exploit.
  • Reconstruction and obfuscation: Anthropic says two attack classes beat its first-generation classifiers, one that splits harmful content into benign pieces and one that disguises the output.
  • Unchecked paths: guardrails in agent frameworks can skip handoffs between agents and built-in tools.

The International AI Safety Report 2026, published on 3 February 2026, recommends layered defence in depth and says such measures “will likely fail to prevent some AI-related incidents”. Plan for the incident.

Which Teams and Use Cases Need Guardrails Most

The teams that need guardrails most are those running agents with tool access that read untrusted content, because a successful attack there ends in an action instead of a bad answer. Exposure falls as you take away tools, outside inputs, and sensitive data.

AlphaCorp AIonline
Let's talk

Curious what AI could do for your business?

No jargon and no hard sell. Just a friendly look at where AI fits, and where it doesn't.

View Services

Ranked by exposure:

  1. Agents with tools and untrusted inputs: tool-call checks, prompt injection defence, least privilege, and human approval for high-risk actions all apply. Designs that isolate untrusted data are worth their cost in task completion here.
  2. RAG systems over sensitive data: every retrieved document is an input, so screen retrieved content for injected instructions and check outputs for leakage and groundedness.
  3. Public-facing chat: anyone can type anything, so content safety and jailbreak checks on input and output come first, with topic control to keep the assistant on its job.
  4. Open-weight deployments: the UK AI Security Institute’s Frontier AI Trends Report calls open-weight models the hardest to defend because their safeguards are cheap to remove. Put your controls in runtime checks that you operate.
  5. Regulated, high-stakes domains: in healthcare or financial services a wrong action carries legal and financial consequences, so keep a person in the approval path for the riskiest ones.

The last two items stack with the first three. A tool-using agent built on an open-weight model inside a hospital faces all five at once.

What sits at the bottom? A single-user internal tool with no tool access, no outside content, and no sensitive data. Basic input and output checks are usually enough there, and I’d spend the saved effort on the agent.

What Changed for AI Guardrails This Year: Agents, Tool Use, and Regulation

In 2026, AI guardrails got cheaper to run, expert consensus settled on layered defence, and security testing moved from chatbots toward agents that use tools. The binding rules sit in the EU AI Act. The main US reference documents are voluntary.

DateDevelopmentWhat it signals
January 2026Anthropic publishes its next-generation Constitutional ClassifiersAlways-on screening at about 1% compute overhead
3 February 2026International AI Safety Report 2026Layered defence as the expected baseline
23 March 2026NIST CAISI reports on an agent red-teaming competitionAgent hijacking as a main test target
8 April 2026NIST updates its page for AI 600-1, the Generative AI ProfileA voluntary framework still being maintained

Anthropic also reported more than 1,700 hours of red-teaming on its January 2026 system with no universal jailbreak found. That figure is self-reported.

The March entry is the one I’d watch. NIST CAISI’s account of a large-scale agent red-teaming competition covers more than 400 participants and over 250,000 attack attempts against 13 frontier models, and it reports that some attacks transferred from one model to another. Swapping models is a weak fix if the same attack works on the next one.

Users can still sometimes get harmful outputs “by rephrasing requests or breaking them into smaller steps,” according to the International AI Safety Report 2026.

The regulatory backdrop has three parts:

  • EU law: Article 15 of the EU AI Act (Regulation (EU) 2024/1689) requires high-risk systems to meet appropriate standards of accuracy and cybersecurity and to be “resilient against attempts by unauthorised third parties”.
  • Named threats: the article lists data poisoning, adversarial examples, and model evasion, and it expects measures that prevent, detect, respond to, resolve, and control for attacks.
  • US guidance: NIST’s AI 600-1 (first published 26 July 2024) and its March 2025 adversarial machine learning taxonomy, AI 100-2 E2025, which covers prompt injection and agent security, are both voluntary.

Confirm the application dates for your system’s risk category against the official text before you plan around a deadline.

Frequently Asked Questions About AI Guardrails

These are the questions people ask most often about AI guardrails, each answered in a few sentences that stand on their own.

Are AI guardrails the same as content filters?

No. A content filter is one type of guardrail, and it screens text for harmful or disallowed material. Guardrails also include prompt injection defence, data leakage checks, groundedness checks on RAG answers, topic control, and checks on the tool calls an agent wants to make.

Can AI guardrails be bypassed?

Yes. The UK AI Security Institute reported finding universal jailbreaks for every system it had tested, in a report covering models released up to October 2025. A 2026 NIST CAISI competition found at least one successful hijack against all 13 frontier models it tested. What guardrails change is the effort required: AISI saw expert jailbreak time rise from about 10 minutes to over 7 hours between two systems released six months apart.

Do AI guardrails stop prompt injection?

They reduce prompt injection without ending it. OWASP’s 2025 guidance says it’s unclear whether fool-proof prevention exists, and a 2025 study by Hackett et al. evaded six injection and jailbreak detectors with up to 100% success in some cases. The strongest published defence is architectural: CaMeL, in 2025, solved 77% of AgentDojo tasks with provable security, against 84% for an undefended system. Pair detection with least privilege and human approval for risky actions.

Do guardrails slow down an LLM?

Yes, and the size of the delay depends on design. The 2025 SoK survey on jailbreak guardrails found that checks run after the model responds add more delay than checks run before it, even with identical detectors. On compute, Anthropic reported a 23.7% overhead for its classifiers in February 2025 and about 1% in January 2026. Classifiers that reason over a written policy are slower, as OpenAI said of gpt-oss-safeguard in October 2025.

Are AI guardrails required by law?

It depends on where you operate and how your system is classified. Article 15 of the EU AI Act requires high-risk AI systems to be resilient against attempts by unauthorised third parties, and it names data poisoning, adversarial examples, and model evasion. In the US, NIST’s AI 600-1 and AI 100-2 E2025 are voluntary guidance.

What is the difference between guardrails and AI alignment?

Alignment is safety behaviour trained into a model’s weights, as in Anthropic’s 2022 Constitutional AI paper. Guardrails are separate checks that run around the model on live traffic. You can change a guardrail’s policy or move it to a different model without retraining, which NVIDIA’s 2023 NeMo Guardrails paper describes as being independent of the underlying LLM. The two work together: alignment sets default behaviour, and guardrails catch what gets past it.

Where to Start With Guardrails in Your Own Application

Start with guardrails in your own application by mapping where untrusted content enters, then adding controls in this order:

  1. Map the risks: list every input you don’t control (user prompts, retrieved documents, tool results) and every action the system can take.
  2. Add input and output checks first, because every request passes through them.
  3. Apply least privilege to each tool, and require human approval for high-risk actions, as OWASP’s prompt injection guidance recommends.
  4. Layer the defences so a missed attack still meets a second check.
  5. Test with adaptive attackers who change tactics after each block. A fixed list of known attacks tells you little.

Then write down what happens when a check fails, because one will.

If you’d like a second opinion on where your AI guardrails have gaps, AlphaCorp AI can review the systems you already run through an AI integration audit.

Share
Newsletter · Weekly

Stay Ahead of AI

One email per week with the AI engineering insights, agent builds, and tools that actually matter.

No spamUnsubscribe anytimeFree forever

In every issue
  1. 01One agent build, taken apart step by step
  2. 02The tools that earned a place in our stack this week
  3. 03What broke in production, and what we changed

Written by Ignas Vaitukaitis, founder of AlphaCorp AI.

Wireframe cubes of circuitry linked by glowing strands above a dark circuit-board floor

Ready to Ship
Your AI System?

Book a free call and let's talk about what AI can do for your business. No sales pitch, just a real conversation.