LLM Guardrails and Prompt Injection: Securing and Monitoring AI in Production

Your assistant passed every eval you wrote. Then a customer pastes a web page into the chat, and your assistant quietly starts sending private account details to a stranger. Nothing crashed, and no error was logged. This gap between a working demo and a safe production system is what we close today.

• Green evals, silent failure
• No crash, no error

Last time we made serving fast with continuous batching and PagedAttention. Fast is not the same as safe. In this final production lesson we build a threat model, stack defenses in layers, write a small guardrail pipeline in Python, and then learn to watch the system with tracing, metrics, online evals and incident playbooks.

• Last time: serving with vLLM
• Today: safe and observable

Here is the root problem. A language model reads one stream of tokens. Your instructions, the user's message, a retrieved document and a tool result all arrive as the same kind of text. Classic software separates code from data. A model has no hard wall between them, and almost every attack in this lesson exploits that fact.

• One text channel for everything
• Model cannot truly tell them apart

A threat model starts by naming what can go wrong. We will use five buckets: misuse of your product for harmful purposes, leakage of private data, jailbreaks that talk the model out of its policy, prompt injection that hijacks its instructions, and excessive agency, where the model can do more than it should. Each needs a different defense.

• Misuse and data leakage
• Jailbreaks and injection
• Excessive agency

Misuse asks who is using your product and for what. A customer support bot should not write malware or give dangerous instructions. Leakage asks what the model can see. If your prompt, your retrieved documents or your tool results contain secrets or another user's data, then a clever question might surface them. Anything in context should be treated as potentially revealable.

• Who can use it, for what
• What can the model see

A jailbreak is a user trying to pressure the model into ignoring its safety training, through role play, long persuasion or odd framing. We will not write any, because the defensive lesson is simple. Model alignment lowers the odds but is statistical, and new phrasings keep appearing. So you never let the model's good behavior be your only barrier.

• Pressure on the model's policy
• Training helps, never guarantees

Excessive agency is the quiet multiplier. Think of risk as likelihood times impact. Suppose a chat-only bot has a one percent chance of being tricked, but the worst outcome is an embarrassing sentence, impact one. Give the same bot a refund tool and an email tool, and the impact jumps to ten or a hundred. Same likelihood, far more risk.

• Risk equals likelihood times impact
• Tools raise the impact

Direct prompt injection is when the person typing is the attacker. They write something like: ignore your earlier rules and do something else. It is the easiest to picture and, in many apps, the least dangerous, because the attacker only controls their own session and sees only what they are already allowed to see.

• The user writes the attack
• Goal: override instructions

Indirect injection is the serious one. The attacker never talks to your bot. They plant instructions inside a web page, a PDF, an email or a support ticket. Later your retrieval system or a tool fetches that content, the model reads it as context, and the planted text tries to steer the assistant on behalf of the attacker, against an innocent user.

• Attack hides in retrieved content
• Victim is a different user

A useful mental model says the dangerous setup needs three ingredients together. First, access to private data. Second, exposure to untrusted content. Third, a channel to communicate outward, such as sending email or loading a URL. Remove any one of the three and the worst attacks lose their power.

• Private data
• Untrusted content
• A way to send data out

Take an assistant that reads your inbox and can send email. It sees private mail, it reads messages from strangers, and it can send. All three ingredients are present. The fix is not a cleverer prompt. Make sending require your approval, or restrict recipients to a short allowlist. Now stolen data has nowhere to go, even if the model is fooled.

• Reads inbox, sends mail
• All three ingredients present
• Remove one leg

This brings us to the most common mistake: relying on the system prompt alone for security. Writing never reveal secrets or never call this tool unless asked is a polite request to a statistical model. It is not enforcement. Anything that must never happen should be blocked by code outside the model, where a persuasive paragraph cannot argue with it.

• System prompt is a request
• Not an enforcement mechanism

Since every single defense leaks, we stack them. Imagine each layer catches eighty percent of bad attempts. One layer misses twenty percent. Three independent layers miss point two cubed, which is point zero zero eight, so ninety nine point two percent are caught. Real layers are not fully independent and attackers adapt, so treat this as an optimistic ceiling, not a promise.

• Every layer has holes
• Stack independent layers

The first layer inspects input. Cheap rules catch obvious things like enormous inputs, known bad patterns or disallowed file types. A moderation model, which is a small classifier or a dedicated safety model, scores nuanced categories. Both add latency and can block good users, so tune thresholds with real examples and run cheap checks before expensive ones.

• Cheap rules first
• Moderation model for nuance
• Adds latency and cost

The second layer inspects what the model produced, before anything acts on it. If you asked for structured output, as in our earlier schema lesson, validate it strictly: valid JSON, expected fields, allowed values, sensible ranges. A refund amount of negative five hundred or an unknown action name is rejected in code. Output filters can also scan text for leaked secrets or disallowed content.

• Validate before acting
• Schema must parse and fit
• Fail closed on doubt

Personally identifiable information deserves its own layer. Redact emails, phone numbers and identifiers before they reach third-party services, before they enter logs, and before they appear in outputs that should not contain them. Pattern matching is quick but imperfect, so pair it with access controls, and never assume a regex found everything.

• Mask emails, phones, IDs
• Before logs, prompts, outputs

Privilege separation is architecture, not filtering. Use one quarantined model call to read untrusted content, and give it no tools and no secrets. It can only return a constrained result, like a short summary or a label from a fixed list. A separate privileged component holds the tools and never sees the raw untrusted text. A hijacked reader has nothing to steal and nothing to press.

• Quarantined model reads untrusted text
• Privileged model holds tools

Apply least privilege to tools. Expose only the tools the task needs, from an explicit allowlist. Give each a credential scoped to the minimum: read-only database access instead of admin, one folder instead of the whole drive. Validate every argument in code, because the model chooses the arguments and an attacker may be influencing that choice.

• Only named tools
• Scoped credentials
• Validate every argument

If the model can run code, assume the code is hostile. Run it in a sandbox: a container or lightweight virtual machine with no access to your secrets, no network unless required, limited memory and disk, and a hard time limit. Throw the environment away afterwards. The sandbox turns a possible disaster into a failed job.

• Isolated container or VM
• No network, limited disk, time limit

For actions that are hard to undo, such as payments, deletions or outbound messages, put a human in the loop. Show the exact action and arguments, not a friendly summary. Keep approvals rare. If people click approve a hundred times a day they stop reading, so reserve the gate for the actions where a mistake really hurts.

• Approve irreversible actions
• Show exact action, not summary
• Beware approval fatigue

Be honest about the limits. No filter, model or prompt gives complete protection against injection today. The practical goal is to assume some attacks will succeed, then limit what a successful one can do, detect it quickly, and recover. Test regularly with your own red team exercises and keep a growing regression set of failures you have seen.

• Assume some attacks succeed
• Limit blast radius
• Red-team and re-test

Now let us write it in plain Python, independent of any provider. The pipeline is four steps. Check the input. Call the model. Validate the output against a schema and an allowlist. Redact personal data from whatever goes back to the user. Each step is a small function that returns either a clean value or a rejection with a reason.

• Check input
• Call model
• Validate output
• Redact PII

The input check does two cheap things. It rejects messages longer than a limit, say four thousand characters, because huge inputs hide payloads and burn money. It also checks a short list of suspicious phrases using lowercase matching. Return a pair: a boolean and a reason. A blocklist is easy to evade, so treat it as a speed bump that feeds your logs, never as the defense.

• Length limit
• Simple blocklist
• Return ok and reason

For redaction we use re dot sub with two patterns. One matches an email: word characters, an at sign, a domain, a dot, a suffix. Another matches phone numbers made of seven to fifteen digits with optional spaces or dashes. The text contact jane at example dot com or 555 0100 becomes contact EMAIL or PHONE. Regex misses unusual formats and sometimes hits harmless numbers, so measure both errors.

• Email pattern
• Phone pattern
• Replace with tags

The output validator parses the model's reply with json dot loads inside a try block. Then it checks that the action is in an allowlist, such as answer, refund or escalate, and that fields like amount lie between zero and one hundred. Anything else returns a rejection. A reply of refund with amount negative five hundred never reaches your payment system.

• Parse JSON strictly
• Check action allowlist
• Check value ranges

Wire the steps together with a clear failure policy. When a check fails, fail closed: return a safe fallback message, or hand off to a human, instead of passing the raw output along. Log the reason every time. Failing open feels friendly during a demo and becomes the incident report later.

• On doubt, refuse or escalate
• Never pass raw output through

Guardrails reduce risk, but you still need to see what is happening. Language model failures are usually silent: the service returns a confident, well-formed, wrong answer with a healthy status code. Prompts change, documents change, models get updated, and users surprise you. So we log structured traces of every request, including inputs, outputs, tool calls, timings, token counts and guardrail decisions.

• Outputs are nondeterministic
• Failures are silent
• Prompts and data change

Here is a tracing decorator. It wraps your pipeline function, records the start time, calls the function, and catches any exception so it can log the error and re-raise. In a finally block it appends one JSON object per line to a file: request id, duration in milliseconds, status, token counts and the guardrail outcome. JSON lines files are easy to append to and trivial to analyse.

• Wrap any function
• Time it, catch errors
• Append one JSON line

Reading the file back gives a dashboard. Say we logged one thousand requests, and thirty were refusals. Refusal rate is thirty over one thousand, three percent. If each request used eight hundred tokens at a hypothetical price of two dollars per million, total tokens are eight hundred thousand and cost is one dollar sixty. Sort the latencies and read the median and the ninety fifth percentile, because averages hide slow tails.

• 1000 requests, 30 refusals
• Refusal rate 3 percent
• Cost from token counts

Four core numbers to chart over time are latency, cost per request, error rate and refusal rate. A rising refusal rate can mean an overactive filter, and a falling one can mean a guardrail broke. Add user feedback signals too: thumbs up and down, regenerate clicks, copy actions, follow-up complaints and human edits. They are noisy, but they point you toward the traces worth reading.

• Latency, cost, error, refusal
• Thumbs, regenerate, edits

Drift is when the world moves under a system that has not changed. Users start asking about a new product. The document corpus is updated and retrieval shifts. The provider updates a model version behind the same name. Track input length, topic mix, retrieval scores and output style over time, and alert when a distribution moves away from its baseline.

• Inputs change
• Corpus changes
• Model versions change

You cannot read every trace, so sample. With one hundred thousand requests a day, sampling two percent gives two thousand to score. A judge model rates each on a rubric: grounded in sources, policy violations, helpfulness. Judges have biases, such as favoring long answers, so periodically compare their scores with human labels on a small set and fix the rubric when they disagree.

• Sample 2 percent of traffic
• Judge scores quality and safety
• Calibrate against humans

Ship changes to prompts, models and guardrails gradually. A canary release sends five percent of traffic to the new version while watching refusal rate, errors, cost and judge scores. Noise matters: with one thousand requests and a three percent rate, the standard error is the square root of point zero three times point nine seven over one thousand, about half a percent. So a move from three to four percent is within noise.

• Send 5 percent to the new version
• Compare guard metrics
• Roll back automatically

Write the incident playbook before you need it. Step one, contain: a kill switch that disables a tool or the whole feature, and a fast rollback. Step two, investigate with the traces: who, which input, which content, which tool call. Step three, fix the root cause, rotate any exposed credentials, notify whoever you must, and add the case to your regression set so it never returns quietly.

• Contain, investigate, fix
• Kill switch and rollback
• Add a regression test

Let us collect the mistakes. Trusting the system prompt as your only security. Giving agents broad tools and credentials. Treating retrieved or tool content as trusted instructions. Logging raw personal data into your traces. Shipping without a kill switch. And never turning past incidents into tests. Each one is cheap to avoid and expensive to learn from the hard way.

• Prompt-only security
• Too much agency
• Logging raw secrets
• No regression set

To recap: models cannot separate instructions from data, so assume injection will happen. Limit agency, separate privileges, validate in code, redact personal data, and keep humans on irreversible actions. Then trace everything, watch latency, cost, errors, refusals and drift, sample with a judge, release gradually, and keep a playbook. That closes the production section. Next we start the capstones, beginning with a document question answering bot.

• Threat model, layers, traces
• Next: capstone document Q&A bot