What do “Guardrails” Mean?

Guardrails are controls that run before, during, and after a model reply. They reduce the chance of bad inputs, wrong or unsafe outputs, and data leaks. Think of it as five layers.

• Input guardrails. Clean and screen the input. Remove dangerous instructions. Flag sensitive entities.
• Retrieval guardrails. Only fetch from the right sources with the right scope. Enforce tenant and role limits during search.
• Policy guardrails. Apply allow and deny rules. Do it before and after the model runs.
• Output guardrails. Check for sensitive content. Require citations if you promise a grounded answer.
• Action guardrails. If the reply triggers an action, confirm the actor, the data, and the target system. Add human review for high-risk paths.

This does not slow you down when you design it well. It frees your team to move faster because the rules are clear and repeatable. It also lines up with the NIST AI RMF idea that teams should govern risk, map the system and context, measure what matters, and manage the risk over time.
(NIST Publications)

Guardrails Stack

• Front door
  • What it is: The place where requests enter and users sign in.
  • Why it matters: Keeps strangers out and directs each request to the right place.
• User profile and permissions
  • What it is: A list of who the person is and what they are allowed to see or do.
  • Why it matters: Answers change based on role, location, and purpose. This prevents mix-ups between teams or customers.
• The library
  • What it is: Your approved documents and data, stored with labels like tenant, team, topic, and date.
  • Why it matters: The AI is only allowed to read from here. Labels make sure it does not fetch the wrong file.
• The rules board
  • What it is: A small service that reads simple rules like “Managers can view summaries of salaries but not raw numbers.”
  • Why it matters: The rules are applied before and after the AI writes a reply. Unsafe requests are blocked or redacted.
• The assistant
  • What it is: The AI model that drafts the reply from the allowed sources.
  • Why it matters: This is the “writer,” not the “judge.” It does not choose
  • What is allowed. The rules board and the library decide that.
• The proof checker
  • What it is: A quick check that the reply has sources and no sensitive text that should be hidden.
  • Why it matters: Catches bad content before users see it.
• The human desk
  • What it is: A small review screen for tricky or high-risk replies.
  • Why it matters: A person approves or edits when the impact is high or confidence is low.
• The notebook
  • What it is: A log of what happened. Who asked. Which sources were used? Which rule allowed it?
  • Why it matters: This is how you learn, fix issues, and pass audits.
  • That is the full stack in simple words. You can build each “desk” with many tool choices, but the roles stay the same.

Retrieval Guardrails

Why this matters: When AI gives you a bad or wrong answer, it's usually not because the AI model is broken. The real problem is that the AI is looking at the wrong information.

Think of it like this – imagine you're asking someone a question, but they're reading from the wrong book to give you an answer. The person isn't the problem – they're just working with bad reference material.

Here's what typically goes wrong

• The AI pulls in a document that doesn't actually answer your question
• The AI accidentally uses information meant for a different customer or project
• The search system grabs irrelevant content and treats it as relevant

Retrieval guardrails are like quality checks that make sure the AI only sees the right information before it tries to answer your question. They act as filters that catch these problems before they reach you.

It's much easier to fix what information goes into the AI than to try to fix how the AI thinks about that information.

What to put in place

• Access control – Bind tenant, product, and role to every query. In OpenSearch, you can set doc-level and field-level rules. That keeps disallowed rows or fields out of results and out of prompts. Do not only filter after retrieval. Filter before and during search as well. (OpenSearch Docs)
• Metadata – At ingest time, tag documents with tenant, branch, role, data class, and expiration. Keep tags small and consistent.
• Freshness windows – Skip outdated documents by default.
• Source allowlists – Only allow certain sources for a given question type.
• PII scrubbing – If the reply never needs raw PII, remove or mask it before indexing.
• Citations – If a reply cannot cite from approved sources, block the display or downgrade to a safe summary.
• No safe answer path – If the system cannot answer safely, say so. Do not guess.

Where we used it

• In PhysioTrack, the app helps clinicians review exercise notes. Retrieval uses treatment notes from the current patient only. The prompt includes policy tags so the model never sees other patients.
• In EduSure, retrieval runs only over documents from the user’s board and license type. We add a freshness rule. Old circulars do not affect a current answer.
• In Mortgage LOS and POS, proof docs drive income summaries. The app requires a source link for every figure. If no valid doc is present, the reply cannot show a number.

Auditable Policy Guardrails

Policies answer four simple questions.

• Who can ask?
• What data can be used?
• What kind of answer is allowed?
• Where may the output go?

Express these rules with attributes. That is ABAC (Attribute-Based Access Control). It fits multi tenant systems well.

Key design choices

• Attributes – Use tenant ID, role, purpose, country, data class, and channel.
• Pre-model and post-model checks – Check before you call the model. Check again on the reply. You can block, redact, or downgrade to a safe summary.
• Version your policies – Record which policy version made each decision.
• Keep decisions explainable – Log the input attributes, the rule that matched, the action, and the reason.

Choosing the Right Tool for Your Rules

If you're building something basic, you can just write your own simple service to handle your rules. That works fine for straightforward projects. But if you need more features, use a dedicated policy engine instead. Here's why

• You can easily test your rules to make sure they work
• You get a record of what rules you used and when you changed them
• You can run automatic checks every time you update your code

Think of it like choosing between a basic calculator and a spreadsheet program. The calculator works great for simple math, but if you need to track changes and run complex formulas, you'll want the spreadsheet.

The same logic applies here; start simple, but upgrade to a proper policy engine when your needs get more complex.

Input and output hardening

Stop prompt injection early – Check all incoming requests for suspicious patterns that might try to trick your AI model. Never trust instructions that come from documents or text you've retrieved – they could be traps. Remove or weaken any content that tries to take control of your system. Security experts at OWASP list prompt injection as the biggest threat to AI apps. Microsoft has also found cases where attackers hide malicious instructions in web pages or emails that your AI might read. You should treat all retrieved text as potentially dangerous.

Treat the AI's response like untrusted input – Always check the AI's answer before showing it to users. Look for leaked passwords, personal information, or words that shouldn't appear. If the answer is supposed to include links or sources, make sure those links actually exist and work properly. OWASP warns that poor output handling is a major security risk. Build a simple checking system that runs quickly on every response. Also, manually proofread the responses (this cannot be stressed enough).

Learn from real incidents – In June 2025, researchers found a serious flaw in Microsoft 365 Copilot's document retrieval system. The bug could let attackers steal sensitive data without the user even clicking anything. Microsoft fixed the problem, but it teaches us an important lesson. Document retrieval can access information in unexpected ways that surprise development teams. In 2025, Google accidentally indexed thousands of private ChatGPT conversations that users had shared via links, exposing sensitive personal discussions about mental health, work, and relationships to public search results. OpenAI quickly removed the sharing feature after discovering users didn't realize their "private" shared chats could become publicly searchable. Good guardrails help prevent these kinds of security breaches from happening.

Human in the loop without slowing teams down

Human review isn't meant to slow things down or catch mistakes as a markdown. It's about using people's judgment where it counts most. You can let safe, low-risk answers go straight to users automatically. But when something seems risky or important, send it to a human first.

You might want a human review when

• The AI can't find good sources to back up its answer
• The AI seems unsure about its response
• The answer mentions sensitive topics or people
• The AI wants to take a big action like sending emails to people outside your company

What You Need to Build

• A simple work queue – Organize tasks by which product they're for, which customer they serve, and what type of request they are.
• A side-by-side review screen – Show the reviewer both the AI's answer and the original sources it used. Also, show any warning tags your system flagged.
• Three simple choices – Let reviewers approve the answer, edit it, or send it back for a complete do-over. Keep track of what they decided and who made the choice.
• A way to learn from changes – When reviewers edit answers, use those improvements to make your AI prompts and rules better over time.

The best human review systems focus on clear responsibilities, honest communication about what's happening, and giving people the power to override the AI when needed. They also track how well things are working so teams can keep improving. This keeps oversight meaningful without creating delays that frustrate everyone.

Where we use it

• In Mortgage LOS and POS, income numbers show only after a reviewer confirms the source and the math.
• In PhysioTrack, discharge summaries go to a clinician for a light review. The app helps with a checklist so the review is fast.
• In EduSure, a small sample of answers is reviewed every day. This gives early signals when a policy or data source needs a fix.

Life Cycle of a Safe Answer

Let us walk a single request through the pipeline.

• The question arrives – A bank officer asks, "What is Joan Smith's current debt-to-income ratio and how does it compare to our program rules?" The question hits our system.
• The system figures out who's asking – It identifies the officer, their bank branch, their job role, what country they're in, and which banking product this relates to.
• The system searches for information – It looks through documents but only shows what this officer is allowed to see based on their role. It finds Joan's financial documents and the current program rules.
• The policy checker decides what's okay to share – It says the officer can get a summary of the information but can't see raw personal details like Social Security numbers.
• The AI gets clear instructions – It must cite at least two sources for its answer and can't include any personal identifying information in its response.
• The AI gives its answer – The system double-checks that the answer includes proper citations and doesn't accidentally leak sensitive information.
• A human reviews the answer – Since this involves a financial decision that could affect a loan offer, the system sends it to the officer or a colleague for approval before showing the final answer.
• Everything gets recorded – The system saves who asked, which rules were used, what sources were checked, the final decision, and who approved it.
• The officer sees the final answer – They get Joan's ratio, the program rule, links to the source documents, and a "Reviewed" banner showing a human checked it first.

This is what safe and fast looks like. The user gets their answer, and the system keeps everyone protected.

Rollout Plan

Here is a staged path you can start from today.

Quick Wins

• Turn on tenant and role filters in retrieval.
• Add allowlists for sources.
• Mask sensitive fields at ingest.
• Add a light output checker.
• Route sensitive categories to a manual queue.

Strategic Initiatives

• Move rules into a small policy service with tests.
• Require citations for grounded answers.
• Add a reviewer app with queues and SLAs.
• Do a canary release for one product and one tenant.

Transformational Changes

• Add automated red team prompts in CI.
• Add policy regression tests.
• Route traffic based on risk.
• Add drift alerts when behavior or cost moves outside a band.

This keeps risk low while value goes up. It also gives your team a rhythm that fits the NIST Govern, Map, Measure, Manage loop.

Common mistakes and fast fixes

Only prompt work – If your safety plan is only prompt changes, you will chase issues forever. Add retrieval and policy layers that do not depend on model behavior. OWASP puts this risk in clear terms.

One big index – A single shared vector index for every tenant sounds simple. It is hard to secure. Use strong doc and field level security with per tenant filters. In many cases, split indices by tenant or by product to make rules simpler.

Answer at any cost – When the data is not safe or is missing, say so. A clean “no safe answer” is better than a risky guess.

No audit trail – If you do not log the decision path, you cannot learn or pass an audit. Log policy inputs, the decision, the reason, and the version.

Skipping adversarial tests – Add prompt injection tests in CI. Microsoft’s public notes on indirect prompt injection show how easy it is to hide instructions in retrieved content. Test for this.

What you get with Aakash

We design and ship AI features that live inside real, regulated workflows. Our team has built retrieval with OpenSearch, attribute based policies, and human review across products like MotionX, PhysioTrack, EduSure and Mortgage LOS & POS. We blend product work with security and data work. You get a reference architecture, a working pilot, policy packs tuned to your data classes, and an operations playbook that your team can run. We keep things simple. We keep them practical. We help your engineers and your business owners speak the same language.

If you plan a new AI feature or if you want to fix an existing one, reach out to the Aakash team. We will run a short guardrails assessment, map your risks, and outline a build plan that fits your stack and budget. We can also run a workshop to adapt the checklist to your domain. It takes a small effort to start and it pays back quickly.