Document Level Access in Azure AI Search: A Complete Guide to Secure RAG

Introduction

Hey all!

Moving a RAG application from a demo to production requires a fundamental shift from focusing solely on relevance to prioritizing RAG Security. In the enterprise, the context we provide to an LLM must be strictly governed by Document Level Access.

In this post, I explore how to architect these security boundaries within Azure AI Search. We will move beyond simple retrieval to look at how metadata like permissionFilter (userIds , groupIds , rbacScope ) allows for granular control.

I will also demonstrate how to implement the x-ms-query-source-authorization header to ensure that “security trimming” happens natively at the search layer, keeping unauthorized data out of your AI’s reach.

Security is critical, but it is even more important to understand the basics of RAG and Azure AI Search first, so be sure to check the foundational posts linked below to build that core knowledge:

• Azure AI Search Indexing Strategies: Pull vs Push Approach
• Naive RAG Explained: The Core Pattern
• Semantic Ranking in Azure AI Search: How Cross-Encoders Improve RAG Retrieval
• Foundry IQ for Agentic Retrieval: Intro to enterprise RAG
• Foundry IQ Masterclass: The Agentic Retrieval Pipeline Explained

The Problem: The Identity Gap in Retrieval

The fundamental goal of any RAG system is relevance. We want the engine to find the most contextually appropriate information to answer a user’s query… but as the picture shows, in each company, finding the right information is a security risk if the system does not check WHO YOU ARE first.

To build intuition for this, imagine a standard RAG pipeline. As seen in the image, a user performs a vector search for “SALARY RANGES“. The search engine does exactly what it was built to do > it finds the most relevant document which happens to be “BOARD SALARIES“.

In the naive implementation, these chunks are injected into the prompt and sent to the LLM. The LLM then provides a clear, accurate answer based on that sensitive context. The LLM didn’t fail, it followed instructions. The failure happened earlier in the pipeline. This is what I call the Identity Gap, the disconnect between the search engine’s ability to find “relevant” data and its ability to understand if the user is authorized to see it.

The Two Pillars of Secure Retrieval

To bridge this gap, we have to look at the problem through two lenses: WHAT and HOW.

1. WHAT context is injected?

At a high abstract level, documents in your index are just blocks of text and vectors. To make them “identity aware,” they need digital “ID badges” or metadata attached to them.

When a query is made, we aren’t just matching text, we are matching the user’s digital passport (their identity token) against these badges. If the “badges” don’t match, even if the text is a 100% semantic match, the document must remain invisible. This is the essence of Document Level Access.

2. HOW is it synced?

Data and permissions are not static. The challenge is keeping the security “badges” in your Azure AI Search index in sync with the source system.

If the sync fails, you face two risks:

• The False Negative: A user has access to a file, but the AI says “I don’t know” because the index is outdated.
• The Information Leak (The Critical Risk): A user’s access is revoked at the source, but the AI still “remembers” the old permissions, leading to a major RAG Security breach.

Ultimately, the identity of the user must be the primary “key” that unlocks the knowledge base. Without this, your AI is a potential internal data leak waiting to happen.

Security Filters: The Basic Approach

Before we look at the advanced native features, I want to explain the “manual” way to handle permissions. This is the foundation of Document Level Access, and it helps build the intuition you need for more complex patterns.

In this approach (Vector Search with Role‑Based Filtering section), you take the full control of both, the data ingestion pipeline and the search query.

Adding Security Metadata

To make your index identity aware, you must add a new field to your Azure AI Search schema. Usually, this is a Collection(Edm.String) field named something like allowed_groups or roles.

When you index a document, you must push a list of IDs (users or groups or roles) into that field. These IDs usually come from your identity provider, like Microsoft Entra ID.

Manual Query Filtering

When a user asks a question, your application does the heavy lifting:

1. Get Identity: Your backend reads the user’s “digital passport” (the identity token) to find their User ID and the list of groups they belong to or roles they have assigned.
2. Apply the Filter: You modify the search query by adding a filter parameter. This tells the engine to only look at documents where the group IDs or roles match.

private async Task<TResult> SearchAsync(string? text, SearchOptions searchOptions, IEnumerable<string> userRoles)
{
	var delimiter = "|";
	var formattedRoles = string.Join(delimiter, userRoles);
	searchOptions.Filter = $"Roles/any(r: search.in(r, '{formattedRoles}','{delimiter}'))";
	
	// trimmed for brevity
}

private async Task<TResult> SearchAsync(string? text, SearchOptions searchOptions, IEnumerable<string> userRoles)
{
	var delimiter = "|";
	var formattedRoles = string.Join(delimiter, userRoles);
	searchOptions.Filter = $"Roles/any(r: search.in(r, '{formattedRoles}','{delimiter}'))";
	
	// trimmed for brevity
}

This is the most direct form of RAG Security. It gives you total control over the logic, and it works with almost any data source. All the remaining techniques which are described below are based on the same pattern > just applying a proper filter when a query is send to Azure AI Search.

I bet that if you have RAG implemented based on Azure AI Search, this is exactly the approach you use.

PermissionFilter and PermissionFilterOption

The manual approach is a good starting point, but in a real company, we need something more powerful. This is where native Azure AI Search features come in. Instead of building your own filter strings, you use a special property called permissionFilter.

It allows you to use Document Level Access without the headache of managing manual strings!

Putting Security in the Schema

In the native approach, the security logic moves from your application code into the search index itself. You tell the index exactly which fields hold the security information by using the permissionFilter attribute in your index definition.

There are three types of permission filters you can set on your fields:

• userIds: For a field that store specific user identities from your Microsoft Entra tenant > Collection(Edm.String).
• groupIds: For a field that store IDs of Microsoft Entra groups > Collection(Edm.String).
• rbacScope: For a field that store RBAC scope (for example a scope of a storage account container) > Edm.String.

A few important things to remember:

• You can define just a single field with a specific permissionFilter, which is understandable.
• If there is a match found for any of these permissionFilter fields, then such a document can be returned in the result.
• You can insert up to 1000 values into the userIds and groupIds fields.
• A single index can store just 5 unique values for the rbacScope field! That is a very small number, especially when you have structured data in your storage account in a way that you create many containers instead of having just a single one or a few.
• filterable must be set to true because, as I mentioned before, behind the scenes it just builds an ordinary filter expression and adds it to the filter expression you define.
• It is recommended to set retrievable as false to ensure that these permissionFilter values cannot be returned when specified in the “select” clause, but for troubleshooting, it might be very helpful, so just decide what is the best option for you.
• You should set searchable and facetable as false.

The Master Switch: permissionFilterOption

You also need to set the permissionFilterOption at the index level. Think of this as the “On” switch for your security guard. It tells Azure AI Search to strictly check permissions for every single search query to ensure RAG Security.

Passing User’s Context to Search Engine

Once the “security guard” is active in your index, you need a way to show the user’s identity when they make a search. In a secure RAG pipeline, we can do it using a specific request header: x-ms-query-source-authorization.

By passing the user’s Microsoft Entra ID token through this header, you connect the application directly to the search engine’s security layer. It is important to know that this header is not just for one type of request. It can be passed to:

Search Operation: When you call the standard Azure AI Search API to find the most relevant documents. Below is an example of how to pass it in .NET using Azure.Search Documents NuGet.

Retrieve Operation: When you use “Knowledge Bases” which I described here. In this case, the system uses a special “retrieve” endpoint to get context for an agent or a chat model.

Using this header in both places is a game changer for RAG Security:

• Automatic Resolution: You don’t have to manually find which groups a user belongs to in your code. The search engine resolves the identity from the token itself.
• Security at the Source: Because the filtering happens inside the engine, unauthorized data is blocked before it even leaves the service.
• Clean Queries: Search requests stay simple because we don’t need to build long strings with many different IDs.

When the search engine receives the x-ms-query-source-authorization header, it automatically checks the fields you marked with the permissionFilter attribute. This ensures that Document Level Access is enforced every single time a user asks a question.

Troubleshooting: Ignoring Security Filters

You might have noticed one additional field on the screenshot with the search operation, which is “enableElevatedRead”. When you set it, behind the scenes it sends the x-ms-enable-elevated-read: true header, which is interpreted by Azure AI Search as: “ignore applying any security filters even if permissionFilterOption is enabled in your index settings.

This is extremely useful when you want to troubleshoot various issues. In order to enable that elevated read, you must use the Search Index Data Contributor RBAC role instead of the Search Index Data Reader role, which you likely use when your service is responsible just for read operations. The data plane permission (read more about control plane vs data plane permissions here) which allows that action is Microsoft.Search/searchServices/indexes/contentSecurity/elevatedOperations/read .

Currently that “ElevatedRead” functionality is available only for the Search API and therefore you cannot see it on the screenshot with the Retrieve API.

Push approach with userIds and groupIds

When you work with Azure AI Search, one of the most important decisions you have to make is whether to use a push approach, which is more challenging, but gives you the full control or rely on the “automatic” approach a.k.a. the pull approach which greatly simplifies a lot of things (it uses data sources, indexers and skillsets behind the scenes).

In case you use a push approach you will be likely relying on the permissionFilters > userIds and groupIds. This is how you can push that data into an index, you will be likely using some SDK (like Azure.Search.Documents for .NET) to do that but it’s always good to know how such a request which SDK generates for you looks like:

I think that Documents with Id = 1 and Id = 2 do not require additional explanation but Document with Id 3 uses “all” and “none” which is a special syntax.

• “all” means that any user can access such document (it is sufficient to specify “all” for just one of the permissionFilter fields)
• “none” (or an empty array) means no user or group is granted access through that specific field.

I created “Contoso – CEO” user which is a member of “Contoso – Executive Board” Entra group (ID: 719bf3ee-ec99-44b1-adb6-2f53874fa90a).

Now, I will obtain a JWT for that user such AZ CLI command. az account get-access-token --resource https://search.azure.com --query accessToken --output tsv

Now, if I use that token and assign it to x-ms-query-source-authorization such documents are returned:

As expected, only document with ID 2 and ID 3 were returned which proves the security filtering works as expected.

Now, to return the document with ID 1 and ID 3 I will use another security principal.

And let’s verify that x-ms-enable-elevated-read header which I described before works as expected too (please note that when you use that header you cannot send x-ms-query-source-authorization header at the same time + that action requires Search Index Data Contributor RBAC role).

All 3 documents were returned! You should remember about that option especially when you troubleshoot some issues.

Pull Approach with rbacScope

Now let me show you how to populate rbacScope field automatically using the pull approach. The first change we must apply is modyfing the definition of a data source by adding indexerPermissionOptions property.

It is not sufficient though because now we have to map the metadata field metadata_rbac_scope which stores that RBAC scope to the RbacScope field which is defined in the index.

I will do it using indexProjections in the skillset which is used by the indexer I defined.

Now I can run the indexer and it will populate that RbacScope field automatically. Below is an example of how it looks like (please note I set retrievable to true in order to be able to return that value during the search operation).

Now the search engine is able to figure out whether given user identity is allowed to access such document or not. I could access it because my Entra security pricipal has Storage Blob Data Reader RBAC role assigned.

Syncing permissions changes

Data and permissions are not static. The real challenge is keeping the security “badges” in your Azure AI Search index in sync with the source system.

Push Approach

When you use a Push approach, you have to maintain userIds and groupIds manually. This is challenging because every time a file’s specific permissions change, you must update the document in the search index. In a large organization, this means a lot of manual code and constant updates to keep the metadata accurate.

Pull Approach: The Stability of rbacScope

The Pull approach (specifically with Azure Storage) is much more stable because of how the rbacScope field works. Instead of storing a list of every user who can see a file, the indexer simply syncs the scope where the document originates.

This is a game changer for stability:

• User changes are transparent: If a user’s permission to access a container changes in Azure RBAC, you don’t need to update the search index at all. The rbacScope in the index stays the same, and the search engine simply checks if the user currently has access to that scope during the query.
• Moving files: If you move a file to a different container with different permissions, the indexer treats it as a brand new file. Thanks to the delete detection policy, the old entry with the old scope is removed, and a new one with the correct rbacScope is created automatically.

Beyond Azure Storage: Other Secure Data Sources

While I focused here on the stability of Azure Storage and rbacScope, the pull approach also handles other complex sources like SharePoint ACLs, ADLS Gen2 POSIX permissions, or even Microsoft Purview sensitivity labels.

Here is a quick look at these advanced options:

• SharePoint Online ACLs: The SharePoint indexer is built to handle the complex, hierarchical permissions used in Microsoft 365. It automatically maps site, library, and folder level permissions into the search index, so your RAG system knows exactly who can see which document.
• ADLS Gen2 POSIX Permissions: This is essential for big data scenarios where you need very detailed control over files and directories. The search indexer can sync these specific POSIX style rules to ensure that even deep folder structures remain secure in your RAG pipeline.
• Microsoft Purview Sensitivity Labels: This goes a step further by protecting data based on its classification. If a file is marked as “Confidential” or “Secret,” the search engine can use that label to make sure only the right people can find it in a chat interface.

I skipped the deep technical details of these topics on purpose because this post is meant to be a gentle introduction to RAG Security. However, it is good to know these options are available for more advanced enterprise projects.

Summary

The goal of this blog post was to demonstrate the built-in capabilities Azure AI Search offers for RAG security. Moving beyond simple relevance to an identity-aware architecture is what separates a demo from a production-ready system.

If your organization already stores data within the Microsoft and Azure ecosystem, leveraging this native mechanism like Document Level Access is the most efficient way to ensure your AI doesn’t become a security liability.

I hope this guide helps you build RAG systems that are not just smart, but truly enterprise-secure.

Thanks for reading and see you in the next post.