Permission-Aware RAG: Access Control Belongs in the Retriever, Not the Prompt

Here's the scenario I walk every client through before we index a single folder.

The internal assistant demo goes well. Two weeks after launch, someone in sales asks it "what's the salary band for a senior engineer?" It answers, confidently, with a citation to a spreadsheet in an HR folder that person has never been able to open. Nobody hacked anything. The retriever did its job: it found the most relevant chunk. The model did its job: it answered from the context it was given. The only thing that failed was the thing nobody built.

Most RAG systems I'm asked to review handle permissions with one sentence in the system prompt: only use documents the user has access to. The model has no idea what the user has access to. Even if it did, a sentence is not enforcement.

This post is about the rule I hold every enterprise RAG build to: a chunk the user couldn't open in the source system never enters the context window. Not "the model shouldn't mention it." Never enters. That rule lives in the retriever, and it's a lot more work than a filter clause.

This isn't a drop-in package. The sketches below show the shape of what I build. Your source systems, identity provider, and vector store will be different, and the connectors are where most of the real effort goes.

Why "the prompt says don't" is not access control

Once a chunk is in the context window, it has been disclosed. To the model, to whatever logs the request, and potentially to the user. Prompt injection, an innocent "summarize everything you found," a citation link, or a paraphrase can all surface it. Asking the model to keep a secret it's holding is the wrong layer.

The deeper problem is that ingestion flattens permissions. A typical pipeline crawls Drive, SharePoint, Confluence, or a document management system with a service account that can read everything, chunks it, embeds it, and drops it into one index. The source system knew who could read each file. The index doesn't. Every user now queries the union of everyone's documents.

This isn't a niche concern. The OWASP Top 10 for LLM Applications 2025 lists it under LLM08, Vector and Embedding Weaknesses: unauthorized access and cross-context leaks when different groups share a vector store, with permission-aware stores and strict partitioning as the mitigation. AWS's security team makes a sharper point in Authorizing access to data with RAG implementations: metadata filtering alone lags behind permission changes because the index only syncs periodically, and it gets hard when principals belong to many groups. Their recommendation is to check authorization against the data source that actually owns the rules.

I agree with both, and I build for both problems.

The architecture

Connector and ACL sync. The ingest side pulls permissions along with content. Every chunk is stamped with the ACL of its source document at index time. Permission changes get their own fast lane, separate from content re-embedding: re-stamping metadata on existing chunks is cheap, re-embedding is not, so there's no reason to wait for the next full crawl to revoke access. Where a source system emits permission-change events, I subscribe. Where it doesn't, I poll ACLs more often than content.

One namespace per tenant. Tenant isolation is a partition, not a WHERE clause. A filter is one refactor away from being dropped. A separate namespace or index per tenant means a bug in filter construction leaks within a tenant at worst, never across tenants.

Principal resolver. The verified identity comes from your auth layer, never from the request body and never from the model. The resolver expands it into a principal set: user id, direct groups, nested groups, roles. It's cached with a short TTL and invalidated on group-membership events, because "removed from the finance group an hour ago" should mean an hour ago, not next week.

Retriever with ACL pre-filter. The permission filter runs inside the vector query, so top-k is computed over documents the user is allowed to see. More on why that ordering matters below.

Authorization check at source. For corpora where a leak is an incident — HR, legal, finance, client matters — candidates go through a batch check against the system of record or a policy engine before anything reaches the model. That might be the source's own permission API, or a relationship-based authorization service like OpenFGA or SpiceDB if you already model permissions there. Deny by default. A timeout is a deny.

Context builder. Only admitted chunks go in. Citations link to the source document, which the user can open, which doubles as a sanity check: if they click a citation and get "access denied," you have a bug, and you want users to be able to tell you.

Audit log. Who asked, a hash of their principal set, the query, candidate chunk ids, admitted ids, denied ids with the reason, and the ACL version each decision used. When someone asks "could this person have seen that document through the assistant?", this is how you answer with evidence instead of a guess.

The chunk metadata I require

Every chunk in a permissioned index carries:

• tenant_id, doc_id, chunk_id, source_system, source_url
• acl_allow: principal ids allowed to read (users and groups, in one namespaced format like user:123, group:finance)
• acl_deny: explicit denies, if the source system supports them — some do, and ignoring them is a leak
• classification: public, internal, confidential, restricted — used for policy, not as a substitute for ACLs
• acl_version and acl_synced_at: so you know how stale a decision could have been
• inherits_from: the folder or space the ACL came from, so a folder-level change can be fanned out to every chunk under it

The rule for anything derived: derived artifacts inherit the most restrictive ACL of their inputs. A summary built from three documents is readable only by people who can read all three. This is the leak people miss. A "document summaries" index built across folders, an entity table extracted from contracts, a FAQ generated from support tickets — each one is a new copy of the data with the permissions stripped off unless you carry them forward deliberately.

Pre-filter, post-filter, or both

There are two places to apply permissions in retrieval, and each one alone has a failure mode.

Post-filter only. Retrieve top-k from everything, then drop what the user can't see. If a user can read a small slice of a large corpus, the top results may contain nothing they're allowed to see, so they get an empty or useless answer and assume the system is bad. Worse, restricted chunks were already loaded into your application, and possibly into your logs, before you dropped them.

Pre-filter only. Put the ACL filter in the vector query. Recall is good and nothing restricted leaves the store. But the decision is only as fresh as your last ACL sync. Someone whose access was revoked this morning can still retrieve those chunks until the sync catches up. For a wiki that's often acceptable. For an HR or legal corpus it usually isn't.

Both. Pre-filter for recall and cheap correctness, then check the candidates against the source of truth for freshness. Over-fetch on the first step so post-check denials don't starve the context. This is my default for anything sensitive.

# Sketch, not a drop-in. vector, authz, and audit are your clients.
from dataclasses import dataclass


@dataclass(frozen=True)
class Principal:
    tenant_id: str
    user_id: str
    groups: frozenset[str]


@dataclass
class Candidate:
    chunk_id: str
    doc_id: str
    text: str
    score: float
    acl_version: int


class AuthzUnavailable(Exception):
    pass


async def retrieve_for_user(query: str, principal: Principal, k: int = 8) -> list[Candidate]:
    ids = [f"user:{principal.user_id}", *(f"group:{g}" for g in principal.groups)]

    candidates = await vector.search(
        namespace=principal.tenant_id,   # tenant isolation is the partition, not a filter
        query=query,
        top_k=k * 3,                     # over-fetch so post-check denials don't starve context
        filter={"acl_allow": {"$in": ids}, "acl_deny": {"$nin": ids}},
    )

    doc_ids = sorted({c.doc_id for c in candidates})
    try:
        decisions = await authz.batch_check(
            subject=f"user:{principal.user_id}",
            relation="viewer",
            objects=[f"doc:{d}" for d in doc_ids],
            timeout_s=0.3,
        )
    except (TimeoutError, AuthzUnavailable):
        decisions = {}                   # fail closed: no decision means no access

    admitted = [c for c in candidates if decisions.get(f"doc:{c.doc_id}") is True][:k]

    await audit.write(
        user_id=principal.user_id,
        principals_hash=hash(frozenset(ids)),
        query=query,
        candidate_ids=[c.chunk_id for c in candidates],
        admitted_ids=[c.chunk_id for c in admitted],
        acl_versions={c.chunk_id: c.acl_version for c in candidates},
    )
    return admitted

That function is the easy 10%. The real work is everything feeding it: connectors that read each source's permission model correctly (inheritance, nested groups, "anyone with the link" sharing, external guests, explicit denies), group expansion for users in a very large number of groups without building a filter the vector store chokes on, permission-change events that actually fan out to every affected chunk, and a principal cache that's invalidated when it should be. Most of the bugs I find live in connectors, not in the retrieval function.

One detail in that sketch matters more than it looks: when the authorization service is down, the answer is "nothing admitted," not "skip the check." A RAG system that fails open during an outage is a RAG system with no access control during an outage.

Leaks that don't come through the retriever

Getting retrieval right closes the obvious hole. These are the ones I find next.

Semantic answer caches keyed only on the query. User A, in HR, asks a question and the answer is cached. User B asks something similar and gets A's answer, built from A's documents. Cache keys need the principal set, or the cache has to be scoped per user. Often the right answer is just not caching permissioned answers.

Shared channels. A RAG bot in a team chat channel answers one person, but everyone in the channel reads the reply. The effective permission there isn't the asker's. It's the intersection of everyone who can see the answer. Most bots get this wrong.

Conversation memory. If a thread's earlier turns pulled restricted chunks into memory, and a different user continues that thread or a memory summary is reused, the restricted content travels with it. Memory inherits ACLs like any other derived artifact.

Traces and logs. If your traces store retrieved chunk text — and for replayable runs they should — the trace store now holds restricted content and needs the same access controls and redaction, not wider ones.

Agent tools that fetch by id. An agent that can call get_document(doc_id) bypasses retrieval entirely. Every tool that returns content must run the same authorization check. That belongs in the tool contract, not in the tool's prompt description.

Existence side channels. "I found three relevant documents, but you don't have access to them" tells the user those documents exist. Sometimes that's fine. For a confidential matter or a pending reorg, the existence is the secret. Don't reveal denied titles or counts unless the business has decided that's acceptable.

Testing it: leak tests belong in the eval gate

Permissions are a property you can test, so I test them the way I test answer quality.

Canary documents. Plant a document with a unique, meaningless string in a restricted location. Run queries designed to retrieve it as test users who should and shouldn't have access. The assertion is simple: the canary string never appears in retrieved context, the answer, or the citations for an unauthorized user.

Revocation tests. Grant a test user access, confirm retrieval works, revoke access in the source system, and query again on a schedule. This measures your real revocation lag instead of the one in the design doc.

Per-principal retrieval evals. Retrieval quality changes with permissions. A question that gets a great answer for an admin may get a thin answer for a contractor, and both can be correct. When I evaluate RAG properly, the eval set includes the persona asking, not just the question.

All of it runs in the golden-set eval gate. A build that leaks a canary doesn't ship, regardless of how good its answers are.

Choosing the enforcement pattern

Not every corpus needs the full stack. I start with one question: who can read what?

Everyone can read everything. A public docs site, an internal handbook every employee can open. You still need tenant isolation if you serve more than one customer, but per-document ACLs would be complexity with no payoff. Be honest about whether this is really true, though. "Everyone can read the wiki" usually turns out to mean "everyone except the restricted spaces nobody remembered."

Stable group ACLs, and a sync lag is acceptable. Department-level access that changes rarely. A metadata pre-filter on ACL groups plus a scheduled ACL sync is enough, as long as the sync interval is a number someone on the client side has explicitly signed off on.

ACLs change fast, or a leak is an incident. HR, legal, finance, client-confidential material, anything regulated. Pre-filter for recall, then a live authorization check at the source before the model sees anything. Fail closed. Audit everything.

If you can't tell which branch you're on, you're on the third one.

Failure modes I keep seeing

Permissions in the system prompt. The model can't enforce what it can't see, and it shouldn't be holding the restricted text in the first place.

Ingesting with an admin service account and never reading ACLs. The index is now the most over-privileged system in the company.

Tenant isolation as a filter. One missing clause and customer A retrieves customer B's contracts.

Post-filtering a plain top-k. Users with narrow access get empty answers, and restricted chunks pass through your app on the way to being dropped.

ACL sync on the content crawl schedule. Revocations wait for re-embedding. They shouldn't wait for anything.

Derived indexes without inherited ACLs. Summaries, extracted entities, generated FAQs, caches, and memory become copies with the permissions stripped off.

Failing open when the authz service times out. Your access control has an off switch, and the outage flips it.

No leak tests. You believe the system is permission-aware because the design says so. A canary document is how you find out.

Close

A RAG system over a permissioned corpus is a new way to read every document in that corpus. If it doesn't enforce the same permissions as the systems it reads from, it isn't a search assistant. It's a privilege escalation with a chat interface.

Sync ACLs with content and revoke them on their own fast path. Partition by tenant. Resolve the user's principals from verified identity. Pre-filter inside the vector query, then check candidates against the source of truth for anything sensitive. Fail closed. Carry ACLs onto every derived artifact. Log every allow and deny. Put canary documents in the eval gate so a leak blocks the release.

I build retrieval systems for legal, finance, and operations teams where "the assistant showed someone a file they couldn't open" is not an acceptable bug. If you're about to point RAG at a permissioned corpus, or you already did and aren't sure what it can see, hire me.