What is AI access control?

Summary slides

Why access control for AI systems can’t rely on the model to enforce it

Conventional access control has always worked by placing enforcement at a system boundary a request simply can’t bypass under any circumstance, a database query filtered by the requesting user’s permissions, an API endpoint that rejects a call lacking the right credential, and the entire model rests on that enforcement point being something the requester cannot get around no matter what it asks for. An AI system introduces a tempting but unreliable alternative: simply instructing the model to respect a stated boundary, telling it not to reveal certain information or not to take certain actions on behalf of a user who lacks the relevant permission, and trusting the model’s judgment to honor that instruction consistently.

This alternative is unreliable for the same underlying reason that makes prompt injection a risk throughout AI system design more broadly: a model’s behavior is shaped by everything currently in its context, and an instruction telling it to respect a boundary sits in that same context alongside user input, retrieved content, and its prior reasoning, all of which can, deliberately or by accident, pull the model toward violating exactly the boundary it was told to respect. A model that’s told not to reveal a piece of information can still be maneuvered into revealing it through a sufficiently creative rephrasing of the request, an indirect question that doesn’t superficially resemble the original forbidden one, or content encountered elsewhere in its context that effectively instructs it to disregard the original restriction, none of which a purely instructional boundary has any mechanism to reliably resist.

Access control for an AI system, then, has to be enforced at the same layer conventional access control has always been enforced at, the point where data is retrieved or an action is executed, filtering what the model can even receive or trigger based on the actual permissions of the actual requester, rather than trusting the model’s downstream judgment to correctly apply a restriction it was merely told about. This doesn’t mean instructing a model about boundaries is worthless, since a model aware of a restriction can behave more sensibly around it in ordinary, non-adversarial cases, but it means that instruction can never be the only thing standing between a request and a resource it shouldn’t be able to reach.

How to enforce access control before retrieval rather than after generation

Arguably the single most consequential design decision in building access control for a retrieval-based AI system is deciding precisely at which stage a permission gets checked, and enforcing it before retrieval, filtering what a user is even allowed to search against before any results are returned, is considerably more robust than enforcing it after generation, checking whether a model’s already-produced output happens to contain something the requester shouldn’t have seen.

Filtering only after generation requires building a check sophisticated enough to catch every single way a restricted fact might surface somewhere in a model’s output, a direct statement, a paraphrase, an inference drawn from restricted material combined with something else entirely permissible, and any gap in that check lets restricted content through regardless of how carefully the rest of the system was designed, since the restricted material was already available to the model and already shaping its response by the time the check runs. Filtering directly before retrieval instead avoids this entire problem structurally, from the ground up, since a model that never receives restricted material as part of its context has no way to leak it, whether directly, through paraphrase, or through inference, because the information simply was never present for the model’s reasoning to draw on in the first place.

This means the access-control check belongs inside the retrieval query itself, filtering the underlying corpus down to what the requesting user is permitted to see before that filtered set is ever ranked, retrieved, or handed to the model, rather than retrieving broadly across the entire corpus and hoping a downstream check, whether a guardrail on the model’s output or an instruction embedded in its prompt, reliably catches every case where something restricted made it into what the model was given to work with.

How access control needs to account for a model’s inferential capability

A subtle, easy-to-overlook risk in AI access control specifically, one that has no direct, clean analogue anywhere in conventional access control built for a purely deterministic system, is that a model can sometimes infer restricted information from a combination of pieces that are each, individually, entirely permissible for a user to see, producing something the access-control system never explicitly granted access to but that the model’s reasoning was nonetheless able to reconstruct from material it was legitimately given access to.

A user permitted to see two entirely separate, individually innocuous pieces of information might, through a model capable of meaningfully connecting them together, effectively gain access to a third, more sensitive conclusion that neither piece alone would have revealed and that the access-control system never explicitly considered when granting access to the two individual pieces on their own separate terms. This risk is difficult to eliminate completely, since doing so completely would require anticipating in advance every single possible combination of permitted information a sufficiently capable model might eventually, successfully manage to connect together, which is a combinatorially much larger space than simply enumerating which individual pieces of information a user is permitted to see, but it can meaningfully be mitigated in practice by deliberately treating certain categories of information as specifically sensitive only in combination with each other, restricting access to any combination already known to be revealing, even when the individual pieces that make up that particular combination are each, considered separately, entirely unrestricted on their own terms.

Building awareness of this particular risk directly into how a team designs its access-control categories from the start, rather than assuming that restricting individually sensitive pieces of information is automatically sufficient protection against a model’s inferential capability, is what closes this gap, and it’s a gap that deserves particular, focused attention specifically in systems handling information where a seemingly innocuous, harmless-looking combination could plausibly reconstruct something sensitive that no individual piece, considered entirely on its own, would ever have revealed by itself.

How to design access control for tool use and agentic action

Access control for an AI system extends considerably beyond merely what data a model can read into what actions it can take on someone’s behalf, and this extension introduces its distinct design considerations, since a permission to merely view information and a permission to act on a user’s behalf are two meaningfully, substantively different kinds of authority that deserve to be modeled, and firmly enforced, as separate, distinct concerns rather than conflated into a single, undifferentiated notion of access.

Tool-level permissions, precisely determining which tools an agent is even allowed to invoke at all regardless of whatever its internal reasoning happens to conclude it should do, provide the first, coarsest layer of this control, and they deserve to be scoped as narrowly as the actual task truly requires, rather than granted broadly and loosely on the general, comfortable assumption that a capable model will simply choose, entirely on its own, not to invoke a tool it doesn’t need for the task at hand. Parameter-level permissions, restricting not merely which tool an agent can invoke but exactly what values or ranges it’s allowed to supply to that same tool’s parameters, add a further, more granular layer, since a tool that’s broadly appropriate for an agent to use at all might still need deliberate constraints on the narrow scope of exactly what it’s permitted to do within any single invocation of it, a data-access tool deliberately scoped to one account or to a narrow record range rather than simply left wide open to querying anything at all the underlying system happens, technically, to expose.

Delegation-aware permissions, specifically and quite deliberately accounting for the particular case where an agent is acting on behalf of one user rather than under some standing, broader system-level authority of its own, need the agent’s actual permission set to be strictly bounded by whatever that one user is authorized to do in the first place, rather than by whatever considerably broader technical capability the underlying system or the agent’s tool integrations simply happen to technically, incidentally support, since an agent acting on a user’s behalf should never, under any circumstance, be able to exceed what that particular user could have done directly and entirely legitimately entirely on their own.

How to handle access control in multi-tenant systems specifically

Systems that serve multiple separate organizations or separate customers through shared underlying infrastructure introduce a whole category of access-control risk considerably more severe than an ordinary, garden-variety permission mismatch occurring within a single tenant alone, since a failure here isn’t merely a user seeing something they weren’t quite supposed to within their organization, it’s a breach of the isolation boundary between entirely separate customers, each of whom reasonably expects their data to remain completely invisible to every other tenant sharing the same underlying system.

Tenant isolation deserves to be enforced as a hard, entirely structural boundary directly at the storage and retrieval layer itself, without exception, with tenant identity functioning as a truly mandatory, enforced filter applied to every single query made against any shared infrastructure, rather than as a soft, informal, application-level convention that depends entirely on every part of the system consistently, reliably remembering to apply it correctly across every single access path it exposes. This particular distinction matters considerably in practice because a soft, informal convention has as many separate opportunities to fail as there are distinct code paths that touch the shared, underlying data at all, a new feature added later, a new external integration, a new internal tool built by someone entirely unfamiliar with the original, established isolation convention from the start, each one representing yet another fresh opportunity for an isolation boundary that depends entirely on ongoing manual discipline to eventually, somewhere along the way, be forgotten entirely or simply implemented incorrectly the first time.

And rigorously testing tenant isolation deliberately and adversarially, specifically, actively attempting to access one particular tenant’s data using another tenant’s credentials across every single code path a system exposes to the outside world, deserves the same seriousness to testing any other critical security boundary, since the consequences of a tenant isolation failure occurring in an AI system carry precisely the exact same severity they would carry in any other, more conventional multi-tenant system entirely without any AI component involved, an often quite seriously reputationally damaging, breach of a promise every single tenant reasonably, fairly assumed was being kept faithfully on their own behalf the entire time.

How access control interacts with memory and personalization

Systems that build persistent memory about individual users, already discussed at length as its own distinct design concern in AI memory system design more broadly elsewhere, introduce their own access-control question: whether a piece of remembered information is scoped correctly to the user or context it should be visible to, and whether that scoping is enforced as a hard boundary rather than an implicit assumption that memory retrieval will simply behave itself and stay within the bounds it was originally intended for.

Per-user memory isolation treated as an access-control boundary entirely in its own right rather than merely an organizational convenience for keeping different users’ data tidily, superficially separated, needs the same kind of hard, storage-layer enforcement discussed for tenant isolation more broadly, since a memory system that leaks one user’s remembered information into a response served to a different user is an access-control failure with consequences, not merely an odd, cosmetic quality glitch that happens to look strange from the outside.

Shared or organizational memory, made deliberately visible across multiple separate, distinct users within the same team or the same account deserves its explicit, dedicated access-control design specifically because it represents a deliberate, intentional relaxation of the otherwise strict per-user boundary, and that relaxation needs to be scoped precisely to the group of users meant to share that particular memory, rather than defaulting to broader visibility simply because the per-user boundary was already being intentionally loosened for this case and it seemed simpler to loosen it further than to define the shared scope with precision.

How to audit and monitor AI access control once it’s built

Access control that’s carefully implemented but never rigorously verified against live behavior can quite easily look entirely correct on paper while still failing in practice in ways that only become visible once usage patterns and audit data are examined closely, which means building ongoing audit capability deserves the same investment as the access-control mechanisms themselves.

Access logging, built specifically to capture exactly what an AI system retrieved and what actions it took, tied explicitly and directly to the user or agent that triggered each individual instance, gives a team the concrete, retrospective evidence needed to verify that access control is behaving as intended, rather than merely as designed on paper, and this logging matters considerably more for AI systems than for conventional software specifically because a model’s behavior can vary across calls in ways that a deterministic system’s behavior never does, which means a single successful test of an access-control rule provides considerably weaker ongoing assurance than the equivalent test would for a conventional, fully deterministic system.

Anomaly detection applied specifically and deliberately to access patterns themselves, flagging a user or agent whose actual retrieval or action pattern deviates meaningfully from what’s typically expected for their role or their established historical pattern of behavior, catches the kind of access-control failure that a purely rule-based permission check, however well designed, structurally can’t catch on its own, since a permission check can only verify that a request was technically authorized under the rules as written, it can’t independently notice that an authorized user’s behavior has started looking unusual in a way that might indicate their account has been compromised, or that a permission granted for one legitimate purpose is now being systematically exploited for a different, considerably less legitimate one.

How access control needs to evolve as an AI system’s capabilities grow

An access-control design that’s entirely appropriate for a system’s actual capabilities at the moment of launch doesn’t necessarily remain equally appropriate as that same system’s capabilities continue to steadily and considerably expand over the full course of its ongoing operational life, since a system that gains a new tool, a new data source, or a new kind of agentic action at some point over time inherits a correspondingly new access-control surface that the system’s original design was never built to anticipate or account for at all.

Seriously treating access control as something that has to be explicitly, deliberately, and consistently extended alongside every single meaningful capability expansion a system undergoes, rather than assumed to automatically and correctly cover whatever new capability happens to get added on top of an existing system, is what prevents a team from discovering, only after the fact, that a new tool or a new data source was quietly integrated without the corresponding necessary access-control extension that new capability required from the very start. This is exactly the kind of discipline that benefits from an explicit checklist or an explicit review step specifically triggered by any new capability addition, treating “what access-control implications does this new capability introduce” as a standard, mandatory question asked deliberately every time, rather than a consideration that only gets raised reactively, and often only after a damaging incident has already forced the question that a proactive review would have caught well in advance.

How access control connects to the broader guardrail and human-in-the-loop layers

Access control doesn’t operate in complete isolation from the other protective mechanisms an AI system relies on more broadly, and understanding how it connects to guardrails and to human-in-the-loop review specifically clarifies which risks each layer is responsible for catching, since treating all three as interchangeable ways of achieving the same general notion of “safety” leads to gaps where a team assumes one layer is handling something that was never within its scope at all.

Guardrails, as already discussed at some length elsewhere in AI system design more broadly, generally address content and behavior that’s problematic regardless of who exactly happens to be asking for it, a hallucinated fact, a policy-violating response, an attempted prompt injection, while access control specifically addresses whether a particular requester is entitled to a particular piece of information or a particular action in the first place, which means a request can cleanly pass every guardrail check while still representing an access-control violation, and a request can just as easily be fully authorized under access control while still triggering a guardrail for entirely unrelated reasons. Recognizing this as two distinct concerns, rather than treating access control as simply one more guardrail among many, is what ensures both get the dedicated design attention they each require, since a permission check folded loosely into a general guardrail layer tends to receive considerably less rigor than a check that’s explicitly understood to be enforcing a hard, non-negotiable security boundary rather than merely catching a quality or policy issue.

Human-in-the-loop review, similarly and quite naturally, connects back to access control in a related but nonetheless distinct way of its own, since a review point is generally the right response to uncertainty about whether an action is wise given everything the system currently knows, while access control is meant to establish a hard boundary that shouldn’t be crossed regardless of how confident or well-reasoned the system’s judgment happens to be in a moment. An access-control violation that gets caught and routed to human review, rather than blocked outright, treats a hard security boundary as though it were merely a soft judgment call, which is exactly the wrong response to this category of risk, since the entire point of access control is to establish limits that don’t bend based on how convincing a particular request’s justification happens to be.

How to design access control for agentic reasoning that spans multiple data sources

Agentic systems that draw on several distinct data sources or several distinct tools within a single, larger task introduce a particular access-control challenge beyond what any single source’s permission model was ever designed to handle in isolation, since an agent synthesizing information across sources needs its overall access to be bounded by the intersection of what it’s permitted to use from each source, not simply by whichever single source happens to have the most permissive access policy of the group.

A naive implementation that simply checks each individual source’s access rules independently, without any meaningful awareness of exactly how the retrieved material from several different sources will end up combined further downstream, can inadvertently let a synthesis draw on restricted material from one source that gets blended into an output that superficially reads as though it were entirely drawn from a different, more permissive source, effectively laundering restricted information through the synthesis process itself. Designing against this risk means carefully tracking provenance explicitly and consistently through the entire pipeline from start to finish, from each individual piece of retrieved material through however it eventually contributes to the final synthesized output, so a downstream access-control check can trace a claim in the final response back to the source, and the permission level, it originally came from, rather than losing that lineage entirely once several sources have already been blended together into what looks like a single, undifferentiated response.

This same provenance tracking also directly supports the audit and monitoring practices already discussed at length elsewhere in this article, since a system that can trace an output claim back to its originating source and the access level that source required gives an auditor something concrete to verify, whether that particular claim was drawn from material the requesting user was authorized to see, rather than asking the auditor to somehow reconstruct that lineage after the fact from an already-synthesized response that gives no explicit indication of where any piece of its content originated.

Common mistakes teams make around AI access control

A first mistake, and the foundational one nearly every other mistake on this list traces back to in some form, is relying on simply instructing the model to respect a stated access boundary rather than enforcing that boundary at the point where data is retrieved or an action is concretely executed, leaving the entire boundary vulnerable to any sufficiently creative rephrasing of the original request or any content encountered elsewhere in the model’s context.

A second mistake is filtering for access-control violations only after generation, checking whether a model’s already-produced output happens to leak something restricted, rather than filtering directly before retrieval so the model never receives that restricted material as part of its context in the first place, and therefore has no way whatsoever to leak it regardless of how it’s later asked.

A third mistake is restricting only individually sensitive pieces of information while entirely overlooking that a model can quite plausibly infer a sensitive conclusion from a combination of individually permissible pieces that were never explicitly considered together at the moment access to each piece was originally, separately granted.

A fourth mistake is granting broad tool-level permissions on the comfortable assumption that a capable model will simply choose, entirely on its own, not to invoke a tool it doesn’t need for the task at hand, rather than scoping tool access as narrowly as the task requires from the very start.

A fifth mistake is granting an agent that’s acting on a user’s behalf a considerably broader permission set than that user is authorized for in their own right, rather than firmly bounding delegation-aware permissions strictly by whatever the delegating user could have done directly and legitimately entirely on their own without the agent’s involvement at all.

A sixth mistake is enforcing tenant isolation as merely a soft, application-level convention rather than as a hard, structural filter applied directly at the storage and retrieval layer itself, leaving that isolation entirely dependent on every single new code path consistently, reliably remembering to apply it correctly on its own.

A seventh mistake is never testing tenant isolation adversarially at all, missing the particular code paths where an isolation boundary that looks entirely correct in the ordinary, common case fails badly under a deliberate attempt to access another tenant’s data.

An eighth mistake is treating per-user memory isolation as merely an organizational convenience rather than as an access-control boundary deserving exactly the same hard, storage-layer enforcement to any other critical permission check anywhere else in the system.

A ninth mistake is defaulting shared or organizational memory to considerably broader visibility than intended, simply because the per-user boundary was already being deliberately relaxed for that one case and it seemed simpler, in the moment, not to scope that relaxation with any deliberate precision at all.

A tenth mistake is never building access logging tied specifically to the exact user or agent behind each individual retrieval and each individual action, leaving a team with no concrete, retrospective evidence to verify that access control is behaving as intended in practice, rather than merely as it was originally designed on paper.

An eleventh mistake is relying entirely, exclusively on rule-based permission checks with no anomaly detection layered on top of them at all, missing the kind of access-control failure where an authorized account’s behavior has quietly started looking unusual in a way that a purely rule-based check structurally, fundamentally can’t ever catch on its own.

A twelfth mistake is treating access control as a one-time design exercise completed at launch, never explicitly extending it alongside new tools, new data sources, or new agentic capabilities that each introduce their new access-control surface the original design was never built to anticipate.

A thirteenth mistake is folding access control loosely into a general, broad guardrail layer, treating a hard, non-negotiable security boundary as though it were merely one more content or behavior check among many similar ones, which tends in practice to leave that permission check receiving considerably less dedicated rigor than a boundary explicitly, deliberately understood to be non-negotiable from the start.

A fourteenth mistake is routing a detected access-control violation to human review rather than blocking it outright and immediately, treating a hard, non-negotiable boundary that shouldn’t bend regardless of context as though it were merely a soft judgment call deserving the same graduated response that’s appropriate specifically for well-founded uncertainty.

A fifteenth and truly final mistake is checking each individual data source’s access rules entirely independently within a multi-source agentic system, with no tracking of how the retrieved material from each source eventually combines further downstream, letting restricted information originating from one source effectively get laundered into a final output that superficially reads as though it came entirely from some other, considerably more permissive source instead.

What connects all fifteen of these mistakes is treating access control for an AI system as though it could rely on the model’s good behavior somewhere in the chain, rather than enforcing it at the same hard, structural layer conventional access control has always depended on for any system handling sensitive data or consequential action. Systems that filter before retrieval, bound delegated permissions strictly, isolate tenants and memory as hard structural boundaries, and keep extending access control deliberately as new capabilities are added tend to hold up reliably even as they scale and as their capabilities continue to grow, while systems that lean on the model’s judgment as a substitute for enforcement tend to discover the gap only once a damaging failure reveals exactly how much was resting on an assumption that access control, well-designed on paper, would extend reliably beyond exactly the case it was originally, narrowly, tested against.

The underlying principle that ties all of this together is one access control has always rested on, long before AI systems ever entered the picture: a boundary that depends on the requester’s good behavior for its enforcement isn’t a boundary, it’s a hope, and a hope is not a security property a team can responsibly build a system around when the actual stakes involve sensitive data consequential action, or a promise made to a tenant, a user, or an entire organization that their information will stay exactly where it’s supposed to stay. What’s new in the AI setting isn’t the principle itself, it’s the unfamiliar ways a model’s reasoning can be maneuvered around an instructional boundary that would have held perfectly well against a purely mechanical, non-reasoning system, which is exactly why the enforcement point has to move to where it’s always belonged, the data and action layer itself, rather than being entrusted to a component whose behavior, however generally well-intentioned, simply can’t be guaranteed to hold in every case an adversary, or even an entirely ordinary user asking an entirely ordinary question in an unexpected way, might eventually present it with.

Teams that internalize this early, building access control as hard, structural enforcement rather than as an instruction layered on top of a model’s reasoning, tend to end up with AI systems that earn the trust placed in them by the users and organizations relying on them, holding boundaries even under conditions nobody explicitly anticipated during initial design, while teams that treat the model’s good behavior as sufficient protection tend to discover, at some point, that the boundary they believed was there was never enforced anywhere a determined enough request couldn’t eventually find its way around.