What is tool calling security?

Summary slides

Why tool calling introduces a distinct security surface beyond the model’s text generation

A model that only generates text has its output reviewed, however briefly and however imperfectly, by whatever process consumes that text, giving even a flawed or manipulated response a chance to be caught before it ever causes lasting harm. Tool calling removes that implicit review step by design, since the entire value of giving a model tool access comes from letting it invoke functions directly rather than requiring a human to translate its intent into an actual API call each time. This particular directness is exactly what makes tool calling valuable in practice, and exactly what makes it a distinct security concern in its own right, since the actual path from a model’s internal reasoning to an executed action has gotten considerably shorter, with fewer natural checkpoints where a problem might otherwise have been caught before it mattered.

The security question specific to tool calling isn’t merely whether a model should be allowed to use a tool at all, that’s the permission question covered elsewhere, it’s whether the mechanics of how a call gets constructed, validated, and executed can be manipulated independently of whether the underlying permission was ever appropriate in the first place. An agent correctly permitted to use a refund tool can still be manipulated into calling that tool with a manipulated amount, a manipulated recipient, or a manipulated set of conditions, none of which the permission system itself, focused narrowly on whether the tool can be called at all, was ever designed to catch in the first place.

How malicious or malformed arguments can be smuggled into a tool call

A model constructs the arguments for a tool call based on its own reasoning about the current situation, which means anything that can influence that reasoning can, in principle, influence what arguments get passed. Content encountered earlier in the model’s context, a retrieved document, a piece of user input, content returned from an earlier tool call, can contain text specifically, deliberately crafted to shift what the model decides to pass as an argument, effectively, quietly smuggling a value into a tool call that the legitimate, original flow of the interaction never intended to produce at all.

This risk means argument construction can’t be trusted simply because it emerged from a model that’s generally well-behaved, since the vulnerability here isn’t the model behaving badly in general, it’s the model being nudged, through content it was exposed to, into constructing one argument incorrectly in one instance. Defending against this risk requires validating tool call arguments against explicit, independently, deliberately defined expectations, an allowed range for a numeric argument, an allowed set of values for a categorical one, a defined schema for a structured one, rather than simply trusting that whatever the model happened to generate is safe purely because the model itself is generally, usually reliable in ordinary circumstances.

Argument validation also needs to account for values that are individually, technically valid but collectively, considerably suspicious taken together, a refund amount within the normal allowed range but paired with a recipient account that’s never been associated with this customer before, a pattern that argument-level validation alone won’t catch unless the underlying validation logic is specifically, deliberately designed to consider combinations of arguments together, rather than merely checking each individual one entirely in isolation.

How tool descriptions themselves become an attack vector

The description of what a tool does, how it should be used, and what its various parameters mean is itself content the model reads and reasons about, which means a tool description is, in a sense, part of the model’s context just as much as a retrieved document or a piece of user input is. A tool description that’s been compromised, whether through a supply chain issue with a third-party tool provider or through an internal process that let an inaccurate or manipulated description slip through unnoticed, can shift how the model uses that tool in ways that have nothing to do with the tool’s actual functionality at all.

This particular vulnerability is especially concerning for tools sourced from external providers, since an organization typically has far less visibility into how carefully a third-party tool’s description was written and reviewed than it has into its internally authored tools built in-house, and a description that subtly, quietly encourages the model toward an unsafe pattern of use, encouraging it to pass more data than necessary, encouraging it to call the tool in situations where a more conservative approach would be appropriate, can shape the model’s actual behavior without any single interaction ever looking obviously, visibly wrong entirely in isolation.

Treating tool descriptions as security-relevant content means they deserve the exact same kind of review and version control that actual code itself already receives, since a tool description is, quite functionally closer to a set of instructions the model will follow than to mere documentation a human simply reads for reference purposes only and a change to a tool description that isn’t reviewed with this concern in mind can introduce a meaningful behavioral shift that nobody ever explicitly, formally signed off on at all.

How to validate and sanitize tool call arguments before execution

Argument validation happens at the actual boundary between the model’s generated call and the execution of that call, actively checking the arguments against explicit rules before the underlying function ever runs, rather than simply trusting the model’s output to already, somehow be inherently safe on its own. This particular validation needs to be specific and tailored to each tool’s actual requirements individually, a generic validation layer that only checks for obviously malformed input, wrong data types, missing required fields catches considerably less than a validation layer built specifically around what values are legitimate for this tool in this context.

Sanitization deserves its separate attention distinct from validation itself, since an argument can be technically valid in raw format while still containing content that shouldn’t be passed through entirely unmodified, a text field that’s technically a valid string but that contains an embedded instruction meant to manipulate whatever system processes that string next, a risk that’s particularly relevant when a tool’s output is later fed back into the model itself or into another downstream tool, letting a manipulated argument propagate further than the original tool call that introduced it.

This particular validation and sanitization work benefits considerably from being centralized, rather than duplicated inconsistently and redundantly across every individual tool implementation separately, since a shared validation layer applied consistently across all of an agent’s tools is both easier to get right and easier to audit than expecting each individual tool’s implementation to independently reinvent the same careful validation logic, with the inevitable inconsistency that comes from many separate implementations each making slightly different judgment calls about what counts as safe.

How confused deputy problems arise specifically in tool calling

A confused deputy problem occurs when a system with legitimate authority to perform an action is tricked into using that authority on behalf of a request that shouldn’t honestly be entitled to it in the first place, and tool calling creates a particularly clean version of this classic problem, since the model itself functions exactly as the deputy, holding authority to call tools but potentially confused, through manipulated context, about whose actual request it’s currently serving. An agent legitimately authorized to look up account information for the user it’s currently helping can be manipulated, through content encountered mid-conversation, into looking up a different account entirely, using its legitimate tool access on behalf of a request that was never part of the original, authorized interaction.

Defending against this pattern means the tool call itself, not just the model’s reasoning about the call, needs to carry context about whose request is being served, allowing the underlying system to verify that a tool call is consistent with the current, legitimate interaction rather than trusting that the model’s internal reasoning correctly tracked this the whole way through. A tool call that includes an explicit, independently verified reference to the current user or session, checked directly against the tool’s access rules at the actual moment of execution, gives the system a check that doesn’t depend entirely on the model’s reasoning having stayed correctly oriented throughout what might be a potentially long, considerably complex interaction.

How to handle tool output that the model then treats as trusted input

Once a tool executes and returns a result, that result becomes part of the model’s context for whatever comes next, and the model generally treats this returned content with a level of implicit trust that a purely external, entirely unvetted piece of content wouldn’t otherwise receive, since the tool call itself was something the system itself deliberately initiated in the first place. This implicit trust is exactly what makes tool output a meaningful risk when the underlying data source a tool queries isn’t itself fully trustworthy, a web search tool returning content from an arbitrary external page, a database query returning content that itself came from user-submitted data, since content flowing back through a trusted, legitimate tool call inherits a level of trust from the model that the content’s actual origin doesn’t necessarily, honestly deserve on its own merits at all.

This means tool output deserves the exact same scrutiny as any other untrusted content entering the model’s context, even though it arrived through a mechanism, a tool the system itself invoked, that feels considerably more trustworthy on the surface than a raw, external input would on its own. Treating tool output as inherently safe simply because it came back through a legitimate tool call, rather than evaluating what the tool queried and whether that underlying source is itself trustworthy, creates exactly the kind of gap that content specifically crafted to exploit this implicit trust can exploit.

How tool selection itself can be manipulated

Beyond simply manipulating the arguments passed to a tool the model was always going to call anyway, an adversarial actor can attempt to manipulate which tool the model decides to call in the first place steering it away from a safer, more appropriate tool and toward a riskier one that happens to be available, purely through content encountered in context that makes the riskier tool seem like the natural, obvious choice for the current situation. This risk is subtler than argument manipulation because the resulting tool call can look entirely legitimate in isolation, a correctly formed call to a tool the agent is authorized to use, with the actual manipulation living entirely in the upstream decision about which tool to reach for.

Defending against tool selection manipulation benefits from designing an agent’s available tool set so that the riskiest tools require additional, explicit justification or additional scrutiny before use, rather than treating every available tool as an equally easy, equally natural choice for the model to reach for. A system where the highest-risk tool is exactly as easy to invoke as the lowest-risk one offers no natural resistance whatsoever to being steered toward the riskier choice, while a system that builds in friction, additional confirmation, additional logging, additional review, specifically and deliberately around its highest-risk tools gives any adversarial attempt to steer tool selection something solid and concrete to push against.

How to secure the tool registry and prevent unauthorized tool injection

An agent’s actual set of available tools is typically defined through some form of registry, a configuration that specifies exactly which tools exist and what they’re allowed to do, and this registry itself deserves security attention, since an agent’s capability is entirely bounded by what’s present in that registry regardless of how carefully every individual tool within it was designed. A registry that can be modified without proper review, whether through a compromised deployment pipeline, an overly permissive configuration management system, or simply a lack of any change control around it, represents a path for an entirely new, potentially dangerous tool to be introduced into an agent’s available set without anyone specifically approving that addition.

This means the tool registry deserves the exact same change management discipline applied to any other security-critical piece of configuration version control, a review process for any addition or modification whatsoever, and a clear auditable record of exactly what tools were available to an agent at any point in time, since without this record an incident investigation has no reliable way of even determining what tool capabilities an agent had access to when an incident occurred.

How multi-tool chains compound individual tool risks

A single tool call, however carefully secured on its own exists within a much broader sequence where its output can become the actual input to another tool call further downstream, and this chaining introduces risk that securing each individual tool call in isolation doesn’t fully address, since a combination of individually acceptable tool calls can accomplish something that no single call in the chain would have been permitted to do on its own. A read tool that returns sensitive data, followed by a formatting tool, followed by a send tool, can together accomplish data exfiltration that would have been blocked immediately if the send tool alone had been asked to transmit that exact same sensitive data directly and openly in one single step.

Securing against this compounding risk means evaluating tool chains as a unit of analysis in their own right, not merely evaluating each individual tool call as it happens entirely in isolation, tracking what data has flowed through a chain of calls within a single interaction and applying policy against that accumulated flow, not merely against each individual step considered on its own. This particular kind of chain-aware monitoring is considerably more complex to build than mere per-call validation alone, but it’s specifically what catches the category of risk that emerges only from combination, exactly the category that per-call security, however careful, structurally simply cannot catch by its design.

How to test and monitor tool calling for security specifically

Security testing for tool calling means actively, deliberately attempting the manipulations this article has already described, crafting content specifically, deliberately designed to smuggle unintended arguments into a call, to manipulate tool selection, to exploit confused deputy patterns, rather than only ever testing that tool calls work correctly under ordinary, well-behaved, expected conditions alone. This adversarial testing needs to be ongoing, since new tools get added, existing tools get modified, and the vulnerabilities that matter for a tool set can shift considerably as that tool set itself evolves over time.

Ongoing monitoring in production deserves its dedicated attention entirely distinct from pre-launch testing, actively tracking actual tool call patterns for anomalies, an unusual combination of arguments, a tool being invoked in a context where it’s rarely used, a chain of calls that doesn’t match typical, legitimate usage patterns, since these operational signals can surface an actual attempted manipulation that no amount of pre-launch testing, however thorough, could have ever specifically anticipated well in advance.

How tool calling security connects to permissions, sandboxing, and broader guardrails

Tool calling security sits at the mechanical layer between an agent’s broader permission model, which determines what a tool is authorized to do, and the sandboxing that contains the consequence if something upstream of it still goes wrong, and understanding this position matters for building a coherent overall defense rather than several layers that each assume some other layer is handling the concern that’s theirs to handle. Permissions answer whether a tool call should be allowed at all; tool calling security ensures that the mechanics of constructing and executing that call can’t be manipulated independently of whether the underlying permission was appropriate; sandboxing contains whatever consequence follows if a manipulated call nonetheless gets through.

This layered view matters because a tool calling security failure and a permission failure can produce results that look identical from the outside looking in, an agent taking an action it shouldn’t have taken, but the actual fix is different depending on precisely which layer failed in that instance. A permission failure calls for revisiting what the agent is authorized to do at all, while a tool calling security failure calls for revisiting how the mechanics of argument construction, validation, and execution can be manipulated even when the underlying permission was entirely appropriate, and conflating the two during an incident investigation risks fixing the wrong layer while leaving the actual vulnerability untouched.

How schema design itself can either help or hurt tool calling security

The schema that defines a tool’s expected arguments does considerably more than merely tell a model what shape of input to produce, it also determines how much room a manipulated piece of context has to influence what ultimately gets passed, and schema design deserves to be treated as a security decision rather than merely a functional one. A schema with a loosely typed free-text field where a tightly constrained enumeration would suffice gives manipulated context far more room to smuggle an unintended value through than a schema that restricts the argument to a small, explicit set of legitimate options in the first place.

This means tool schema design benefits from actively minimizing the space of technically valid arguments to only what a tool’s legitimate use cases require, even when a broader, more flexible schema would be marginally more convenient to build against. A recipient field constrained to a validated list of known, legitimate accounts closes off an entire category of argument-smuggling risk that a free-text recipient field leaves wide open by comparison, and this kind of deliberate narrowing, applied consistently across a tool’s full set of arguments, does security work well before any runtime validation logic ever executes.

Schema design also benefits from making a tool’s most consequential parameters, the ones that determine scale, recipient, or irreversibility, structurally harder to set to a dangerous value than a tool’s more routine parameters, rather than treating every argument as equally easy to populate regardless of how much consequence rides on getting it right. A schema that requires a separate, explicit confirmation flag before a high-consequence parameter takes effect adds a structural checkpoint that a schema treating all parameters identically simply doesn’t provide.

How to think about tool calling security across a growing library of available tools

An organization’s tool library tends to grow considerably, meaningfully over real time as new capabilities get added specifically to serve new use cases, and this growth introduces its security concern distinct from any single tool’s design, since an agent’s actual attack surface is a function of its entire available tool set considered together, not merely the sum of each individual tool’s isolated risk. A tool library that’s grown organically, one addition at a time, each individually reasonable, can end up containing combinations of tools that considered together, enable exactly the kind of chained, compounding risk discussed earlier, without anyone specifically reviewing the library as a coherent whole once it reached its current, considerably larger size.

This means a growing tool library benefits from periodic, deliberate review at the level of the whole library, not merely at the level of each individual addition, specifically asking whether the current combination of available tools introduces risk that wasn’t present, or wasn’t as severe, when the library was smaller and each tool’s interactions with every other tool were easier to reason about directly. A library that’s never reviewed this way tends to accumulate risk quietly, through the accumulation of individually reasonable additions whose collective interaction nobody ever specifically, deliberately examined.

How to think about the security implications of parallel or concurrent tool calls

Modern agent architectures increasingly allow a model to invoke several tools in parallel within a single reasoning step, rather than strictly, always one at a time in sequence, and this concurrency introduces its security wrinkle distinct from the sequential chaining discussed earlier, since two tool calls that each look entirely acceptable when evaluated independently and simultaneously can still interact in ways that neither call’s validation logic was ever built to anticipate. A read call and a write call issued in the same reasoning step, each individually permitted, can race against each other in a way that produces a result neither call alone would have produced, a category of risk that purely sequential, one-call-at-a-time thinking simply doesn’t surface.

Securing against this means the validation and monitoring layer covering an agent’s tool calls needs awareness of what else is happening concurrently within the same reasoning step, not merely a view of each individual call considered in isolation, since a validation check that only ever examines one call at a time has no natural way of noticing that two calls, issued together, interact in a way that neither would on its own. This is considerably harder to build correctly than sequential, chain-aware monitoring, since concurrent calls can complete in an order that isn’t fully predictable in advance, and validation logic that assumes a fixed, predictable ordering can miss exactly the race condition that concurrent execution introduces.

How to design fallback behavior for a tool call that fails validation

What happens when a tool call fails validation deserves just as much deliberate design attention as the underlying validation logic itself, since a poorly designed fallback can quietly reintroduce exactly the risk the validation was built to prevent. A system that silently retries a failed call with slightly adjusted arguments, hoping to find some version that passes validation, risks the model effectively probing the validation boundary itself through repeated attempts, each one incrementally closer to whatever the validation logic will accept, a pattern that looks like ordinary error recovery but that functions, in practice, as an, if unintentional, attack against the validation layer.

A considerably safer fallback design instead treats a validation failure as a signal worth surfacing, rather than merely a technical hiccup to quietly, silently work around, logging the failed attempt, its actual arguments, and the reason it failed, and routing repeated failures from the same interaction toward human review rather than allowing indefinite, automated retry. This particular design choice trades away some of the graceful, self-healing behavior that automated retry would otherwise provide, but that tradeoff is worth making precisely because the alternative, treating repeated validation failures as merely routine gives an adversarial actor room to iterate against the validation boundary until something eventually gets through undetected.

Common mistakes teams make around tool calling security

Several distinct patterns recur often enough across organizations building tool-using AI systems that naming them directly is worth doing before they’re discovered through an actual incident.

1. Trusting that arguments generated by a generally well-behaved model are safe by default, rather than validating them against explicit, independently defined expectations before execution.

2. Validating individual arguments in isolation without checking for combinations that are collectively suspicious even when each individual value looks acceptable on its own.

3. Treating tool descriptions as mere documentation rather than as security-relevant content that shapes model behavior and deserves the same review rigor as code.

4. Building generic, one-size-fits-all argument validation rather than validation logic specific to what’s legitimate for each individual tool in its context.

5. Duplicating validation logic inconsistently across many separate tool implementations rather than centralizing it into a shared, consistently applied layer.

6. Relying entirely on the model’s reasoning to correctly track whose request is currently being served, without an independent, system-level check verifying that a tool call is consistent with the actual current interaction.

7. Treating tool output as inherently trustworthy simply because it arrived through a legitimate tool call, without evaluating whether the tool’s underlying data source deserves that trust.

8. Making the riskiest available tools exactly as easy to invoke as the lowest-risk ones, offering no natural resistance to an attempt to steer tool selection toward a more dangerous choice.

9. Allowing the tool registry to be modified without proper review or change control, letting a new tool enter an agent’s available set without anyone specifically approving that addition.

10. Keeping no auditable record of exactly what tools were available to an agent at a point in time, leaving incident investigations unable to even establish what capabilities were in play.

11. Evaluating each tool call in a chain in isolation, missing risk that only emerges from the accumulated combination of several individually acceptable calls.

12. Testing tool calling only under ordinary, well-behaved conditions rather than actively attempting the manipulations that argument smuggling, tool selection steering, and confused deputy patterns rely on.

13. Relying entirely on pre-launch testing without ongoing production monitoring for anomalous tool call patterns that only become visible once usage accumulates.

14. Conflating a tool calling security failure with a permission failure during incident response, fixing the wrong layer while leaving the actual vulnerability untouched.

15. Assuming a well-secured permission model alone is sufficient, without recognizing that the mechanics of how a call gets constructed and executed can be manipulated entirely independently of whether the underlying permission was ever appropriate.

16. Using loosely typed, free-text schema fields where a tightly constrained enumeration would suffice, giving manipulated context far more room to smuggle an unintended value through than necessary.

17. Letting a tool library grow organically, one individually reasonable addition at a time, without ever periodically reviewing the whole library together for combinations of tools that introduce risk none of them carried in isolation.

18. Validating each concurrent tool call in isolation without any awareness of what else is happening in the same reasoning step, missing race conditions that only emerge when calls interact.

19. Silently retrying a failed tool call with adjusted arguments rather than treating a validation failure as a signal worth surfacing, effectively letting the model probe the validation boundary through repeated attempts.

What connects all nineteen of these mistakes is a single underlying pattern: treating the moment a model decides to call a tool as the end of the security question, rather than as the actual beginning of a distinct set of mechanical concerns, argument construction, description integrity, output trust, chain composition, that exist independently of whether the model was ever authorized to use that tool in the first place. A permission system can be flawless and a tool calling implementation can still be exploited, precisely because these are different layers addressing different failure modes, and an organization that secures one while assuming the other is automatically, quietly covered is leaving exactly the gap between them wide open.

The deeper principle underneath all of this is that a model deciding to call a tool is fundamentally an act of reasoning, not an act of independently verified fact, and everything about how that reasoning gets translated into an executed function call deserves the exact same skepticism applied to the model’s reasoning anywhere else in the system, since a tool call that looks mechanically, superficially correct, properly formatted, calling an authorized tool, can still be the product of reasoning that was subtly, successfully manipulated somewhere further upstream, and only independent verification performed at the point of execution, never mere trust in the model’s apparent correctness catches that gap before it becomes a lasting consequence.