What is function calling?

Summary slides

Why a strict schema matters more than it might first seem

The core technical challenge function calling solves is getting a language model, which naturally produces free-form, unstructured text, to instead produce output that another piece of software can parse reliably every single time, with no ambiguity about what field means what or what format a value should take. A model asked to “look up the weather in a given city” in plain language might respond in dozens of different phrasings, but a function call needs to produce exactly the same structured shape every time, a specific field name for the city, a specific format for how that value is represented, so the application receiving that structured output can process it programmatically without needing to interpret free-form prose.

This is why function definitions are written using a formal schema, a structured specification defining each parameter’s name, its type, whether it’s required, and often a description of what it represents, rather than a loose, natural-language description alone. The schema gives the model an exact target to match, and modern function-calling implementations are trained specifically to produce output conforming to that schema with a high degree of reliability, which is what makes it practical to build software that depends on parsing the model’s output programmatically rather than needing a human to read and interpret it.

How a function definition is actually structured

A typical function definition includes a name uniquely identifying the function, a description explaining what it does and when it should be used, and a parameters specification listing each argument the function accepts, its data type, whether it’s required or optional, and any constraints on what values it can take. This structure closely mirrors how a function signature works in ordinary programming, and that similarity isn’t a coincidence, function calling is deliberately designed to let a developer expose an already-existing piece of code to a model using a specification format that closely resembles how that code’s own interface would already be documented.

Getting each part of this definition right matters directly for how reliably the model uses the function correctly. A parameter’s type needs to be specified precisely, a value the function expects as a number rather than a string can produce a subtly broken function call if the schema doesn’t make that distinction clear, and a parameter description that’s ambiguous about what values are actually acceptable leaves the model to guess, producing calls that pass schema validation but still supply an argument that doesn’t correctly capture the caller’s actual intent.

What happens after the model produces its structured output

Once a model produces a function call matching its target schema, the application receiving that output still has real work to do before treating it as a legitimate function invocation. Well-built systems validate the structured output against the schema a second time on the receiving end, confirming required fields are actually present, that types match what was specified, and that the values fall within any constraints the schema defined, rather than assuming the model’s output is automatically correct simply because it was trained to follow this format. Only after this validation passes does the application actually execute the underlying function and pass its result back into the conversation.

This validation step matters because a model producing output that’s syntactically well-formed, valid structure, correct field names, doesn’t guarantee that output is semantically correct, a syntactically valid function call can still contain a value that doesn’t make sense for the situation, a date outside a reasonable range, an amount that’s clearly a typo-scale error, and catching this kind of problem requires validation logic specific to what the function actually does, beyond what generic schema conformance checking alone can catch.

Why reliability at the schema level enabled much of what agentic AI depends on

Before function calling became a reliable, widely supported capability, building an application that needed a model to trigger external actions meant parsing free-form text output and trying to extract structured intent from it, a fragile, error-prone approach that broke in unpredictable ways whenever the model phrased something slightly differently than expected. The shift toward models producing reliably schema-conforming output directly enabled the broader agentic capabilities covered throughout this collection’s discussions of AI coding agents and agent harnesses, since an agent taking a sequence of real actions depends fundamentally on each individual action being specified precisely enough for the surrounding system to execute it correctly without ambiguity.

This reliability improvement is easy to take for granted now that it’s a standard, well-supported capability across major model providers, but it represents a genuine technical advance over earlier approaches, and understanding that history helps explain why function calling’s strict schema conformance is treated as such a foundational capability rather than a minor implementation convenience, it’s specifically what makes dependable, automated action-taking possible at all.

Handling the cases where a model’s output doesn’t quite fit

Even with well-trained models and carefully written schemas, function calling doesn’t achieve perfect reliability, a model can still occasionally produce output that fails to conform to the expected schema, supplies a value of the wrong type, or omits a required field. A well-built system anticipates this rather than assuming schema conformance is guaranteed, handling a malformed function call gracefully, either by asking the model to retry with corrected output, by applying reasonable defaults where appropriate, or by failing in a way that’s clearly communicated rather than causing a confusing downstream error.

This connects directly to the broader discussion of AI native testing and debugging covered elsewhere in this collection, a system that assumes function calls will always be perfectly formed tends to produce exactly the kind of rare, hard-to-reproduce failure that’s difficult to diagnose after the fact, while a system built with the expectation that occasional malformed output will happen handles that reality gracefully rather than being caught off guard by it in production.

Choosing what belongs in a function definition versus what doesn’t

Not every piece of functionality an application offers needs to be exposed as a callable function, and deciding what to expose is a real design decision with consequences for how reliably a model uses what’s available. A function whose purpose and required arguments are genuinely ambiguous without extensive context is a poor candidate for direct exposure, since a model has no reliable way to determine the correct arguments from a request alone, while a function with a narrow, well-defined purpose and clearly specified arguments gives a model exactly the information it needs to use it correctly and consistently.

This connects to the broader discussion of curated tool selection covered throughout this collection’s agent design discussions, a small number of well-defined, clearly scoped functions tends to produce more reliable behavior than a large number of loosely defined ones, even when the loosely defined set technically covers more total functionality, since the model’s ability to use each function correctly depends directly on how precisely that function’s purpose and interface have been specified.

Common mistakes teams make around function calling

1. Writing loose, ambiguous parameter descriptions and schemas, leaving the model to guess at details that should have been specified precisely in the function definition itself.

2. Trusting a model’s structured output as automatically correct simply because it conforms to the expected schema, missing that schema conformance doesn’t guarantee semantic correctness.

3. Assuming function calls will always be perfectly well-formed and having no graceful handling for the occasional malformed or incomplete output that still occurs even with well-trained models.

4. Exposing a large number of loosely defined functions rather than a smaller, more carefully scoped set, producing less reliable function selection despite covering more total functionality.

5. Confusing function calling’s narrower, schema-focused mechanism with the broader concept of tool calling, missing that not every tool a model can use is necessarily built around a custom-defined function schema.

What connects these mistakes is underestimating how much of function calling’s real-world reliability depends on the precision of the schema definitions themselves, the mechanism gives a model a structured way to request an action, but the quality of that request, how correctly its arguments capture what was actually intended, depends directly on how carefully each function’s interface was specified in the first place.

The deeper point about function calling is that it’s a specific, technically precise solution to the general problem of getting a language model’s naturally free-form output to interoperate reliably with the strict, structured world of ordinary software, and the schema-based approach it’s built around is exactly what makes it possible to treat a model’s requested action with the same programmatic confidence a developer would extend to a function call in any conventional codebase.