What Is Tool Use and Function Calling? The Definitive Guide

What is tool use in AI?

Tool use is the ability of an AI model to reach outside itself. A model on its own can only generate text based on patterns in its training data. It cannot look up today's weather, read a row from your database, send an email, or run a calculation it has not memorized. Tool use removes that ceiling. It gives the model a defined set of external tools it can invoke, and a structured way to request them, so that the model's reasoning can be extended by real software running real operations.

The term covers two layers that are easy to confuse. Function calling is the specific technical mechanism, the API-level contract by which a model emits a structured request to run a named function with typed arguments. Tool use is the broader concept, the capability that function calling delivers, including the surrounding loop of deciding when a tool is needed, executing it, interpreting the result, and continuing. In everyday usage the two terms are often treated as synonyms. Most engineers say tool use when they mean the whole pattern and function calling when they mean the message format, and the phrases AI tool use, LLM tool use, and tool calling all point at the same idea.

The distinction that matters most is this. A model does not execute anything itself. When a model decides to use a tool, it does not run code or touch a database. It produces a structured description of the call it wants made, a name and a set of arguments, and hands that back to the developer's application. The application runs the function, then returns the output to the model. The model treats that output as new information and reasons forward. This separation is deliberate and central to safety, because it keeps the model as the planner and the surrounding system as the executor with its own permissions, validation, and limits.

How does function calling work?

Function calling is the four-step loop that turns a request into an action. The mechanics have converged across OpenAI, Anthropic, Google, and the open-source ecosystem to the point where the same mental model applies everywhere.

The process begins when the developer defines the available tools. Each tool is described as a schema, almost always JSON Schema, that names the function, explains in plain language what it does, and specifies its parameters with their types and which are required. This description is the only thing the model knows about the tool. Good descriptions are the difference between a model that calls the right function with the right arguments and one that fails, because the model reasons entirely from the words in the schema.

Next, the model receives the user's request alongside the catalog of tool definitions. It decides whether a tool is needed at all. If the answer lies in its own knowledge, it simply responds. If the request requires external data or an action, the model returns a structured tool call rather than a normal message, naming the function and supplying typed arguments it has inferred from the conversation. Developers can steer this decision with a control setting, commonly called tool choice, that can force the model to call a tool, forbid it, or leave the decision automatic.

The developer's application then executes the requested function. This is ordinary code, an API request, a database query, a calculation, whatever the tool represents, and it runs with the application's own credentials and safeguards, not the model's. The result is captured and sent back to the model as a new message tied to that specific call.

Finally, the model incorporates the result and continues. It may answer the user directly, or it may decide another tool call is needed, repeating the loop until the task is complete. Two refinements have made this reliable enough for production. The first is strict schema adherence, which guarantees the model's arguments conform exactly to the declared JSON Schema rather than being best effort. The second is parallel function calling, which lets a model fan out to several independent tools in a single turn. Together they turned function calling from a promising demo in 2023 into dependable infrastructure by 2026.

A simple tool use example

A concrete example makes the mechanics clear. Suppose you want an AI assistant to answer questions about the weather. The model cannot know the current weather, so you give it a tool. The tool is described with JSON Schema, the standard format for OpenAI function calling, Anthropic tool use, and every other major provider. The developer defines the tool like this:

{
  "name": "get_weather",
  "description": "Get the current weather for a city",
  "parameters": {
    "type": "object",
    "properties": {
      "city": { "type": "string", "description": "City name" },
      "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
    },
    "required": ["city"]
  }
}

When a user asks "What is the weather in Bengaluru right now?", the model does not answer from memory. It emits a structured tool call naming the function and the arguments it inferred:

{
  "tool": "get_weather",
  "arguments": {
    "city": "Bengaluru",
    "unit": "celsius"
  }
}

The developer's application runs the actual get_weather function, which calls a real weather API, and returns the result to the model. The model then writes the final answer for the user in natural language. This is the entire pattern of external tools and API function calling. The model chooses and fills in the call, your code executes it, and the result flows back. Everything from database queries to sending email works exactly this way. Only the tool definition changes.

When should a model not use a tool?

Tool calling is not always the right move, and part of good tool use is knowing when to skip it. Controlling this behavior is what the tool choice setting exists for, and it is a frequent source of confusion for developers building agentic AI tools. A model should generally not call a tool in three situations.

• When it can answer from internal knowledge. If the user asks a general question the model already knows, calling an external tool adds latency and cost for no benefit. A good model recognizes this and answers directly.
• When it should ask a clarifying question first. If the request is ambiguous or missing a required parameter, the model should ask the user rather than guessing arguments and making a wrong call. Guessing is how error compounding begins.
• When the cost or latency is not justified. Every tool execution adds a round trip and consumes tokens. For low-value or time-sensitive interactions, an immediate answer can be the better choice even when a tool exists.

Benchmarks explicitly test this judgment through irrelevance-detection categories, where the correct behavior is to call no function at all. Getting this wrong in either direction, calling a tool when none is needed or answering from memory when a tool was required, is one of the most common failure modes in production LLM tool use.

How is tool use different from related concepts?

Tool use sits close to several ideas it is often confused with. The distinctions clarify where it fits in the modern AI stack.

The pattern across every row is the same. Tool use is a foundational capability, and the related concepts are either the mechanism beneath it, an application built on it, or a standard for connecting it at scale.

How does tool use turn a language model into an agent?

The clearest way to understand tool use is to see what it unlocks. A language model with no tools is a conversation partner. It can explain, summarize, draft, and reason, but every output is text and nothing leaves the chat. The moment you give that same model a set of tools and a loop to call them, it becomes something categorically different. It can now take an instruction, decide what actions the instruction requires, carry them out through software, observe what happened, and adjust.

This is precisely the definition of an AI agent. An agent is an autonomous system that combines model reasoning with tool execution, memory, and feedback to complete multi-step tasks. Strip tool use away and the agent collapses back into a chatbot, because it can plan but never act. Tool use is the load-bearing capability. Every other agent component, the planner, the memory, the reflection step, exists to decide which tools to call, in what order, and what to do with the results.

Consider a concrete task: find all overdue invoices from the last quarter, total the outstanding amount, and email the finance team a summary. A bare model can describe how it would do this. An agent with tools actually does it. It calls a database query tool to pull the invoices, a calculation step to sum them, and an email tool to send the summary, checking the result of each call before moving to the next. Reasoning selects the tools, and tool execution performs the work.

As tasks grow more complex, the number and sequencing of tool calls become harder to manage, which is where AI agent orchestration enters. Orchestration is the control layer that decomposes a goal into steps, routes each step to the right tool or sub-agent, and assembles the results into an auditable outcome. In this framing, tool use is the action primitive and orchestration is the coordination logic that governs how those actions are chained. One without the other is incomplete: tool use without orchestration cannot handle long tasks, and orchestration without tool use has nothing to coordinate.

How good are AI models at tool use today?

Tool use has its own benchmarks, and they reveal a field that has matured quickly while leaving real gaps. Two evaluations dominate serious discussion.

The Berkeley Function-Calling Leaderboard, maintained by the Gorilla research group at the University of California, Berkeley, is the canonical test of raw function-calling ability. It measures whether a model selects the correct function, fills in the correct parameters, and sequences calls correctly across thousands of scenarios. These span simple single calls, parallel calls where several independent functions are requested at once, multi-turn cases where the result of one call informs the next, and edge cases such as detecting when no function should be called at all. As of mid-2026, the strongest models cluster tightly in the high seventies. The distribution of scores is telling. Simple single-call tasks are effectively saturated for frontier models, while multi-turn interactions remain the category where the gap between the best models and the rest is widest.

The second benchmark, tau-bench, from Sierra Research, tests something harder and more realistic. Rather than checking a single call in isolation, it evaluates a model acting as an agent in a sustained conversation with a simulated user, using tools against a database in domains like retail and airline customer service. On the original tau-bench, the best result belonged to Claude 3.5 Sonnet at 69.2 percent on retail and 46.0 percent on airline tasks, measured as pass at one, meaning the agent had to succeed on the first attempt. The airline number is the more sobering figure, because it shows that even a leading model completes fewer than half of realistic multi-step service tasks reliably on the first try. Sierra has since frozen the original tasks and moved evaluation to tau2-bench, which fixes task errors and adds a telecom domain with 114 tasks where both the user and the agent can call tools independently.

The gap between the two benchmarks is the whole story. Models are excellent at the mechanics of a single well-specified call and still unreliable at stringing many calls together across a long, ambiguous interaction with a real user. The mechanics are close to solved. The judgment of when to call, when to ask, and when to stop is not.

What is the Model Context Protocol and how does it relate?

For the first two years of tool use, connecting a model to a tool was bespoke work. Every integration was hand-built for a specific model API and a specific tool, and the same connector had to be rewritten for each new model. This did not scale as the number of models and tools exploded.

The Model Context Protocol, introduced by Anthropic in November 2024, was the response. MCP standardizes how AI models connect to external tools, APIs, databases, and enterprise software. It is an open standard for how tools, data sources, and applications expose themselves to any AI model. Instead of writing a custom integration for each model, a developer builds one MCP server for a tool, and any compatible model can use it. The relationship to function calling is worth stating precisely, because the two are frequently framed as rivals when they are layers. Function calling is the model-side mechanism, the way a single model requests a tool through its API. MCP is the integration layer above it. Function calling is how the model asks. MCP is how the tool is wired in.

Adoption moved unusually fast for an infrastructure standard. By late 2025 the ecosystem included more than 10,000 public MCP servers and support across major AI products from multiple competing vendors. The clearest signal came on December 9, 2025, when Anthropic donated MCP to the newly formed Agentic AI Foundation under the Linux Foundation, with OpenAI and Block as co-founders and companies including Amazon, Google, and Microsoft joining as members. When direct competitors agree to govern a standard jointly, it has stopped being one company's project and become industry infrastructure.

Why does tool use matter for enterprise AI?

The business case for tool use follows directly from the business case for agents, because agents are simply tool use applied at scale. Gartner projects that 40 percent of enterprise applications will include task-specific AI agents by the end of 2026, a sharp rise from less than 5 percent in 2025. Every one of those agents depends on tool use to reach the systems that hold a company's data and run its operations. Without it, an enterprise model can describe a workflow but cannot touch the CRM, the ERP, the ticketing system, or the data warehouse where the actual work happens.

The longer-range projections make the same point at greater magnitude. Gartner expects at least 15 percent of day-to-day work decisions to be made autonomously through agentic AI by 2028, up from effectively zero in 2024, and in its most optimistic scenario sees agentic AI accounting for roughly 30 percent of enterprise application software revenue by 2035, a figure it places above 450 billion dollars. These forecasts are bets on autonomous software acting in the world, and acting in the world is exactly what tool use provides.

There is a sobering counterweight in the same research. Gartner also predicts that more than 40 percent of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. Read alongside the benchmark results, this is not a contradiction. It is the predictable consequence of a capability that works beautifully in single well-specified calls and remains fragile across long, ambiguous, multi-step tasks. The organizations that succeed will be the ones that scope tool use tightly, validate every call, and put human approval where the stakes are high.

What are the limitations and risks of tool use?

Tool use introduces failure modes that a pure text model does not have, and understanding them is essential before deploying it.

• Error compounding. In a multi-step task, a mistake early in the chain propagates. If the model calls the wrong function, misreads a returned value, or fills in a subtly wrong argument, every subsequent step builds on that error. This is why single-call benchmarks are near saturation while multi-turn scores lag.
• Calling the wrong tool, or none. Deciding when a tool is appropriate is a judgment task, and models still misjudge it. They sometimes invoke a function for a question they could answer directly, and sometimes answer from memory when they should have checked.
• Security and prompt injection. Because tool use connects a model to systems that can read data and take actions, it widens the attack surface. Malicious instructions hidden in a document, web page, or tool result can hijack the model into calling tools in ways the user never intended. In early 2026, security researchers filed dozens of vulnerability disclosures against MCP servers, including cross-tenant data exposure and path-traversal flaws affecting thousands of applications.
• Cost and latency. Every tool call adds a round trip and consumes tokens, so a task requiring many sequential calls can be slow and expensive. The economics only work when tool use is scoped to tasks where the value clearly exceeds the compounding cost.

None of these are reasons to avoid tool use, but they establish the non-negotiable practices around it: least-privilege credentials for every tool, strict validation of all arguments, human approval gates for high-stakes actions, and treating every tool result as untrusted input.

Where is tool use used in practice?

Tool use is already the quiet engine behind most useful AI products, even when users never see the term.

1. Retrieval-augmented generation: When an assistant answers from a company's documents rather than its training data, a retrieval tool is being called behind the scenes, fetching relevant passages the model then reasons over.
2. Customer service: Support agents use tools to look up orders, process returns, and update account records, which is exactly what the tau-bench evaluations simulate.
3. Coding assistants: Developer tools call functions to read files, run tests, execute code, and search a codebase.
4. Data analysis: Analytics assistants convert a natural-language question into a database query, run it, and interpret the result.
5. Productivity and scheduling: Assistants read and write calendars, send messages, and file tickets through tool calls.

The common thread is that in every one of these products, the model's value comes not from what it knows but from what it can reach. The knowledge in the model is a starting point. The tools are what connect that knowledge to live data and real actions, and that connection is where the utility lives.

The future of tool use

Tool use moved from a single API feature to core AI infrastructure in roughly three years. The timeline below traces that shift.

Three directions are shaping where tool use goes next. The first is standardization settling into place. With MCP now governed by a neutral foundation backed by competing vendors, the era of bespoke per-model integrations is ending, and the expectation is a large shared library of tools that any model can use. The second is reliability on long tasks, the frontier the benchmarks expose. The mechanics of a single call are close to solved, and research is now focused on sustaining correct tool use across dozens of steps, including methods that help models recover from execution errors rather than compounding them. The third is efficiency, with new patterns that reduce the cost of tool-heavy workflows, including approaches where the model writes and executes code that calls many tools at once rather than making a long chain of individual round trips.

Tool use began as a convenience feature in a single API in 2023. By 2026 it is the foundation of the entire agentic layer of AI, the capability that turns models into systems and text into action. Everything built above it, agents, orchestration, and the autonomous enterprise applications the analysts are forecasting, rests on the simple loop of a model asking software to do something and reasoning about what came back.

Frequently asked questions

Sources and further reading

1. OpenAI. Function calling and other API updates. June 13, 2023. openai.com
2. OpenAI. Function calling. OpenAI API Documentation, 2026. platform.openai.com
3. Berkeley Function-Calling Leaderboard (BFCL), Gorilla Project, University of California, Berkeley. gorilla.cs.berkeley.edu
4. Patil, S. et al. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models. 2026.
5. Sierra Research. tau2-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. 2026. github.com/sierra-research/tau2-bench
6. Anthropic. Introducing the Model Context Protocol. November 2024. modelcontextprotocol.io
7. Anthropic. Code execution with MCP: building more efficient AI agents. Anthropic Engineering, 2026.
8. Gartner. Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026. August 26, 2025. gartner.com
9. Gartner. Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. June 25, 2025.
10. Stanford Institute for Human-Centered AI. 2026 AI Index Report. hai.stanford.edu