How LLM tool use works: function calling under the hood
What tool use actually is
Language models are text-in, text-out systems. Left alone, they can reason and write, but they cannot look up the current stock price, execute a database query, or send an email. Tool use — also called function calling — is the mechanism that bridges this gap.
The idea is simple in principle. You describe a set of functions to the model using a structured schema. When the model determines that calling one of those functions would help it answer the user, it does not fabricate an answer — it emits a structured request that your code can execute. Your code runs the function, returns the result to the model, and the model incorporates that result into a final response.
The subtlety is in the implementation details. The model never actually runs code. It produces structured output that looks like a function call. Your application is responsible for parsing that output, invoking the real function, and feeding the result back. The model is the orchestrator; your runtime is the executor.
This distinction matters a lot when things go wrong. If the model calls a function with a wrong argument, that is a prompt engineering or schema design problem. If the function throws an exception, that is your problem to catch and report back to the model gracefully.
The JSON schema contract
Tool definitions are written in JSON Schema. Each tool has a name, a description, and a parameters object. The description is arguably the most important part — it is what the model reads to decide when to call the function and how to interpret the results.
Here is a minimal tool definition for a weather lookup function:
{
"name": "get_current_weather",
"description": "Returns the current weather for a city. Use this when the user asks about weather conditions, temperature, humidity, or forecasts.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city name, e.g. 'Lagos' or 'Berlin'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit. Default to celsius unless the user specifies otherwise."
}
},
"required": ["city"]
}
}
In the Anthropic SDK this becomes:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const tools: Anthropic.Tool[] = [
{
name: "get_current_weather",
description:
"Returns the current weather for a city. Use this when the user asks about weather conditions, temperature, humidity, or forecasts.",
input_schema: {
type: "object",
properties: {
city: {
type: "string",
description: "The city name, e.g. 'Lagos' or 'Berlin'",
},
unit: {
type: "string",
enum: ["celsius", "fahrenheit"],
description:
"Temperature unit. Default to celsius unless the user specifies otherwise.",
},
},
required: ["city"],
},
},
];
const response = await client.messages.create({
model: "claude-opus-4-5",
max_tokens: 1024,
tools,
messages: [{ role: "user", content: "What is the weather in Accra?" }],
});
When the model decides to call get_current_weather, it sets stop_reason to "tool_use" and includes a tool_use content block in the response:
{
"stop_reason": "tool_use",
"content": [
{
"type": "tool_use",
"id": "toolu_01XFZUADTBb8G6JfKBH7ABCD",
"name": "get_current_weather",
"input": {
"city": "Accra",
"unit": "celsius"
}
}
]
}
The request-response cycle
The full cycle has four distinct phases. Understanding each one helps you build reliable integrations and debug failures systematically.
Phase 1: Initial request. You send the conversation history plus tool definitions to the model. The model reasons about what it needs.
Phase 2: Tool call emission. The model responds with stop_reason: "tool_use" and one or more tool_use content blocks. Each block has a unique id, the function name, and the input object matching your schema.
Phase 3: Tool execution. Your code receives the tool call, executes the actual function (a database query, an HTTP request, whatever), and gets a result.
Phase 4: Result injection. You append the assistant's tool call message to the conversation, then append a user message with a tool_result content block containing the result and the matching id. You send this back to the model, which now has the information it needs to produce a final response.
Here is the complete cycle in TypeScript:
async function runWithTools(userMessage: string): Promise<string> {
const messages: Anthropic.MessageParam[] = [
{ role: "user", content: userMessage },
];
while (true) {
const response = await client.messages.create({
model: "claude-opus-4-5",
max_tokens: 1024,
tools,
messages,
});
// If the model is done, return the text
if (response.stop_reason === "end_turn") {
const textBlock = response.content.find((b) => b.type === "text");
return textBlock?.type === "text" ? textBlock.text : "";
}
// Collect all tool calls from this response
const toolUseBlocks = response.content.filter(
(b): b is Anthropic.ToolUseBlock => b.type === "tool_use"
);
if (toolUseBlocks.length === 0) break;
// Append assistant's message (including tool calls) to history
messages.push({ role: "assistant", content: response.content });
// Execute each tool and collect results
const toolResults: Anthropic.ToolResultBlockParam[] = await Promise.all(
toolUseBlocks.map(async (toolCall) => {
const result = await executeTool(toolCall.name, toolCall.input);
return {
type: "tool_result" as const,
tool_use_id: toolCall.id,
content: JSON.stringify(result),
};
})
);
// Inject tool results back into the conversation
messages.push({ role: "user", content: toolResults });
}
return "";
}
async function executeTool(name: string, input: unknown): Promise<unknown> {
if (name === "get_current_weather") {
const { city, unit = "celsius" } = input as {
city: string;
unit?: string;
};
return fetchWeather(city, unit);
}
throw new Error(`Unknown tool: ${name}`);
}
Parallel tool calls and how to handle them
The model can emit multiple tool_use blocks in a single response. This happens when it determines that several pieces of information are needed simultaneously and neither depends on the other.
{
"stop_reason": "tool_use",
"content": [
{
"type": "tool_use",
"id": "toolu_01",
"name": "get_current_weather",
"input": { "city": "Lagos" }
},
{
"type": "tool_use",
"id": "toolu_02",
"name": "get_current_weather",
"input": { "city": "Nairobi" }
}
]
}
You must return a tool_result for each tool_use id. Returning results for only some of the calls will cause an error or confuse the model. The Promise.all pattern in the loop above handles this correctly — all tool calls are executed in parallel and all results are returned together.
If your function can throw, make sure to catch the error and return it as a tool_result with is_error: true:
const toolResults: Anthropic.ToolResultBlockParam[] = await Promise.all(
toolUseBlocks.map(async (toolCall) => {
try {
const result = await executeTool(toolCall.name, toolCall.input);
return {
type: "tool_result" as const,
tool_use_id: toolCall.id,
content: JSON.stringify(result),
};
} catch (err) {
return {
type: "tool_result" as const,
tool_use_id: toolCall.id,
content: `Error: ${err instanceof Error ? err.message : String(err)}`,
is_error: true,
};
}
})
);
When the model receives an error result, it can decide to try a different approach, ask the user for clarification, or report the failure gracefully. This makes the error path as important as the success path.
Controlling tool use with tool_choice
By default, the model decides whether to call a tool or answer directly. You can override this with the tool_choice parameter.
auto (default) — the model decides.
any — the model must call at least one tool. Useful when you are building a router that classifies user intent.
tool with a specific name — the model must call that exact tool. Useful for structured extraction where you always want the output in a specific shape.
// Force the model to always call the search tool
const response = await client.messages.create({
model: "claude-opus-4-5",
max_tokens: 1024,
tools,
tool_choice: { type: "tool", name: "search_knowledge_base" },
messages,
});
none — the model cannot call any tools, even if they are defined. This is useful for the final response turn when you want to prevent the model from making additional calls after you have already fetched everything it needs.
Schema design that leads to fewer hallucinations
The quality of tool calls is tightly coupled to the quality of your schema and descriptions. Several patterns consistently produce better results.
Be specific in the description. Vague descriptions lead to erratic calling behaviour. Instead of "Gets data", write "Queries the product catalog by SKU. Call this when the user mentions a specific product code, part number, or asks about inventory for a known item." The description should tell the model the precondition for calling the function and the kind of information it returns.
Use enums for constrained inputs. If a parameter can only be one of a fixed set of values, declare that with an enum. The model will almost never produce an out-of-range value when the options are enumerated.
Mark only truly required parameters as required. If you mark a parameter required but the user's message does not always contain the information to fill it, the model will hallucinate a value. Optional parameters with good defaults are better than required parameters the model has to guess.
Return rich results. The result you pass back in tool_result is raw context for the model's next reasoning step. Returning a flat string like "28 degrees" makes the model work harder than returning { "temperature": 28, "unit": "celsius", "condition": "partly cloudy", "humidity": 72 }. The richer the result, the better the final answer.
Streaming and tool use
Streaming changes the mechanics of receiving tool calls. Instead of getting a complete response, you receive incremental events. The tool call arrives in pieces: a content_block_start event announces the tool use block, followed by content_block_delta events that stream the input JSON incrementally, and finally a content_block_stop event.
const stream = await client.messages.stream({
model: "claude-opus-4-5",
max_tokens: 1024,
tools,
messages,
});
let currentToolCallId: string | null = null;
let inputBuffer = "";
for await (const event of stream) {
if (
event.type === "content_block_start" &&
event.content_block.type === "tool_use"
) {
currentToolCallId = event.content_block.id;
inputBuffer = "";
} else if (
event.type === "content_block_delta" &&
event.delta.type === "input_json_delta"
) {
inputBuffer += event.delta.partial_json;
} else if (event.type === "content_block_stop" && currentToolCallId) {
const input = JSON.parse(inputBuffer);
// Execute tool with currentToolCallId and input
console.log("Tool call complete:", currentToolCallId, input);
currentToolCallId = null;
}
}
With streaming, you can begin executing tool calls as soon as each one completes rather than waiting for all of them. This reduces overall latency in applications with multiple tools.
Production pitfalls and how to avoid them
Infinite loops. If your tool always fails and the model keeps retrying, you can end up in a loop. Always cap the number of iterations in your agentic loop — typically 10 to 20 is sufficient — and surface an error to the user when the cap is hit.
Context window exhaustion. Tool results accumulate in the conversation history. A long-running agentic task with many tool calls can fill the context window. Consider summarising earlier tool results or using prompt caching to reduce costs.
Non-deterministic tool selection. Temperature affects tool selection, not just text generation. For critical tool calls where correctness matters more than creativity, use temperature: 0. For exploratory tasks, a small positive temperature can help the model try different approaches.
Leaking sensitive data in results. Tool results go straight into the context and may be referenced in future turns. Avoid returning full API keys, passwords, or PII in tool results. Scrub sensitive fields before injecting them.
Schema versioning. If you change a tool's schema after deployment — renaming a parameter, removing a field — existing conversations with that tool in their history will break. Treat tool schemas like API contracts: version them carefully and migrate deliberately.
Tool use is what transforms a language model from a sophisticated text generator into a genuine agent that can take actions in the world. Getting the mechanics right is the prerequisite for every more sophisticated pattern: multi-step reasoning, self-correcting pipelines, and autonomous agents that handle real tasks without human supervision at each step.