← All chapters
Tool Use & Function Calling
Connecting models to tools, APIs, and structured function calls.
24 terms
API ToolAn API tool is a tool that wraps an external HTTP API, exposing one or more endpoints as callable functions within an agent's action space. The tool handles authentication, serialization, and error normalization so the agent can invoke real-world services — such as weather data, CRM systems, or payment processors — with typed arguments and reliable return values, without the model needing to know raw HTTP details.Argument ValidationArgument validation is the process of checking that the arguments a model supplies in a tool call conform to the tool's schema — verifying required fields are present, types match, and values fall within acceptable ranges. Validation can happen in the runtime before dispatching the call, in the tool itself, or at both layers. Returning clear, structured validation errors to the model often enables it to self-correct and retry with valid arguments.Code Execution ToolA code execution tool allows an agent to run programs — typically Python or shell scripts — in a sandboxed environment and receive the output as a tool result. This capability makes agents dramatically more powerful for numeric computation, data analysis, file manipulation, and algorithm implementation, because the agent can offload computation to a deterministic interpreter rather than attempting to simulate it through language model inference alone.Function CallingFunction calling is a model capability that allows a language model to emit a structured request to invoke a named function with typed arguments, rather than free-form text. The runtime intercepts the request, executes the function, and returns the result as an observation. Function calling makes tool integration reliable by replacing text parsing with a machine-readable protocol, and it is the foundation of most production tool-use implementations.Grounding via ToolsGrounding via tools is the practice of anchoring model outputs to verifiable, up-to-date information by having the agent actively fetch data through tool calls rather than relying solely on parametric knowledge. A grounded agent retrieves current prices, live documents, or database records at inference time, substantially reducing hallucination on factual questions. Grounding is complementary to RAG: retrieval tools give the agent the ability to ground itself on demand rather than via a fixed pre-fetched context.Human-in-the-Loop ApprovalHuman-in-the-loop approval inserts a manual confirmation checkpoint before an agent executes a high-impact or irreversible tool call, such as sending money or deleting records. The agent proposes the action and its arguments, and a person reviews and authorizes it before execution proceeds. This gate catches misinterpretations and manipulation, trading some autonomy for safety on consequential side-effecting operations.IdempotencyIdempotency means that calling a tool multiple times with the same arguments produces the same result as calling it once, with no additional side effects on repeated calls. Designing tools to be idempotent makes agents dramatically more resilient: if a network failure or timeout causes the runtime to retry a call, no duplicate actions — such as double charges or duplicate records — result. Idempotency keys are a common implementation technique for mutation-heavy APIs.JSON ModeJSON mode is a model inference setting that guarantees the output is syntactically valid JSON. It does not guarantee semantic correctness or schema conformance — only that the text parses without error. JSON mode is useful when a downstream consumer requires parseable output but when the full structure cannot be specified in advance. For strict schema adherence, schema-guided decoding or function calling with explicit parameter types is preferred.Parallel Tool CallsParallel tool calls allow a model to emit multiple tool invocation requests in a single generation step, which the runtime then executes concurrently before returning all results together. This dramatically reduces latency when subtasks are independent — for example, querying two databases simultaneously. The model must correctly identify which calls are independent; sequential dependencies cannot be parallelized and must remain ordered.Retrieval ToolA retrieval tool gives an agent the ability to search an external knowledge source — such as a vector database, document store, or search engine — and return relevant passages as a tool result. Unlike static context injected at the start of a conversation, retrieval tools can be called on demand and with targeted queries, allowing the agent to fetch exactly the information it needs at each step of its reasoning.Schema-Guided DecodingSchema-guided decoding constrains the token sampling process at inference time so that the model can only generate tokens that keep the partial output on a path toward a valid instance of the target schema. By masking logits for invalid tokens at each decoding step, the runtime guarantees schema conformance without relying on the model to self-correct post hoc. This enables reliable structured output even from models that would otherwise produce invalid JSON.Side-Effecting ToolsSide-effecting tools take actions that change external state — sending an email, committing code, executing a database write, or placing an order. Unlike read-only retrieval tools, side-effecting tools produce consequences that cannot be undone without additional action. They require extra caution in agent design: argument validation, confirmation steps, and idempotency checks are all especially important to prevent costly or irreversible mistakes.Structured OutputStructured output is the practice of constraining a model's generation to conform to a defined schema — such as JSON, XML, or a typed Pydantic model — rather than producing free-form text. It eliminates brittle text parsing and makes downstream processing deterministic. Structured output can be achieved through schema-guided decoding at the logit level, careful prompting, or a combination of both, and it is widely used for tool arguments, API responses, and data extraction.Tool BudgetA tool budget is a configurable limit on how many total tool calls — or calls to a specific tool — an agent may make during a task run. It controls cost, latency, and the blast radius of potential errors. When the budget is exhausted the runtime can force the agent to produce a final answer from whatever information it has gathered, or it can raise an exception so the caller can decide how to proceed.Tool DescriptionA tool description is the natural-language text within a tool schema that explains what the tool does, when to use it, and what its parameters mean. Because the language model relies on this text to decide whether and how to call the tool, description quality has an outsized impact on overall agent performance. Effective descriptions are precise, concrete, and include notes on common misuse cases or edge-case constraints that the model should be aware of.Tool Error HandlingTool error handling defines how an agent runtime manages failures during tool execution — including network errors, invalid arguments, timeouts, and permission denials. A robust strategy returns a structured error message to the model so it can decide whether to retry with different arguments, switch to an alternative tool, or surface the failure to a human operator. Silently swallowing errors typically leads to silent failures and incorrect agent outputs.Tool ObservabilityTool observability is the instrumentation that records each tool invocation — its name, arguments, result, latency, and any error — so engineers can trace and debug agent behavior. Detailed tool traces reveal why an agent chose a tool, whether arguments were well-formed, and where a task went wrong. This visibility is essential for diagnosing failures and tuning tool descriptions and selection logic.Tool ResultA tool result is the data returned by a tool execution that is fed back into the model's context as an observation. The model then incorporates this information into its next generation. Tool results must be serialized to text or structured content that the model can process; for binary data like images or large files, results are typically summarized or referenced by a handle rather than embedded verbatim in the prompt.Tool Result CachingTool result caching stores the output of a tool call keyed by its arguments so that repeated identical calls return the cached result instead of re-executing. This cuts latency and cost, and reduces load on external services. Caching is only safe for read-only or pure tools whose output depends solely on inputs; side-effecting tools must be excluded to avoid skipping real actions.Tool Retry PolicyA tool retry policy defines how an agent responds when a tool call fails transiently — how many times to retry, how long to wait between attempts, and when to give up or escalate. Exponential backoff spaces retries to avoid overwhelming a struggling service. A well-designed policy distinguishes retryable errors like timeouts from permanent ones like invalid arguments, which should not be retried.Tool RouterA tool router is the logic — implemented in prompt, code, or a dedicated model — that selects which tool to invoke given the current state of the agent's task. Simple routers let the model choose freely from all available tools; more sophisticated routers filter the candidate set based on task context or enforce policies about which tools are permitted in a given situation, reducing the chance of irrelevant or unsafe tool selection.Tool SandboxA tool sandbox is an isolated execution environment in which potentially dangerous tool calls — especially code execution — are run with restricted access to the host system. Sandboxes typically limit network access, file-system permissions, process creation, and execution time. They prevent an agent from causing damage outside the intended scope of its task and are a non-negotiable safety layer for any agent that can run arbitrary code.Tool SchemaA tool schema is the formal description of a tool that the language model receives at inference time: it includes the tool's name, a natural-language description of what it does, and a typed parameter specification — typically expressed as JSON Schema. The model uses the schema to decide whether to call a tool and to produce correctly structured arguments. A well-written schema is one of the most important levers for improving tool-selection accuracy.Tool SelectionTool selection is the model's decision about which tool — if any — to invoke given its current reasoning state and the available tool schemas. Quality tool selection depends on well-written tool descriptions, appropriate context in the prompt, and a manageable number of tools presented at once. When the tool set is large, retrieval-based tool routing or two-stage selection (first pick a category, then a specific tool) improves accuracy.