Function Calling and Tool Use in LLMs: A 2026 Engineering Guide

Function calling is the mechanism that turns an LLM from a text generator into something that can act — query a database, call an API, run code. The concept is simple to describe and surprisingly easy to get wrong in production: models omit required parameters, hallucinate tool names, pass strings where you defined integers, and call the wrong tool with total confidence. None of this is provider-specific; it's universal across OpenAI, Anthropic, and local models (AI/TLDR).

How it actually works

You describe a set of available functions using JSON Schema. The model receives the user prompt plus the tool catalog, decides whether a tool call is needed, and — if so — emits a structured tool call with typed arguments rather than free text (FutureAGI). The model never executes anything itself; your application code receives the structured call, runs the actual function, and feeds the result back into the conversation for the model to use in its next response.

{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Look up the current status of a customer order by ID",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": { "type": "string" },
        "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
      },
      "required": ["order_id"]
    }
  }
}

Schema design: the mistakes that actually cause failures

Two schema mistakes account for most real-world tool-call errors:

• Leaving required parameters out of the required array. This is the single most common cause of invocation errors — the model omits the parameter and the call fails silently rather than erroring loudly (AI/TLDR).
• Not using enum for constrained string fields. If a parameter can only take a fixed set of values, declare it as an enum. The model will never hallucinate a value outside that set once it's constrained (AI/TLDR).

Even with a correct schema, expect type coercion errors at the boundary — models may pass the string '5' instead of the integer 5, omit required params despite the schema declaring them required, or ignore enum constraints outright (AI/TLDR). Validate and coerce at the application boundary; never assume schema compliance is guaranteed just because you declared a schema.

Parallel tool calls and the strict-mode tradeoff

Parallel tool calls let a model emit multiple independent tool requests in a single response instead of sequentially waiting on each one — this reduces end-to-end latency by up to 3.7x compared to sequential execution when the requested actions don't depend on each other (FutureAGI).

There's a real tradeoff here worth knowing before you design around it: OpenAI's strict: true mode enforces exact schema compliance on arguments, but strict mode is incompatible with parallel tool calls — you have to pick one guarantee or the other (FutureAGI).

Message ordering matters more than it looks

A subtle but common bug: when a model returns multiple tool calls in one message, you must add exactly one assistant message containing all the tool_calls, followed by N separate result messages (one per tool call result) — reversing or interleaving this order breaks the model's context window handling (FutureAGI). This is easy to get wrong when building your own agent loop from scratch rather than using an SDK that handles it.

The seven recurring tool-use error types

Research analyzing LLM agent tool-use failures (ToolScan) identifies seven recurring error categories (apxml):

These fall into three broader buckets: format errors (unparsable output), tool selection errors (wrong tool for the task), and parameter errors (missing/invalid/malformed arguments) (apxml).

Error recovery patterns that work

The practical fix isn't trying to eliminate these errors upstream with better prompting alone — it's designing for graceful recovery:

• Return structured, typed error messages the model can parse and reason about, rather than a raw stack trace or a generic "error" string. Distinct, structured errors for distinct failure modes let the model self-correct on the next turn (apxml).
• Validate agent-generated arguments before executing any external action. Structured output gives you a typed contract, but a typed contract is not the same as a safe one — always validate before a tool call triggers a real side effect (payment, deletion, external API write) (apxml).
• Use schema-constrained tool wrappers that reject malformed calls before they reach your business logic, combined with runtime argument validation and prompt-time format enforcement (apxml).

def execute_tool_call(call):
    try:
        validated_args = ToolSchema.model_validate(call.arguments)
    except ValidationError as e:
        # Return a structured, parseable error — not a raw traceback
        return {"error": "invalid_arguments", "detail": str(e), "retryable": True}

    if call.name == "issue_refund" and validated_args.amount > MAX_AUTO_REFUND:
        return {"error": "requires_human_approval", "retryable": False}

    return run_tool(call.name, validated_args)

Provider differences that actually matter

The core mechanism (JSON Schema tool catalog in, structured call out) is consistent across OpenAI, Anthropic, and Gemini, but there are practical differences worth knowing:

• OpenAI's strict mode vs. parallel calling tradeoff (above) doesn't have a direct equivalent constraint in every provider — check current docs before assuming portability of a strict-schema design.
• Message-ordering conventions for tool results differ slightly between providers' APIs, so agent loops built against one provider's SDK generally need adjustment, not a straight swap, when porting to another.

Actionable takeaway

Get the schema right first — every required field actually marked required, every constrained string as an enum — since that alone eliminates a large share of invocation errors before you write any error-handling code. Then build recovery around structured, typed error responses rather than trying to prompt your way to zero failures; tool-use errors at the 2-5% rate are a normal, expected part of production LLM systems, not a sign your prompt is broken. Validate every agent-generated argument before it triggers a real-world side effect, full stop — that's the one rule with no shortcut.

Sources: FutureAGI: LLM Function Calling 2026, AI/TLDR: Tool Calling Best Practices for LLMs, apxml: Error Handling for LLM Agent Tools