loading…

Tool Calling Loop

Tools (also called functions) are the mechanism that turns an LLM from a text generator into an agent that can act on the world and get feedback from it.

The Loop

sequenceDiagram
    participant App as Application
    participant LLM as LLM
    App->>LLM: System prompt with tool definitions (name, description, JSON-schema args)
    App->>LLM: User message ("write a .gitignore file")
    LLM-->>App: Tool call message (id, tool name, arguments)
    App->>App: Execute tool locally (e.g. write file)
    App->>LLM: Tool result message (id, "wrote .gitignore successfully")
    LLM-->>App: Summary text of what was done

1. Specify tools in the system prompt: each is a name, a description, and JSON-schema-typed arguments. Nothing more — tool definitions are just extra prompt information.
2. Call: the LLM replies with a special tool-call message carrying an ID, the tool name, and the argument object. Nothing has happened yet in the world at this point.
3. Execute: the application runs its own code to perform the action.
4. Result: the application returns a tool-result message with the matching ID; the LLM follows with a human-facing summary.

A tool call is “really like a conversation with the LLM — it’s just the LLM communicating with the system that creates the file instead of communicating with us.”

Tool Budget Caution

Too many tools is actively detrimental: with 40+ tool definitions the context window drowns and selection accuracy drops (context rot, lost-in-the-middle). Many frameworks recommend staying under ~6 tools; issues can appear as low as 12. Tool-count discipline is therefore a first-class context engineering concern, not a nice-to-have.

Failure Modes

Symptom	Root cause	Fix
Agent calls tools in wrong order	No dependency awareness	Prompt declares tool preconditions explicitly
Infinite retry on failing tool	Error treated as transient	Cap retries; require a diagnosis after 2 failures
Hallucinated parameters	Schema ambiguity	Strict schemas; validate before execution
>6 tools confuse selection	Tool-choice accuracy degrades with count	Group or gate tools by task phase

Related

llm message protocol, model context protocol basics, context rot, agent harness engineering, generator evaluator loop.