loading…

LLM Message Protocol

Every interaction with an LLM is a message history: an ordered array of typed messages exchanged with a stateless model-provider endpoint. The LLM has no memory between requests — the entire conversation is resent each turn, which is why token costs and context rot compound with history length.

system   -> invariant instructions
user     -> task
assistant-> reasoning + tool calls
tool     -> results (append-only log, never reordered)

Message Types

1. System prompt — persistent instructions defining the assistant’s role and constraints. Tools are declared here as name + description + JSON-schema arguments (see tool calling loop).
2. User message — input from the human or calling application.
3. Assistant message — model output; either text, a tool call, or both.
4. Tool call / tool result — the executable half of the agentic loop (see tool calling loop).

Tokens are the unit of this protocol — the “currency” of LLM communication — and every message in the history is billed on every request. Understanding the protocol makes capabilities like structured outputs and streaming legible: they are conventions layered on top of the same message array.

Design Consequences

• Statelessness: conversation memory is an application-side concern (the model re-reads the full array each turn).
• History management: chatbot follow-up questions only work because prior turns are resent; trimming or summarizing history is a context-engineering act (see context engineering, handoff artifacts).
• Reasoning tokens: some models emit hidden intermediate reasoning within their turn, billed as output.

Failure Modes

Symptom	Root cause	Fix
Role confusion across turns	Ad-hoc message assembly	One canonical builder; roles typed at construction, not stringly-checked
Tool results leaking into wrong turns	Ordering bugs in the loop	Sequential message log; append-only; never reorder
Context bloat from protocol overhead	Full history resent every call	Prune by policy, keep the protocol invariant

Rule of Thumb

The protocol is the harness contract: if two components disagree on message shape, fix the protocol, not the call sites.

Related

• llm app improvement ladder — the ordered improvement techniques built on this protocol. tool calling loop, structured outputs, context engineering, smart zone, model provider abstraction.