---
title: "LLM Message Protocol"
description: "The message-based conversation protocol between application and LLM: system prompts, user/assistant messages, tool calls, and tool results."
section: "concepts"
type: "concept"
created: "2026-08-30"
updated: "2026-08-30"
confidence: "high"
tags: ["agents", "context-engineering", "llm-fundamentals"]
canonical: "https://pyweb.dev/wiki/llm-message-protocol"
---
# LLM Message Protocol

Every interaction with an LLM is a **message history**: an ordered array of typed messages exchanged with a stateless model-provider endpoint. The LLM has no memory between requests — the entire conversation is resent each turn, which is why token costs and [context rot](/wiki/context-rot) compound with history length.


```text
system   -> invariant instructions
user     -> task
assistant-> reasoning + tool calls
tool     -> results (append-only log, never reordered)
```

## Message Types
1. **System prompt** — persistent instructions defining the assistant's role and constraints. Tools are declared here as name + description + JSON-schema arguments (see [tool calling loop](/wiki/tool-calling-loop)).
2. **User message** — input from the human or calling application.
3. **Assistant message** — model output; either text, a tool call, or both.
4. **Tool call / tool result** — the executable half of the agentic loop (see [tool calling loop](/wiki/tool-calling-loop)).

Tokens are the unit of this protocol — the "currency" of LLM communication — and every message in the history is billed on every request. Understanding the protocol makes capabilities like structured outputs and streaming legible: they are conventions layered on top of the same message array.

## Design Consequences
- **Statelessness:** conversation memory is an application-side concern (the model re-reads the full array each turn).
- **History management:** chatbot follow-up questions only work because prior turns are resent; trimming or summarizing history is a context-engineering act (see [context engineering](/wiki/context-engineering), [handoff artifacts](/wiki/handoff-artifacts)).
- **Reasoning tokens:** some models emit hidden intermediate reasoning within their turn, billed as output.

## Failure Modes

| Symptom | Root cause | Fix |
|---|---|---|
| Role confusion across turns | Ad-hoc message assembly | One canonical builder; roles typed at construction, not stringly-checked |
| Tool results leaking into wrong turns | Ordering bugs in the loop | Sequential message log; append-only; never reorder |
| Context bloat from protocol overhead | Full history resent every call | Prune by policy, keep the protocol invariant |

## Rule of Thumb

The protocol is the harness contract: if two components disagree on
message shape, fix the protocol, not the call sites.

## Related
- [llm app improvement ladder](/wiki/llm-app-improvement-ladder) — the ordered improvement techniques built on this protocol.
[tool calling loop](/wiki/tool-calling-loop), [structured outputs](/wiki/structured-outputs), [context engineering](/wiki/context-engineering), [smart zone](/wiki/smart-zone), [model provider abstraction](/wiki/model-provider-abstraction).

---

## Agent Navigation

cluster: llm-fundamentals (8 pages) | betweenness: 347.7

### References (outbound)
- [Context Rot](https://pyweb.dev/wiki/context-rot.md)
- [Tool Calling Loop](https://pyweb.dev/wiki/tool-calling-loop.md)
- [Context Engineering](https://pyweb.dev/wiki/context-engineering.md)
- [LLM App Improvement Ladder](https://pyweb.dev/wiki/llm-app-improvement-ladder.md)

### Referenced by (inbound)
- [Model Provider Abstraction](https://pyweb.dev/wiki/model-provider-abstraction.md)
- [Structured Outputs](https://pyweb.dev/wiki/structured-outputs.md)
- [Think Tool](https://pyweb.dev/wiki/think-tool.md)
- [Tool Calling Loop](https://pyweb.dev/wiki/tool-calling-loop.md)
- [Matt Pocock](https://pyweb.dev/wiki/matt-pocock.md)
- [Vercel](https://pyweb.dev/wiki/vercel.md)

### Evidence (verified primary sources)
- raw/aihero-video/e00k46QugaOBLd4ZrIj3hR8DFSm01xZL2WhSAym3xNoa00.md (internal workspace doc)
- raw/aihero-video/ogTZkRIjv00qmTh6B8MpqpSdKOVl2lYQHUcZOohLg007o.md (internal workspace doc)
- [aihero-ai-coding-dictionary-2026](https://pyweb.dev/wiki/raw/articles/aihero-ai-coding-dictionary-2026.md) | origin: https://www.aihero.dev/ai-coding-dictionary | ingested: 2026-08-27 | sha256: 52b0a5da7c9f6dd60dfdbf29031bcf6107c2765b6ef5b3a7f96cc607f42b06c7

### Machine endpoints
- Knowledge graph: https://pyweb.dev/api/graph.json
- Graph analysis: https://pyweb.dev/api/graph-analysis.json
- Context index: https://pyweb.dev/llms.txt