On this page
What an AI agent actually is
An AI agent is a program in which a language model chooses the next action from a fixed set of tools, executes it, reads the result, and repeats until a stop condition fires. The defining property is that the model decides the order of operations at run time and your code does not. Everything people find hard about agents follows from that: you cannot test a path you never wrote, and you cannot price a run whose length you do not know in advance.
That rules out most of what is sold as an agent. A prompt that answers a question is not one, because nothing happens after the answer. Three prompts in sequence are not one, because a person decided the sequence. A classifier routing tickets into five queues is not one either, because the branch set is finite and written down. None of those are worse than an agent. Choosing between the shapes is the subject of agent or workflow.
Two things sold as agents are worth naming. The single-tool agent is one prompt, one tool, called once, in a loop that never runs twice: that is a function call with extra latency, so write it as a function. The other is the multi-agent system that is really a for loop over three prompts with no supervisor and no shared state, which buys three places for the context to be wrong.
- AI agent
- A system in which a language model selects and executes actions from a defined tool set in a loop, using the result of each action to choose the next, until a stop condition is reached.
- Tool call
- A structured request from the model naming one function and its arguments. The model does not run code. It asks your program to run code, and its arguments are unvalidated until you validate them.
- Turn
- One pass through the agent loop: one model call, zero or more tool calls, and the results appended to the message history that the next model call reads in full.
- Stop condition
- The rule that ends the loop. Four kinds exist: a completion signal from the model, a turn cap, a cost ceiling, and a wall clock timeout. A production agent needs all four, because each catches a failure the others miss.
- Trace
- The ordered record of every prompt, tool call, argument, result and error in one run, stored so a human can reconstruct why the agent acted that way months later.
- 1The property that makes it an agent: the model, not your code, picks the next step.
- 4Stop conditions a production agent needs: completion signal, turn cap, cost cap, wall clock.
- n(n+1)/2How billed input tokens grow across n turns, since the whole history is resent each turn.
- 3Layers where a run can fail: the model's choice, the tool's execution, the loop's bookkeeping.
The loop, one turn at a time
The loop itself is about forty lines of code. The difficulty is deciding what happens at the six points where a turn can go sideways, and most teams decide four of them.
- Assemble the context
Policy prompt, tool schemas, the whole message history so far, plus anything retrieved. You construct this payload and send it again on every turn, which is where the cost curve comes from.
- Call the model with the tool schemas attached
It returns text, or one or more tool calls, or both. Handle two calls at once, because it will do that, and a loop reading only the first silently drops half the plan.
- Validate the call before executing it
Four checks: the tool name exists, the arguments parse against the schema, every identifier in them appeared in an earlier tool result this run, and the action sits inside the permitted scope. Models hand over well formed order IDs that never existed.
- Execute and capture the result verbatim
Return the real error, not a generic string. An error saying the date range is capped at 90 days lets the model fix itself next turn. An error saying request failed guarantees a repeat.
- Append the result and decide whether to continue
Increment the turn count, add token spend to the ledger, check elapsed wall clock, and check whether this exact tool with these exact arguments has already been called. That last check is cheap and catches the commonest runaway.
- Terminate, then verify the claim
A model saying the task is complete is a claim, not evidence. Query the record independently and confirm the world changed. If no verification query appears in the trace, the run did not succeed.
There is no hidden memory between turns. What the agent knows on turn seven is exactly what was in the message list you sent on turn seven. Truncate that list to save money and you have handed it selective amnesia, and it will redo work it already finished. Choosing what to keep is the whole of agent memory design.
The four dials of agency
Agent is not a binary. It is a region in a space with four dials, and the useful move in a design review is to state the dial settings rather than argue about the noun. Every question about cost and risk attaches to exactly one dial, and each turns independently.
The Four Dials of Agency
Describe any proposed system with these four numbers before anyone says the word agent. The settings, not the label, predict cost and blast radius.
Count the decisions per run made by the model instead of your code. One is a classifier. Twenty is an agent. That count predicts your variance and how much review time goes on reading traces.
List every tool and split it into reads and writes. Reads are recoverable. Writes are not, unless you built the undo, and almost nobody builds the undo. Reach decides how bad a bad run can be, and it is the first dial to turn down when you are nervous.
Turns between a human seeing the input and a human seeing the output. A horizon of twenty means the system acted nineteen times before anyone looked. Errors compound along the horizon, because turn eight reads turn seven's mistake as established fact.
Three options exist: retry, escalate, roll back. Most teams build retry, half build escalate, almost nobody builds roll back. Decide per tool, in writing, before the agent touches production data rather than during the incident.
The most underused setting is Choice high and Reach low. Let the model plan freely against read-only tools, then have it emit a proposed action list that a deterministic executor or a human carries out. The irreversible part stays in code you can read, and a proposed plan is far easier to evaluate than a side effect.
The parts you actually have to build
The model is the smallest part of an agent. Seven other components decide whether it survives production, and every one is ordinary software your team already knows how to write. That is why schedules slip: the demo needs two of these rows and the system needs all seven.
| Component | What it does | What happens when you skip it |
|---|---|---|
| Policy prompt | States the job, the boundaries and the stop rules in one place | The agent invents its own scope and does work nobody asked for |
| Tool schema layer | Declares each tool's name, arguments and description | The model guesses arguments, and guesses fail deep inside a write |
| Validator | Checks arguments, identifier provenance and permissions first | An invented ID reaches your database as a genuine update |
| Budget ledger | Counts turns, tokens, currency and wall clock per run | One pathological run costs more than a normal month |
| Trace store | Records every prompt, call, argument, result and error | Nobody can answer why it did that, including whoever built it |
| Output contract | Validates the final result against a schema before it leaves | Downstream systems get prose where they expected JSON |
| Escalation path | Routes a stuck or low confidence run to a named human | The run fails quietly until a customer finds out |
A demo needs the policy prompt and the tools. Production needs the other five, plus retries, plus the ability to replay a failed run against a new prompt version. A proposal that prices the demo and calls it the build delivers the rest as change requests later.
The policy prompt an agent actually needs
An agent's system prompt is not a personality. It is an operating policy, and it should read like a runbook for a competent contractor who cannot phone you with a question. Every line below exists because of a specific failure.
ROLE
You are the [order operations] agent for [company]. You finish one task per run,
then you stop. You do not do adjacent work that nobody asked for.
WHAT YOU MAY ASSUME
Nothing. Every identifier you put in a tool call must have appeared in a tool
result earlier in this run. If you need an ID you do not have, call the lookup
tool. Never construct or guess an identifier.
ORDER OF WORK
1. Restate the task in one sentence, naming the record you will act on.
2. Gather facts with read-only tools until you can state the action.
3. Before any write, state the write, the record, and the result you expect.
4. Call the tool, read the result, confirm it matches what you predicted.
5. Emit the final JSON and stop.
STOP AND ESCALATE IF
- A read tool returns empty twice for the same query.
- A write tool returns an error you have already seen in this run.
- The task needs an action that is not in your tool list.
- The record is flagged [vip] or the amount is over [threshold].
- You have used [8] tool calls without a write that succeeded.
Escalating is not a failure. Emit the escalation JSON with your reason.
FORBIDDEN
- Retrying a failed call with identical arguments. Change something, or escalate.
- Reporting a task complete unless a tool result confirms the change.
- Summarising a tool error. Quote it.
OUTPUT
Return only this JSON and no prose:
{"status":"done|escalated","record_id":"...","action_taken":"...",
"evidence":"verbatim tool result proving the change","notes":"..."}Four lines are load-bearing and the rest is scaffolding.
- Every identifier must have appeared in a tool result. This removes the largest single class of agent bug, the confidently invented ID. Pair it with a validator enforcing the same rule in code, because a prompt is a preference and a validator is a guarantee.
- Do not retry with identical arguments. Without it, a model that hits an error apologises, repeats the call unchanged, apologises again, and burns the turn budget being polite.
- Escalation framed as a success. A model steered hard toward finishing will fabricate a completion rather than admit it is stuck.
- The evidence field. Making the model quote the tool result that proves the change turns silent success into something a validator catches, because a run that changed nothing has nothing to quote.
How the loop breaks
Agent failures cluster into six recognisable shapes. A named shape is something you can write a test for, and most teams only test the version where everything works.
- The invented identifier. The agent needs an order number, does not have one, and produces something that matches the format perfectly and refers to nothing. Caught by provenance checks, not by asking the model to be careful.
- The apology loop. A tool errors, the model apologises, then issues the same call with the same arguments. Caught by hashing tool name plus arguments per run and refusing a repeat.
- Silent success. The agent reports the job done having called only read tools. Nothing changed anywhere. Caught by independent verification after termination, never by reading the final message.
- Context saturation. By turn twelve the history is mostly stale tool output and the instruction that mattered was on turn one. Behaviour drifts toward whatever sits nearest the end.
- The partial write. The agent updates the CRM, then fails before billing. No transaction spans two APIs, so you need a compensating action or an approval gate before the first write. Choose which before launch.
- Scope creep by helpfulness. Asked to refund one line item, it decides the whole order looks wrong and refunds all of it. Prevented by narrowing Reach, not by adding a sentence to the prompt.
The other five are loud. They throw, they time out, they burn turns, and monitoring sees them. Silent success looks like a green run and reaches the customer. The fix is structural: require the verbatim evidence quote, verify the record independently after the loop ends, and treat any run whose trace holds no verification query as failed. More in how agents fail.
What one run actually costs
An agent run does not cost one model call. It costs the sum of every message history you resend, and that history grows each turn, so billed input scales with roughly the square of the turn count rather than linearly with it.
On turn one you send the policy prompt and the tool schemas. On turn two you send those again plus turn one's call and result. By turn ten you are sending nine turns of history for the tenth time. With a preamble of b tokens and t added per turn, billed input across n turns is about n times b, plus t times n times n plus one, over two.
The two price fields are empty slots, not quoted rates. Put your provider's current per-million prices in before reading the outputs, because those move constantly.
Two conclusions follow. A turn cap is a cost control before it is a safety feature, and it should come from your numbers rather than a tutorial. And the cheapest optimisation is rarely a cheaper model: a tool handing back four thousand tokens of JSON when the agent needed three fields pays that tax again on every remaining turn. Return shape is covered in designing tools an agent can use.
How to tell whether you need one
You need an agent when the sequence of steps genuinely cannot be written down in advance, and when you can afford to be wrong sometimes. Both halves matter. Plenty of processes satisfy the first and fail the second.
If most of that list is already answered, the build is a normal software project with an unusual component in the middle. If none of it is, the honest next step is a week of measurement rather than a sprint of prototyping. That is what AI agent development starts with.
Questions readers ask next
Is an AI agent the same thing as a chatbot?
How many tools should an agent have?
Does an agent need a large context window to work properly?
Can an agent run without a human reviewing its output?
What is the difference between an agent and a multi-agent system?
How long does it take to build a production agent?
ChatGPTalker. "What an AI Agent Actually Is, Minus the Hype." chatgptalker.com, 2026-08-26. https://chatgptalker.com/guides/what-an-ai-agent-actually-is/