An AI agent is a software system that is given a goal, decides for itself which tools to call to reach it, and keeps calling them in a loop until it finishes or hits a limit you set. The defining feature is not that it uses a language model. It is that the model, rather than your code, chooses the next step.
Anthropic draws the same line, and the wording is worth borrowing because it is precise:
Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. (Verified July 2026. Source: Anthropic, Building effective agents.)
That distinction is not academic. The two things cost different amounts to build, fail in different ways, and need different kinds of supervision.
How is an agent different from automation, a chatbot, or a single AI call?
The short answer: by who decides the next step, and by what happens when the situation is not the one anybody planned for.
| What it is | What decides the next step | What it can touch | How it typically fails |
|---|---|---|---|
| Fixed automation (a Zap, an n8n flow, a cron job) | Rules a person wrote in advance | Exactly what was wired up | Stops, or does the wrong thing quietly, when reality does not match the rule |
| A single model call | Nothing. One input, one output | Nothing outside its prompt | Answers fluently from whatever context it was given, including stale context |
| A chatbot | The user's next message | Usually a knowledge base, sometimes a lookup | Talks well and changes nothing in your systems |
| An AI agent | The model, at each step, based on what the last step returned | The tools you granted, within the permissions you set | Chooses wrong and keeps going, quickly |
The chatbot row is the one that causes the most confusion in buying conversations, and it has enough detail to deserve its own piece: see AI agent vs chatbot.
What is an AI agent made of?
Five parts. Any vendor description that omits the last two is describing a demo, not a system you can run.
- A model. The component that decides. Everything it knows in the moment has to fit inside its context window, which is a hard, published number: Claude Opus 5 and Claude Sonnet 5 carry 1M-token context windows and Claude Haiku 4.5 carries 200k. (Verified July 2026, Anthropic models overview.) Context is not memory. It is working space, and it is finite.
- Tools. Functions you describe to the model so it can act on real systems. The mechanism is a round trip: the model returns a tool call and stops, your code executes that call against your CRM or your accounting system, and you hand the result back so the model can continue. Some tools instead run on the model vendor's infrastructure, such as web search. (Verified July 2026, Anthropic tool use documentation.) A model with no tools cannot change anything. Tools are where an agent stops being a conversation.
- Memory. The context window empties between runs. Anything that has to survive, a customer's stated preference, a decision made last Tuesday, a note that this supplier always invoices in a different currency, has to be written somewhere you control and read back deliberately.
- The loop. Decide, act, read the result, decide again. Anthropic's guidance on this is worth quoting because it is the part demos skip: it is crucial for agents to gain ground truth from the environment at each step, and it is common to include stopping conditions such as a maximum number of iterations. (Verified July 2026.) A loop with no stopping condition is not autonomy, it is a runaway process with a billing account.
- Guardrails. Which tools it may call without asking, which actions need a human to approve, what it does when it is unsure, and a log you can read afterwards to find out what it did and why.
What does an AI agent look like in a business?
Two examples, both ordinary, both written in the conditional because they describe the shape of a build rather than a named client.
Supplier invoice matching. An invoice arrives in a shared inbox. The agent reads it, pulls the supplier and the line items, looks up the matching purchase order in the accounting system, and compares them. If everything reconciles within a variance threshold you set, it files the invoice against the order. If the invoice says Freight and the purchase order says Delivery, that is a judgment: a rule cannot say whether they are the same line, but a system that can read both and check the amounts can form a view. If it cannot, or if the variance is over the threshold, it routes the invoice to a person with both documents lined up and its own reasoning attached. It never pays anything.
Inbound enquiry handling. A message arrives on WhatsApp asking whether a unit is still available. The agent looks the unit up in the listing system, answers from that record rather than from anything it remembers, offers the two viewing slots that are actually free in the calendar, and books one. The moment the conversation turns into a price negotiation, it hands over to a person with the full transcript. The handover is the design, not an admission of failure.
In both cases the agent's value sits in one specific judgment that nobody could write down as a rule, wrapped in a lot of ordinary integration work. That ratio is typical.
What are AI agents still bad at?
Four things, and none of them are close to solved.
Knowing when they are wrong. A model that has the wrong answer states it with exactly the same confidence as the right one. There is no internal signal you can read. This is why the agents that work in production are built around outputs that can be checked against something, a total that must reconcile, a record that must exist, a schema that must validate, and why the ones built around unverifiable prose stay in the pilot phase.
Long chains with nothing to check against. Errors compound. The arithmetic is illustrative rather than measured, but it makes the shape clear: if each step is independently right 95% of the time, twenty steps in sequence are all right about 36% of the time. Anthropic names the same risk directly, calling out compounding errors as a cost of autonomy. (Verified July 2026.) The practical response is fewer steps, checkpoints between them, and ground truth read back from the environment rather than assumed.
Anything unrecoverable. Sending money, sending a message to a customer under your brand, deleting a record, filing something with a regulator. Not because a model cannot form the action, but because the recovery cost sets the supervision level. If you cannot undo it, a person approves it.
Work that is already fully specified. If the process is high volume, stable, and every branch is known, an agent is the expensive way to do it. A rule is cheaper, faster, and behaves the same way every time. Anthropic's own guidance opens on this point: for many applications, optimizing single model calls with retrieval and good examples is usually enough. (Verified July 2026.) That is a vendor telling you to buy less, and it is correct.
Why autonomous is oversold
The word implies a system that runs unattended and figures things out. What ships in production is narrower: a small set of tools, a permission policy on the ones that cannot be undone, an escalation path to a named human, and a hard cap on iterations. Anthropic's guidance recommends both the stopping conditions and pausing for human feedback at checkpoints. (Verified July 2026.) Every serious agent framework ships an approval mechanism, which tells you what the people building them actually expect.
There is also a cost story that marketing tends to leave out. Agentic systems trade latency and cost for task performance, in Anthropic's phrasing, and that trade is only worth making on tasks where the performance matters. An agent that reasons its way to an answer a lookup table already had is a slower, more expensive lookup table.
The honest framing is that autonomy is a dial. Where you set it is a commercial decision about what an error costs you, not a technical achievement to be maximised.
How to decide whether you need one
Two questions settle it, and neither requires a vendor. First: in this process, which single decision cannot be written down as a rule? Second: what should happen when the system is not sure about that decision, and who owns the queue it goes into? If you can answer both, you have the beginnings of a specification. If the answer to the first is that every decision can be written down, build the rule, keep the money, and revisit this in a year.
If the answer is that you do need one, the next question is what it takes to make it work: the tools, the guardrails, the evaluation set, and the escalation path are all covered in how to build an AI agent step by step.
If you can answer both and want the decision costed rather than pitched, that is what our AI agent development service exists for.