Why AI Agents Need an Architecture
AI-agent architecture is the subject of this series, with a specific concern: how much autonomy should a system receive for a given task, and what must surround that autonomy so the system remains useful, observable, and able to stop? The practical judgment is not whether a language model can produce an impressive answer. It is whether a design can navigate an uncertain workflow without allowing its complexity, cost, or authority to outrun the problem.
That question matters because an agent is not simply a chatbot with a more confident tone. It is a system that can decide what to do next, select tools, inspect what those tools return, revise its approach, and continue until it reaches a defined stopping condition. Once software can act across customer records, databases, documents, or financial systems, architecture determines more than performance. It determines who or what controls the next decision, which state can change, what evidence remains after a run, and when a person gets control back.
From generating answers to executing workflows
Conventional automation is strongest when the path is known in advance. A developer can specify a sequence of conditions and actions, test its branches, and run it repeatedly. A single-turn language-model application has the opposite strength: it can interpret a request and generate a response without requiring every sentence to be anticipated. Neither description is yet enough for an agent.
An agent combines language-model reasoning with control over workflow execution. It receives a goal, forms a next step, uses a tool to obtain information or take an action, observes the result, and decides whether another step is necessary. It can recognize completion, correct an unsuccessful attempt, halt after a failure, or transfer control when the task reaches a boundary it should not cross. The supplied guides define this ability to act on a user's behalf, through dynamically selected tools and within guardrails, as the distinguishing feature of an agent. A classifier that labels a message, or a chatbot that answers without directing the workflow, may be useful software, but it is not an agent in this sense.
The distinction is consequential because tool use makes the model part of a larger operating system. A model may propose a refund, but a refund tool changes an account. It may summarize a support issue, but a ticket-routing tool changes who owns the case. It may search a document repository, but the quality and permissions of that repository shape what the agent can know. The model supplies flexible judgment; the surrounding interfaces supply access and consequence. Treating the model as the whole system hides the boundary where a plausible sentence becomes an external action.
The workflow is therefore not just a conversation stretched over several turns. It has state. A request may begin with an unresolved question, acquire information from a customer record, pass through a policy check, and end with either a recommendation or a changed record. At each transition, the design must preserve enough context for the next decision without silently broadening the agent's authority. The difference between reading an account and altering it is architectural; so is the difference between drafting an escalation and sending one. The more consequential the transition, the less acceptable it is to rely on an implicit assumption that the model will know when to stop.
This loop also explains why an agent cannot be judged only by the quality of its final prose. A system may produce a polished answer after consulting the wrong record, using stale context, or taking an action it was not authorized to take. A trustworthy design must make the path to the answer—and the exits from that path—part of the subject.
That path has at least two audiences. The user needs a useful result, but the people responsible for the system need to know what the agent attempted, what it observed, and why it stopped. A final response can conceal a failed lookup, an unnecessary retry, or a handoff that never reached the right team. This is why an agent's architecture includes ownership and evidence, not merely a prompt and a model. The system must make it possible to tell the difference between a correct decision reached efficiently, a lucky answer reached through an unsafe route, and an incomplete task that should have been handed back.
Where autonomy adds value
The best reason to build an agent is not that the task is fashionable or that a model is available. It is that a deterministic approach encounters a kind of uncertainty that matters to the outcome. The supplied guidance points to three recurring conditions: the work requires nuanced judgment or exception handling; a ruleset has become difficult to maintain; or the system must interpret natural language and unstructured material.
Payment-fraud analysis makes the difference clear. A rules engine can flag a transaction when a known combination of conditions appears. That is valuable when the conditions are stable and the desired response is clear. An agent can instead examine surrounding context, compare less obvious signals, and investigate documents or account history when the suspicious pattern does not fit a preset checklist. That flexibility may help with ambiguous cases, but it also means the system needs stronger boundaries. If the workflow is simple, stable, and fully expressible as rules, an agent may add latency, cost, and uncertainty without adding corresponding value.
The same reasoning applies outside fraud. Incident response can require a system to gather signals from several sources, test hypotheses, and adapt when an initial explanation fails. Customer onboarding can involve incomplete information and different paths through internal systems. Development workflows can use automated tests as feedback, allowing an agent to revise a change rather than follow one predetermined sequence. In each case, the value comes from navigating a path that cannot be completely specified beforehand—not from replacing every fixed step with a model call.
There are several kinds of uncertainty here, and they should not be collapsed into one argument for autonomy. The facts may be incomplete, the correct sequence may depend on what a tool returns, or the policy may require judgment at an exception. A workflow can also be large without being uncertain: a long but stable series of transformations may be better represented as ordinary software. Conversely, a short workflow can be high-stakes if one decision changes money, access, or a person's case. Architecture must therefore consider both how hard the path is to predict and how costly a wrong transition would be. Those dimensions explain why an agent can be valuable in one part of a process and inappropriate in the next.
Production examples in the Anthropic guide show what this value can look like when an organization connects autonomy to a real operating constraint. The guide reports that Coinbase uses Claude-powered agents for customer support at the scale of thousands of messages per hour and cites 99.99% availability for that platform. It reports 80–90% resolution rates for a financial-services customer-support agent from Gradient Labs, and a 20-fold reduction in fraud-review time in an Inscribe example. These figures are useful as illustrations of the outcomes teams choose to measure: availability, resolution, turnaround time, and human effort.
They are not portable guarantees. The numbers are vendor and customer case-study claims tied to particular systems, baselines, workloads, and operating practices. The supplied material does not provide independent evaluation methods, control groups, full cost accounting, or comparable failure rates. A careful architecture therefore treats such outcomes as hypotheses to test locally. The meaningful question is not whether an agent can claim a high resolution rate somewhere else. It is whether the proposed design improves the relevant outcome while preserving acceptable error, cost, latency, and control in this workflow.
The smallest architecture that fits the problem
This leads to the central principle: choose the smallest architecture that can handle the task's uncertainty while preserving a credible path to oversight. “Smallest” does not mean simplistic. It means that every additional agent, handoff, parallel branch, memory mechanism, or evaluator has to answer a specific need. Complexity should be a response to a demonstrated limit, not a proxy for ambition.
A useful starting assessment asks four questions. How much control and traceability does the work require? How broad and unpredictable is the problem domain? What do latency, token, staffing, and budget constraints permit? Does the workflow need one coherent expertise or coordination among genuinely distinct domains? A high-control financial or compliance process may be better served by a single agent with explicit tools or by a structured sequence. An open-ended research problem that spans independent domains may justify more distributed work. A single domain with a stable operating procedure may need a specialized capability, not a team of agents.
These questions also prevent a misleading comparison between architectures that solve different problems. A central manager may improve synthesis but add another decision-maker and another context boundary. A parallel design may shorten elapsed time only when its work is independent and its results can be reconciled. A simple agent may be slower on one difficult case but easier to audit across thousands of ordinary ones. The decision is consequently a systems judgment: weigh flexibility against predictability, specialization against coordination overhead, and possible benefit against the cost of explaining and containing failure.
These choices are not permanent. A system can begin with one agent handling customer inquiries, add routing when categories become meaningfully different, introduce specialists when one context becomes unwieldy, and escalate only unusual cases to a more expensive coordinated design. The important property is not that the system eventually becomes multi-agent. It is that the transition is tied to observed performance, context limits, tool confusion, or a real need for parallel expertise. Emerging designs such as runtime-generated agents or peer-to-peer networks remain conditional in the supplied evidence; their promise does not remove the need to establish a simpler baseline first.
Safe exit belongs in the first design, not at the end of this progression. An agent should know what counts as completion, how many retries or actions are acceptable, which operations require review, and what happens when it cannot resolve the task. A system that can cancel an order, authorize a large refund, or make a payment needs a different boundary from one that only retrieves information. Returning control to a person is not an admission that the architecture failed. It is a way to contain uncertainty, protect the user, and create evidence for improving the system.
The architecture question can now be stated plainly: what must be allowed to adapt, and what must remain explicit? The model may adapt the route through an ambiguous task. Tools should make access and consequence visible. Instructions should define the operating boundary. Evaluation should show whether the design works. Stopping conditions should prevent persistence from becoming recklessness. The next step is to examine those smallest components and see how they turn a capable model into a controlled workflow executor.