Inside One Agentao Session: From Prompt to Tool to Reply
Agentao’s most important unit is not the individual API call but the stateful session that surrounds it. This chapter follows one request through construction, prompt assembly, model iterations, tool execution, host decisions, persistence, and cleanup. The goal is practical: once you can see what changes at each transition, you can decide what a host must serialize, restore, time out, or close instead of treating an agent as a stateless text endpoint.
A turn is a loop, not a function call
A session begins when the host constructs an Agentao instance. That object binds a conversation, a tool registry, a memory manager, a skill manager, and a working directory. The working directory is more than a convenience for file paths. It is the root against which file and shell operations, project instructions, skills, memory, and logs are resolved. The practical consequence is immediate: two concurrent users should not casually share one instance or allow both instances to fall back to the same process-wide directory.
Construction is not the same as a turn. The normal lifecycle is to create one instance, call chat() repeatedly for that conversation, and call close() when the session ends. Rebuilding the object for every message may appear simple, but it discards the in-memory message history and any active skill state. Keeping one object alive preserves continuity, but it creates obligations around concurrency and resource cleanup that a plain request/response function does not have.
When the host submits a user message, Agentao rebuilds the system prompt for that turn. The prompt combines project instructions, the runtime’s capability description, the current skill catalog and active skills, memory recall, task state, and other runtime context. This rebuilding is intentional. The date, available capabilities, active skills, and recalled memories can change; they should not be fossilized inside a transcript. The conversation history is one input to the next call, not the complete definition of the current runtime.
The language model then has two broad ways to move the turn forward. It can emit a visible response, or it can propose a tool call. In the second case, Agentao normalizes the call, checks the tool registry, and routes the invocation through the host/runtime policy path. A Transport can emit events for streamed text, tool start, tool output, and completion. If the call requires a decision, the same Transport can synchronously ask the host whether to continue. The host’s answer is not a side note: until it arrives, the agent loop is waiting.
If the tool runs, its result is added to the conversation as evidence for another model iteration. The model may use that result to call a second tool, revise its plan, or answer the user. This is why a tool-using agent is better understood as a feedback loop than as “one prompt plus one completion.” The model proposes; the runtime executes or refuses; the resulting observation changes the next proposal.
Consider a project-management request: “Find the overdue work and create a follow-up task.” The first model response might call a read-only project lookup. Agentao emits the tool-start and output events while the host keeps the user informed. The result then returns to the model as part of the turn, not as a hidden side effect. If the model proposes creating a task, that second call has a different consequence. The host may require confirmation, and the business tool should still validate the project and user on the server side. If the user rejects it, the runtime can return a cancellation result to the model; the model may explain that it found the overdue work but did not create anything. The visible answer is therefore the end of a sequence of state transitions, not evidence that every proposed action happened.
The sequence also clarifies what Transport is doing. Streaming text and tool progress are observations that can be forwarded to a browser or terminal. Confirmation and user questions are decisions that hold the loop until the host answers. They have different failure modes: a dropped event can make the interface less informative, while a confirmation callback that never returns can leave the agent unable to proceed. A host should keep those paths separate in its design, even when one callback object carries both. It should also treat tool results as data from an external capability, not as new authority. A page, file, or business API response can inform the next model step; it does not automatically rewrite the host's authorization policy.
The loop can produce several legitimate endings. The model can answer after no tool call, answer after one or more tool results, stop because the host rejected an action, or reach a configured iteration boundary. Cancellation can end the turn while leaving the session available for a later request, depending on how the host handles the signal. These endings may all return text, but they do not carry the same operational meaning. A UI can show a reply; an audit record should retain whether the turn completed, was cancelled, or stopped without a complete answer.
The diagram contains a small but important asymmetry. The model is inside the loop, but it does not directly call the business system. Agentao mediates the call, Transport gives the host a decision point, and the tool implementation remains responsible for its own argument checks and backend behavior. A refusal is also part of the loop: the model receives a result explaining that an action did not run and can adapt or stop.
The loop has a boundary. chat() accepts a maximum number of tool iterations, and the Transport can decide whether to stop, continue, or inject an instruction when that limit is reached. Cancellation can interrupt an in-flight turn. These controls matter because a tool sequence that never converges is not merely inefficient; it can consume tokens, hold a user request open, and leave the host unsure whether the final text represents a completed task or a partial one. A production host should preserve that distinction in its own status and audit model rather than labeling every returned string “success.”
State changes on different clocks
The most common persistence mistake is to treat everything associated with an agent as one kind of history. Agentao’s documentation separates several state buckets because they have different lifetimes and owners.
Conversation messages are the turn-by-turn record in agent.messages. They include user messages, assistant messages, tool-call requests, matching tool results, and the final assistant response. If the host wants the conversation to survive process termination, it must serialize this list or an equivalent representation. The tool-call pairs are structurally important: an assistant message that requests a tool and the corresponding tool result must be retained together. Dropping the result because it looks like implementation noise can make the next provider request invalid or remove the evidence the model needs to understand what has already happened.
The system prompt does not belong in that persisted list. Agentao rebuilds it from the current working directory, runtime configuration, active skills, memory recall, and task state. Persisting the old prompt would preserve stale instructions and dates while missing changes to the tool schema or skill catalog. A restore operation should recreate the agent with the intended working directory and configuration, then replay the conversation messages into the new instance before calling chat() again.
Persistent memory is different from conversation history. Agentao documents project-scoped and user-scoped SQLite stores for facts, preferences, and rules that can be recalled across sessions. The runtime can place relevant memories into the prompt and can expose a write path for the model, but the model is not given unrestricted management of the entire memory library. This separation is useful: a user preference should not need to be repeated in every conversation, while a transcript of the current task should not automatically become a permanent fact.
Memory also has a different reset behavior. Clearing conversation history does not clear persistent memory. That is intentional for a “new conversation” action, but it means a product that promises “forget everything” must invoke explicit memory-clearing and retention logic as well. In a multi-tenant host, the distinction becomes a privacy boundary. A stable project directory can isolate project memory, while a process-global user store may require disabling or re-keying so that one tenant’s memories are never recalled for another.
Session summaries are another form of durable state. When the context becomes too large, the runtime can compress older conversation material and store a summary. The summary is not a replacement for every original message in the host’s persistence model; it is part of the mechanism that keeps a long session usable. The important design principle is that compression changes what the model sees next, so a host should observe or record the event without trying to treat the summary table as an application-level conversation API.
Active skills live in the running skill manager rather than in the conversation list. If a host restores a session into a fresh instance, it should restore the active skill set as well as the messages. Otherwise the transcript may say that a workflow was underway while the new runtime lacks the instructions that shaped the earlier turns. This is one reason the host’s persistence record may need both conversation data and a small amount of runtime state.
These buckets create three different restore situations. A warm session needs no message replay: the host can serialize the latest turn for durability while the live object continues with its in-memory state. A restarted session needs a new Agentao object, the same working-directory identity, the saved messages, and any active skills that are part of the conversation’s meaning. A deliberately new conversation may reuse the same project memory while starting with an empty message list. Treating all three as “load the transcript” either loses useful memory or carries more context forward than the user intended.
There is also a difference between restoring state and trusting it. The host should verify that the session belongs to the requesting tenant or user before replaying it, and should rebuild tools with the current authorization context rather than restoring executable objects from storage. It should preserve the message structure needed by the model, but it should not let old conversation text silently grant a capability that the current policy no longer allows. This is the practical meaning of rebuilding the prompt and tool registry on a new turn: continuity is preserved without making yesterday’s runtime configuration an unreviewed authority token.
The working directory and external processes form a further practical layer. Files created by tools, logs, MCP connections, and project-local configuration can outlive the Python object even when the in-memory conversation does not. That persistence is helpful for continuity, but it also means a session directory should be scoped deliberately and cleaned or retained according to the product’s privacy and recovery policy.
Concurrency and cleanup are part of correctness
An Agentao instance is not thread-safe. Two requests that call chat() concurrently on the same object can interleave messages, tool results, and host decisions. The safe pattern for an asynchronous web host is to keep one instance per session, run the blocking chat call in a worker thread, and guard that instance with a per-session lock. Different sessions can proceed independently; turns within one session must be serialized.
That rule is easy to violate when a session pool is introduced. A pool keyed only by a short session identifier can accidentally reuse an agent for the wrong tenant or workspace. The key must include the identity boundary the application actually enforces, and the pooled object must retain the working directory and tool bindings with which it was created. Replacing a Transport callback for a new request does not make it safe to swap the underlying session between users.
A useful mental test is to imagine two requests arriving at the same moment for one chat session. The first may be waiting for a tool result while the second appends a new user message. Without serialization, the next LLM call can see a history in which the assistant’s tool request belongs to one user turn and the tool result belongs to another. Even if the provider accepts the payload, the result can be semantically wrong. A per-session lock is therefore not merely a performance choice; it preserves the ordering that gives messages and tool calls their meaning. The host can still run different sessions concurrently, because their working directories, message lists, and policy decisions are separate.
A pool also needs an eviction story. Keeping every live Agentao object forever preserves context but retains memory, open databases, event subscriptions, and any MCP subprocesses. Evicting an object without calling close() releases the Python reference but may leave external resources behind. The pool’s lifecycle must therefore pair admission, serialization, idle eviction, and cleanup. The exact pool policy belongs to the host, but the runtime’s one-session and explicit-close invariants do not disappear when the objects are stored in a dictionary.
Cancellation has a similar ownership question. The host may expose a Stop button or a request timeout, but the runtime needs a cancellation token or equivalent signal that reaches the current turn. A user-visible timeout should not simply abandon a worker thread while leaving an MCP subprocess or pending confirmation behind. The host should record that the turn was cancelled, release its lock in a finally path, and decide whether the session can safely accept another turn.
close() completes the lifecycle by disconnecting MCP clients and stopping the manager’s event-loop resources. Without it, a service can leak child processes and threads even if the visible reply was correct. The same discipline applies to sessions evicted from a pool and to processes shutting down under deployment orchestration. Cleanup is not housekeeping after the “real” answer; it is the transition that determines whether the next session starts with a clean boundary.
A correct reply therefore has two dimensions. There is the text the user sees, and there is the state transition that produced it: which tools ran, which were refused, whether the loop reached its cap, whether cancellation occurred, what memory changed, and whether resources were released. Agentao exposes different event and state surfaces for these concerns, while the host decides which facts to persist and which to show. This is the foundation for choosing the next boundary. An in-process Python SDK keeps these transitions close and easy to inspect; ACP moves them across a subprocess protocol and makes the host responsible for a different set of lifecycle messages.