When a Workflow Runs: State, Queues, and Partial Progress
When an AutoGPT task starts, the saved graph does not become a result in one indivisible call. The Platform creates execution state, queues work whose inputs are ready, records node progress, and decides how the run ends. This chapter follows that lifecycle and separates the roles of the database, message queue, workers, Redis coordination, and event stream so readers can tell what a run remembers—and what a restart or cancellation cannot repair.
A run is a chain of recorded state transitions
A graph definition describes a reusable workflow. A graph execution is one attempt to run a particular graph version with a particular set of inputs. A node execution is one attempt to run one node within that graph execution. These three objects answer different questions. The definition tells us what the workflow permits. The graph execution tells us which task was attempted. The node execution tells us which action received which data and what it produced.
That separation is essential for an agent that can branch, iterate, pause, or call another graph. One node in the definition can have several node executions during one task if a list is expanded or a loop-like workflow sends it more than one item. A graph execution can also have child graph executions when an Agent Executor block invokes a reusable agent. Treating the whole task as one model response would erase the distinctions the Platform needs in order to show progress and account for partial work.
The lifecycle begins before a worker touches the model or an external service. AutoGPT validates the graph and constructs inputs for its starting nodes. An execution record is created with the graph ID and version, the task inputs, optional credential metadata, and the initial node executions. Those node records can begin in a queued or incomplete state depending on whether their inputs are ready. The execution context carries the identities needed to connect the work back to its user, graph, node, workspace, safety settings, and—where relevant—parent or child execution.
The main states tell a compact story:
- INCOMPLETE means an execution record exists, but the inputs needed for a node or graph are not yet complete.
- QUEUED means the work is ready to be picked up by an executor.
- RUNNING means a graph or node is actively being processed.
- REVIEW means execution is waiting for a human decision at a configured review point.
- COMPLETED means the relevant work reached its terminal success path.
- FAILED means an error or classified business failure ended the work.
- TERMINATED means cancellation or interruption stopped it.
These are not merely labels for a dashboard. They are guarded transitions. The data model defines which prior states may lead to a target state, and terminal graph states record an end time. A graph can therefore be inspected as a sequence of decisions and events rather than as a blank “running” indicator until the final answer appears.
Readiness turns into work
The graph rules described earlier become scheduling decisions here. A node is not dispatched because it is visually next. The executor can start an independent node when it has no inbound dependency, but a dependent node must wait until its connected inputs and required configured values have arrived and passed validation. When an output is persisted, the runtime registers it against the downstream input. If that makes the downstream node complete, the node execution is queued.
This lets a single graph make partial progress. Consider a briefing workflow with two independent retrieval branches and one synthesis branch. The retrieval nodes may be dispatched without waiting for each other. The synthesis node remains incomplete until the required results reach its inputs. If one retrieval branch fails and the graph has an explicit error path, the other branch can still produce useful evidence. If the synthesis node requires both normal outputs, it will not run merely because one branch succeeded.
The database preserves the evidence of these transitions. Graph executions store task-level inputs and statistics. Node executions store their execution status, timestamps, input data, output data, and node statistics. Output blocks provide the final values that the Platform reconstructs as task outputs, while intermediate node records remain available for detailed inspection. A partial result is therefore not necessarily the same as a completed agent result: it may show what a branch produced before a later dependency or terminal policy stopped the run.
The distinction between node failure and graph failure also appears at this level. The Platform's normal error handling lets a block's failure be represented and routed through the workflow. The graph can continue along an explicitly designed path, while dependent normal-output nodes wait because their required values were never satisfied. Some failures are more decisive. The executor promotes trusted conditions such as insufficient balance or a paywall decision to graph-level failure, so a run does not appear successful merely because an individual error was stored somewhere inside it.
A review state is different again. A Human-in-the-Loop block can pause a node while its data awaits approval or rejection. The graph is not completed, but it is not simply broken. A later decision may send the actual data through an approved or rejected output. This is why a task history that shows REVIEW represents a live dependency on a person, not a slow model call.
The same stateful design supports nested work. A child graph has its own graph execution and node records, but its execution context can carry the parent execution identity. The parent Agent Executor listens for child updates and turns child Output-block values into outputs of the parent block. That arrangement preserves the child run's detailed lifecycle while allowing the enclosing workflow to treat the child as one composable action.
The platform splits persistence, transport, coordination, and observation
A durable run needs more than a database, but the services do different jobs. PostgreSQL is the durable record of graph definitions and execution state. RabbitMQ transports a request for graph work from the service that enqueues it to an executor. The executor owns the active work and dispatches ready nodes within a bounded worker pool. Redis supplies coordination primitives such as cluster-wide locks and also carries execution events to listeners. WebSocket and related event paths let the interface show progress without repeatedly reconstructing the entire run from scratch.
The order of operations matters. On the normal enqueue path, the Platform first creates or loads the graph execution, updates its state to QUEUED, and only then publishes the run message. That ordering is a defense against two concurrent requests publishing the same execution before either has recorded that it is already queued. If publication fails after the database update, the error path marks the execution as failed rather than leaving an apparently healthy task with no work message.
The executor consumes durable messages with bounded prefetch and submits graph work to a worker pool. Before accepting a run, it checks whether the same execution is already active locally and attempts to acquire a Redis-backed cluster lock. This prevents two executor processes from treating the same graph execution as their own at the same time. The lock is coordination state, not the durable result: the execution record remains the source for what work exists and which state it has reached.
Within a graph worker, ready node executions are placed into an execution queue. Node work can run asynchronously, and outputs are persisted as they become available. The executor then evaluates those outputs and registers the next inputs. That design gives the runtime two useful properties. First, independent branches need not wait for a global sequential loop. Second, the system can expose an output while other parts of the graph are still in progress. The worker pool and configuration still bound actual concurrency; “parallel” means eligible independent work can proceed, not that the Platform creates unlimited workers.
The event stream is an observation path, not a second source of truth. The execution model publishes graph and node updates to channels scoped to the user, graph, and execution. Payloads are truncated before publication when necessary. A client can watch one execution or listen for activity across a graph, while the persisted records remain available for later task history and output reconstruction.
This division also explains why a message queue cannot replace the database. A queue says that work should be delivered; it does not contain the complete graph, every input/output value, review decision, or final statistic. Nor can an event stream replace either one. Events are useful notifications for a live interface, but a disconnected client must be able to query the durable execution state later.
Acknowledgements and locks support recovery, not immortality
The run consumer does not acknowledge a graph message before its execution has finished. If a worker process dies or the run handler fails, the message can be rejected and requeued according to the consumer's policy. The queue uses durable infrastructure and long-running-consumer settings so an ordinary graph is not treated like a short request that must finish inside a narrow acknowledgement window. Connection code also includes heartbeats, timeouts, and bounded reconnect attempts for transient RabbitMQ and Redis problems.
When an executor receives a run, it can pre-populate its node queue from node executions that are still in active or resumable states. This makes persisted state useful when a supported requeue or resume path is invoked. It does not mean that every interruption is transparent. A lock can prevent duplicate ownership while it is held, but a provider call can still fail, a process can be terminated, and a state update can still require a retry or administrative recovery.
Cancellation travels through a separate path. A stop request can publish a cancellation event, which the active executor turns into a cancellation signal. The runtime can recursively stop child graph executions, stop running node tasks, and mark pending review work as cancelled when appropriate. Terminal graph updates also sweep running child node records so the database does not continue to report work as active after the parent has ended. The purpose is consistency across related state, not an attempt to pretend that an external side effect never happened.
Shutdown policy makes the boundary especially visible. The multi-container executor has a graceful-shutdown path that waits for active executions within a configured limit and releases cluster locks. The experimental one-container deployment documents a stricter limitation: runs still in flight when that appliance stops may be abandoned, their messages may be dropped, and their execution rows may remain RUNNING; they do not automatically resume after restart. That statement belongs to the appliance distribution and should not be generalized into a description of every hosted or separately operated deployment.
There is also a practical difference between “the task has an error” and “the system has lost all evidence.” Because inputs, outputs, statuses, timestamps, and statistics are persisted incrementally, a failed task can still reveal which nodes started and what they produced. But partial evidence does not make the task complete, and it does not reverse an external action already performed by a provider. A fresh run may be necessary when the unfinished work depends on a new input, a repaired credential, a restored service, or a decision about whether side effects should be repeated.
Reading a run as evidence
A useful operator does not ask only whether an agent succeeded. They ask which graph version ran, which starting inputs were accepted, which node became ready, what output was recorded, whether a branch failed or waited, whether a review decision was pending, and how the graph-level status was chosen. The answer may be visible across task history, node details, live events, and the final Output blocks rather than in one sentence generated by a model.
The runtime's design supports that style of reading because it keeps definition, execution, transport, coordination, and observation distinct. It also imposes limits. Durable state cannot guarantee provider availability. Requeueing cannot safely duplicate an irreversible external action without a workflow-specific idempotency decision. A cluster lock cannot prove that a third-party call did not complete just before a network failure. A WebSocket update can be missed even though the database has the result.
The most important conclusion is therefore conditional. AutoGPT makes a workflow more inspectable by recording state at the graph and node levels and by separating the path that carries work from the paths that coordinate and display it. That is a strong foundation for long-running, branching work, but it is not the same as an immortal execution. To use the design well, a reader must combine what the graph says with what the runtime records and then distinguish wiring, readiness, coordination, and terminal policy as possible causes of an unexpected outcome.
The next synthesis connects those layers into one diagnostic model: what the graph promises, what the executor waits for, what the persisted run proves, and what a reader can reasonably infer from its outcome.