Inside ReAct: How Thought, Action, and Observation Work Together
Inside ReAct, the decisive boundary is not between a model and a tool. It is between language that changes the model's working context and an action that changes, or queries, an external environment. This chapter traces that boundary through a complete trajectory so the reader can tell what a thought does, what an action does, and why the observation between them is the part that closes the loop.
That distinction turns the paper's central idea into an operating mechanism. ReAct does not ask a model to produce a long explanation and then act once. It gives the model several kinds of intermediate move, each with a different effect on the state available at the next step. Understanding those effects is necessary before judging whether the pattern improves answers or decisions.
A thought is part of the context, not the world
A useful way to describe ReAct is to say that it expands the model's action space with language. In an ordinary interactive setup, the model chooses from actions that belong to the environment: search for an entity, move to a location, select an option, or finish the task. ReAct adds another space of possible outputs: a thought expressed in ordinary language. The model can now choose either an environment action or a language-space action as it continues the trajectory.
The two choices have different consequences. A thought does not directly alter the room, webpage, database, or knowledge base. It produces no new observation from the environment. Its effect is internal to the trajectory: it appends useful language to the context the model will consult later. That language can state a plan, identify the next subgoal, extract a fact from an earlier observation, point out that a search was uninformative, or revise an assumption.
An environment action crosses the other boundary. It is interpreted by the task environment and may change the environment's state. A search returns a result, a navigation command moves the agent, or a product-selection action changes which options are available. The resulting observation is not the model's interpretation of the action. It is feedback from the environment. ReAct places that feedback back into the context so the model can reason over what actually happened.
This is why calling every line a “step” hides the mechanism. A thought changes what the model can consider; an action changes what the environment can return; an observation tells the model what the environment returned. If a model writes “I should check the refrigerator next,” the refrigerator has not been checked. If it executes a command to open the refrigerator, the environment may report that it is closed, open, or unavailable. Only then can the model decide whether the original plan still applies.
The relationship can be summarized as a repeated exchange between two owners of state. The language model owns the working interpretation: its current plan, assumptions, subgoals, and proposed next move. The environment owns the external state: what can be found, what action succeeded, and what changed. A ReAct trajectory is useful only when information can cross that boundary in both directions. The model must send an action that reflects a reasoned need, and it must allow the resulting observation to constrain the next thought.
In the paper's formal description, the model's context is the accumulated trajectory: the task, earlier observations, earlier actions, the current observation, and any newly generated thought. Adding a thought extends that context without pretending that the world has moved. Adding an environment action does something different: it requests a transition or query, after which the new observation becomes another piece of the accumulated state. This distinction prevents a common category error. A model can write that it has found an object before the environment confirms it, but the statement alone does not make the object available for the next action.
The boundary also explains why the available environment actions matter. In the paper's knowledge setup, actions are not arbitrary software calls; they are a small interface for searching an entity, looking up a phrase, or finishing with an answer. In a household setting, actions have different names and preconditions. A thought can be expressed flexibly across both domains, but the external action must belong to the environment's vocabulary and be valid for its current state. ReAct joins these spaces without erasing their difference: language supplies a general control layer, while the environment still decides which transitions can occur.
This makes failure observable at the right boundary. If a thought is confused, the model may select an irrelevant action. If the action is invalid or premature, the environment can return an unhelpful result or no change. If the observation is clear but the next thought ignores it, the error lies in updating the working interpretation. Separating these cases is more informative than calling the whole trajectory a reasoning failure, because each boundary suggests a different remedy: better task decomposition, valid action selection, more informative feedback, or stronger state tracking.
The diagram also shows the termination condition. ReAct is not a loop for its own sake. The trajectory must eventually produce an answer or complete the goal, and the environment must not be treated as if every action succeeded merely because the model intended it to. A thought can prepare a finish action, but it cannot make the task finished by describing success.
How language becomes useful state
The paper identifies several jobs for intermediate thoughts, and they are best understood as operations on the model's working state. The first is decomposition. A high-level request is rarely identical to one environment action. A question may require finding an entity and then finding a related fact. A household instruction may require locating an object, taking it, cleaning it, and putting it somewhere else. A thought can make those dependencies explicit before the model commits to the first move.
The second job is observation extraction. An environment may return more information than the next action needs. The thought can select the relevant part and attach it to the current problem. In a knowledge task, the observation may identify the intermediate entity that the question depends on. In a text-based household, it may establish that an object is present at one location while another location has already been ruled out. The thought is not new evidence; it is a working representation of evidence that has already entered the trajectory.
The third job is progress tracking. A long task contains facts about what has been done, not just what remains possible. After an object has been found, the next subgoal is no longer “find the object.” After it has been cleaned, the model must not act as though cleaning is still pending. A progress thought can make the transition explicit, which reduces the chance that the model will select an action appropriate to an earlier state.
The fourth job is exception handling. A plan formed before the first action is a hypothesis about how the task will unfold. A search may return similar entities rather than the requested one. A location may be empty. A product may match the requested category but not the requested size or color. When that happens, a thought can identify the mismatch and reformulate the next action. ReAct therefore treats planning as revisable rather than as a fixed sequence generated once at the beginning.
A retrieval trajectory makes the interaction concrete. The model can begin by stating that it needs an intermediate fact, issue a search, and receive an observation that only partially answers the question. It can then use a lookup action for a relevant phrase, recognize that a broad query was unhelpful, or search a more specific entity. The observation supplies the fact; the thought determines whether that fact is enough and what to seek next. If the model skips the interpretive step, it may either stop too early or continue searching without a clear relationship to the question.
At each cycle, the model is implicitly answering four different questions: What is the goal? What is already established? What is still missing? Which available action can reduce that uncertainty? The paper's prompt examples place those questions in ordinary language rather than in a separate planner with a fixed schema. That makes the trajectory adaptable across tasks, but it also means that the model must maintain distinctions that a specialized controller might represent explicitly. A thought that says “the result does not answer the question” is useful because it preserves the gap between an observation and a conclusion; a thought that silently treats the result as sufficient collapses that gap.
The same pattern appears in a household environment, but the state is more physical and the horizon is longer. Suppose the goal is to place a clean object on a surface. The model must maintain at least three distinctions: the object may not yet be found, it may have been found but not taken, or it may have been taken and cleaned but not placed. Each observation narrows the state, while each useful thought translates that state into a next subgoal. The paper's ALFWorld examples use thoughts for exactly this kind of decomposition, progress tracking, likely-location reasoning, and next-subgoal selection.
These examples explain why the authors do not require a thought before every action. A dense trace is useful when the task requires frequent interpretation of evidence, as in multi-hop question answering. In a long household trajectory, repeating a thought after every routine movement would consume context without adding a new decision. The paper instead allows thoughts to appear sparsely and asynchronously where a plan must be formed, a subgoal must be checked, or an exception must be handled. The useful unit is not narration frequency; it is a meaningful change in what the model needs to know or decide.
How the prompting setup teaches the interface
In the reported prompting setup, the underlying language model is frozen. It is shown human-written examples in which thoughts, actions, and environment observations appear in the same trajectory, then it is asked to continue that pattern on a new task. The paper uses between one and six in-context examples depending on the task and domain. This arrangement teaches more than the names of available actions. It demonstrates when a person would state a plan, what kind of language would mark progress, how an observation can redirect a search, and when a finish action is appropriate.
That flexibility is also a burden. Ordinary environment actions usually come from a domain-specific set, but the language-space action can take almost any form. The model must learn both how to produce a valid action and how to produce a thought that is useful rather than repetitive or misleading. A prompt example can show a good pattern, but it cannot enumerate every possible exception in a large action space. This is one reason the paper treats ReAct as straightforward to design but difficult to scale: the same free-form channel that makes the paradigm adaptable also enlarges the space the model must control.
Prompt design also determines how the interface exposes responsibility. If a search returns nothing, the environment has supplied a result, but the model still owns the decision to reformulate the query, try a related entity, or stop. If a household command produces no change, the environment has reported a failed transition, but the model must decide whether the object is in the wrong location, the action is premature, or the task state was misunderstood. A trajectory makes those decisions inspectable because the proposed interpretation appears before the next action. It does not make them correct automatically.
This is the practical meaning of a complete ReAct trajectory. It preserves a chain of state changes: what the model thought the goal required, what it tried, what the environment returned, and how the model revised its working plan. A final answer can hide whether a fact was retrieved or guessed, and a final action can hide whether the model understood a subgoal or merely stumbled into success. The trajectory supplies more material for diagnosis, but the material still has to be checked against the environment and the task.
The interface therefore has a clear promise and a clear limit. It promises a place where planning can target action and feedback can revise planning. It does not provide a learned theory of the best thought frequency, guarantee that every thought is faithful to hidden computation, or remove the need for stopping rules and valid environment feedback. Once this boundary is clear, the next test is empirical: when ReAct uses the interface to retrieve evidence, does the loop produce more reliable answers, and what new kinds of error does the search process introduce?