The First Reframe: Read an Agent Run as a Trajectory
The LHTB task environment, its dense reward, and its termination analysis support one practical judgment: an agent run is a trajectory of changing state, not a single answer with a score attached. This synthesis connects those ideas so readers can inspect a long-horizon run fairly—separating what the task demanded, what the agent did, what the verifier observed, and how the run ended. That distinction is the foundation for choosing a reliability measure instead of treating every incomplete result as the same kind of failure.
A run is a chain of evidence, not a single score
A long terminal task begins with an instruction, but the instruction is only the entry point. The task environment supplies files, data, tools, and constraints. The agent turns that starting state into a sequence of actions and observations. Those actions alter the workspace: a script is changed, an artifact is generated, a test result appears, or an error narrows the possibilities. The verifier then examines selected evidence in the resulting state and decides how much of the intended goal was met.
The chain can be written in plain language:
- Demands and dependencies: What must ultimately be true, and which intermediate conditions depend on one another?
- Actions and observations: Which commands, edits, executions, and interpretations move the work forward?
- State and evidence: Which files, outputs, tests, or simulator responses demonstrate that a subtask has been completed?
- Judgment and termination: Did the final evidence satisfy the verifier, or did the run end through a timeout, early exit, or infrastructure failure?
The order matters. A final artifact cannot be interpreted without knowing what requirement it was meant to satisfy. An action cannot be judged in isolation if its value depends on an earlier observation. A timeout cannot be called a reasoning failure without checking whether the agent was still making progress, whether the task had reached a verified boundary, and whether the environment remained available.
This is the operating model that LHTB makes visible. The workspace owns the conditions and the mutable state. The agent owns the decisions that cross that state. The verifier owns the encoded standard for success. None of these responsibilities fully substitutes for another. A well-designed environment cannot compensate for an agent that repeatedly pursues a disproven hypothesis. A capable agent cannot receive credit for a requirement the verifier does not encode. A rigorous verifier cannot turn an unavailable provider into a fair test of model behavior.
Consider a representative data-processing task in the suite. The instruction may require the agent to repair a pipeline and produce a checked artifact from supplied inputs. The task demand is not simply “make the script run”; it includes the relationships that make the artifact meaningful. The agent may need to discover how the input is represented, identify why one stage disagrees with another, change the implementation, rerun the pipeline, and inspect the resulting output. Each observation changes the state of the investigation. A successful intermediate result can narrow the next decision, while a misleading or ignored error can send later actions down an unproductive path.