The Click Was Right—Until the Page Changed
Autonomous multimodal browser agents are the subject of this series, and its opening question is deceptively simple: why can an agent see the right-looking control and still fail to advance the task? The answer is often not that the model lacks the relevant fact or cannot issue a click. The deeper problem is that its perception, its representation of the current page, its memory of the goal, and the action it dispatches may no longer refer to the same situation. Understanding that mismatch gives the reader a practical way to diagnose browser-agent failures instead of treating every bad outcome as a vague failure of “reasoning.”
A browser agent does not operate on a static picture. It moves through pages that load, scroll, expand, rerender, and redirect. A visual label can remain in roughly the same place while its meaning changes; a selector can remain in memory after the element it once identified has disappeared; a search result can be correctly read but attached to the wrong entity several hops later. The central concern here is therefore continuity: how to keep a chain of perception and action aligned with the page state and with the evidence needed for the final answer. BrowseComp-V3 measures how badly current systems lose that chain in demanding multimodal search. RunAgent Super Browser proposes one architecture for keeping it intact. They are complementary lenses, not a head-to-head comparison.
When the same screen stops meaning the same thing
Consider a compound filter row on a flight-search page. The row displays “United States” and a small downward chevron. A vision model may reasonably draw one box around the whole row and give it a label. If the agent clicks the geometric center, however, the event may land on the text span. The label is visible but not interactive; the small arrow is the control that opens the regional choices. Nothing changes. The agent takes another screenshot, sees the same row, and tries again. From the model’s point of view, the target was plausible and the action was executed. From the website’s point of view, no meaningful state transition occurred.
The failure is easy to describe as poor visual grounding, but that description is incomplete. Several representations have drifted apart. The screenshot says what the page looks like. The DOM says which element can respond. The active subgoal says whether the agent needs the label or the expansion control. The action result says whether the page actually changed. A reliable system has to reconcile all four rather than trust whichever representation is easiest to pass into the next model call.
The same issue appears in less visible forms. A page transition can move a button, replace a result list, or change the contents of an iframe while leaving a familiar visual pattern behind. A cached coordinate may still point to a clickable element, just not the one that satisfies the current goal. A direct URL may load successfully while encoding the wrong country, category, or filter. Even memory can become misleading: a fact extracted on an earlier page may be accurate but no longer belong to the entity the agent is following.
This gives us four continuity links to watch. Goal continuity asks whether the action still serves the task’s current subgoal. Evidence continuity asks whether a fact or visual clue remains attached to the right person, object, page, or hop. Page-state continuity asks whether the percept and the DOM describe the same version of the interface. Transition continuity asks whether the action produced the intended change, rather than merely producing an event that the browser accepted. A failure at any link can look like a failure at another. The useful diagnosis is to locate the broken link.
The distinction between an accepted event and an effective transition is especially important. A browser can report that a click was dispatched even when the wrong element received it, an overlay intercepted it, or the page required an event sequence that a synthetic shortcut did not provide. Conversely, a visually small change can be the correct progress signal when it opens the next evidence-bearing control. Reliability is not the same as making the interface move; it is making the intended state change and preserving its meaning for the next step.
A browser task is a chain of state transitions
Ordinary factual lookup encourages a misleading mental model. The model receives a question, retrieves a passage, and produces an answer. A deep browser task is different. It may begin with an image, use a text search to identify a person, follow a page to a competition, inspect another image, and then combine several facts into a short answer. Each hop changes what counts as a relevant target. The agent must not only find information; it must preserve identity, modality, and relationships while moving through a changing environment.
BrowseComp-V3 was designed to make that burden visible. Its 300 questions span 24 secondary domains and five broad categories, and the collection includes 383 images. The authors deliberately interleave critical evidence across text and visual material within and across pages. Their three levels of cross-modal difficulty range from aligning information within a region to integrating separate regions and reasoning across images. The benchmark also requires supporting evidence to be publicly searchable, favors temporally stable and objective answers, and records expert-validated subgoals and search trajectories. These choices matter because an inaccessible source or a moving web page could otherwise make a failure impossible to interpret.
The benchmark’s most useful distinction is between whole-task success and process progress. Success Rate asks whether the task was completed. Process Score counts the share of required subgoals that the agent achieved. A system can therefore identify the right intermediate person, find the relevant event, and still fail on the last relational step. The final answer is wrong, but the trajectory is not empty. When Process Score is higher than Success Rate, the gap is evidence that the system can perform pieces of the search while losing logical consistency across the full chain.
The reported results show why adding tools is necessary but insufficient. Human browser participants reached an average Success Rate of 68.03% and Process Score of 82.93% in the paper’s evaluation. Tool-augmented model services remained below 40% Success Rate; with OmniSeeker’s standardized tools, GPT-5.2 reached 36.00% and Doubao-Seed-1.8 reached 33.67%. Tool-free results were generally far lower. The narrow conclusion is strong enough: parameterized knowledge alone cannot reliably supply current, cross-modal evidence. The broader conclusion—that any tool-using agent is now reliable—does not follow. Search, screenshots, and clicks expand what the agent can access; they do not ensure that each new observation is joined to the correct prior state.
The benchmark’s failure analysis points in the same direction. Visual grounding and image perception errors dominate the reported model failures. Performance declines as tasks require more complex integration, while planning becomes a more visible constraint once basic perception improves. This is not a claim that every model fails for precisely the same reason, and the paper is a preprint rather than an independent audit. It is, however, a useful map of the problem: a browser agent can fail before it acts, while it acts, or after several individually reasonable actions have accumulated into an incoherent path.
Two lenses on the same reliability problem
BrowseComp-V3 is the measurement lens. It asks whether an agent can acquire and connect public multimodal evidence over a deep search path, and it gives partial progress a place in the evaluation. RunAgent Super Browser is the systems lens. It treats browsing as a perception–cognition–action loop and assigns different responsibilities to candidate generation, strategic planning, operational action, memory, and event dispatch. The latter paper’s cognitive language is a design analogy, not a claim that the software is conscious or that an LLM literally thinks like a person.
Its central proposal is best understood as a contract. Perception should produce candidates that carry enough semantic and DOM context to be actionable. Memory should preserve the goal, relevant facts, recent actions, checkpoints, and dead ends without filling the live prompt with obsolete screenshots. Action should be allowed to reuse a prior target only while lightweight checks show that the page remains compatible with that perception. After navigation or meaningful mutation, the old visual epoch should lose authority. The result of an action should be checked as a state transition, not assumed from the fact that a tool returned.
That contract does not eliminate uncertainty. A state fingerprint can miss a semantic change. A compact ledger can summarize the wrong fact. A URL shortcut can be fast and mis-scoped at the same time. A better visual model can still leave long-horizon planning as the bottleneck. The value of the proposal is that each risk becomes explicit enough to test: Was the target fresh? Was it tied to the active subgoal? Did the page change as expected? Was the failed route remembered without preserving all of its noise? Did the final answer remain traceable to the evidence chain?
Those are the questions that turn “the agent made a mistake” into an engineering diagnosis. The benchmark shows why deep, multimodal search exposes the weakness; the architecture suggests how a system might keep its references synchronized. The next step is to examine the benchmark itself as an instrument: what its tasks demand, what its two metrics reveal, and where the reported gaps locate the browser agent’s weakest link.