When a Strong Result Is Still Only a Lead

The evidence around browser agents needs diagnosis of its own. BrowseComp-V3 reports where multimodal systems lose progress in deep search, while RunAgent Super Browser reports a strong end-to-end result for a state-aware architecture. The question here is how to separate a symptom from a cause, solvability from execution economy, and a promising system correlation from proof that one mechanism produced the gain. That separation gives us a more defensible way to evaluate browser-agent claims.

A high success rate is important, but it is not self-explanatory. It may reflect better perception, better planning, more reliable event dispatch, more efficient use of a fixed scaffold, or a favorable task distribution. A low rate is equally ambiguous: the agent may never have found the evidence, may have acted on stale page state, or may have completed nearly every subgoal and failed only at the final relation. Diagnosis begins by preserving those distinctions.

From symptoms to causes

The first symptom is a visual error: the agent selects the wrong region, confuses two similar entities, or fails to read a small visual clue. BrowseComp-V3 reports Visual Grounding Error and Image Perception Failure as dominant categories in its analysis, and performance declines as tasks require more complex cross-region and cross-image integration. The natural hypothesis is a perception problem. The appropriate test is not simply to ask whether the model's answer was wrong. It is to inspect whether the relevant evidence was present, whether the candidate region was identified, and whether the selected region matched the active subgoal. If the needed image was never grounded, later planning cannot repair the missing observation.

A second symptom is repeated re-grounding or repeated attempts at the same target. That pattern points toward stale state, but not conclusively. The worker may be uncertain about the task, the page may be changing without an obvious visual difference, or the dispatch layer may be returning no meaningful effect. Super Browser's state-aware design supplies the right discriminating observations: compare the current page fingerprint with the last visual epoch, inspect mutation and URL changes, and record whether a target produced a no-op. If the page state drifted, the cached target is suspect. If the state did not drift but the element was semantically wrong, target resolution is the better candidate. If the right element was addressed but no state changed, event delivery or widget behavior deserves attention.

A third symptom is a high share of process progress paired with a failed final task. BrowseComp-V3's Process Score versus Success Rate distinction makes this pattern visible. The agent may have acquired several correct subgoals and still lose the evidence chain at a later relation. That is consistent with a planning or identity-continuity failure, but the metric alone cannot tell us which. The trajectory should show where the last valid fact was established and what decision followed. A late failure after accurate early steps is a different engineering problem from a run that never found a usable page.

A fourth symptom is surplus tool use.