Turning a Fresh Reference Into a Safe Action
A fresh observation gives BrowserSkill a current semantic target, but it does not yet tell the browser where to send input or what the page will do with it. This chapter follows the action path from reference or selector resolution through frame-aware geometry, native browser input, verification, and cleanup. The goal is to interpret a click, fill, scroll, or wheel result precisely: what was checked, what was sent, what was confirmed, and what still requires inspection.
From semantic target to browser coordinates
Every interaction begins by resolving the target tab through the session rules. The session must exist, the tab must be visible to that session, and the tool's effect must be allowed for that tab. In particular, write tools are constrained to the Agent Window; a target that is merely visible in a user window is not a valid place for page input. Remote sessions add the explicit-created-or-borrowed requirement before even passive page operations proceed.
Once the tab is in scope, BrowserSkill resolves the element. A fresh reference is checked against the session's reference store and the tab for which it was observed. A CSS selector follows a different path: it searches the main document and obtains a live node identity. A selector is not a way around reference freshness or tab scope. It still has to resolve in the authorized target, and it cannot reach an iframe or shadow-root element through a main-document search when the relevant target was not selected that way.
The next problem is geometry. A target may live inside an iframe, a nested scrolling container, or a shadow-root structure. The browser's local coordinates are not automatically the top-level viewport coordinates needed for a native mouse event. BrowserSkill follows the target's frame owners, brings the relevant element into view, projects its live geometry into the top-level viewport, and chooses an action point or visible region. The process is not decorative positioning. It reconciles the semantic identity from observation with the physical surface on which Chrome will dispatch input.
That reconciliation has failure conditions. The element may have been detached, its frame may have navigated, an ancestor may clip it completely, or the session may be cancelled while the geometry is being resolved. A result that says the target is not visible or no longer exists is more useful than a guessed coordinate. Even a visible rectangle is not a full hit-test guarantee: another page element may cover it, the layout may change after measurement, or moving the pointer may trigger hover behavior. Geometry is evidence for the next browser operation, not a lock on the page.
The frame distinction is especially important for an educated reader who sees only one visual page. An iframe may have its own document and debugging target, and an out-of-process frame may require a different connection context. A coordinate inside that frame cannot be treated as if it were a coordinate in the top-level document. BrowserSkill's frame-aware path preserves the target's identity while projecting its visible geometry outward.