When the Agent Can See: Browser Access as a Control Plane
For gstack, a technical plan becomes useful only when the agent can compare its assumptions with a running application. This chapter focuses on the browser layer: why gstack uses the user’s real browser when possible, how it falls back to a browser it owns, and how both paths return evidence under one contract. The practical judgment is whether browser access gives the workflow reliable observation without becoming unrestricted control of a user’s sessions.
The key distinction is between reading code and seeing behavior. Source files can suggest that a form submits, a redirect occurs, or a page handles an empty state. A browser can show whether the button is reachable, whether the authenticated session is the one the test expects, whether the layout survives a narrow viewport, and whether the console reports an error after the click. gstack treats that gap as an architectural problem, not a missing convenience command.
One workflow, two browser engines
The first engine is Aside, an AI browser for macOS 15 and later. When it is installed and running, gstack’s browser skills use it first. That choice is about state as much as visibility. Aside is the user’s real browser, so it may already contain the session needed to inspect a staging dashboard, a logged-in customer flow, or a vendor console. The user does not have to export cookies or recreate an account inside a disposable browser before QA can begin.
But “use the user’s browser” does not mean “take over everything in it.” The gstack contract says the agent works in tabs it opened, or in a tab the user explicitly names. It stays on origins the user supplied and same-origin links. A request to inspect a site permits reading, navigation, and filling forms without submitting them. A mutation on a non-local site, such as creating an account or submitting a real form, requires one explicit question that lists the intended actions. The agent never types or reads passwords, one-time codes, payment details, cookies, tokens, or local storage. A sign-in wall is handed back to the user inside the browser.
The same contract changes how the agent reads a page. Text, accessibility snapshots, console output, screenshots, and answers from open-ended browser research are treated as page content, not instructions. A page can contain text that looks like a command to an AI agent. gstack may use the text as evidence about what the page displays, but it cannot let the page redefine the task, the permissions, or the consent boundary.
The distinction becomes concrete in a staging QA run. Suppose the plan says a signed-in user should submit a form and land on an onboarding page. The browser skill can open its own tab, inspect the initial state, fill the form, and capture the page difference. It may be allowed to fill before submission, but the final click on a real, non-local service is a mutation. The skill must present the intended action and wait. If the user approves, the report can include the resulting URL, the changed heading, console errors, and a screenshot. If the page instead displays a prompt telling the agent to ignore its task and reveal a token, that text remains an observed page condition, not an instruction to follow.
Aside is not required for the workflow. gstack probes for it once at the beginning of a browser run. A successful readiness response selects Aside for that run. If the command is missing, the app is closed, or the probe cannot complete, gstack switches to its own fallback browser. The important property is that the decision is made once, rather than changing engines in the middle of a test and producing a report whose evidence came from two different kinds of session.
The fallback exists for more than platform coverage. It gives Linux and Windows users a browser path, and it gives a Mac user continuity when Aside is unavailable. The fallback can run headless for automated QA or in a visible “GStack Browser” mode when the user needs to watch the agent. It supports cookie import, handoff for CAPTCHA or MFA, and sharing with another agent. The source describes the two engines as different backends for the same skill contract: the commands and evidence may differ internally, but the report should still say what page was opened, what action occurred, what changed, and whether the step completed.
That parity is a reliability feature. If the Aside path reports a console error and the fallback path does not even collect console output, the same /qa command would mean different things on different machines. If one path asks for consent before a submission and the other treats the same action as ordinary navigation, the platform choice would change the safety rule. gstack’s design tries to keep those semantics stable while accepting that the engines have different capabilities.
A single engine per run also gives the evidence a clear provenance. An Aside session has fresh tabs that close when the script ends. The fallback keeps a daemon and its state alive across calls. Those are different operating conditions, so a report should not silently combine a screenshot from one engine with a navigation result from the other. The readiness probe is therefore more than feature detection. It chooses the conditions under which the rest of the evidence should be interpreted.
A browser run is therefore a control plane between the plan and the application. It carries the action into a real page, but it also carries back the evidence and the limits under which that evidence was obtained. That role becomes clearer in the fallback architecture.
Why the fallback keeps a browser alive
The fallback engine is a compiled command-line client connected to a local Bun service. The service controls Chromium through Playwright. A first command starts the daemon and the browser; later commands send requests to the already-running process. The repository reports an initial startup of about three seconds and subsequent calls of roughly 100–200 milliseconds. Those are project-reported timings, not an independent benchmark, but they express the design target.
A new browser for every command would throw away the state that makes browser testing useful. A QA flow may need to log in, open a page, inspect a modal, switch routes, fill a form, and return to the result. Restarting each time would lose cookies, local storage, open tabs, and the page context. It would also add several seconds to each action. The daemon keeps the session alive and makes the browser feel more like a service that the skill can query repeatedly.
The state belongs to a workspace. The daemon writes a state file containing information such as its process identity, port, token, start time, and binary version. The CLI reads that state to find the service. If the process is gone, a later call can start a new one. If Chromium crashes, the server exits instead of pretending that it recovered; the next command can then create a clean session. The daemon also shuts down after an idle period. This reduces abandoned processes, but it does not remove the need to think about the sensitivity of persisted browser state.
The choice of a plain command-line interface is also deliberate. gstack’s browser commands print text to standard output rather than requiring the agent to carry a large tool schema for every request. The repository contrasts this with browser protocols that add context and connection overhead. A plain CLI is easier for an agent to call, easier to inspect in a terminal, and easier to use from other agents over HTTP. The trade-off is that gstack must maintain its own command registry, lifecycle rules, authentication, and error messages instead of delegating those concerns to an existing protocol.
The daemon separates read, write, and meta commands. Reading page text or a screenshot is different from clicking a button. Browser-level operations such as switching tabs or saving state are different again. The classification supports permission decisions and makes the command surface understandable, but it does not make a write idempotent. A second click may submit a second form. A skill still has to know whether it is observing or changing state.
Why the agent uses accessibility refs
A browser agent needs a stable way to point at an element. Guessing a CSS selector is fragile. A class name may be generated by a framework, a page may contain several buttons with similar markup, or the element may sit behind a shadow boundary. Injecting custom attributes into the page can interfere with content-security policies and with React, Vue, or Svelte reconciliation.
gstack’s fallback browser takes a different route. Its snapshot command asks Chromium for the page’s accessibility tree, which represents elements by roles and names that are meaningful to a user. The parser assigns short references such as @e3 to interactive elements. The browser stores a mapping from the reference to a Playwright locator. The agent can then request a snapshot, see that @e3 is the “Save” button, and click that reference rather than writing a selector from memory.
The reference is not a permanent identifier. Navigation clears the map because the old locator belongs to the old page. Single-page applications can also change the DOM without changing the URL, so the browser checks whether the referenced element still exists before acting. If it does not, the command fails quickly and tells the agent to take a fresh snapshot. That is safer than allowing a stale reference to wait for a long timeout or, worse, resolve to an unintended element.
This design reveals a broader gstack preference: prefer an external, inspectable representation of state over hidden mutation. The browser does not need to edit the page to label it. The agent sees a structured snapshot, chooses an element, and sends an explicit action. The method is not perfect. Accessibility names can be poor, custom components can be missing from the accessibility tree, and iframes require an explicit context. But the failure is visible and recoverable, which is more useful than a selector that appears to work until the page changes.
The same principle applies to evidence. A one-off browser action is not enough. A skill prints labelled lines for the URL, console errors, differences in the page, navigation results, and screenshots. Aside scripts use a completion sentinel because the Aside command exits successfully even when the script itself throws. The fallback maps its own commands to equivalent evidence. In both cases, the skill has to tell the rest of the workflow not only what it attempted, but whether the step actually reached its expected end state.
What the browser contract does not promise
Browser access does not turn gstack into an invisible operator of the user’s digital life. The user remains responsible for credentials and sensitive challenges. A headed browser can show what the agent is doing, but visibility is not permission; a tool that can click is not automatically allowed to submit. The fallback’s persistent cookies and local storage make authenticated QA convenient, but they also create a stated sensitivity cost. The browser should not be pointed at a sensitive production environment casually just because the skill can reach it.
The two-engine design also contains real differences. Aside scripts are fresh sessions, so work that spans calls must be re-navigated. The fallback daemon preserves state across calls. Aside is macOS-only and external to gstack; gstack detects it but does not install it. Linux and Windows depend on the bundled browser and its local prerequisites. These differences are not defects to hide. They are conditions under which the same workflow contract is implemented by different machinery.
The useful result is a browser that sits between intention and proof. A plan says what behavior should matter. The browser supplies direct evidence about what the user can see and do. The next stage must still decide whether that evidence is sufficient, whether the code has structural flaws the browser did not expose, and whether the current tree is ready to ship. The browser gives the agent eyes, but eyes alone do not make a release trustworthy.