navigate and act results that changed the page document now include the
loaded page's compact snapshot inline (honoring browser.snapshotDefaults
and standard caps/wrapping), removing the follow-up snapshot round-trip.
Snapshot execution moves to browser-tool.snapshot.ts to keep action
executors under the file size cap.
* fix(browser): retire durable tab rows whose browser never returns
Durable cleanup defers whenever ownership cannot be proven, so a browser
that never comes back at the same cdpUrl leaves its rows behind forever:
each sweep re-claims them, fails the identity lookup, warns, and defers
again. Nothing in the subsystem removes a row by age.
The `browser.session-tabs` namespace is opened with a 5000-row cap and
`reject-new`, so once those rows accumulate to the cap, tracking a new tab
throws PLUGIN_STATE_LIMIT_EXCEEDED. That propagates into the compensation
path in browser-tool-session-tabs.ts, which closes the tab the user just
opened and rethrows -- every `browser open` on that profile then opens a
tab, closes it again, and errors, with no self-healing path.
Bound the retry: when a close attempt reports the target unavailable and
the tab has been unused for longer than the retire window, drop the row
instead of deferring again. A browser returning after that long almost
always carries a fresh instance fingerprint, which retires the row through
the ownership-mismatch path anyway.
closeTrackedBrowserTabsForSessions now accepts `now` like the sweep does,
so lifecycle cleanup can be exercised on a coherent clock.
* refactor(browser): split session tab cleanup claim and test harness
check-lint failed on max-lines: session-tab-registry.ts was at 699 of its
700-line budget and the durable registry test at 982 of 1000, so the retire
branch and its regression test pushed both over.
Extract the cleanup claim bookkeeping (claim, ownership match, delete) into
session-tab-cleanup-claim.ts, and the durable registry test shapes into
session-tab-registry.sqlite.test-helpers.ts, matching the existing
*.test-helpers.ts convention in this directory. No behavior change.
* test(browser): protect unreachable retirement races
Co-authored-by: Yigtwxx <yigiterdogan023@gmail.com>
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat(browser): restore driver "extension" via loopback Chrome extension relay
Reintroduces browser profile driver "extension" (removed in 2026.3.22) as a
loopback relay that drives the user's signed-in Chrome through an MV3 extension
instead of the remote-debugging port. This avoids Chrome's blocking "Allow
remote debugging?" prompt, which cannot be clicked when the operator drives
OpenClaw from a phone. Automated tabs live in an "OpenClaw" tab group (the
consent boundary), mirroring the Codex/Claude-in-Chrome model.
- relay bridge synthesizes the CDP browser target surface for Playwright
connectOverCDP and forwards session-scoped commands to chrome.debugger
- relay server binds loopback only; both sides authenticate with a token
derived (HMAC-SHA256) from gateway auth, so the raw credential never reaches
Chrome; extension origin + loopback Host checks guard the upgrade
- built-in "chrome" profile; distinct relay ports per extension profile;
relay reconciles on auth rotation / cdpPort change and prunes removed profiles
- doctor + status surface the extension transport; doctor keeps repairing the
retired relay endpoint URL on legacy "extension" profiles
Refs #53599
* feat(browser): bundle the OpenClaw MV3 Chrome extension
Thin MV3 extension (chrome-extension/): a WebSocket client to the loopback
relay plus chrome.debugger forwarding and OpenClaw tab-group management. All
CDP target synthesis lives server-side in the relay bridge, so the extension
stays a dumb transport (the removed 2026.3 extension put that logic in a
1000-line untestable service worker). Popup handles pairing and per-tab share
toggle; `openclaw browser extension path|pair` load and pair it. A build copy
hook stages it into dist so the load path is stable.
Refs #53599
* docs(browser): document the Chrome extension profile
Adds docs/tools/chrome-extension for the restored extension driver (install,
pair, tab-group consent model, security posture) and wires it into the browser
docs profile section and nav.
Refs #53599
* feat(browser): make the extension relay work on remote browser nodes
Derives the relay auth token from a host-local secret in the credentials dir
(created on first use) instead of gateway auth. Each machine that runs a
browser — the gateway host and every browser node host — owns its own token, so
the extension pairs with whichever machine hosts its Chrome and no gateway
credential travels to a node. The node host already runs the shared browser
control bootstrap, so this is all that was missing for cross-machine control.
Also removes the "relay needs gateway auth before it can start" failure mode:
startup and `openclaw browser extension pair` ensure the secret exists.
Refs #53599
* fix(browser): harden relay secret creation and satisfy CI lint/typecheck
- Make the host-local relay secret creation atomic (O_CREAT|O_EXCL + adopt the
winner on EEXIST) so the gateway service and `extension pair` CLI cannot mint
divergent tokens on a fresh host (would 401 until restart); credentials dir
created mode 0700. (adversarial review finding)
- Resolve type-aware oxlint findings across the relay + extension: unknown catch
vars, addEventListener over ws.on* in the MV3 worker, void async listeners,
drop useless returns/spreads, Object.assign over map-spread, safe ws frame
decode (Buffer[]/ArrayBuffer), toSorted.
- Add extensionRelayDefaultPort/extensionRelayPorts to remaining test config
literals; type the extension relay-core module (.d.ts, excluded from dist);
regenerate docs_map.
* fix(browser): satisfy OpenGrep security policy on the relay
- Hash both operands before timingSafeEqual so token comparison has no
length short-circuit (GHSA-JJ6Q-RRRF-H66H).
- Bound the relay WebSocketServer with maxPayload (64 MiB, headroom for CDP
screenshots/bodies) against oversized frames (GHSA-VW3H-Q6XQ-JJM5).
- Rewrite the config test env helper to avoid the skill-env-host-injection
shape (GHSA-82G8-464F-2MV7); it is a test-only env swap.
* fix(browser): drain download saves and use monotonic cursor for act response
- Add drainPendingDownloadSaves() to wait for in-flight saveAs before sampling
- Use monotonic downloadSeq cursor instead of bounded-list length
- Propagate downloads info in POST /act response for click/batch/evaluate
Fixes#93250
* fix(lint): add braces around single-line if returns
Fixes eslint(curly) failures in drainPendingDownloadSaves and
pickNewDownloads. PR #93307 required CI gate.
Ref: ClawSweeper P1 review finding
* fix(browser): scope act download metadata to action
* fix(browser): broadcast downloads to all active captures to prevent misattribution
When concurrent /act calls overlap on the same page, using
state.actionDownloadCaptures.at(-1) assigned downloads to the wrong
action's capture. Push managed save promises to all active captures
so the triggering action always receives its download metadata.
Also adds regression tests: broadcast-to-all-captures, sequential
capture isolation, and strengthen the dispose test to assert no
re-capture after disposal.
* fix(browser): prevent unhandled rejection when download capture action throws before drain
Move managedSave.catch() before the captures-branch check so the
rejection handler always runs, preventing an unhandled promise
rejection when the action throws before drain() is called. Simplify
the handler by removing the now-dead return + catch at the bottom.
* fix(browser): type downloads in BrowserActResponse, fix lint unused var
- Add optional downloads field to BrowserActResponse type contract so
typed callers of browserAct() can consume the new payload without casts.
- Remove unused afterDispose variable in pw-session tests (lint fix).
* fix(test): correct post-dispose download assertion
capture.promises is not cleared by dispose(), so re-draining a disposed
capture still returns pre-dispose results. This is benign — callers
should drain before disposing. Update the test to assert the actual
behavior (still shows old results, new download not captured).
* test(browser): cover /act download metadata response
* refactor(browser): report action-owned downloads safely
* fix(browser): close action download ownership races
* test(browser): type action download capture mock
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Clarify that `networkidle` is supported for managed and raw-CDP browser sessions but rejected for existing-session mode.
Fixes#80587.
Co-authored-by: ZengWen-DT <ceng.wen@xydigit.com>
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Notte exposes a CDP-compatible WebSocket gateway at
wss://us-prod.notte.cc/sessions/connect?token=<NOTTE_API_KEY> that
auto-creates a session on connect — the same shape OpenClaw's existing
"Direct WebSocket CDP providers" section was generically framed for
(per #31085).
Real behaviour proof (against wss://us-prod.notte.cc/sessions/connect):
$ openclaw browser --browser-profile notte open https://example.com
opened: https://example.com/
tab: t4
id: 7FE04AC44931A6E1C799DE4ABF0DC807
A screenshot captured against the same session is a 1254x1111 PNG of
the rendered example.com page.
Playwright connectOverCDP flow against the same URL (today):
connectOverCDP 695ms
context.newCDPSession(page) 169ms
session.send('Target.getTargetInfo') → targetId 87ms
page.goto('https://example.com') 631ms
total 1.8s
AI-assisted (Claude Opus 4.7). codex review --base origin/main returned
clean. See PR description for the full pre-flight checklist.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(browser): note tilde expansion also covers per-profile paths
The 95a2c9b fix expanded "~" for both `browser.executablePath` and
per-profile `profiles.<name>.executablePath` (config.ts:382 calls
`normalizeExecutablePath` for profile overrides). Per-profile
`userDataDir` on existing-session profiles is also tilde-expanded
(config.ts:391 via `resolveUserPath`). The configuration reference
only mentioned the top-level `browser.executablePath` case.
* docs(browser): align tilde path config help
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* docs(browser): document local startup timeout bounds
The new browser.localLaunchTimeoutMs and browser.localCdpReadyTimeoutMs
options are clamped to MAX_BROWSER_STARTUP_TIMEOUT_MS (120000 ms) by
normalizeStartupTimeoutMs in extensions/browser/src/browser/config.ts,
and zero/negative/non-finite values fall back to the defaults. Without
this in the configuration reference, users setting a higher value see
no error and silently get the 120 s ceiling, or set 0 expecting 'no
timeout' and silently get the default.
* docs(browser): clarify startup timeout validation
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Vincent's commit ab1d1a5c9e (#71560) added user-facing config keys to
existing-session profiles for the Chrome DevTools MCP launch path:
- browser.profiles.<name>.mcpCommand
- browser.profiles.<name>.mcpArgs
Plus runtime behavior changes:
- cdpUrl http(s) -> --browserUrl, cdpUrl ws(s) -> --wsEndpoint
- endpoint flags and userDataDir are mutually exclusive
The CHANGELOG entry covered the change, but docs/tools/browser.md
existing-session reference did not. Add a 'Custom Chrome MCP launch'
subsection describing the new fields and the cdpUrl endpoint mapping
rules.
Sweep recent (last ~5h) doc edits for two readability/uniformity issues:
- Replace 42 path-as-text links of the form '[/foo/bar](/foo/bar)' with
descriptive labels derived from each target page's frontmatter title
(e.g. '[Anthropic]', '[Token use and costs]', '[OpenAI-compatible
endpoints]'). Affected files include gateway/troubleshooting,
concepts/oauth, reference/session-management-compaction, and
reference/transcript-hygiene.
- Sentence-case Title-Cased headings and link text in Related sections
across codex-harness, model-providers, tools/plugin, sdk-runtime,
sdk-setup, prompt-caching, ci, cli/config, google-meet, browser,
rich-output-protocol, subagents, web/control-ui, while preserving
brand and proper-noun capitalization (OpenAI, Codex, Chrome, Parallels,
Z.AI, etc.).
