* feat(browser): restore driver "extension" via loopback Chrome extension relay
Reintroduces browser profile driver "extension" (removed in 2026.3.22) as a
loopback relay that drives the user's signed-in Chrome through an MV3 extension
instead of the remote-debugging port. This avoids Chrome's blocking "Allow
remote debugging?" prompt, which cannot be clicked when the operator drives
OpenClaw from a phone. Automated tabs live in an "OpenClaw" tab group (the
consent boundary), mirroring the Codex/Claude-in-Chrome model.
- relay bridge synthesizes the CDP browser target surface for Playwright
connectOverCDP and forwards session-scoped commands to chrome.debugger
- relay server binds loopback only; both sides authenticate with a token
derived (HMAC-SHA256) from gateway auth, so the raw credential never reaches
Chrome; extension origin + loopback Host checks guard the upgrade
- built-in "chrome" profile; distinct relay ports per extension profile;
relay reconciles on auth rotation / cdpPort change and prunes removed profiles
- doctor + status surface the extension transport; doctor keeps repairing the
retired relay endpoint URL on legacy "extension" profiles
Refs #53599
* feat(browser): bundle the OpenClaw MV3 Chrome extension
Thin MV3 extension (chrome-extension/): a WebSocket client to the loopback
relay plus chrome.debugger forwarding and OpenClaw tab-group management. All
CDP target synthesis lives server-side in the relay bridge, so the extension
stays a dumb transport (the removed 2026.3 extension put that logic in a
1000-line untestable service worker). Popup handles pairing and per-tab share
toggle; `openclaw browser extension path|pair` load and pair it. A build copy
hook stages it into dist so the load path is stable.
Refs #53599
* docs(browser): document the Chrome extension profile
Adds docs/tools/chrome-extension for the restored extension driver (install,
pair, tab-group consent model, security posture) and wires it into the browser
docs profile section and nav.
Refs #53599
* feat(browser): make the extension relay work on remote browser nodes
Derives the relay auth token from a host-local secret in the credentials dir
(created on first use) instead of gateway auth. Each machine that runs a
browser — the gateway host and every browser node host — owns its own token, so
the extension pairs with whichever machine hosts its Chrome and no gateway
credential travels to a node. The node host already runs the shared browser
control bootstrap, so this is all that was missing for cross-machine control.
Also removes the "relay needs gateway auth before it can start" failure mode:
startup and `openclaw browser extension pair` ensure the secret exists.
Refs #53599
* fix(browser): harden relay secret creation and satisfy CI lint/typecheck
- Make the host-local relay secret creation atomic (O_CREAT|O_EXCL + adopt the
winner on EEXIST) so the gateway service and `extension pair` CLI cannot mint
divergent tokens on a fresh host (would 401 until restart); credentials dir
created mode 0700. (adversarial review finding)
- Resolve type-aware oxlint findings across the relay + extension: unknown catch
vars, addEventListener over ws.on* in the MV3 worker, void async listeners,
drop useless returns/spreads, Object.assign over map-spread, safe ws frame
decode (Buffer[]/ArrayBuffer), toSorted.
- Add extensionRelayDefaultPort/extensionRelayPorts to remaining test config
literals; type the extension relay-core module (.d.ts, excluded from dist);
regenerate docs_map.
* fix(browser): satisfy OpenGrep security policy on the relay
- Hash both operands before timingSafeEqual so token comparison has no
length short-circuit (GHSA-JJ6Q-RRRF-H66H).
- Bound the relay WebSocketServer with maxPayload (64 MiB, headroom for CDP
screenshots/bodies) against oversized frames (GHSA-VW3H-Q6XQ-JJM5).
- Rewrite the config test env helper to avoid the skill-env-host-injection
shape (GHSA-82G8-464F-2MV7); it is a test-only env swap.
The /set/media control route accepts dark|light|no-preference|none, but the
CLI rejected no-preference, so prefers-color-scheme: no-preference could not
be emulated from the CLI. Align the CLI argument, validation, and error text
with the route contract and update the browser-control docs media line.
Summary:
- The branch adds Vitest coverage for browser action-input CLI request bodies across element, navigation/resize, fill/evaluate, and upload paths, plus blank-ref validation.
- PR surface: Tests +278. Total +278 across 4 files.
- Reproducibility: yes. for a source-level coverage gap: current main exposes the browser action-input command ... isting tests still lack broad success-path request-body assertions. This is not a runtime bug reproduction.
Automerge notes:
- PR branch already contained follow-up commit before automerge: test(browser): cover click-coords action body
Validation:
- ClawSweeper review passed for head c070a8d51b.
- Required merge gates passed before the squash merge.
Prepared head SHA: c070a8d51b
Review: https://github.com/openclaw/openclaw/pull/92574#issuecomment-4697124920
Co-authored-by: Stellar鱼 <2182712990@qq.com>
Co-authored-by: yu-xin-c <2182712990@qq.com>
Remove the unused requireRef import and void anchor from Browser navigation command registration while keeping navigate/resize registration covered by regression tests. Fixes#83878.
Summary:
- The branch documents friendly browser tab references across docs, the browser skill, CLI help, and tool schema descriptions, and adds tests for target reference resolution and tab alias behavior.
- PR surface: Source +24, Tests +328, Docs +9. Total +361 across 21 files.
- Reproducibility: yes. for the documentation mismatch by source inspection: current main supports friendly ta ... schema/help surfaces still emphasize raw CDP target ids. Runtime behavior itself is not a new failing path.
Automerge notes:
- PR branch already contained follow-up commit before automerge: refactor(browser): share tab reference CLI help
Validation:
- ClawSweeper review passed for head 118af80b0b.
- Required merge gates passed before the squash merge.
Prepared head SHA: 118af80b0b
Review: https://github.com/openclaw/openclaw/pull/88393#issuecomment-4583558133
Co-authored-by: FMLS <kfliuyang@gmail.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: hxy91819
Co-authored-by: hxy91819 <8814856+hxy91819@users.noreply.github.com>
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Summary:
- The branch adds `openclaw browser evaluate --timeout-ms`, forwards it to the evaluate body and request timeo ... ents and tests it, adds a changelog entry, and includes a config.patch no-op shortcut from the repair pass.
- Reproducibility: not applicable. this is a feature PR rather than a bug report. Source inspection shows current main lacks the CLI flag while the branch wires it into an already-supported evaluate `timeoutMs` payload.
Automerge notes:
- PR branch already contained follow-up commit before automerge: feat(browser): add evaluate timeout CLI option
Validation:
- ClawSweeper review passed for head 0d81d3d93e.
- Required merge gates passed before the squash merge.
Prepared head SHA: 0d81d3d93e
Review: https://github.com/openclaw/openclaw/pull/83696#issuecomment-4479900502
Co-authored-by: fred <fengruifree@gmail.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
