* feat(browser): restore driver "extension" via loopback Chrome extension relay
Reintroduces browser profile driver "extension" (removed in 2026.3.22) as a
loopback relay that drives the user's signed-in Chrome through an MV3 extension
instead of the remote-debugging port. This avoids Chrome's blocking "Allow
remote debugging?" prompt, which cannot be clicked when the operator drives
OpenClaw from a phone. Automated tabs live in an "OpenClaw" tab group (the
consent boundary), mirroring the Codex/Claude-in-Chrome model.
- relay bridge synthesizes the CDP browser target surface for Playwright
connectOverCDP and forwards session-scoped commands to chrome.debugger
- relay server binds loopback only; both sides authenticate with a token
derived (HMAC-SHA256) from gateway auth, so the raw credential never reaches
Chrome; extension origin + loopback Host checks guard the upgrade
- built-in "chrome" profile; distinct relay ports per extension profile;
relay reconciles on auth rotation / cdpPort change and prunes removed profiles
- doctor + status surface the extension transport; doctor keeps repairing the
retired relay endpoint URL on legacy "extension" profiles
Refs #53599
* feat(browser): bundle the OpenClaw MV3 Chrome extension
Thin MV3 extension (chrome-extension/): a WebSocket client to the loopback
relay plus chrome.debugger forwarding and OpenClaw tab-group management. All
CDP target synthesis lives server-side in the relay bridge, so the extension
stays a dumb transport (the removed 2026.3 extension put that logic in a
1000-line untestable service worker). Popup handles pairing and per-tab share
toggle; `openclaw browser extension path|pair` load and pair it. A build copy
hook stages it into dist so the load path is stable.
Refs #53599
* docs(browser): document the Chrome extension profile
Adds docs/tools/chrome-extension for the restored extension driver (install,
pair, tab-group consent model, security posture) and wires it into the browser
docs profile section and nav.
Refs #53599
* feat(browser): make the extension relay work on remote browser nodes
Derives the relay auth token from a host-local secret in the credentials dir
(created on first use) instead of gateway auth. Each machine that runs a
browser — the gateway host and every browser node host — owns its own token, so
the extension pairs with whichever machine hosts its Chrome and no gateway
credential travels to a node. The node host already runs the shared browser
control bootstrap, so this is all that was missing for cross-machine control.
Also removes the "relay needs gateway auth before it can start" failure mode:
startup and `openclaw browser extension pair` ensure the secret exists.
Refs #53599
* fix(browser): harden relay secret creation and satisfy CI lint/typecheck
- Make the host-local relay secret creation atomic (O_CREAT|O_EXCL + adopt the
winner on EEXIST) so the gateway service and `extension pair` CLI cannot mint
divergent tokens on a fresh host (would 401 until restart); credentials dir
created mode 0700. (adversarial review finding)
- Resolve type-aware oxlint findings across the relay + extension: unknown catch
vars, addEventListener over ws.on* in the MV3 worker, void async listeners,
drop useless returns/spreads, Object.assign over map-spread, safe ws frame
decode (Buffer[]/ArrayBuffer), toSorted.
- Add extensionRelayDefaultPort/extensionRelayPorts to remaining test config
literals; type the extension relay-core module (.d.ts, excluded from dist);
regenerate docs_map.
* fix(browser): satisfy OpenGrep security policy on the relay
- Hash both operands before timingSafeEqual so token comparison has no
length short-circuit (GHSA-JJ6Q-RRRF-H66H).
- Bound the relay WebSocketServer with maxPayload (64 MiB, headroom for CDP
screenshots/bodies) against oversized frames (GHSA-VW3H-Q6XQ-JJM5).
- Rewrite the config test env helper to avoid the skill-env-host-injection
shape (GHSA-82G8-464F-2MV7); it is a test-only env swap.
* fix(browser): drain download saves and use monotonic cursor for act response
- Add drainPendingDownloadSaves() to wait for in-flight saveAs before sampling
- Use monotonic downloadSeq cursor instead of bounded-list length
- Propagate downloads info in POST /act response for click/batch/evaluate
Fixes#93250
* fix(lint): add braces around single-line if returns
Fixes eslint(curly) failures in drainPendingDownloadSaves and
pickNewDownloads. PR #93307 required CI gate.
Ref: ClawSweeper P1 review finding
* fix(browser): scope act download metadata to action
* fix(browser): broadcast downloads to all active captures to prevent misattribution
When concurrent /act calls overlap on the same page, using
state.actionDownloadCaptures.at(-1) assigned downloads to the wrong
action's capture. Push managed save promises to all active captures
so the triggering action always receives its download metadata.
Also adds regression tests: broadcast-to-all-captures, sequential
capture isolation, and strengthen the dispose test to assert no
re-capture after disposal.
* fix(browser): prevent unhandled rejection when download capture action throws before drain
Move managedSave.catch() before the captures-branch check so the
rejection handler always runs, preventing an unhandled promise
rejection when the action throws before drain() is called. Simplify
the handler by removing the now-dead return + catch at the bottom.
* fix(browser): type downloads in BrowserActResponse, fix lint unused var
- Add optional downloads field to BrowserActResponse type contract so
typed callers of browserAct() can consume the new payload without casts.
- Remove unused afterDispose variable in pw-session tests (lint fix).
* fix(test): correct post-dispose download assertion
capture.promises is not cleared by dispose(), so re-draining a disposed
capture still returns pre-dispose results. This is benign — callers
should drain before disposing. Update the test to assert the actual
behavior (still shows old results, new download not captured).
* test(browser): cover /act download metadata response
* refactor(browser): report action-owned downloads safely
* fix(browser): close action download ownership races
* test(browser): type action download capture mock
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Clarify that `networkidle` is supported for managed and raw-CDP browser sessions but rejected for existing-session mode.
Fixes#80587.
Co-authored-by: ZengWen-DT <ceng.wen@xydigit.com>
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Notte exposes a CDP-compatible WebSocket gateway at
wss://us-prod.notte.cc/sessions/connect?token=<NOTTE_API_KEY> that
auto-creates a session on connect — the same shape OpenClaw's existing
"Direct WebSocket CDP providers" section was generically framed for
(per #31085).
Real behaviour proof (against wss://us-prod.notte.cc/sessions/connect):
$ openclaw browser --browser-profile notte open https://example.com
opened: https://example.com/
tab: t4
id: 7FE04AC44931A6E1C799DE4ABF0DC807
A screenshot captured against the same session is a 1254x1111 PNG of
the rendered example.com page.
Playwright connectOverCDP flow against the same URL (today):
connectOverCDP 695ms
context.newCDPSession(page) 169ms
session.send('Target.getTargetInfo') → targetId 87ms
page.goto('https://example.com') 631ms
total 1.8s
AI-assisted (Claude Opus 4.7). codex review --base origin/main returned
clean. See PR description for the full pre-flight checklist.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(browser): note tilde expansion also covers per-profile paths
The 95a2c9b fix expanded "~" for both `browser.executablePath` and
per-profile `profiles.<name>.executablePath` (config.ts:382 calls
`normalizeExecutablePath` for profile overrides). Per-profile
`userDataDir` on existing-session profiles is also tilde-expanded
(config.ts:391 via `resolveUserPath`). The configuration reference
only mentioned the top-level `browser.executablePath` case.
* docs(browser): align tilde path config help
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* docs(browser): document local startup timeout bounds
The new browser.localLaunchTimeoutMs and browser.localCdpReadyTimeoutMs
options are clamped to MAX_BROWSER_STARTUP_TIMEOUT_MS (120000 ms) by
normalizeStartupTimeoutMs in extensions/browser/src/browser/config.ts,
and zero/negative/non-finite values fall back to the defaults. Without
this in the configuration reference, users setting a higher value see
no error and silently get the 120 s ceiling, or set 0 expecting 'no
timeout' and silently get the default.
* docs(browser): clarify startup timeout validation
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Vincent's commit ab1d1a5c9e (#71560) added user-facing config keys to
existing-session profiles for the Chrome DevTools MCP launch path:
- browser.profiles.<name>.mcpCommand
- browser.profiles.<name>.mcpArgs
Plus runtime behavior changes:
- cdpUrl http(s) -> --browserUrl, cdpUrl ws(s) -> --wsEndpoint
- endpoint flags and userDataDir are mutually exclusive
The CHANGELOG entry covered the change, but docs/tools/browser.md
existing-session reference did not. Add a 'Custom Chrome MCP launch'
subsection describing the new fields and the cdpUrl endpoint mapping
rules.
Sweep recent (last ~5h) doc edits for two readability/uniformity issues:
- Replace 42 path-as-text links of the form '[/foo/bar](/foo/bar)' with
descriptive labels derived from each target page's frontmatter title
(e.g. '[Anthropic]', '[Token use and costs]', '[OpenAI-compatible
endpoints]'). Affected files include gateway/troubleshooting,
concepts/oauth, reference/session-management-compaction, and
reference/transcript-hygiene.
- Sentence-case Title-Cased headings and link text in Related sections
across codex-harness, model-providers, tools/plugin, sdk-runtime,
sdk-setup, prompt-caching, ci, cli/config, google-meet, browser,
rich-output-protocol, subagents, web/control-ui, while preserving
brand and proper-noun capitalization (OpenAI, Codex, Chrome, Parallels,
Z.AI, etc.).
