* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Notte exposes a CDP-compatible WebSocket gateway at
wss://us-prod.notte.cc/sessions/connect?token=<NOTTE_API_KEY> that
auto-creates a session on connect — the same shape OpenClaw's existing
"Direct WebSocket CDP providers" section was generically framed for
(per #31085).
Real behaviour proof (against wss://us-prod.notte.cc/sessions/connect):
$ openclaw browser --browser-profile notte open https://example.com
opened: https://example.com/
tab: t4
id: 7FE04AC44931A6E1C799DE4ABF0DC807
A screenshot captured against the same session is a 1254x1111 PNG of
the rendered example.com page.
Playwright connectOverCDP flow against the same URL (today):
connectOverCDP 695ms
context.newCDPSession(page) 169ms
session.send('Target.getTargetInfo') → targetId 87ms
page.goto('https://example.com') 631ms
total 1.8s
AI-assisted (Claude Opus 4.7). codex review --base origin/main returned
clean. See PR description for the full pre-flight checklist.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* docs(browser): note tilde expansion also covers per-profile paths
The 95a2c9b fix expanded "~" for both `browser.executablePath` and
per-profile `profiles.<name>.executablePath` (config.ts:382 calls
`normalizeExecutablePath` for profile overrides). Per-profile
`userDataDir` on existing-session profiles is also tilde-expanded
(config.ts:391 via `resolveUserPath`). The configuration reference
only mentioned the top-level `browser.executablePath` case.
* docs(browser): align tilde path config help
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* docs(browser): document local startup timeout bounds
The new browser.localLaunchTimeoutMs and browser.localCdpReadyTimeoutMs
options are clamped to MAX_BROWSER_STARTUP_TIMEOUT_MS (120000 ms) by
normalizeStartupTimeoutMs in extensions/browser/src/browser/config.ts,
and zero/negative/non-finite values fall back to the defaults. Without
this in the configuration reference, users setting a higher value see
no error and silently get the 120 s ceiling, or set 0 expecting 'no
timeout' and silently get the default.
* docs(browser): clarify startup timeout validation
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Vincent's commit ab1d1a5c9e (#71560) added user-facing config keys to
existing-session profiles for the Chrome DevTools MCP launch path:
- browser.profiles.<name>.mcpCommand
- browser.profiles.<name>.mcpArgs
Plus runtime behavior changes:
- cdpUrl http(s) -> --browserUrl, cdpUrl ws(s) -> --wsEndpoint
- endpoint flags and userDataDir are mutually exclusive
The CHANGELOG entry covered the change, but docs/tools/browser.md
existing-session reference did not. Add a 'Custom Chrome MCP launch'
subsection describing the new fields and the cdpUrl endpoint mapping
rules.
Sweep recent (last ~5h) doc edits for two readability/uniformity issues:
- Replace 42 path-as-text links of the form '[/foo/bar](/foo/bar)' with
descriptive labels derived from each target page's frontmatter title
(e.g. '[Anthropic]', '[Token use and costs]', '[OpenAI-compatible
endpoints]'). Affected files include gateway/troubleshooting,
concepts/oauth, reference/session-management-compaction, and
reference/transcript-hygiene.
- Sentence-case Title-Cased headings and link text in Related sections
across codex-harness, model-providers, tools/plugin, sdk-runtime,
sdk-setup, prompt-caching, ci, cli/config, google-meet, browser,
rich-output-protocol, subagents, web/control-ui, while preserving
brand and proper-noun capitalization (OpenAI, Codex, Chrome, Parallels,
Z.AI, etc.).
* fix(browser): discover CDP websocket from bare ws:// URL before attach
When browser.cdpUrl is set to a bare ws://host:port (no /devtools/ path), ensureBrowserAvailable would call isChromeReachable -> canOpenWebSocket against the URL verbatim. Chrome only accepts WebSocket upgrades at the specific path returned by /json/version, so the handshake failed immediately with HTTP 400. With attachOnly: true, that surfaced as:
Browser attachOnly is enabled and profile "openclaw" is not running.
even though the CDP endpoint was reachable and the profile was healthy. Reproduced by the new tests in chrome.test.ts and cdp.test.ts (#68027).
Fix: introduce isDirectCdpWebSocketEndpoint(url) — true only when a ws/wss URL has a /devtools/<kind>/<id> handshake path. Route any other ws/wss cdpUrl (including the bare ws://host:port shape) through HTTP /json/version discovery by normalising the scheme via the existing normalizeCdpHttpBaseForJsonEndpoints helper. Apply this in isChromeReachable, getChromeWebSocketUrl, and createTargetViaCdp. Direct WS endpoints with a /devtools/ path are still opened without an extra discovery round-trip.
Fixes#68027
* test(browser): add seeded fuzz coverage for CDP URL helpers
Adds property-based / seeded-fuzz tests for the URL helpers the
attachOnly CDP fix depends on (#68027):
- isWebSocketUrl
- isDirectCdpWebSocketEndpoint
- normalizeCdpHttpBaseForJsonEndpoints
- parseBrowserHttpUrl
- redactCdpUrl
- appendCdpPath
- getHeadersWithAuth
Follows the existing repo convention (see
src/gateway/http-common.fuzz.test.ts): no fast-check dep, small
mulberry32 PRNG + hand-rolled generators, deterministic per-describe
seeds so failures are reproducible.
Lifts cdp.helpers.ts coverage from 77.77% -> 89.54% statements,
67.9% -> 80.24% branches, 78% -> 90% lines. Remaining uncovered
lines are inside the WS sender internals (createCdpSender,
withCdpSocket, fetchCdpChecked rate-limit branch), which require
integration-style mocks and are unrelated to the attachOnly fix.
* test(browser): drive cdp.helpers/cdp/chrome to 100% coverage
Lifts the three files touched by the #68027 attachOnly fix to 100% statements/branches/functions/lines across the extensions test suite. Adds cdp.helpers.internal.test.ts, cdp.internal.test.ts, and chrome.internal.test.ts covering error paths, branch matrices, CDP session helpers, Chrome spawn/launch/stop flows, and canRunCdpHealthCommand. Defensively unreachable guards are annotated with c8 ignore + inline justifications.
* fix(browser): restore WS fallback for non-/devtools ws:// CDP URLs
When /json/version discovery is unavailable (or returns no
webSocketDebuggerUrl), fall back to treating the original bare ws/wss
URL as a direct WebSocket endpoint. This preserves the #68027 fix for
Chrome's debug port while restoring compatibility with Browserless/
Browserbase-style providers that expose a direct WebSocket root without
a /json/version endpoint.
Priority order for bare ws/wss cdpUrl inputs:
1. /devtools/<kind>/<id> URL \u2192 direct handshake, no discovery (unchanged)
2. bare ws/wss root \u2192 try HTTP discovery first; if discovery returns a
webSocketDebuggerUrl use it; otherwise fall back to the original URL
as a direct WS endpoint
3. HTTP/HTTPS URL \u2192 HTTP discovery only, no fallback (unchanged)
Affected call sites: isChromeReachable, getChromeWebSocketUrl,
createTargetViaCdp.
Also renames a misleading test ('still enforces SSRF policy for direct
WebSocket URLs') to accurately describe what it tests: SSRF enforcement
on the navigation target URL, not on the CDP endpoint.
New tests added for all three fallback paths. Coverage remains 100% on
all three touched files (238 tests).
* fix: browser attachOnly bare ws CDP follow-ups (#68715) (thanks @visionik)
Add cross-linking Related sections to tool pages that were dead ends:
- exec, exec-approvals, browser, pdf, skills, lobster
Each page now links to 2-4 related topics for navigation continuity.
