* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Refactor OpenAI provider identity so OpenAI remains the canonical provider for API-key and OAuth-backed flows while legacy openai-codex state is doctor/migration-only.
Keeps OpenAI Codex Responses as an API/transport class rather than a provider identity, moves auth aliases through providerAuthAliases, updates doctor repair sequencing for old auth/profile state, and refreshes tests/docs around the canonical OpenAI behavior.
Notte exposes a CDP-compatible WebSocket gateway at
wss://us-prod.notte.cc/sessions/connect?token=<NOTTE_API_KEY> that
auto-creates a session on connect — the same shape OpenClaw's existing
"Direct WebSocket CDP providers" section was generically framed for
(per #31085).
Real behaviour proof (against wss://us-prod.notte.cc/sessions/connect):
$ openclaw browser --browser-profile notte open https://example.com
opened: https://example.com/
tab: t4
id: 7FE04AC44931A6E1C799DE4ABF0DC807
A screenshot captured against the same session is a 1254x1111 PNG of
the rendered example.com page.
Playwright connectOverCDP flow against the same URL (today):
connectOverCDP 695ms
context.newCDPSession(page) 169ms
session.send('Target.getTargetInfo') → targetId 87ms
page.goto('https://example.com') 631ms
total 1.8s
AI-assisted (Claude Opus 4.7). codex review --base origin/main returned
clean. See PR description for the full pre-flight checklist.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Add a general typed tool-progress contract so long-running non-exec tools can emit public channel progress without overloading model-facing tool content.
`web_fetch` now uses the generic delayed progress helper: it shows `Fetching page content...` only when the fetch is still pending after five seconds, clears the timer on completion/abort, passes the abort signal into guarded fetch, and avoids provider fallback or cached success after cancellation. The subscriber path accepts only explicit `visibility: "channel"` and `privacy: "public"` progress metadata, while untyped tool partials and exec output keep their existing behavior.
Docs now explain typed progress, delayed producer examples, and the `web_fetch` timing behavior.
Proof: `pnpm test src/agents/tools/web-tools.fetch.test.ts src/agents/embedded-agent-subscribe.handlers.tools.test.ts -- --run`; `pnpm docs:check-mdx`; changed-file `pnpm exec oxlint ...`; `git diff --check`; autoreview clean.
Remove stale Telegram-only wording from the reasoning stream acknowledgement and docs so channel-neutral behavior is reflected.
Fixes#68305.
Co-authored-by: Lanzhi <lizhan3@xiaomi.com>
* feat: add Claude Opus 4.8 support
* fix: omit Vertex Opus sampling overrides
* fix: preserve Opus adaptive thinking levels
* fix: clamp Anthropic max effort support
* fix: use sha256 for QA mock call ids
* fix: type Anthropic transport test model metadata
* test: update PDF model default for Opus 4.8
Support grouped skill folders while keeping skill invocation flat via frontmatter names.
Includes bounded nested SKILL.md discovery, refresh/watch coverage for grouped folders, plugin symlink containment, and docs for grouped skill organization.
Verification:
- Node 24 targeted skill discovery and refresh tests passed locally.
- Docs checks passed locally and in CI.
- Autoreview clean.
- Crabbox live OpenAI proof showed nested foo/bar skills listed and visible in the agent system prompt.
- CI run 26595118581 passed.
Split the diffs viewer Shiki language pack into an external publishable plugin.
The diffs plugin keeps the default curated syntax set, while the new @openclaw/diffs-language-pack package carries the extended Shiki languages for npm and ClawHub distribution. The install metadata includes the external ClawHub spec, and the curated C# alias set keeps both c# and cs supported without the language pack.
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
Move vLLM Qwen thinking control onto configured model compat metadata and carry it through catalog/model-selection/runtime thinking contexts.
Also migrate legacy provider/default request params in doctor and keep Pi/runtime model rows buildable with explicit reasoning defaults.
Thanks @rendrag-git.
Co-authored-by: rendrag-git <253747599+rendrag-git@users.noreply.github.com>
* fix(agents): answer Claude live control_request can_use_tool via exec policy
Claude CLI emits stream-json control_request frames with subtype
can_use_tool when it wants to use a native tool. The Claude live-session
bridge previously dropped these frames, leaving Claude waiting for a
control_response until the 180/600s no-output timeout fired (see #80819).
Resolve the effective OpenClaw exec policy (per-agent tools.exec -> global
tools.exec -> allowlist/on-miss defaults) once at session-start time and
thread it through fingerprinting and the session record. When a
can_use_tool request arrives:
- Allow native Bash when the resolved policy is security=full, ask=off
(matching the bypassPermissions semantics OpenClaw already documents).
- Otherwise deny with a message that names the resolved policy and
points the agent at OpenClaw MCP tools.
Unsupported control_request subtypes get a structured error response
instead of a silent no-op, and stray control_response frames are
silently dropped. Adds spawn-test coverage for both allow and deny paths.
Fixes#80819
* fix(agents): align Claude live control_request policy with backend defaults
Resolve the effective exec policy through the same defaults that
extensions/anthropic/cli-shared.ts:isOpenClawRequestedYolo and
src/agents/exec-defaults.ts:resolveExecDefaults already use (security
?? "full", ask ?? "off") instead of falling back to a hand-rolled
allowlist/on-miss default that disagreed with the rest of the codebase.
Without this, a default-config OpenClaw deployment launches Claude with
--permission-mode bypassPermissions but the bridge would still deny
Bash control_requests, re-creating the #80819 stall for the very
default-config case the issue reports.
Also thread the effective Claude permission mode into the policy
decision. Prefer the operator's explicit --permission-mode in argv,
falling back to what normalizeClaudePermissionArgs would have inserted
for an un-overridden launch. Native Bash is auto-allowed only when the
effective mode is bypassPermissions AND tools.exec resolves to
full/no-ask, so explicit raw-arg overrides like --permission-mode
default or acceptEdits broaden Claude's native prompting and are
honored by routing through deny.
Adds a no-config regression test (default deployment allows Bash, no
stall) and a permission-mode-override test (tools.exec full/off plus
explicit --permission-mode default in raw args denies). Existing
allow/deny tests continue to pass via the synthesized-mode fallback.
* fix(agents): honor effective exec policy for Claude live Bash
---------
Co-authored-by: Guillaume Thirry <g.thirry@gmail.com>
