Adds Xiaomi MiMo voicedesign TTS support by registering the v2.5 voicedesign model and omitting audio.voice for that model's prompt-driven voice design flow.
Also accepts generic TTS aliases modelId, speakerVoice, and speakerVoiceId for Xiaomi provider config and request overrides.
Fixes exec timeout classification so a process that exits after a missed timeout callback is still reported as timed out, using monotonic deadlines to avoid wall-clock skew.
Verification:
- node scripts/run-vitest.mjs extensions/xiaomi/speech-provider.test.ts
- node scripts/run-vitest.mjs src/process/supervisor/supervisor.test.ts
- node scripts/run-vitest.mjs src/agents/bash-tools.exec-foreground-failures.test.ts
- git diff --check
- autoreview --mode local
- live Xiaomi MiMo voicedesign call returned wav RIFF/WAVE output, 169004 bytes
- GitHub CI success on fb3018ef31: CI 26708919072, CodeQL Critical Quality 26708919082, CodeQL 26708919091, OpenGrep PR Diff 26708919089, Workflow Sanity 26708919083, Dependency Guard 26708918574, Real behavior proof 26708921767
Thanks @GimingRao.
Co-authored-by: Raoyu <2425198313@qq.com>
Co-authored-by: giming <53329020+GimingRao@users.noreply.github.com>
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Refactor OpenAI provider identity so OpenAI remains the canonical provider for API-key and OAuth-backed flows while legacy openai-codex state is doctor/migration-only.
Keeps OpenAI Codex Responses as an API/transport class rather than a provider identity, moves auth aliases through providerAuthAliases, updates doctor repair sequencing for old auth/profile state, and refreshes tests/docs around the canonical OpenAI behavior.
Notte exposes a CDP-compatible WebSocket gateway at
wss://us-prod.notte.cc/sessions/connect?token=<NOTTE_API_KEY> that
auto-creates a session on connect — the same shape OpenClaw's existing
"Direct WebSocket CDP providers" section was generically framed for
(per #31085).
Real behaviour proof (against wss://us-prod.notte.cc/sessions/connect):
$ openclaw browser --browser-profile notte open https://example.com
opened: https://example.com/
tab: t4
id: 7FE04AC44931A6E1C799DE4ABF0DC807
A screenshot captured against the same session is a 1254x1111 PNG of
the rendered example.com page.
Playwright connectOverCDP flow against the same URL (today):
connectOverCDP 695ms
context.newCDPSession(page) 169ms
session.send('Target.getTargetInfo') → targetId 87ms
page.goto('https://example.com') 631ms
total 1.8s
AI-assisted (Claude Opus 4.7). codex review --base origin/main returned
clean. See PR description for the full pre-flight checklist.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Add a general typed tool-progress contract so long-running non-exec tools can emit public channel progress without overloading model-facing tool content.
`web_fetch` now uses the generic delayed progress helper: it shows `Fetching page content...` only when the fetch is still pending after five seconds, clears the timer on completion/abort, passes the abort signal into guarded fetch, and avoids provider fallback or cached success after cancellation. The subscriber path accepts only explicit `visibility: "channel"` and `privacy: "public"` progress metadata, while untyped tool partials and exec output keep their existing behavior.
Docs now explain typed progress, delayed producer examples, and the `web_fetch` timing behavior.
Proof: `pnpm test src/agents/tools/web-tools.fetch.test.ts src/agents/embedded-agent-subscribe.handlers.tools.test.ts -- --run`; `pnpm docs:check-mdx`; changed-file `pnpm exec oxlint ...`; `git diff --check`; autoreview clean.
Remove stale Telegram-only wording from the reasoning stream acknowledgement and docs so channel-neutral behavior is reflected.
Fixes#68305.
Co-authored-by: Lanzhi <lizhan3@xiaomi.com>
* feat: add Claude Opus 4.8 support
* fix: omit Vertex Opus sampling overrides
* fix: preserve Opus adaptive thinking levels
* fix: clamp Anthropic max effort support
* fix: use sha256 for QA mock call ids
* fix: type Anthropic transport test model metadata
* test: update PDF model default for Opus 4.8
Support grouped skill folders while keeping skill invocation flat via frontmatter names.
Includes bounded nested SKILL.md discovery, refresh/watch coverage for grouped folders, plugin symlink containment, and docs for grouped skill organization.
Verification:
- Node 24 targeted skill discovery and refresh tests passed locally.
- Docs checks passed locally and in CI.
- Autoreview clean.
- Crabbox live OpenAI proof showed nested foo/bar skills listed and visible in the agent system prompt.
- CI run 26595118581 passed.
Split the diffs viewer Shiki language pack into an external publishable plugin.
The diffs plugin keeps the default curated syntax set, while the new @openclaw/diffs-language-pack package carries the extended Shiki languages for npm and ClawHub distribution. The install metadata includes the external ClawHub spec, and the curated C# alias set keeps both c# and cs supported without the language pack.
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
Move vLLM Qwen thinking control onto configured model compat metadata and carry it through catalog/model-selection/runtime thinking contexts.
Also migrate legacy provider/default request params in doctor and keep Pi/runtime model rows buildable with explicit reasoning defaults.
Thanks @rendrag-git.
Co-authored-by: rendrag-git <253747599+rendrag-git@users.noreply.github.com>
