The talk.speak synthesis path passed raw agent text to providers, so talk
mode could read fenced code, links, and tables aloud. Speech normalization
now happens once at the shared synthesis boundary (speech-mode stripMarkdown:
label-only links, decorative bullet/glyph cleanup, punctuation collapsing).
Code-heavy replies (>=50% fenced content, CommonMark-aware scanner incl.
blockquote/list containers and unclosed fences) get routed by surface:
talk.speak speaks a short on-screen fallback line, channel auto-TTS skips
the voice note since the visible text already carries the content, and
explicit conversions (tts.convert, tagged hidden TTS, chat Listen) always
speak stripped best-effort content.
* fix(tts): wrap tts.status provider enumeration with configuredOrFalse
`tts.status` enumerates provider states by calling
`getResolvedSpeechProviderConfig` then `isConfigured` for each
provider. If a provider's config resolution throws (e.g. Gradium
with an invalid baseUrl), the entire tts.status call fails.
Wrap the provider-state closure with `configuredOrFalse` (already
used by `talk.ts`) so a misconfigured provider is reported as
`configured: false` instead of crashing the enumeration.
Addresses review feedback on #108569.
* fix(tts): isolate provider configuration probes
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* test: fix environment-dependent failures on built and partial-clone checkouts
Canvas A2UI fixtures now pin the resolved root through a new canvas
test-api hook: dev checkouts with a locally built a2ui.bundle.js won
candidate resolution and shadowed the fixture root. Speech-core prefs
tests resolve the canonical tmpdir (macOS /tmp symlink breaks fs-safe
store roots), and the bundled plugin install test asks discovery for
the canonical bundled path instead of hardcoding the source tree.
* fix(release): keep evidence verifier blob reads local-deterministic
git show against a forged or missing SHA lazily fetched from the
promisor remote on partial clones, hanging a security check on network
state (266s observed). GIT_NO_LAZY_FETCH pins verification to local
objects; missing blobs fail closed as before.
* perf(test): pay the command-handler barrel once and merge subagent command tests
commands-name and commands-login imported the full handler barrel
(~1,700 modules) for registration assertions; those now live in a
dedicated registration test, dropping the two files from 12.6s/4.6s to
1.4s/1.0s locally. Five subagent command test files merged into one so
the auto-reply module graph evaluates once instead of five times.
* perf(ci): stripe node test bins by measured file cost
Round-robin striping packed the whale files together, leaving
auto-reply-commands-1 at ~220s while sibling stripes idled at ~30s.
Greedy LPT over advisory per-file seconds hints balances the commands,
cli-runner, and tooling stripes deterministically; group hints for the
restriped bins are equalized at the old totals.
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
