Add a bounded `chat.message.get` gateway method so Control UI can fetch one display-normalized transcript message by id when an assistant history preview was truncated. Keep `chat.history` lightweight, reject oversized/hidden/missing rows with explicit unavailable reasons, and wire the WebChat side reader to request full content only for visible truncated assistant messages.
Also refresh the generated Swift gateway protocol models and document the new assistant-message side-reader behavior.
Closes#84651.
Related #53242.
Co-authored-by: NianJiuZst <3235467914@qq.com>
Extract shared normalization/coercion helpers into private @openclaw/normalization-core workspace package while preserving existing plugin SDK helper subpaths.\n\nAlso keeps direct normalization-core imports internal, wires UI/build/loader resolution, and replaces the slow PR network CodeQL lane with a fast added-line boundary scan while retaining full CodeQL for scheduled/manual runs.\n\nVerification: local moved tests, plugin SDK boundary tests, extension loader tests, agents-support shard, UI build/test, build artifacts, lint, workflow guards, autoreview, and GitHub CI passed on PR head 963d893715.
Move Workboard durable data into a relational SQLite database and add extension doctor migration for .28 plugin-state rows. Preserve attachment lifecycle behavior, SQLite permissions/WAL settings, and scoped plugin migration access.
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Add MCP server add/configure/login/reload flows plus config/runtime support for enablement, filters, timeouts, OAuth, TLS, and parallel execution hints. Update docs and tests for the expanded MCP operator surface.
Treat OpenAI-compatible streaming tool deltas as executable only when the final finish reason is `tool_calls`. This prevents malformed provider streams from triggering spurious tool execution while preserving normal tool-call responses.
Fixes#85161.
Verification:
- Local OpenAI-compatible SSE replay: spurious stop stream `finalToolCalls: 0`; valid tool-call stream `finalToolCalls: 1`.
- `pnpm test src/agents/openai-transport-stream.test.ts src/llm/providers/openai-completions.test.ts -- --reporter=verbose`
- PR CI green on `cdc2fc34753492c862cae99b37f8cf3761d9bbed`.
Co-authored-by: 忻役 <xinyi@mininglamp.com>
Co-authored-by: Jerry-Xin <jerryxin0@gmail.com>
Preserve plugin-resolved cron delivery targets after target resolution so provider-looking canonical target prefixes are not stripped before outbound delivery.
Adds regression coverage for plugin canonical targets returned directly and via aliases, plus a guard that generic normalized fallback targets still strip the selected prefix.
Fixes#87905
Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com>
Surface gateway chat failures as visible assistant messages in the Control UI, with regression coverage and Crabbox/WebVNC proof.
(cherry picked from commit 31a46638ad)
Fixes#63558.
Adds a Dreaming-tab agent selector and propagates the selected agent through Dreaming status, diary, and diary actions while preserving default-agent fallback when agentId is omitted. Also keeps report Memory Palace cards in the Control UI wiki-preview flow and documents the optional Dreaming agentId gateway parameters.
Verification:
- GitHub CI run 26693682975 passed on 43a2b17243.
- CodeQL Critical Quality run 26693682971 passed.
- CodeQL / Security High run 26693682957 passed.
- Workflow Sanity run 26693682949 passed.
- OpenGrep PR Diff run 26693682947 passed.
- Dependency Guard run 26693682003 passed.
- Real behavior proof run 26693860539 passed.
- git diff --check origin/main...refs/remotes/origin/pr/78748 passed.
- git merge-tree --write-tree origin/main refs/remotes/origin/pr/78748 passed.
Thanks @stevenepalmer.
Co-authored-by: Steven Palmer <6134396+stevenepalmer@users.noreply.github.com>
Extracts serialized plaintext tool-call parsing, scrubbing, stream normalization, and standalone promotion into the private internal @openclaw/tool-call-repair package.
Provider wrappers and the embedded runner now share one repair path for standalone serialized tool calls, including adjacent text-block splits, while preserving exact argument bytes when already valid. The public plugin SDK payload module remains as the compatibility facade.
Verification:
- pnpm test src/plugin-sdk/provider-stream-shared.test.ts src/plugin-sdk/tool-payload.test.ts src/agents/embedded-agent-runner/run/attempt.tool-call-normalization.test.ts -- --reporter=verbose
- env -u OPENCLAW_TESTBOX pnpm check:changed
- PR CI: all reported checks green/skipped/neutral on ff0b3c0a5c
Refs #86924
Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
* fix(responses): drop orphaned assistant msg_* id when reasoning is dropped (#88019)
When an Azure/OpenAI Responses session falls back to a non-Responses model
and later resumes a Responses model, sanitizeSessionHistory drops the
replayable reasoning (rs_*) item via downgradeOpenAIReasoningBlocks. The
paired assistant text block still carried its textSignature (the msg_* id),
so the transport replayed an assistant message item referencing msg_* with
no accompanying rs_* reasoning item. Azure Responses then rejected the next
turn with:
400 Item 'msg_...' provided without its required 'reasoning' item: 'rs_...'
permanently poisoning the session.
Fix:
- downgradeOpenAIReasoningBlocks now strips the textSignature from a turn's
text blocks whenever it drops a replayable reasoning item, so the msg_* id
and its rs_* reasoning are removed together. The transport then falls back
to a synthetic, unpaired id that Azure accepts.
- Because the synthetic fallback id is derived from the per-message msgIndex,
multiple id-less text blocks in one assistant turn (e.g. commentary +
final_answer) would collide on the same id. Make the fallback unique per
text block in both Responses conversion sites
(openai-transport-stream.ts and the shared llm provider
openai-responses-shared.ts).
Tests:
- sanitize-session-history: model-switch path drops the paired msg_* id.
- embedded-agent-helpers: downgrade strips paired text signature(s).
- reasoning-replay: multiple id-less text blocks get distinct item ids.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(responses): preserve phase metadata and guard malformed blocks (#88019)
Address PR review feedback on the orphaned msg_* replay fix:
- Preserve Responses phase metadata: dropping the paired msg_* id when its
rs_* reasoning is removed previously stripped the entire textSignature,
which also discarded the phase (commentary/final_answer). Phased text now
keeps a phase-only signature ({v:1,phase}) so commentary is not replayed
as user-visible output. Both parseTextSignature copies (shared provider and
embedded transport) now accept id-less phase-only signatures and fall back
to a synthetic id while preserving the phase.
- Guard malformed content blocks: the post-drop map no longer dereferences
contentBlock.type unconditionally, so a corrupted transcript with a
null/primitive block can still sanitize through a model switch.
Tests:
- sanitize-session-history: phase metadata is preserved while the paired id
is dropped on a model switch.
- reasoning-replay: id-less phase-only signatures get distinct synthetic ids
and retain their phase.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Payload-less session write-lock files now get a 30s grace for default/long acquire timeouts and cleanup sweeps, while short acquire timeouts keep 5s recovery. This avoids reclaiming a lock while the owner is suspended between exclusive create and metadata write.
Verified with:
- git diff --check origin/main...HEAD
- .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --parallel-tests "node scripts/run-vitest.mjs src/agents/session-write-lock.test.ts"
- gh pr checks 80686 --repo openclaw/openclaw --watch=false
Thanks @wAngByg.
Extract web-content shared runtime helpers into packages/web-content-core, move the focused tests with the new package, and split quiet CI shards so the node matrix no longer stalls past the no-output watchdog.\n\nVerification: node scripts/run-vitest.mjs test/scripts/ci-node-test-plan.test.ts test/scripts/run-vitest.test.ts src/infra/restart.test.ts src/infra/os-summary.test.ts src/infra/gateway-processes.test.ts src/infra/inline-option-token.test.ts src/infra/map-size.test.ts src/infra/machine-name.test.ts src/commands/doctor-whatsapp-responsiveness.test.ts; autoreview clean; manual CI https://github.com/openclaw/openclaw/actions/runs/26693962844; dependency guard https://github.com/openclaw/openclaw/actions/runs/26693959937. Admin merge used because optional Mantis Telegram Desktop proof was cancelled after blocking merge outside this PR's required proof.
Bound MCP channel bridge pending Claude permission and approval maps with TTL sweep and close cleanup.
Also sweep before listing pending approvals so expired requests are not exposed between periodic ticks.
Fixes#71646.
Thanks @Feelw00.
Adds `openclaw sessions tail` as an operator-facing progress view over session trajectory events, with conservative redaction for prompt text, tool arguments, and tool result bodies. The command supports explicit session keys, store/agent scope, follow mode, relocated trajectory pointer files, and cursor-safe follow across bounded trajectory window rewrites.
Documents the new sessions tail CLI surface in `docs/cli/sessions.md`.
Fixes#83441.
Co-authored-by: zhengzuo0-ai <zheng.zuo0@gmail.com>
Skip browser lifecycle cleanup when root browser support or the browser plugin entry is disabled, and make the browser maintenance facade respect activation before cached surface use.
Also stabilize the resource-only MCP runtime test by waiting for the async rejection log that CI can observe late.
Verification:
- pnpm test src/plugin-sdk/browser-maintenance.test.ts src/browser-lifecycle-cleanup.test.ts src/auto-reply/reply/session.test.ts src/gateway/server.sessions.reset-cleanup.test.ts src/agents/auth-profiles/usage.test.ts
- pnpm test src/agents/agent-bundle-mcp-runtime.test.ts
- git diff --check
- pnpm build
- autoreview local: no accepted/actionable findings
- GitHub Actions: CI 26693713166, CodeQL 26693713159, CodeQL Critical Quality 26693713157, OpenGrep PR Diff 26693713125, Workflow Sanity 26693713149, Dependency Guard 26693712478
Co-authored-by: Nicolas Van Eenaeme <nicolas@poison.be>
Deduplicate the browser lifecycle cleanup wrapper for embedded subagent completions while preserving retire and announce finalization for duplicate callers.\n\nAdds regression coverage for parallel completion callers and the held-first-cleanup duplicate-tail path.\n\nFixes #68668.\n\nCo-authored-by: Feelw00 <dhrtn1006@naver.com>
