Extract shared normalization/coercion helpers into private @openclaw/normalization-core workspace package while preserving existing plugin SDK helper subpaths.\n\nAlso keeps direct normalization-core imports internal, wires UI/build/loader resolution, and replaces the slow PR network CodeQL lane with a fast added-line boundary scan while retaining full CodeQL for scheduled/manual runs.\n\nVerification: local moved tests, plugin SDK boundary tests, extension loader tests, agents-support shard, UI build/test, build artifacts, lint, workflow guards, autoreview, and GitHub CI passed on PR head 963d893715.
* feat(browser): add optional vision understanding to screenshot tool
* fix(browser): wrap vision output as external content, enforce maxBytes, forward auth profiles
* fix(browser): remove no-op scope/attachments config, drop profile pass-through lacking runtime support
* feat(media-understanding): add profile/preferredProfile to DescribeImageFileWithModelParams and forward to describeImage
* style(browser): add curly braces to satisfy eslint curly rule
* fix(browser): correct tools.browser.enabled help text to match actual behavior
* fix(browser): thread agentDir/workspaceDir from plugin tool context into browser vision
* refactor(browser): move vision config from tools.browser to browser.models
The browser plugin's vision configuration now lives on the top-level
`browser` config namespace (browser.models, browser.visionEnabled,
browser.visionPrompt, etc.) instead of `tools.browser`. This aligns
with the plugin's existing config location and avoids confusion between
tool-level and plugin-level settings.
- Remove tools.browser from ToolsSchema and ToolsConfig
- Add models/vision* fields to BrowserConfig and its zod schema
- Update getBrowserVisionConfig to read from cfg.browser
- Update schema help, labels, and quality test
- Update vision.test.ts to use new config shape
* docs(browser): add screenshot vision configuration section
Document the new browser.models config for automatic screenshot
description via vision models, enabling text-only main models to
reason about web page content.
* fix(browser): remove deliverable media markers from vision result, drop unused import
P1: Vision-success path no longer exposes the raw screenshot as
deliverable media (removes MEDIA: line and details.media.mediaUrl).
This prevents channel delivery from auto-sending sensitive page content
when the intended output is a text description.
P2: Remove unused ToolsMediaUnderstandingSchema import that would fail
noUnusedLocals typecheck.
* fix(browser): add command/args fields to browser models schema
The browser vision model schema uses .strict(), so CLI-type entries
with command/args were rejected by TypeScript. Add these fields to
align with MediaUnderstandingModelSchema.
* chore(browser): remove debug console.log statements
* fix(browser): harden screenshot vision result against MEDIA: directive injection and restore image sanitization on failure fallback
ClawSweeper #84247 review round 2:
P1 (security, high): neutralize line-start MEDIA: directives in vision descriptions
before wrapping with wrapExternalContent. The agent media extractor scans every
browser tool-result text block via splitMediaFromOutput which treats line-start
MEDIA: as a trusted local-media delivery directive, and browser is on the
trusted-media allowlist. Without neutralization, page or vision-provider output
containing 'MEDIA:/tmp/secret.png' could synthesize a channel-deliverable media
artifact from untrusted content. wrapExternalContent itself does not strip
line-start directives. Introduce neutralizeMediaDirectives in vision.ts that
prepends '[neutralized] ' to any line whose trimStart() begins with MEDIA:
(case-insensitive), defanging the parser anchor while keeping the original
text human-readable.
P2 (compatibility): pass resolveRuntimeImageSanitization() to imageResultFromFile
in the vision-failure catch fallback. The non-vision screenshot path already
forwards this option (d5cc0d53b7) so configured agents.defaults.imageMaxDimensionPx
takes effect. Without this fix, any provider timeout/error silently bypasses the
sanitization guard and returns a raw full-resolution screenshot.
Regression coverage:
- vision.test.ts: 6 unit cases for neutralizeMediaDirectives (no-op fast path,
mid-line MEDIA: untouched, line-start defanged, leading-whitespace defanged,
case-insensitive, multiple directives per blob).
- browser-tool.test.ts: 2 integration cases that drive the full screenshot
tool execute path:
- 'neutralizes MEDIA: directives in vision text and does not attach media'
asserts no line matches /^\s*MEDIA:/i in returned text, secret path text
is preserved verbatim, details.media is absent, and imageResultFromFile
is not called on the success path.
- 'preserves screenshot image sanitization on vision failure fallback'
mocks describeImageFileWithModel to reject and asserts the fallback
imageResultFromFile call receives imageSanitization: {maxDimensionPx:1600}
plus the 'browser screenshot vision failed' extraText.
* fix(browser): apply clawsweeper fallback media fix from PR #84247
* refactor: reuse media image understanding for browser screenshots
* refactor: use structured media delivery
* test: update music completion media instruction expectation
* fix: trim buffered reply directive padding
* test: refresh codex prompt snapshots for message media aliases
---------
Co-authored-by: scotthuang <scotthuang@tencent.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Forward OpenAI-compatible stop sequences from gateway chat completions through the agent runner into provider transports.
The gateway now normalizes stop into sampling extras, agent transports pass it into the shared stream options, and OpenAI, Anthropic, Mistral, Google, and Vertex-backed simple providers map it to their native request fields. Provider/gateway/agent coverage plus Crabbox live gateway proof verify valid stop dispatch and invalid stop rejection.
Refs #87920
Refactor OpenAI provider identity so OpenAI remains the canonical provider for API-key and OAuth-backed flows while legacy openai-codex state is doctor/migration-only.
Keeps OpenAI Codex Responses as an API/transport class rather than a provider identity, moves auth aliases through providerAuthAliases, updates doctor repair sequencing for old auth/profile state, and refreshes tests/docs around the canonical OpenAI behavior.
Fix Codex app-server native thread overflow recovery and CLI compaction fallback.
- rotate Codex native startup bindings when rollout token pressure leaves too little headroom
- keep byte-size rollout fuses ahead of rollout content reads
- clear stale resumed context-engine bindings only when the stored thread id still matches
- fall back to context-engine compaction when Codex owns/skips native compaction
Verification:
- node scripts/run-vitest.mjs run --config test/vitest/vitest.extension-codex.config.ts extensions/codex/src/app-server/startup-binding.test.ts extensions/codex/src/app-server/run-attempt.context-engine.test.ts extensions/codex/src/app-server/session-binding.test.ts --reporter=verbose
- node scripts/run-vitest.mjs run --config test/vitest/vitest.agents.config.ts src/agents/command/cli-compaction.test.ts --reporter=verbose
- git diff --check origin/main...HEAD
- autoreview --mode branch --base origin/main: clean
- GitHub CI for 466bfbe78c: green
Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
A claude-cli session whose JSONL transcript ends with an assistant
`tool_use` content block that was never answered by a `tool_result` user
message cannot resume — claude-cli will sit waiting for the missing
`tool_result`, hit its no-output watchdog, and the runtime kills it
with `reason=abort`. The dispatcher then sees an empty payload and emits
NO_REPLY, which to the user looks like the agent silently ignored their
message — same end-user symptom as the binding-flush amnesia bug, but a
different root cause.
The orphan can be left behind when:
- Gateway restarts mid-tool (brew upgrade, manual kickstart, OOM,
crash) — claude was waiting on a tool result that never arrived.
- `claude-live-session.ts` no-output watchdog fires while a tool is
actively running and OC kills the subprocess.
- The tool itself crashed or hung past its own deadline.
In all cases the resumed session is dead until the binding gets cleared,
because every subsequent resume hits the same trailing tool_use and the
same kill cycle. Observed in production on a personal OpenClaw gateway
(3d-engineer agent, 50-message-deep transcript ending in a Bash
`tool_use`; every Telegram message after the orphan landed silently
aborted at the 180s no-output mark).
Add `claudeCliSessionTranscriptHasOrphanedToolUse` to the helpers that
walks the JSONL, finds the last assistant message, and returns true if
any of its `tool_use` ids has no matching `tool_result` later in the
file. Wire into `prepareCliRunContext` as a second invalidator gate
alongside `missing-transcript`. The new `invalidatedReason:
"orphaned-tool-use"` follows the same path as missing-transcript: the
binding is dropped, this turn starts a fresh session, and the prior
context is reseeded into the new session via `RAW_TRANSCRIPT_RESEED`.
Detection only considers TRAILING orphans — an unanswered tool_use
deeper in history is inert because a later assistant message already
moved past it. Only the most recent assistant message's tool_use ids
matter for forward progress.
Probe runs only for claude-cli providers and only when the transcript-
content gate already passed, so we add no I/O on already-invalidated
sessions and no behavior change for non-claude providers.
AI-assisted: yes. Tooling: Claude Opus + claude-cli.
Fix `sessions.json` persistence after compaction transcript rotation.
When the agent runtime rotates from the pre-compaction session transcript to the post-compaction transcript, post-run consumers now receive the effective OpenClaw session id and session file. Backend CLI session ids remain backend metadata and no longer overwrite the top-level OpenClaw session identity.
Refs #88040.
Thanks @1052326311.
Verification:
- `node scripts/run-vitest.mjs src/agents/agent-command.compaction-rotation.test.ts src/agents/agent-command.live-model-switch.test.ts src/agents/command/session-store.test.ts`
- Autoreview clean
- GitHub CI green on PR head `c3d3c77ddf675bbba0b9ba6681b030a2f69a898c`
Fix claude-cli transcript resume so session-id rotation and transcript flush timing do not drop valid resume state.
- Capture the latest claude-cli session_id from JSONL output.
- Resolve Claude project transcript paths through the shared canonical project-dir resolver.
- Probe transcript content from the actual CLI process cwd.
- Thanks @benjamin1492!
Recover Codex compaction paths when a stale app-server thread binding returns an unstructured `thread not found` failure. The raw missing-thread response now shares the same recovery behavior as structured missing/stale binding failures for preflight, queued compaction, and CLI fallback.
Fixes#87736.
Co-authored-by: Paul Frederiksen <paul@paulfrederiksen.com>
Fixes#87016.
Empty preflight compaction recovery now resets stale token snapshots immediately, preserves valid legacy transcript rows during cleanup, and avoids re-persisting stale context-budget or compaction metadata after a successful retry.
Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com>
Avoid stale restart continuation reuse after a session key has rotated.
Queued restart agent turns now carry the session id they were queued for and fall back to a system wake if the key points at a different session by delivery time. Normal completed-run lifecycle fields stay reusable for fresh sessions, while new-session creation clears stale lifecycle markers.
Closes#86593.
Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com>
Forward OpenAI-compatible frequency_penalty, presence_penalty, and seed params through the gateway/chat-completions path while keeping Responses untouched.
Verification:
- pnpm test src/gateway/openai-http.test.ts src/agents/pi-embedded-runner/extra-params.sampling.test.ts src/agents/openai-transport-stream.test.ts
- CI passed on head 9abb9466d9 after rerunning cancelled jobs: preflight, critical quality network-runtime-boundary, security high, checks, docs, Real behavior proof.
Co-authored-by: lellansin <lellansin@gmail.com>
* fix(agents): answer Claude live control_request can_use_tool via exec policy
Claude CLI emits stream-json control_request frames with subtype
can_use_tool when it wants to use a native tool. The Claude live-session
bridge previously dropped these frames, leaving Claude waiting for a
control_response until the 180/600s no-output timeout fired (see #80819).
Resolve the effective OpenClaw exec policy (per-agent tools.exec -> global
tools.exec -> allowlist/on-miss defaults) once at session-start time and
thread it through fingerprinting and the session record. When a
can_use_tool request arrives:
- Allow native Bash when the resolved policy is security=full, ask=off
(matching the bypassPermissions semantics OpenClaw already documents).
- Otherwise deny with a message that names the resolved policy and
points the agent at OpenClaw MCP tools.
Unsupported control_request subtypes get a structured error response
instead of a silent no-op, and stray control_response frames are
silently dropped. Adds spawn-test coverage for both allow and deny paths.
Fixes#80819
* fix(agents): align Claude live control_request policy with backend defaults
Resolve the effective exec policy through the same defaults that
extensions/anthropic/cli-shared.ts:isOpenClawRequestedYolo and
src/agents/exec-defaults.ts:resolveExecDefaults already use (security
?? "full", ask ?? "off") instead of falling back to a hand-rolled
allowlist/on-miss default that disagreed with the rest of the codebase.
Without this, a default-config OpenClaw deployment launches Claude with
--permission-mode bypassPermissions but the bridge would still deny
Bash control_requests, re-creating the #80819 stall for the very
default-config case the issue reports.
Also thread the effective Claude permission mode into the policy
decision. Prefer the operator's explicit --permission-mode in argv,
falling back to what normalizeClaudePermissionArgs would have inserted
for an un-overridden launch. Native Bash is auto-allowed only when the
effective mode is bypassPermissions AND tools.exec resolves to
full/no-ask, so explicit raw-arg overrides like --permission-mode
default or acceptEdits broaden Claude's native prompting and are
honored by routing through deny.
Adds a no-config regression test (default deployment allows Bash, no
stall) and a permission-mode-override test (tools.exec full/off plus
explicit --permission-mode default in raw args denies). Existing
allow/deny tests continue to pass via the synthesized-mode fallback.
* fix(agents): honor effective exec policy for Claude live Bash
---------
Co-authored-by: Guillaume Thirry <g.thirry@gmail.com>
Expose a path-free estimated context budget status on session entries and gateway session rows, render it in status when fresh provider usage is unavailable, and clear stale estimates across reset, refresh, compaction, and session-rotation boundaries.
Verification: focused local Vitest covered session persistence, status rendering, gateway rows, model resets, compaction, and session rotation; GitHub CI passed on clean head cad199e43d.
Refs #80594, #54996, #77992, #84490, #83177, #43009, #83526, #8635.
