diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 8584ad914f6..1f4eead39f7 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -26,7 +26,7 @@ jobs: matrix: include: - language: javascript-typescript - runs_on: blacksmith-16vcpu-ubuntu-2404 + runs_on: blacksmith-32vcpu-ubuntu-2404 needs_node: true needs_python: false needs_java: false diff --git a/CHANGELOG.md b/CHANGELOG.md index 909285547e6..9d8c16fc50b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,7 +69,13 @@ Docs: https://docs.openclaw.ai ### Fixes +- Talk/TTS: resolve configured extension speech providers from the active runtime registry before provider-list discovery, so Talk mode no longer rejects valid plugin speech providers as unsupported. +- Sessions/subagents: stop stale ended runs and old store-only child reverse links from reappearing in `childSessions`, while keeping live descendants and recently-ended children visible. Fixes #57920. - Subagents: stop stale unended runs from counting as active or pending forever, while preserving restart-aborted recovery for recoverable child sessions. Fixes #71252. Thanks @hclsys. +- Gateway/tools: allow `POST /tools/invoke` to reach plugin-backed catalog tools such as `browser` when no core implementation exists, while still preferring built-in tools for real core names. Thanks @chat2way. +- Browser/security: require `operator.admin` for the `browser.request` gateway method, matching the host/browser-node control authority exposed by that route. Thanks @RichardCao. +- Browser/profiles: allow local managed profiles to override `browser.executablePath`, so different profiles can launch different Chromium-based browsers. Thanks @nobrainer-tech. +- Agents/replay: repair displaced or missing tool results before strict provider replay, use Codex-compatible `aborted` outputs for OpenAI Responses history, and drop partial aborted/error transport turns before retries. - Reply media: allow sandboxed replies to deliver OpenClaw-managed `media/outbound` and `media/tool-*` attachments without treating them as sandbox escapes, while keeping alias-escape checks on the managed media root. Fixes #71138. Thanks @mayor686, @truffle-dev, and @neeravmakwana. - CLI/agent: keep `openclaw agent --json` stdout reserved for the JSON response by routing gateway, plugin, and embedded-fallback diagnostics to stderr before execution starts. Fixes #71319. - Agents/Gemini: retry reasoning-only, empty, and planning-only Gemini turns instead of letting sessions silently stall. Fixes #71074. (#71362) Thanks @neeravmakwana. @@ -77,6 +83,8 @@ Docs: https://docs.openclaw.ai - Exec approvals: allow bare command-name allowlist patterns to match PATH-resolved executable basenames without trusting `./tool` or absolute path-selected binaries. Fixes #71315. Thanks @chen-zhang-cs-code and @dengluozhang. - Config/recovery: skip whole-file last-known-good rollback when invalidity is scoped to `plugins.entries.*`, preserving unrelated user settings during plugin schema or host-version skew. Fixes #71289. Thanks @jalehman. - Agents/tools: keep resolved reply-run configs from being overwritten by stale runtime snapshots, and let empty web runtime metadata fall back to configured provider auto-detection so standard and queued turns expose the same tool set. Fixes #71355. Thanks @c-g14. +- Agents/TTS: pass the resolved shared config into the `tts` tool, so tool-triggered speech uses configured providers and voices instead of falling back to a fresh config load. +- Reply media: strip `MEDIA:` attachments from final replies when the same media already went out through block streaming, preventing duplicate Telegram voice notes and files. Fixes #65468. Thanks @aurora-openclaw. - Agents/TTS: preserve voice media when a tool-generated reply is paired with an exact `NO_REPLY` sentinel, stripping the sentinel text instead of dropping the audio payload. Fixes #66092. - Compaction: honor explicit `agents.defaults.compaction.keepRecentTokens` for manual `/compact`, re-distill safeguard summaries instead of snowballing previous summaries, and enable safeguard summary quality checks by default. Fixes #71357. Thanks @WhiteGiverMa. - Sessions: honor configured `session.maintenance` settings during load-time maintenance instead of falling back to default entry caps. Fixes #71356. Thanks @comolago. @@ -102,6 +110,8 @@ Docs: https://docs.openclaw.ai - Providers/OpenRouter: add an OpenRouter TTS provider using the OpenAI-compatible `/audio/speech` endpoint and `OPENROUTER_API_KEY`. Fixes #71268. - macOS Talk Mode: retry failed local ElevenLabs stream playback through gateway `talk.speak` before falling back to the system voice, so configured ElevenLabs voices still play when streaming playback fails. Fixes #65662. - Plugins/Voice Call: reap stale pre-answer calls by default, honor configured TTS timeouts for Twilio media-stream playback, and fail empty telephony audio instead of completing as silence. Fixes #42071; supersedes #60957. Thanks @Ryce and @sliekens. +- Plugins/Voice Call: fail fast when Twilio, Telnyx, or Plivo would fall back to a loopback/private webhook URL, so calls do not start with an unreachable callback endpoint. Thanks @artemgetmann. +- Plugins/Voice Call: resolve queued-but-not-yet-playing Twilio TTS entries when barge-in or stream teardown clears the playback queue, so callers awaiting `queueTts()` do not hang. Thanks @kevinWangSheng. - Plugins/Voice Call: terminate expired restored call sessions with the provider and restart restored max-duration timers with only the remaining duration, preventing stale outbound retry loops after Gateway restarts. Fixes #48739. Thanks @mira-solari. - Plugins/Voice Call: start provider STT after Telnyx outbound conversation greetings and pass configured Telnyx voice IDs through to the speak action. Fixes #56091. Thanks @Roshan. - Skills: honor legacy `metadata.clawdbot` requirements and installer hints when `metadata.openclaw` is absent, so older skills no longer appear ready when required binaries are missing. Fixes #71323. Thanks @chen-zhang-cs-code. @@ -235,6 +245,7 @@ Docs: https://docs.openclaw.ai - Heartbeat: include async exec completion details in heartbeat prompts so command-finished notifications relay the actual output. (#71213) Thanks @GodsBoy. - Memory search: apply session visibility and agent-to-agent policy to session transcript hits, and keep `corpus=sessions` ranking scoped to session collections before result limiting. (#70761) Thanks @nefainl. - Agents/sessions: stop session write-lock timeouts from entering model failover, so local lock contention surfaces directly instead of cascading across providers. (#68700) Thanks @MonkeyLeeT. +- Auto-reply: run inbound reply delivery through `message_sending` hooks so plugins can transform or cancel generated replies before they are sent. (#70118) Thanks @jzakirov. ## 2026.4.23 diff --git a/docs/.generated/config-baseline.sha256 b/docs/.generated/config-baseline.sha256 index 18beac9d4a0..89bd8e1b70c 100644 --- a/docs/.generated/config-baseline.sha256 +++ b/docs/.generated/config-baseline.sha256 @@ -1,4 +1,4 @@ -b6d1e53947fcdfbff1b99f8ec79d3814d243385a1750b7fb40b40bb30f2e2975 config-baseline.json -98c83ce8af9ec4703726d7d673add95279be008a801b1d298982cbd9c1785747 config-baseline.core.json +d885c14dea2c361123a97a0f6c854f6dbae8592f39daa211173ef7f1fe7d554a config-baseline.json +c991bb527d8efffb5c9a2c5e502113260a2873923d469289c82f7029257fddaf config-baseline.core.json d72032762ab46b99480b57deb81130a0ab5b1401189cfbaf4f7fef4a063a7f6c config-baseline.channel.json -86f615b7d267b03888af0af7ccb3f8232a6b636f8a741d522ff425e46729ba81 config-baseline.plugin.json +0d5ba81f0030bd39b7ae285096276cc18b150836c2252fd2217329fc6154e80e config-baseline.plugin.json diff --git a/docs/cli/config.md b/docs/cli/config.md index ea9cc12831c..c35c07f59a7 100644 --- a/docs/cli/config.md +++ b/docs/cli/config.md @@ -36,6 +36,7 @@ openclaw config --section gateway --section daemon openclaw config schema openclaw config get browser.executablePath openclaw config set browser.executablePath "/usr/bin/google-chrome" +openclaw config set browser.profiles.work.executablePath "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" openclaw config set agents.defaults.heartbeat.every "2h" openclaw config set agents.list[0].tools.exec.node "node-id-or-name" openclaw config set agents.defaults.models '{"openai/gpt-5.4":{}}' --strict-json --merge diff --git a/docs/cli/voicecall.md b/docs/cli/voicecall.md index 4ad165c6069..dc42b95879a 100644 --- a/docs/cli/voicecall.md +++ b/docs/cli/voicecall.md @@ -33,6 +33,10 @@ scripts: openclaw voicecall setup --json ``` +For external providers (`twilio`, `telnyx`, `plivo`), setup must resolve a public +webhook URL from `publicUrl`, a tunnel, or Tailscale exposure. A loopback/private +serve fallback is rejected because carriers cannot reach it. + `smoke` runs the same readiness checks. It will not place a real phone call unless both `--to` and `--yes` are present: diff --git a/docs/concepts/streaming.md b/docs/concepts/streaming.md index 5c80ffc2cf4..cc487bc2c1b 100644 --- a/docs/concepts/streaming.md +++ b/docs/concepts/streaming.md @@ -54,6 +54,19 @@ Legend: `message_end` still uses the chunker if the buffered text exceeds `maxChars`, so it can emit multiple chunks at the end. +### Media delivery with block streaming + +`MEDIA:` directives are normal delivery metadata. When block streaming sends a +media block early, OpenClaw remembers that delivery for the turn. If the final +assistant payload repeats the same media URL, the final delivery strips the +duplicate media instead of sending the attachment again. + +Exact duplicate final payloads are suppressed. If the final payload adds +distinct text around media that was already streamed, OpenClaw still sends the +new text while keeping the media single-delivery. This prevents duplicate voice +notes or files on channels such as Telegram when an agent emits `MEDIA:` during +streaming and the provider also includes it in the completed reply. + ## Chunking algorithm (low/high bounds) Block chunking is implemented by `EmbeddedBlockChunker`: diff --git a/docs/gateway/configuration-reference.md b/docs/gateway/configuration-reference.md index dd64ae5d1df..b47885db151 100644 --- a/docs/gateway/configuration-reference.md +++ b/docs/gateway/configuration-reference.md @@ -175,7 +175,11 @@ See [Plugins](/tools/plugin). }, profiles: { openclaw: { cdpPort: 18800, color: "#FF4500" }, - work: { cdpPort: 18801, color: "#0066CC" }, + work: { + cdpPort: 18801, + color: "#0066CC", + executablePath: "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", + }, user: { driver: "existing-session", attachOnly: true, color: "#00AA00" }, brave: { driver: "existing-session", @@ -218,6 +222,9 @@ See [Plugins](/tools/plugin). `responsebody`, PDF export, download interception, or batch actions. - Local managed `openclaw` profiles auto-assign `cdpPort` and `cdpUrl`; only set `cdpUrl` explicitly for remote CDP. +- Local managed profiles can set `executablePath` to override the global + `browser.executablePath` for that profile. Use this to run one profile in + Chrome and another in Brave. - Auto-detect order: default browser if Chromium-based → Chrome → Brave → Edge → Chromium → Chrome Canary. - `browser.executablePath` accepts `~` for your OS home directory. - Control service: loopback only (port derived from `gateway.port`, default `18791`). diff --git a/docs/gateway/doctor.md b/docs/gateway/doctor.md index 20dd0f1123a..3f67df8a56d 100644 --- a/docs/gateway/doctor.md +++ b/docs/gateway/doctor.md @@ -172,9 +172,11 @@ Current migrations: - `routing.agentToAgent` → `tools.agentToAgent` - `routing.transcribeAudio` → `tools.media.audio.models` - `messages.tts.` (`openai`/`elevenlabs`/`microsoft`/`edge`) → `messages.tts.providers.` +- `messages.tts.provider: "edge"` and `messages.tts.providers.edge` → `messages.tts.provider: "microsoft"` and `messages.tts.providers.microsoft` - `channels.discord.voice.tts.` (`openai`/`elevenlabs`/`microsoft`/`edge`) → `channels.discord.voice.tts.providers.` - `channels.discord.accounts..voice.tts.` (`openai`/`elevenlabs`/`microsoft`/`edge`) → `channels.discord.accounts..voice.tts.providers.` - `plugins.entries.voice-call.config.tts.` (`openai`/`elevenlabs`/`microsoft`/`edge`) → `plugins.entries.voice-call.config.tts.providers.` +- `plugins.entries.voice-call.config.tts.provider: "edge"` and `plugins.entries.voice-call.config.tts.providers.edge` → `provider: "microsoft"` and `providers.microsoft` - `plugins.entries.voice-call.config.provider: "log"` → `"mock"` - `plugins.entries.voice-call.config.twilio.from` → `plugins.entries.voice-call.config.fromNumber` - `plugins.entries.voice-call.config.streaming.sttProvider` → `plugins.entries.voice-call.config.streaming.provider` diff --git a/docs/help/testing-live.md b/docs/help/testing-live.md index 08fe18bc617..21c4f00f6f6 100644 --- a/docs/help/testing-live.md +++ b/docs/help/testing-live.md @@ -13,6 +13,35 @@ For quick start, QA runners, unit/integration suites, and Docker flows, see suites: model matrix, CLI backends, ACP, and media-provider live tests, plus credential handling. +## Live: local profile smoke commands + +Source `~/.profile` before ad hoc live checks so provider keys and local tool +paths match your shell: + +```bash +source ~/.profile +``` + +Safe media smoke: + +```bash +pnpm openclaw infer tts convert --local --json \ + --text "OpenClaw live smoke." \ + --output /tmp/openclaw-live-smoke.mp3 +``` + +Safe voice-call readiness smoke: + +```bash +pnpm openclaw voicecall setup --json +pnpm openclaw voicecall smoke --to "+15555550123" +``` + +`voicecall smoke` is a dry run unless `--yes` is also present. Use `--yes` only +when you intentionally want to place a real notify call. For Twilio, Telnyx, and +Plivo, a successful readiness check requires a public webhook URL; local-only +loopback/private fallbacks are rejected by design. + ## Live: Android node capability sweep - Test: `src/gateway/android-node.capabilities.live.test.ts` diff --git a/docs/plugins/voice-call.md b/docs/plugins/voice-call.md index df9a55a1ab7..ad14fc1f571 100644 --- a/docs/plugins/voice-call.md +++ b/docs/plugins/voice-call.md @@ -152,6 +152,11 @@ whether the plugin is enabled, the provider and credentials are present, webhook exposure is configured, and only one audio mode is active. Use `openclaw voicecall setup --json` for scripts. +For Twilio, Telnyx, and Plivo, setup must resolve to a public webhook URL. If the +configured `publicUrl`, tunnel URL, Tailscale URL, or serve fallback resolves to +loopback or private network space, setup fails instead of starting a provider +that cannot receive real carrier webhooks. + For a no-surprises smoke test, run: ```bash @@ -478,6 +483,9 @@ Notes: - Core TTS is used when Twilio media streaming is enabled; otherwise calls fall back to provider native voices. - If a Twilio media stream is already active, Voice Call does not fall back to TwiML ``. If telephony TTS is unavailable in that state, the playback request fails instead of mixing two playback paths. - When telephony TTS falls back to a secondary provider, Voice Call logs a warning with the provider chain (`from`, `to`, `attempts`) for debugging. +- When Twilio barge-in or stream teardown clears the pending TTS queue, queued + playback requests settle instead of hanging callers that are awaiting playback + completion. ### More examples @@ -589,6 +597,9 @@ For outbound `conversation` calls, first-message handling is tied to live playba - Barge-in queue clear and auto-response are suppressed only while the initial greeting is actively speaking. - If initial playback fails, the call returns to `listening` and the initial message remains queued for retry. - Initial playback for Twilio streaming starts on stream connect without extra delay. +- Barge-in aborts active playback and clears queued-but-not-yet-playing Twilio + TTS entries. Cleared entries resolve as skipped, so follow-up response logic + can continue without waiting on audio that will never play. - Realtime voice conversations use the realtime stream's own opening turn. Voice Call does not post a legacy `` TwiML update for that initial message, so outbound `` sessions stay attached. ### Twilio stream disconnect grace diff --git a/docs/reference/rich-output-protocol.md b/docs/reference/rich-output-protocol.md index be790e9defe..4703cfefaf6 100644 --- a/docs/reference/rich-output-protocol.md +++ b/docs/reference/rich-output-protocol.md @@ -15,6 +15,11 @@ Assistant output can carry a small set of delivery/render directives: These directives are separate. `MEDIA:` and reply/voice tags remain delivery metadata; `[embed ...]` is the web-only rich render path. +When block streaming is enabled, `MEDIA:` remains single-delivery metadata for a +turn. If the same media URL is sent in a streamed block and repeated in the final +assistant payload, OpenClaw delivers the attachment once and strips the duplicate +from the final payload. + ## `[embed ...]` `[embed ...]` is the only agent-facing rich render syntax for the Control UI. diff --git a/docs/reference/transcript-hygiene.md b/docs/reference/transcript-hygiene.md index b65e913b205..e195150b9e9 100644 --- a/docs/reference/transcript-hygiene.md +++ b/docs/reference/transcript-hygiene.md @@ -114,9 +114,9 @@ external end-user instructions. - Image sanitization only. - Drop orphaned reasoning signatures (standalone reasoning items without a following content block) for OpenAI Responses/Codex transcripts, and drop replayable OpenAI reasoning after a model route switch. - No tool call id sanitization. -- No tool result pairing repair. +- Tool result pairing repair may move real matched outputs and synthesize Codex-style `aborted` outputs for missing tool calls. - No turn validation or reordering. -- No synthetic tool results. +- Missing OpenAI Responses-family tool outputs are synthesized as `aborted` to match Codex replay normalization. - No thought signature stripping. **Google (Generative AI / Gemini CLI / Antigravity)** diff --git a/docs/tools/browser.md b/docs/tools/browser.md index 1fe77fe2053..51f5d514b62 100644 --- a/docs/tools/browser.md +++ b/docs/tools/browser.md @@ -143,7 +143,12 @@ Browser settings live in `~/.openclaw/openclaw.json`. executablePath: "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser", profiles: { openclaw: { cdpPort: 18800, color: "#FF4500" }, - work: { cdpPort: 18801, color: "#0066CC", headless: true }, + work: { + cdpPort: 18801, + color: "#0066CC", + headless: true, + executablePath: "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", + }, user: { driver: "existing-session", attachOnly: true, @@ -187,6 +192,7 @@ Browser settings live in `~/.openclaw/openclaw.json`. - `attachOnly: true` means never launch a local browser; only attach if one is already running. - `headless` can be set globally or per local managed profile. Per-profile values override `browser.headless`, so one locally launched profile can stay headless while another remains visible. +- `executablePath` can be set globally or per local managed profile. Per-profile values override `browser.executablePath`, so different managed profiles can launch different Chromium-based browsers. - `color` (top-level and per-profile) tints the browser UI so you can see which profile is active. - Default profile is `openclaw` (managed standalone). Use `defaultProfile: "user"` to opt into the signed-in user browser. - Auto-detect order: system default browser if Chromium-based; otherwise Chrome → Brave → Edge → Chromium → Chrome Canary. @@ -205,6 +211,7 @@ auto-detection. `~` expands to your OS home directory: ```bash openclaw config set browser.executablePath "/usr/bin/google-chrome" +openclaw config set browser.profiles.work.executablePath "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" ``` Or set it in config, per platform: @@ -239,6 +246,10 @@ Or set it in config, per platform: +Per-profile `executablePath` only affects local managed profiles that OpenClaw +launches. `existing-session` profiles attach to an already-running browser +instead, and remote CDP profiles use the browser behind `cdpUrl`. + ## Local vs remote control - **Local control (default):** the Gateway starts the loopback control service and can launch a local browser. @@ -246,6 +257,9 @@ Or set it in config, per platform: - **Remote CDP:** set `browser.profiles..cdpUrl` (or `browser.cdpUrl`) to attach to a remote Chromium-based browser. In this case, OpenClaw will not launch a local browser. - `headless` only affects local managed profiles that OpenClaw launches. It does not restart or change existing-session or remote CDP browsers. +- `executablePath` follows the same local managed profile rule. Changing it on a + running local managed profile marks that profile for restart/reconcile so the + next launch uses the new binary. Stopping behavior differs by profile mode: diff --git a/docs/tools/subagents.md b/docs/tools/subagents.md index 236488356fc..5de2bcc3670 100644 --- a/docs/tools/subagents.md +++ b/docs/tools/subagents.md @@ -214,6 +214,11 @@ Operational guidance: - Start child work once and wait for completion events instead of building poll loops around `sessions_list`, `sessions_history`, `/subagents list`, or `exec` sleep commands. +- `sessions_list` and `/subagents list` keep child-session relationships focused + on live work: live children remain attached, ended children stay visible for a + short recent window, and stale store-only child links are ignored after their + freshness window. This prevents old `spawnedBy` / `parentSessionKey` metadata + from resurrecting ghost children after restart. - If a child completion event arrives after you already sent the final answer, the correct follow-up is the exact silent token `NO_REPLY` / `no_reply`. diff --git a/docs/tools/tts.md b/docs/tools/tts.md index 5a6fc117906..30a7104b3c3 100644 --- a/docs/tools/tts.md +++ b/docs/tools/tts.md @@ -347,13 +347,15 @@ Then run: - `mode`: `"final"` (default) or `"all"` (includes tool/block replies). - `provider`: speech provider id such as `"elevenlabs"`, `"google"`, `"gradium"`, `"microsoft"`, `"minimax"`, `"openai"`, `"vydra"`, or `"xai"` (fallback is automatic). - If `provider` is **unset**, OpenClaw uses the first configured speech provider in registry auto-select order. -- Legacy `provider: "edge"` still works and is normalized to `microsoft`. +- Legacy `provider: "edge"` config is repaired by `openclaw doctor --fix` and + rewritten to `provider: "microsoft"`. - `summaryModel`: optional cheap model for auto-summary; defaults to `agents.defaults.model.primary`. - Accepts `provider/model` or a configured model alias. - `modelOverrides`: allow the model to emit TTS directives (on by default). - `allowProvider` defaults to `false` (provider switching is opt-in). - `providers.`: provider-owned settings keyed by speech provider id. - Legacy direct provider blocks (`messages.tts.openai`, `messages.tts.elevenlabs`, `messages.tts.microsoft`, `messages.tts.edge`) are repaired by `openclaw doctor --fix`; committed config should use `messages.tts.providers.`. +- Legacy `messages.tts.providers.edge` is also repaired by `openclaw doctor --fix`; committed config should use `messages.tts.providers.microsoft`. - `maxTextLength`: hard cap for TTS input (chars). `/tts audio` fails if exceeded. - `timeoutMs`: request timeout (ms). - `prefsPath`: override the local prefs JSON path (provider/limit/summary). @@ -402,7 +404,8 @@ Then run: - `providers.microsoft.saveSubtitles`: write JSON subtitles alongside the audio file. - `providers.microsoft.proxy`: proxy URL for Microsoft speech requests. - `providers.microsoft.timeoutMs`: request timeout override (ms). -- `edge.*`: legacy alias for the same Microsoft settings. +- `edge.*`: legacy alias for the same Microsoft settings. Run + `openclaw doctor --fix` to rewrite persisted config to `providers.microsoft`. ## Model-driven overrides (default on) diff --git a/extensions/browser/index.test.ts b/extensions/browser/index.test.ts index a20fa9b9928..a69ed193164 100644 --- a/extensions/browser/index.test.ts +++ b/extensions/browser/index.test.ts @@ -118,6 +118,17 @@ describe("browser plugin", () => { }); }); + it("registers browser.request as an admin gateway method", () => { + const { api, registerGatewayMethod } = createApi(); + registerBrowserPlugin(api); + + expect(registerGatewayMethod).toHaveBeenCalledWith( + "browser.request", + runtimeApiMocks.handleBrowserGatewayRequest, + { scope: "operator.admin" }, + ); + }); + it("declares setup auto-enable reasons for browser config surfaces", () => { const probe = registerBrowserAutoEnableProbe(); diff --git a/extensions/browser/plugin-registration.ts b/extensions/browser/plugin-registration.ts index 2386bbf57fc..f441978bf83 100644 --- a/extensions/browser/plugin-registration.ts +++ b/extensions/browser/plugin-registration.ts @@ -34,7 +34,7 @@ export function registerBrowserPlugin(api: OpenClawPluginApi) { })) as OpenClawPluginToolFactory); api.registerCli(({ program }) => registerBrowserCli(program), { commands: ["browser"] }); api.registerGatewayMethod("browser.request", handleBrowserGatewayRequest, { - scope: "operator.write", + scope: "operator.admin", }); api.registerService(createBrowserPluginService()); } diff --git a/extensions/browser/src/browser/chrome.internal.test.ts b/extensions/browser/src/browser/chrome.internal.test.ts index 74fbdb34553..3b9d340fb24 100644 --- a/extensions/browser/src/browser/chrome.internal.test.ts +++ b/extensions/browser/src/browser/chrome.internal.test.ts @@ -82,6 +82,20 @@ function makeFakeProc(overrides: Partial = {}): FakeProc { return Object.assign(proc, overrides); } +function effectiveSpawnCommand(call: unknown[] | undefined): unknown { + const command = call?.[0]; + const args = call?.[1]; + if ( + command === "/bin/sh" && + Array.isArray(args) && + args[0] === "-c" && + typeof args[2] === "string" + ) { + return args[2]; + } + return command; +} + async function withMockChromeCdpServer(params: { wsPath: string; onConnection?: (wss: WebSocketServer) => void; @@ -387,6 +401,38 @@ describe("chrome.ts internal", () => { }); }); + it("uses profile executablePath over global executablePath when launching", async () => { + const originalPlatform = process.platform; + vi.spyOn(fs, "existsSync").mockImplementation((p) => { + const s = String(p); + if (s === "/tmp/profile-chrome" || s.endsWith("Local State") || s.endsWith("Preferences")) { + return true; + } + return false; + }); + spawnMock.mockImplementation(() => makeFakeProc()); + + Object.defineProperty(process, "platform", { value: "linux" }); + try { + await withMockChromeCdpServer({ + wsPath: "/devtools/browser/PROFILE_EXE", + run: async (baseUrl) => { + const port = new URL(baseUrl).port; + const profile = { ...makeProfile(Number(port)), executablePath: "/tmp/profile-chrome" }; + const resolved = { + ...makeResolved(), + executablePath: "/tmp/global-chrome", + } as ResolvedBrowserConfig; + const running = await launchOpenClawChrome(resolved, profile); + expect(effectiveSpawnCommand(spawnMock.mock.calls[0])).toBe("/tmp/profile-chrome"); + running.proc.kill?.("SIGTERM"); + }, + }); + } finally { + Object.defineProperty(process, "platform", { value: originalPlatform }); + } + }); + it("throws with stderr hint + sandbox hint when CDP never becomes reachable", async () => { const originalPlatform = process.platform; Object.defineProperty(process, "platform", { value: "linux" }); diff --git a/extensions/browser/src/browser/chrome.ts b/extensions/browser/src/browser/chrome.ts index 7e88859d79c..95c046ebe7e 100644 --- a/extensions/browser/src/browser/chrome.ts +++ b/extensions/browser/src/browser/chrome.ts @@ -90,8 +90,14 @@ export type RunningChrome = { proc: ChildProcess; }; -function resolveBrowserExecutable(resolved: ResolvedBrowserConfig): BrowserExecutable | null { - return resolveBrowserExecutableForPlatform(resolved, process.platform); +function resolveBrowserExecutable( + resolved: ResolvedBrowserConfig, + profile: ResolvedBrowserProfile, +): BrowserExecutable | null { + return resolveBrowserExecutableForPlatform( + { ...resolved, executablePath: profile.executablePath ?? resolved.executablePath }, + process.platform, + ); } export function resolveOpenClawUserDataDir(profileName = DEFAULT_OPENCLAW_BROWSER_PROFILE_NAME) { @@ -268,7 +274,7 @@ export async function launchOpenClawChrome( } await ensurePortAvailable(profile.cdpPort); - const exe = resolveBrowserExecutable(resolved); + const exe = resolveBrowserExecutable(resolved, profile); if (!exe) { throw new Error( "No supported browser found (Chrome/Brave/Edge/Chromium on macOS, Linux, or Windows).", diff --git a/extensions/browser/src/browser/config.test.ts b/extensions/browser/src/browser/config.test.ts index abf1fbf0432..ec56f032766 100644 --- a/extensions/browser/src/browser/config.test.ts +++ b/extensions/browser/src/browser/config.test.ts @@ -279,6 +279,50 @@ describe("browser config", () => { expect(remote?.headless).toBe(false); }); + it("inherits executablePath from global browser config when profile override is not set", () => { + const resolved = resolveBrowserConfig({ + executablePath: "~/bin/chrome-global", + profiles: { + remote: { cdpUrl: "http://127.0.0.1:9222", color: "#0066CC" }, + }, + }); + + const remote = resolveProfile(resolved, "remote"); + expect(remote?.executablePath).toBe(path.resolve(os.homedir(), "bin/chrome-global")); + }); + + it("allows profile executablePath to override global browser executablePath", () => { + const resolved = resolveBrowserConfig({ + executablePath: "/usr/bin/chrome-global", + profiles: { + remote: { + cdpUrl: "http://127.0.0.1:9222", + executablePath: " ~/bin/chrome-profile ", + color: "#0066CC", + }, + }, + }); + + const remote = resolveProfile(resolved, "remote"); + expect(remote?.executablePath).toBe(path.resolve(os.homedir(), "bin/chrome-profile")); + }); + + it("falls back to global executablePath when profile executablePath is blank", () => { + const resolved = resolveBrowserConfig({ + executablePath: "/usr/bin/chrome-global", + profiles: { + remote: { + cdpUrl: "http://127.0.0.1:9222", + executablePath: " ", + color: "#0066CC", + }, + }, + }); + + const remote = resolveProfile(resolved, "remote"); + expect(remote?.executablePath).toBe("/usr/bin/chrome-global"); + }); + it("uses base protocol for profiles with only cdpPort", () => { const resolved = resolveBrowserConfig({ cdpUrl: "https://example.com:9443", diff --git a/extensions/browser/src/browser/config.ts b/extensions/browser/src/browser/config.ts index 2eb87ed4c49..ae4b68c7c53 100644 --- a/extensions/browser/src/browser/config.ts +++ b/extensions/browser/src/browser/config.ts @@ -94,6 +94,7 @@ export type ResolvedBrowserProfile = { userDataDir?: string; color: string; driver: "openclaw" | "existing-session"; + executablePath?: string; headless: boolean; attachOnly: boolean; }; @@ -370,6 +371,7 @@ export function resolveProfile( let cdpUrl = ""; const driver = profile.driver === "existing-session" ? "existing-session" : "openclaw"; const headless = profile.headless ?? resolved.headless; + const executablePath = normalizeExecutablePath(profile.executablePath) ?? resolved.executablePath; if (driver === "existing-session") { return { @@ -381,6 +383,7 @@ export function resolveProfile( userDataDir: resolveUserPath(profile.userDataDir?.trim() || "") || undefined, color: profile.color, driver, + executablePath, headless, attachOnly: true, }; @@ -415,6 +418,7 @@ export function resolveProfile( cdpIsLoopback: isLoopbackHost(cdpHost), color: profile.color, driver, + executablePath, headless, attachOnly: profile.attachOnly ?? resolved.attachOnly, }; diff --git a/extensions/browser/src/browser/resolved-config-refresh.ts b/extensions/browser/src/browser/resolved-config-refresh.ts index 1378ca04399..07ce8bbc02d 100644 --- a/extensions/browser/src/browser/resolved-config-refresh.ts +++ b/extensions/browser/src/browser/resolved-config-refresh.ts @@ -27,6 +27,13 @@ function changedProfileInvariants( ) { changed.push("headless"); } + if ( + currentUsesLocalManagedLaunch && + nextUsesLocalManagedLaunch && + current.executablePath !== next.executablePath + ) { + changed.push("executablePath"); + } if (current.attachOnly !== next.attachOnly) { changed.push("attachOnly"); } diff --git a/extensions/browser/src/browser/routes/basic.existing-session.test.ts b/extensions/browser/src/browser/routes/basic.existing-session.test.ts index e61b0d6be8c..631463e3273 100644 --- a/extensions/browser/src/browser/routes/basic.existing-session.test.ts +++ b/extensions/browser/src/browser/routes/basic.existing-session.test.ts @@ -26,6 +26,8 @@ function createExistingSessionProfileState(params?: { isHttpReachable?: () => Pr cdpUrl: "", userDataDir: "/tmp/brave-profile", color: "#00AA00", + executablePath: "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser", + headless: false, attachOnly: true, }, isHttpReachable: params?.isHttpReachable ?? (async () => true), @@ -80,6 +82,7 @@ describe("basic browser routes", () => { cdpPort: null, cdpUrl: null, userDataDir: "/tmp/brave-profile", + executablePath: "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser", pid: 4321, }); }); diff --git a/extensions/browser/src/browser/routes/basic.ts b/extensions/browser/src/browser/routes/basic.ts index f0a823f9b38..8a60899f47c 100644 --- a/extensions/browser/src/browser/routes/basic.ts +++ b/extensions/browser/src/browser/routes/basic.ts @@ -103,7 +103,7 @@ async function buildBrowserStatus(req: BrowserRequest, ctx: BrowserRouteContext) color: profileCtx.profile.color, headless: profileCtx.profile.headless, noSandbox: current.resolved.noSandbox, - executablePath: current.resolved.executablePath ?? null, + executablePath: profileCtx.profile.executablePath ?? null, attachOnly: profileCtx.profile.attachOnly, }; } diff --git a/extensions/browser/src/browser/server-context.hot-reload-profiles.test.ts b/extensions/browser/src/browser/server-context.hot-reload-profiles.test.ts index f19f50c2d68..c7f3f3b3886 100644 --- a/extensions/browser/src/browser/server-context.hot-reload-profiles.test.ts +++ b/extensions/browser/src/browser/server-context.hot-reload-profiles.test.ts @@ -6,6 +6,7 @@ type TestProfileConfig = { cdpUrl?: string; color?: string; headless?: boolean; + executablePath?: string; driver?: "openclaw" | "existing-session"; }; type TestConfig = { @@ -275,6 +276,55 @@ describe("server-context hot-reload profiles", () => { expect(runtime?.reconcile?.reason).toContain("headless"); }); + it("marks local managed runtime state for reconcile when profile executablePath changes", async () => { + mockState.cfgProfiles.openclaw = { + cdpPort: 18800, + color: "#FF4500", + executablePath: "/usr/bin/chrome-old", + }; + mockState.cachedConfig = null; + const cfg = loadConfig(); + const resolved = resolveBrowserConfig(cfg.browser, cfg); + const openclawProfile = resolveProfile(resolved, "openclaw"); + expect(openclawProfile).toBeTruthy(); + expect(openclawProfile?.executablePath).toBe("/usr/bin/chrome-old"); + const state: BrowserServerState = { + server: null, + port: 18791, + resolved, + profiles: new Map([ + [ + "openclaw", + { + profile: openclawProfile!, + running: { pid: 123 } as never, + lastTargetId: "tab-1", + reconcile: null, + }, + ], + ]), + }; + + mockState.cfgProfiles.openclaw = { + cdpPort: 18800, + color: "#FF4500", + executablePath: "/usr/bin/chrome-new", + }; + mockState.cachedConfig = null; + + refreshResolvedBrowserConfigFromDisk({ + current: state, + refreshConfigFromDisk: true, + mode: "cached", + }); + + const runtime = state.profiles.get("openclaw"); + expect(runtime).toBeTruthy(); + expect(runtime?.profile.executablePath).toBe("/usr/bin/chrome-new"); + expect(runtime?.lastTargetId).toBeNull(); + expect(runtime?.reconcile?.reason).toContain("executablePath"); + }); + it("does not reconcile existing-session runtime when only headless changes", async () => { mockState.cfgProfiles.remote = { cdpUrl: "http://127.0.0.1:9222", diff --git a/extensions/microsoft/speech-provider.test.ts b/extensions/microsoft/speech-provider.test.ts index 9e77593fc2b..fbe52717da3 100644 --- a/extensions/microsoft/speech-provider.test.ts +++ b/extensions/microsoft/speech-provider.test.ts @@ -184,27 +184,6 @@ describe("buildMicrosoftSpeechProvider", () => { vi.restoreAllMocks(); }); - it("accepts legacy providers.edge voice config", () => { - const provider = buildMicrosoftSpeechProvider(); - - const resolved = provider.resolveConfig?.({ - cfg: TEST_CFG, - rawConfig: { - provider: "edge", - providers: { - edge: { - voice: "en-US-AvaNeural", - }, - }, - }, - timeoutMs: 1000, - }); - - expect(resolved).toMatchObject({ - voice: "en-US-AvaNeural", - }); - }); - it("switches to a Chinese voice for CJK text when no explicit voice override is set", async () => { const provider = buildMicrosoftSpeechProvider(); const edgeSpy = vi.spyOn(ttsModule, "edgeTTS").mockImplementation(async ({ outputPath }) => { diff --git a/extensions/microsoft/speech-provider.ts b/extensions/microsoft/speech-provider.ts index ce07bd4f7eb..3c021648964 100644 --- a/extensions/microsoft/speech-provider.ts +++ b/extensions/microsoft/speech-provider.ts @@ -59,9 +59,8 @@ function normalizeMicrosoftProviderConfig( const providers = asObject(rawConfig.providers); const rawEdge = asObject(rawConfig.edge); const rawMicrosoft = asObject(rawConfig.microsoft); - const rawProviderEdge = asObject(providers?.edge); const rawProviderMicrosoft = asObject(providers?.microsoft); - const raw = { ...rawEdge, ...rawProviderEdge, ...rawMicrosoft, ...rawProviderMicrosoft }; + const raw = { ...rawEdge, ...rawMicrosoft, ...rawProviderMicrosoft }; const outputFormat = trimToUndefined(raw.outputFormat); return { enabled: asBoolean(raw.enabled) ?? true, diff --git a/extensions/qa-lab/src/agentic-parity-report.test.ts b/extensions/qa-lab/src/agentic-parity-report.test.ts index 8fcee784939..b1a836ab86d 100644 --- a/extensions/qa-lab/src/agentic-parity-report.test.ts +++ b/extensions/qa-lab/src/agentic-parity-report.test.ts @@ -16,6 +16,7 @@ const FULL_PARITY_PASS_SCENARIOS: QaParityReportScenario[] = [ { name: "Image understanding from attachment", status: "pass" as const }, { name: "Subagent handoff", status: "pass" as const }, { name: "Subagent fanout synthesis", status: "pass" as const }, + { name: "Subagent stale child links", status: "pass" as const }, { name: "Memory recall after context switch", status: "pass" as const }, { name: "Thread memory isolation", status: "pass" as const }, { name: "Config restart capability flip", status: "pass" as const }, diff --git a/extensions/qa-lab/src/agentic-parity.ts b/extensions/qa-lab/src/agentic-parity.ts index d997778e85f..e8978cf353a 100644 --- a/extensions/qa-lab/src/agentic-parity.ts +++ b/extensions/qa-lab/src/agentic-parity.ts @@ -36,6 +36,11 @@ export const QA_AGENTIC_PARITY_SCENARIOS = [ title: "Subagent fanout synthesis", countsTowardValidToolCallRate: true, }, + { + id: "subagent-stale-child-links", + title: "Subagent stale child links", + countsTowardValidToolCallRate: false, + }, { id: "memory-recall", title: "Memory recall after context switch", diff --git a/extensions/qa-lab/src/cli.runtime.test.ts b/extensions/qa-lab/src/cli.runtime.test.ts index 98587a3ab98..a7f96c13b24 100644 --- a/extensions/qa-lab/src/cli.runtime.test.ts +++ b/extensions/qa-lab/src/cli.runtime.test.ts @@ -644,6 +644,7 @@ describe("qa cli runtime", () => { "compaction-retry-mutating-tool", "subagent-handoff", "subagent-fanout-synthesis", + "subagent-stale-child-links", "memory-recall", "thread-memory-isolation", "config-restart-capability-flip", @@ -1071,6 +1072,7 @@ describe("qa cli runtime", () => { "compaction-retry-mutating-tool", "subagent-handoff", "subagent-fanout-synthesis", + "subagent-stale-child-links", "memory-recall", "thread-memory-isolation", "config-restart-capability-flip", diff --git a/extensions/qa-lab/src/suite-runtime-types.ts b/extensions/qa-lab/src/suite-runtime-types.ts index c4ba16e7a3d..b16b64846a5 100644 --- a/extensions/qa-lab/src/suite-runtime-types.ts +++ b/extensions/qa-lab/src/suite-runtime-types.ts @@ -7,6 +7,14 @@ export type QaRuntimeGatewayClient = { tempRoot: string; workspaceDir: string; runtimeEnv: NodeJS.ProcessEnv; + restartAfterStateMutation?: ( + mutateState: (context: { + configPath: string; + runtimeEnv: NodeJS.ProcessEnv; + stateDir: string; + tempRoot: string; + }) => Promise, + ) => Promise; call: ( method: string, params?: unknown, diff --git a/extensions/telegram/src/bot-message-dispatch.test.ts b/extensions/telegram/src/bot-message-dispatch.test.ts index ba0b7e2f09d..9b060a74c43 100644 --- a/extensions/telegram/src/bot-message-dispatch.test.ts +++ b/extensions/telegram/src/bot-message-dispatch.test.ts @@ -427,6 +427,9 @@ describe("dispatchTelegramMessage draft streaming", () => { ); expect(dispatchReplyWithBufferedBlockDispatcher).toHaveBeenCalledWith( expect.objectContaining({ + dispatcherOptions: expect.objectContaining({ + beforeDeliver: expect.any(Function), + }), replyOptions: expect.objectContaining({ disableBlockStreaming: true, }), diff --git a/extensions/telegram/src/bot-message-dispatch.ts b/extensions/telegram/src/bot-message-dispatch.ts index 09ae28434fe..ceb9ce507b1 100644 --- a/extensions/telegram/src/bot-message-dispatch.ts +++ b/extensions/telegram/src/bot-message-dispatch.ts @@ -761,6 +761,7 @@ export const dispatchTelegramMessage = async ({ cfg, dispatcherOptions: { ...replyPipeline, + beforeDeliver: async (payload) => payload, deliver: async (payload, info) => { if (isDispatchSuperseded()) { return; diff --git a/extensions/telegram/src/bot-native-commands.session-meta.test.ts b/extensions/telegram/src/bot-native-commands.session-meta.test.ts index c2d33a93719..801782d4f8b 100644 --- a/extensions/telegram/src/bot-native-commands.session-meta.test.ts +++ b/extensions/telegram/src/bot-native-commands.session-meta.test.ts @@ -449,6 +449,9 @@ describe("registerTelegramNativeCommands — session metadata", () => { await runPromise; expect(replyMocks.dispatchReplyWithBufferedBlockDispatcher).toHaveBeenCalledTimes(1); + expect( + replyMocks.dispatchReplyWithBufferedBlockDispatcher.mock.calls[0]?.[0].dispatcherOptions, + ).toEqual(expect.objectContaining({ beforeDeliver: expect.any(Function) })); }); it("does not inject approval buttons for native command replies once the monitor owns approvals", async () => { diff --git a/extensions/telegram/src/bot-native-commands.ts b/extensions/telegram/src/bot-native-commands.ts index 0465e0a0d1f..98fb3f5504f 100644 --- a/extensions/telegram/src/bot-native-commands.ts +++ b/extensions/telegram/src/bot-native-commands.ts @@ -940,6 +940,7 @@ export const registerTelegramNativeCommands = ({ cfg: executionCfg, dispatcherOptions: { ...replyPipeline, + beforeDeliver: async (payload) => payload, deliver: async (payload, _info) => { if ( shouldSuppressLocalTelegramExecApprovalPrompt({ diff --git a/extensions/voice-call/src/media-stream.test.ts b/extensions/voice-call/src/media-stream.test.ts index 16f51aeb4f6..126b78abda0 100644 --- a/extensions/voice-call/src/media-stream.test.ts +++ b/extensions/voice-call/src/media-stream.test.ts @@ -103,7 +103,7 @@ describe("MediaStreamHandler TTS queue", () => { started.push("active"); await waitForAbort(signal); }); - void handler.queueTts("stream-1", async () => { + const queued = handler.queueTts("stream-1", async () => { queuedRan = true; }); @@ -112,10 +112,37 @@ describe("MediaStreamHandler TTS queue", () => { handler.clearTtsQueue("stream-1"); await active; + await withTimeout(queued); await flush(); expect(queuedRan).toBe(false); }); + + it("resolves pending queued playback during stream teardown", async () => { + const handler = new MediaStreamHandler({ + transcriptionProvider: createStubSttProvider(), + providerConfig: {}, + }); + + let queuedRan = false; + const active = handler.queueTts("stream-1", async (signal) => { + await waitForAbort(signal); + }); + const queued = handler.queueTts("stream-1", async () => { + queuedRan = true; + }); + + await flush(); + ( + handler as unknown as { + clearTtsState(streamSid: string): void; + } + ).clearTtsState("stream-1"); + + await withTimeout(active); + await withTimeout(queued); + expect(queuedRan).toBe(false); + }); }); describe("MediaStreamHandler security hardening", () => { diff --git a/extensions/voice-call/src/media-stream.ts b/extensions/voice-call/src/media-stream.ts index 39128a9814e..4a633d3e86c 100644 --- a/extensions/voice-call/src/media-stream.ts +++ b/extensions/voice-call/src/media-stream.ts @@ -561,7 +561,7 @@ export class MediaStreamHandler { */ clearTtsQueue(streamSid: string, _reason = "unspecified"): void { const queue = this.getTtsQueue(streamSid); - queue.length = 0; + this.resolveQueuedTtsEntries(queue); this.ttsActiveControllers.get(streamSid)?.abort(); this.clearAudio(streamSid); } @@ -634,13 +634,21 @@ export class MediaStreamHandler { private clearTtsState(streamSid: string): void { const queue = this.ttsQueues.get(streamSid); if (queue) { - queue.length = 0; + this.resolveQueuedTtsEntries(queue); } this.ttsActiveControllers.get(streamSid)?.abort(); this.ttsActiveControllers.delete(streamSid); this.ttsPlaying.delete(streamSid); this.ttsQueues.delete(streamSid); } + + private resolveQueuedTtsEntries(queue: TtsQueueEntry[]): void { + const pending = queue.splice(0); + for (const entry of pending) { + entry.controller.abort(); + entry.resolve(); + } + } } /** diff --git a/extensions/voice-call/src/runtime.test.ts b/extensions/voice-call/src/runtime.test.ts index ad9aa64b208..b50864af5dd 100644 --- a/extensions/voice-call/src/runtime.test.ts +++ b/extensions/voice-call/src/runtime.test.ts @@ -78,6 +78,33 @@ function createBaseConfig(): VoiceCallConfig { return createVoiceCallBaseConfig({ tunnelProvider: "ngrok" }); } +function createExternalProviderConfig(params: { + provider: "twilio" | "telnyx" | "plivo"; + publicUrl?: string; +}): VoiceCallConfig { + const config = createVoiceCallBaseConfig({ + provider: params.provider, + tunnelProvider: "none", + }); + config.twilio = { + accountSid: "AC123", + authToken: "secret", + }; + config.telnyx = { + apiKey: "key", + connectionId: "conn", + publicKey: "pub", + }; + config.plivo = { + authId: "MA123", + authToken: "secret", + }; + if (params.publicUrl) { + config.publicUrl = params.publicUrl; + } + return config; +} + describe("createVoiceCallRuntime lifecycle", () => { beforeEach(() => { vi.clearAllMocks(); @@ -170,6 +197,36 @@ describe("createVoiceCallRuntime lifecycle", () => { expect(mocks.webhookCtorArgs[0]?.[4]).toBe(fullConfig); }); + it.each(["twilio", "telnyx", "plivo"] as const)( + "fails closed when %s falls back to a local-only webhook", + async (provider) => { + await expect( + createVoiceCallRuntime({ + config: createExternalProviderConfig({ provider }), + coreConfig: {} as CoreConfig, + agentRuntime: {} as never, + }), + ).rejects.toThrow(`${provider} requires a publicly reachable webhook URL`); + expect(mocks.webhookStop).toHaveBeenCalledTimes(1); + }, + ); + + it("accepts an explicit public URL for external voice providers", async () => { + const runtime = await createVoiceCallRuntime({ + config: createExternalProviderConfig({ + provider: "twilio", + publicUrl: "https://voice.example.com/voice/webhook", + }), + coreConfig: {} as CoreConfig, + agentRuntime: {} as never, + }); + + expect(runtime.webhookUrl).toBe("https://voice.example.com/voice/webhook"); + expect(runtime.publicUrl).toBe("https://voice.example.com/voice/webhook"); + + await runtime.stop(); + }); + it("wires the shared realtime agent consult tool and handler", async () => { const config = createBaseConfig(); config.inboundPolicy = "allowlist"; diff --git a/extensions/voice-call/src/runtime.ts b/extensions/voice-call/src/runtime.ts index 4b1fb181f85..eb509a40d20 100644 --- a/extensions/voice-call/src/runtime.ts +++ b/extensions/voice-call/src/runtime.ts @@ -158,6 +158,40 @@ function isLoopbackBind(bind: string | undefined): boolean { return bind === "127.0.0.1" || bind === "::1" || bind === "localhost"; } +function providerRequiresPublicWebhook(providerName: VoiceCallProvider["name"]): boolean { + return providerName === "twilio" || providerName === "telnyx" || providerName === "plivo"; +} + +function isLocalOnlyWebhookHost(hostname: string): boolean { + const host = hostname.trim().toLowerCase(); + if (!host) { + return false; + } + if ( + host === "localhost" || + host === "0.0.0.0" || + host === "::" || + host === "::1" || + host.startsWith("127.") + ) { + return true; + } + if (host.startsWith("10.") || host.startsWith("192.168.") || host.startsWith("169.254.")) { + return true; + } + const private172 = /^172\.(1[6-9]|2\d|3[0-1])\./.test(host); + return private172 || host.startsWith("fc") || host.startsWith("fd"); +} + +function isProviderUnreachableWebhookUrl(webhookUrl: string): boolean { + try { + const parsed = new URL(webhookUrl); + return isLocalOnlyWebhookHost(parsed.hostname); + } catch { + return false; + } +} + async function resolveProvider(config: VoiceCallConfig): Promise { const allowNgrokFreeTierLoopbackBypass = config.tunnel?.provider === "ngrok" && @@ -376,6 +410,17 @@ export async function createVoiceCallRuntime(params: { const webhookUrl = publicUrl ?? localUrl; + if ( + providerRequiresPublicWebhook(provider.name) && + isProviderUnreachableWebhookUrl(webhookUrl) + ) { + throw new Error( + `[voice-call] ${provider.name} requires a publicly reachable webhook URL. ` + + `Refusing to use local-only webhook ${webhookUrl}. ` + + "Set plugins.entries.voice-call.config.publicUrl or enable tunnel/tailscale exposure.", + ); + } + if (publicUrl && provider.name === "twilio") { (provider as TwilioProvider).setPublicUrl(publicUrl); } diff --git a/extensions/voice-call/src/tunnel.test.ts b/extensions/voice-call/src/tunnel.test.ts new file mode 100644 index 00000000000..c60d50097ed --- /dev/null +++ b/extensions/voice-call/src/tunnel.test.ts @@ -0,0 +1,166 @@ +import { EventEmitter } from "node:events"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +class FakeChildProcess extends EventEmitter { + readonly stdout = new EventEmitter(); + readonly stderr = new EventEmitter(); + killedWith: NodeJS.Signals | null = null; + + kill(signal: NodeJS.Signals = "SIGTERM"): boolean { + this.killedWith = signal; + queueMicrotask(() => this.emit("close", null)); + return true; + } + + close(code: number | null = 0): void { + this.emit("close", code); + } + + fail(error: Error): void { + this.emit("error", error); + } +} + +const mocks = vi.hoisted(() => ({ + spawn: vi.fn(), + getTailscaleDnsName: vi.fn(), +})); + +vi.mock("node:child_process", () => ({ + spawn: mocks.spawn, +})); + +vi.mock("./webhook/tailscale.js", () => ({ + getTailscaleDnsName: mocks.getTailscaleDnsName, +})); + +import { isNgrokAvailable, startNgrokTunnel, startTailscaleTunnel, startTunnel } from "./tunnel.js"; + +function nextProcess(): FakeChildProcess { + const proc = new FakeChildProcess(); + mocks.spawn.mockReturnValueOnce(proc as never); + return proc; +} + +function emitNgrokUrl(proc: FakeChildProcess, url: string): void { + proc.stdout.emit("data", Buffer.from(`${JSON.stringify({ msg: "started tunnel", url })}\n`)); +} + +describe("voice-call tunnels", () => { + beforeEach(() => { + vi.clearAllMocks(); + mocks.getTailscaleDnsName.mockReset(); + }); + + it("checks ngrok availability from the version command exit code", async () => { + const proc = nextProcess(); + const result = isNgrokAvailable(); + proc.close(0); + + await expect(result).resolves.toBe(true); + expect(mocks.spawn).toHaveBeenCalledWith("ngrok", ["version"], expect.any(Object)); + }); + + it("treats ngrok spawn failures as unavailable", async () => { + const proc = nextProcess(); + const result = isNgrokAvailable(); + proc.fail(new Error("spawn ngrok ENOENT")); + + await expect(result).resolves.toBe(false); + }); + + it("starts ngrok and appends the webhook path to the public URL", async () => { + const proc = nextProcess(); + const result = startNgrokTunnel({ port: 3334, path: "/voice/webhook" }); + + emitNgrokUrl(proc, "https://abc.ngrok.io"); + + await expect(result).resolves.toMatchObject({ + publicUrl: "https://abc.ngrok.io/voice/webhook", + provider: "ngrok", + }); + expect(mocks.spawn).toHaveBeenCalledWith( + "ngrok", + expect.arrayContaining(["http", "3334"]), + expect.any(Object), + ); + }); + + it("sets ngrok auth token before starting the tunnel", async () => { + const authProc = nextProcess(); + const tunnelProc = nextProcess(); + const result = startNgrokTunnel({ + port: 3334, + path: "/hook", + authToken: "token", + }); + + authProc.close(0); + await vi.waitFor(() => expect(mocks.spawn).toHaveBeenCalledTimes(2)); + emitNgrokUrl(tunnelProc, "https://auth.ngrok.io"); + + await expect(result).resolves.toMatchObject({ + publicUrl: "https://auth.ngrok.io/hook", + }); + expect(mocks.spawn).toHaveBeenNthCalledWith( + 1, + "ngrok", + ["config", "add-authtoken", "token"], + expect.any(Object), + ); + }); + + it("rejects ngrok startup errors from stderr", async () => { + const proc = nextProcess(); + const result = startNgrokTunnel({ port: 3334, path: "/hook" }); + + proc.stderr.emit("data", Buffer.from("ERR_NGROK_3200: invalid auth token")); + + await expect(result).rejects.toThrow("ngrok error:"); + }); + + it("starts Tailscale serve using the resolved tailnet DNS name", async () => { + mocks.getTailscaleDnsName.mockResolvedValue("host.tailnet.ts.net"); + const proc = nextProcess(); + const result = startTailscaleTunnel({ + mode: "serve", + port: 3334, + path: "voice/webhook", + }); + + await vi.waitFor(() => expect(mocks.spawn).toHaveBeenCalled()); + proc.close(0); + + await expect(result).resolves.toMatchObject({ + publicUrl: "https://host.tailnet.ts.net/voice/webhook", + provider: "tailscale-serve", + }); + expect(mocks.spawn).toHaveBeenCalledWith( + "tailscale", + expect.arrayContaining(["serve", "--set-path", "/voice/webhook"]), + expect.any(Object), + ); + }); + + it("rejects Tailscale tunnel startup when the DNS name is unavailable", async () => { + mocks.getTailscaleDnsName.mockResolvedValue(null); + + await expect( + startTailscaleTunnel({ mode: "funnel", port: 3334, path: "/hook" }), + ).rejects.toThrow("Could not get Tailscale DNS name"); + expect(mocks.spawn).not.toHaveBeenCalled(); + }); + + it("dispatches tunnel providers from config", async () => { + await expect(startTunnel({ provider: "none", port: 3334, path: "/hook" })).resolves.toBeNull(); + + const proc = nextProcess(); + const result = startTunnel({ provider: "ngrok", port: 3334, path: "/hook" }); + emitNgrokUrl(proc, "https://dispatch.ngrok.io"); + + await expect(result).resolves.toMatchObject({ + publicUrl: "https://dispatch.ngrok.io/hook", + provider: "ngrok", + }); + }); +}); diff --git a/qa/scenarios/agents/subagent-stale-child-links.md b/qa/scenarios/agents/subagent-stale-child-links.md new file mode 100644 index 00000000000..7f6a18b86dd --- /dev/null +++ b/qa/scenarios/agents/subagent-stale-child-links.md @@ -0,0 +1,175 @@ +# Subagent stale child links + +```yaml qa-scenario +id: subagent-stale-child-links +title: Subagent stale child links +surface: subagents +coverage: + primary: + - agents.subagents + secondary: + - gateway.sessions-list +objective: Verify restarted gateways hide stale persisted subagent child links without hiding live or fresh children. +successCriteria: + - Old ended subagent run records are not exposed as current children. + - Old store-only spawnedBy and parentSessionKey rows are not exposed as current children. + - Child-side ACP store rows from sibling agents are not exposed as current children. + - Live subagent runs and fresh dashboard children remain visible. +docsRefs: + - docs/tools/subagents.md + - docs/concepts/qa-e2e-automation.md + - docs/help/testing.md +codeRefs: + - src/gateway/session-utils.ts + - src/agents/subagent-run-liveness.ts + - extensions/qa-lab/src/gateway-child.ts +execution: + kind: flow + summary: Seed stale subagent session state on disk, restart the real gateway, then assert sessions.list filters only the stale child links. +``` + +```yaml qa-flow +steps: + - name: restarted gateway filters stale subagent child links + actions: + - call: waitForGatewayHealthy + args: + - ref: env + - 60000 + - set: mainKey + value: "agent:qa:main" + - set: staleRunKey + value: "agent:qa:subagent:qa-stale-ended" + - set: staleOrphanKey + value: "agent:qa:subagent:qa-orphan" + - set: staleAcpKey + value: "agent:claude:acp:qa-stale-acp" + - set: freshDashboardKey + value: "agent:qa:dashboard:qa-fresh-child" + - set: liveRunKey + value: "agent:qa:subagent:qa-live-child" + - call: env.gateway.restartAfterStateMutation + args: + - lambda: + params: + - ctx + async: true + expr: |- + await (async () => { + const now = Date.now(); + const old = now - 2 * 60 * 60 * 1000; + const recent = now - 5000; + const qaSessionsDir = path.join(ctx.stateDir, "agents", "qa", "sessions"); + const claudeSessionsDir = path.join(ctx.stateDir, "agents", "claude", "sessions"); + const subagentDir = path.join(ctx.stateDir, "subagents"); + await fs.mkdir(qaSessionsDir, { recursive: true }); + await fs.mkdir(claudeSessionsDir, { recursive: true }); + await fs.mkdir(subagentDir, { recursive: true }); + await fs.writeFile(path.join(subagentDir, "runs.json"), `${JSON.stringify({ + version: 2, + runs: { + "run-stale-ended": { + runId: "run-stale-ended", + childSessionKey: staleRunKey, + controllerSessionKey: mainKey, + requesterSessionKey: mainKey, + requesterDisplayKey: "main", + task: "old ended ghost", + cleanup: "keep", + createdAt: old - 60000, + startedAt: old - 50000, + endedAt: old, + outcome: { status: "ok" }, + }, + "run-live-visible": { + runId: "run-live-visible", + childSessionKey: liveRunKey, + controllerSessionKey: mainKey, + requesterSessionKey: mainKey, + requesterDisplayKey: "main", + task: "live child remains visible", + cleanup: "keep", + createdAt: recent, + startedAt: recent, + }, + }, + }, null, 2)}\n`, "utf8"); + await fs.writeFile(path.join(qaSessionsDir, "sessions.json"), `${JSON.stringify({ + [mainKey]: { + sessionId: "sess-main", + updatedAt: now, + }, + [staleRunKey]: { + sessionId: "sess-stale-run", + updatedAt: old, + spawnedBy: mainKey, + status: "done", + endedAt: old, + }, + [staleOrphanKey]: { + sessionId: "sess-orphan", + updatedAt: old, + parentSessionKey: mainKey, + }, + [freshDashboardKey]: { + sessionId: "sess-fresh-dashboard", + updatedAt: now, + parentSessionKey: mainKey, + }, + [liveRunKey]: { + sessionId: "sess-live-child", + updatedAt: recent, + spawnedBy: mainKey, + }, + }, null, 2)}\n`, "utf8"); + await fs.writeFile(path.join(claudeSessionsDir, "sessions.json"), `${JSON.stringify({ + [staleAcpKey]: { + sessionId: "sess-acp-stale", + updatedAt: old, + spawnedBy: mainKey, + status: "done", + endedAt: old, + }, + }, null, 2)}\n`, "utf8"); + })() + - call: waitForGatewayHealthy + args: + - ref: env + - 60000 + - call: env.gateway.call + saveAs: listed + args: + - "sessions.list" + - {} + - timeoutMs: 60000 + - call: env.gateway.call + saveAs: filtered + args: + - "sessions.list" + - spawnedBy: + ref: mainKey + - timeoutMs: 60000 + - set: mainChildren + value: + expr: "(listed.sessions.find((session) => session.key === mainKey)?.childSessions ?? [])" + - set: filteredKeys + value: + expr: "filtered.sessions.map((session) => session.key)" + - assert: + expr: "mainChildren.includes(freshDashboardKey)" + message: + expr: "`fresh dashboard child missing from main children: ${JSON.stringify(mainChildren)}`" + - assert: + expr: "mainChildren.includes(liveRunKey)" + message: + expr: "`live subagent child missing from main children: ${JSON.stringify(mainChildren)}`" + - assert: + expr: "filteredKeys.includes(freshDashboardKey) && filteredKeys.includes(liveRunKey)" + message: + expr: "`spawnedBy filter dropped live/fresh children: ${JSON.stringify(filteredKeys)}`" + - assert: + expr: "![staleRunKey, staleOrphanKey, staleAcpKey].some((key) => mainChildren.includes(key) || filteredKeys.includes(key))" + message: + expr: "`stale child leaked through sessions.list (main=${JSON.stringify(mainChildren)} filtered=${JSON.stringify(filteredKeys)})`" + detailsExpr: "({ mainChildren, filteredKeys })" +``` diff --git a/qa/scenarios/index.md b/qa/scenarios/index.md index d1d1edd4ef2..c790310c31e 100644 --- a/qa/scenarios/index.md +++ b/qa/scenarios/index.md @@ -25,7 +25,7 @@ Coverage tracking: Theme directories: -- `agents/` - agent behavior, instructions, and subagent flows +- `agents/` - agent behavior, instructions, subagent flows, and persisted child-link regressions - `channels/` - DM, shared channel, thread, and message-action behavior - `character/` - persona and style eval scenarios - `config/` - config patch, apply, and restart behavior diff --git a/src/agents/openai-reasoning-compat.live.test.ts b/src/agents/openai-reasoning-compat.live.test.ts index 09670fb6ff5..ae8bcf5106c 100644 --- a/src/agents/openai-reasoning-compat.live.test.ts +++ b/src/agents/openai-reasoning-compat.live.test.ts @@ -1,10 +1,14 @@ +import type { AgentMessage } from "@mariozechner/pi-agent-core"; import { completeSimple, type Api, type Model } from "@mariozechner/pi-ai"; +import { SessionManager } from "@mariozechner/pi-coding-agent"; +import { Type } from "typebox"; import { describe, expect, it } from "vitest"; import { loadConfig } from "../config/config.js"; import { resolveOpenClawAgentDir } from "./agent-paths.js"; import { isLiveProfileKeyModeEnabled, isLiveTestEnabled } from "./live-test-helpers.js"; import { getApiKeyForModel, requireApiKey } from "./model-auth.js"; import { ensureOpenClawModelsJson } from "./models-config.js"; +import { sanitizeSessionHistory } from "./pi-embedded-runner/replay-history.js"; import { discoverAuthStorage, discoverModels } from "./pi-model-discovery.js"; const LIVE = isLiveTestEnabled(); @@ -169,4 +173,141 @@ describeLive("openai reasoning compat live", () => { }, 3 * 60 * 1000, ); + + it( + "accepts repaired OpenAI Codex parallel tool replay with aborted missing results", + async () => { + const { provider, modelId } = resolveTargetModelRef(); + const cfg = loadConfig(); + await ensureOpenClawModelsJson(cfg); + + const agentDir = resolveOpenClawAgentDir(); + const authStorage = discoverAuthStorage(agentDir); + const modelRegistry = discoverModels(authStorage, agentDir); + const model = modelRegistry.find(provider, modelId) as Model | null; + + if (!model) { + logProgress(`[openai-reasoning-compat] model missing from registry: ${TARGET_MODEL_REF}`); + return; + } + + let apiKeyInfo; + try { + apiKeyInfo = await getApiKeyForModel({ + model, + cfg, + credentialPrecedence: LIVE_CREDENTIAL_PRECEDENCE, + }); + } catch (error) { + logProgress(`[openai-reasoning-compat] skip (${String(error)})`); + return; + } + + if (REQUIRE_PROFILE_KEYS && !apiKeyInfo.source.startsWith("profile:")) { + logProgress( + `[openai-reasoning-compat] skip (non-profile credential source: ${apiKeyInfo.source})`, + ); + return; + } + + const messages = [ + { + role: "user", + content: "Use noop.", + timestamp: Date.now(), + }, + { + role: "assistant", + provider: model.provider, + api: model.api, + model: model.id, + stopReason: "toolUse", + timestamp: Date.now(), + content: [ + { type: "toolCall", id: "call_keep", name: "noop", arguments: {} }, + { type: "toolCall", id: "call_missing_a", name: "noop", arguments: {} }, + { type: "toolCall", id: "call_missing_b", name: "noop", arguments: {} }, + ], + }, + { + role: "user", + content: "Reply with exactly: replay ok.", + timestamp: Date.now(), + }, + { + role: "toolResult", + toolCallId: "call_keep", + toolName: "noop", + content: [{ type: "text", text: "ok" }], + isError: false, + timestamp: Date.now(), + }, + ] as unknown as AgentMessage[]; + + const sanitized = await sanitizeSessionHistory({ + messages, + modelApi: model.api, + provider: model.provider, + modelId: model.id, + sessionManager: SessionManager.inMemory(), + sessionId: "openai-codex-tool-replay-live", + }); + + expect(sanitized.map((message) => message.role)).toEqual([ + "user", + "assistant", + "toolResult", + "toolResult", + "toolResult", + "user", + ]); + expect( + sanitized.slice(2, 5).map((message) => (message as { toolCallId?: string }).toolCallId), + ).toEqual(["call_keep", "call_missing_a", "call_missing_b"]); + expect( + sanitized + .slice(3, 5) + .map((message) => (message as Extract).content), + ).toEqual([[{ type: "text", text: "aborted" }], [{ type: "text", text: "aborted" }]]); + expect(JSON.stringify(sanitized)).not.toContain("missing tool result"); + + const response = await completeSimpleWithTimeout( + model, + { + systemPrompt: "You are a concise assistant. Follow the user's instruction exactly.", + messages: sanitized as never, + tools: [ + { + name: "noop", + description: "Return ok.", + parameters: Type.Object({}, { additionalProperties: false }), + }, + ], + }, + { + apiKey: requireApiKey(apiKeyInfo, model.provider), + reasoning: "low", + maxTokens: 64, + }, + 120_000, + ); + + const text = response.content + .filter((block) => block.type === "text") + .map((block) => block.text.trim()) + .join(" ") + .trim(); + const errorMessage = + typeof (response as { errorMessage?: unknown }).errorMessage === "string" + ? ((response as { errorMessage?: string }).errorMessage ?? "") + : ""; + if (errorMessage && isKnownLiveBlocker(errorMessage)) { + logProgress(`[openai-reasoning-compat] skip (${errorMessage})`); + return; + } + + expect(text).toMatch(/^replay ok\.?$/i); + }, + 3 * 60 * 1000, + ); }); diff --git a/src/agents/openclaw-tools.ts b/src/agents/openclaw-tools.ts index ce3e4eaf48a..e3055ba0a5b 100644 --- a/src/agents/openclaw-tools.ts +++ b/src/agents/openclaw-tools.ts @@ -246,7 +246,7 @@ export function createOpenClawTools( ...(!embedded && messageTool ? [messageTool] : []), createTtsTool({ agentChannel: options?.agentChannel, - config: options?.config, + config: resolvedConfig, }), ...collectPresentOpenClawTools([imageGenerateTool, musicGenerateTool, videoGenerateTool]), ...(embedded diff --git a/src/agents/openclaw-tools.tts-config.test.ts b/src/agents/openclaw-tools.tts-config.test.ts new file mode 100644 index 00000000000..0758ff22f4b --- /dev/null +++ b/src/agents/openclaw-tools.tts-config.test.ts @@ -0,0 +1,62 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import type { OpenClawConfig } from "../config/types.openclaw.js"; + +const mocks = vi.hoisted(() => ({ + textToSpeech: vi.fn(async () => ({ + success: true, + audioPath: "/tmp/openclaw/tts-config-test.opus", + provider: "microsoft", + voiceCompatible: true, + })), +})); + +vi.mock("../tts/tts.js", () => ({ + textToSpeech: mocks.textToSpeech, +})); + +describe("createOpenClawTools TTS config wiring", () => { + beforeEach(() => { + mocks.textToSpeech.mockClear(); + }); + + it("passes the resolved shared config into the tts tool", async () => { + const injectedConfig = { + messages: { + tts: { + auto: "always", + provider: "microsoft", + providers: { + microsoft: { + voice: "en-US-AvaNeural", + }, + }, + }, + }, + } satisfies OpenClawConfig; + + const { __testing, createOpenClawTools } = await import("./openclaw-tools.js"); + __testing.setDepsForTest({ config: injectedConfig }); + + try { + const tool = createOpenClawTools({ + disableMessageTool: true, + disablePluginTools: true, + }).find((candidate) => candidate.name === "tts"); + + if (!tool) { + throw new Error("missing tts tool"); + } + + await tool.execute("call-1", { text: "hello from config" }); + + expect(mocks.textToSpeech).toHaveBeenCalledWith( + expect.objectContaining({ + text: "hello from config", + cfg: injectedConfig, + }), + ); + } finally { + __testing.setDepsForTest(); + } + }); +}); diff --git a/src/agents/pi-embedded-runner.sanitize-session-history.test.ts b/src/agents/pi-embedded-runner.sanitize-session-history.test.ts index 8c9a54b5caa..cc8fb530b5f 100644 --- a/src/agents/pi-embedded-runner.sanitize-session-history.test.ts +++ b/src/agents/pi-embedded-runner.sanitize-session-history.test.ts @@ -688,20 +688,181 @@ describe("sanitizeSessionHistory", () => { expect(result[1]?.role).toBe("assistant"); }); - it("synthesizes missing tool results for openai-responses after repair", async () => { + it("synthesizes Codex-style aborted tool results for openai-responses after repair", async () => { const messages: AgentMessage[] = [ + makeUserMessage("start"), makeAssistantMessage([{ type: "toolCall", id: "call_1", name: "read", arguments: {} }], { stopReason: "toolUse", }), + makeUserMessage("continue"), + ]; + + const result = await sanitizeOpenAIHistory(messages); + + expect(result.map((message) => message.role)).toEqual([ + "user", + "assistant", + "toolResult", + "user", + ]); + expect((result[2] as { toolCallId?: string }).toolCallId).toBe("call1"); + expect((result[2] as Extract).content).toEqual([ + { type: "text", text: "aborted" }, + ]); + expect(JSON.stringify(result)).not.toContain("missing tool result"); + }); + + it("synthesizes Codex-style aborted tool results for openai-codex-responses", async () => { + const messages: AgentMessage[] = [ + makeAssistantMessage( + [ + { type: "toolCall", id: "call_a", name: "exec", arguments: {} }, + { type: "toolCall", id: "call_b", name: "exec", arguments: {} }, + { type: "toolCall", id: "call_c", name: "exec", arguments: {} }, + ], + { stopReason: "toolUse" }, + ), + makeUserMessage("status?"), + ]; + + const result = await sanitizeSessionHistory({ + messages, + modelApi: "openai-codex-responses", + provider: "openai-codex", + sessionManager: mockSessionManager, + sessionId: TEST_SESSION_ID, + }); + + expect(result.map((message) => message.role)).toEqual([ + "assistant", + "toolResult", + "toolResult", + "toolResult", + "user", + ]); + expect( + result.slice(1, 4).map((message) => (message as { toolCallId?: string }).toolCallId), + ).toEqual(["calla", "callb", "callc"]); + for (const message of result.slice(1, 4)) { + expect((message as Extract).content).toEqual([ + { type: "text", text: "aborted" }, + ]); + } + expect(JSON.stringify(result)).not.toContain("missing tool result"); + }); + + it("keeps real parallel tool results for openai-responses and aborts missing siblings", async () => { + const messages: AgentMessage[] = [ + makeAssistantMessage( + [ + { type: "toolCall", id: "call_1", name: "read", arguments: {} }, + { type: "toolCall", id: "call_2", name: "exec", arguments: {} }, + { type: "toolCall", id: "call_3", name: "write", arguments: {} }, + ], + { stopReason: "toolUse" }, + ), + makeUserMessage("continue"), + castAgentMessage({ + role: "toolResult", + toolCallId: "call_2", + toolName: "exec", + content: [{ type: "text", text: "ok" }], + isError: false, + }), ]; const result = await sanitizeOpenAIHistory(messages); - // repairToolUseResultPairing now runs for all providers (including OpenAI) - // to fix orphaned function_call_output items that OpenAI would reject. - expect(result).toHaveLength(2); - expect(result[0]?.role).toBe("assistant"); - expect(result[1]?.role).toBe("toolResult"); + expect(result.map((message) => message.role)).toEqual([ + "assistant", + "toolResult", + "toolResult", + "toolResult", + "user", + ]); + expect( + extractToolCallsFromAssistant(result[0] as Extract), + ).toMatchObject([ + { id: "call1", name: "read" }, + { id: "call2", name: "exec" }, + { id: "call3", name: "write" }, + ]); + expect( + result.slice(1, 4).map((message) => (message as { toolCallId?: string }).toolCallId), + ).toEqual(["call1", "call2", "call3"]); + expect((result[1] as Extract).content).toEqual([ + { type: "text", text: "aborted" }, + ]); + expect((result[2] as Extract).content).toEqual([ + { type: "text", text: "ok" }, + ]); + expect((result[3] as Extract).content).toEqual([ + { type: "text", text: "aborted" }, + ]); + expect(JSON.stringify(result)).not.toContain("missing tool result"); + }); + + it("applies aborted missing-result repair to azure-openai-responses", async () => { + const messages: AgentMessage[] = [ + makeAssistantMessage([{ type: "toolCall", id: "call_azure", name: "read", arguments: {} }], { + stopReason: "toolUse", + }), + makeUserMessage("continue"), + ]; + + const result = await sanitizeSessionHistory({ + messages, + modelApi: "azure-openai-responses", + provider: "azure-openai-responses", + sessionManager: mockSessionManager, + sessionId: TEST_SESSION_ID, + }); + + expect(result.map((message) => message.role)).toEqual(["assistant", "toolResult", "user"]); + expect((result[1] as { toolCallId?: string }).toolCallId).toBe("callazure"); + expect((result[1] as Extract).content).toEqual([ + { type: "text", text: "aborted" }, + ]); + }); + + it("drops duplicate and orphan OpenAI outputs while preserving the first real result", async () => { + const messages: AgentMessage[] = [ + castAgentMessage({ + role: "toolResult", + toolCallId: "call_orphan", + toolName: "read", + content: [{ type: "text", text: "orphan" }], + isError: false, + }), + makeAssistantMessage([{ type: "toolCall", id: "call_keep", name: "read", arguments: {} }], { + stopReason: "toolUse", + }), + castAgentMessage({ + role: "toolResult", + toolCallId: "call_keep", + toolName: "read", + content: [{ type: "text", text: "first" }], + isError: false, + }), + castAgentMessage({ + role: "toolResult", + toolCallId: "call_keep", + toolName: "read", + content: [{ type: "text", text: "duplicate" }], + isError: false, + }), + makeUserMessage("continue"), + ]; + + const result = await sanitizeOpenAIHistory(messages); + + expect(result.map((message) => message.role)).toEqual(["assistant", "toolResult", "user"]); + expect((result[1] as { toolCallId?: string }).toolCallId).toBe("callkeep"); + expect((result[1] as Extract).content).toEqual([ + { type: "text", text: "first" }, + ]); + expect(JSON.stringify(result)).not.toContain("orphan"); + expect(JSON.stringify(result)).not.toContain("duplicate"); }); it.each([ diff --git a/src/agents/pi-embedded-runner/compact.ts b/src/agents/pi-embedded-runner/compact.ts index 30f042b8e4c..703a72a7dfe 100644 --- a/src/agents/pi-embedded-runner/compact.ts +++ b/src/agents/pi-embedded-runner/compact.ts @@ -810,6 +810,12 @@ export async function compactEmbeddedPiSessionDirect( config: params.config, contextWindowTokens: ctxInfo.tokens, allowSyntheticToolResults: transcriptPolicy.allowSyntheticToolResults, + missingToolResultText: + model.api === "openai-responses" || + model.api === "azure-openai-responses" || + model.api === "openai-codex-responses" + ? "aborted" + : undefined, allowedToolNames, }); checkpointSnapshot = captureCompactionCheckpointSnapshot({ @@ -965,6 +971,11 @@ export async function compactEmbeddedPiSessionDirect( const limited = transcriptPolicy.repairToolUseResultPairing ? sanitizeToolUseResultPairing(truncated, { erroredAssistantResultPolicy: "drop", + ...(model.api === "openai-responses" || + model.api === "azure-openai-responses" || + model.api === "openai-codex-responses" + ? { missingToolResultText: "aborted" } + : {}), }) : truncated; if (limited.length > 0) { diff --git a/src/agents/pi-embedded-runner/replay-history.ts b/src/agents/pi-embedded-runner/replay-history.ts index 938f063f1ff..b912b4b1234 100644 --- a/src/agents/pi-embedded-runner/replay-history.ts +++ b/src/agents/pi-embedded-runner/replay-history.ts @@ -493,13 +493,17 @@ export async function sanitizeSessionHistory(params: { allowedToolNames: params.allowedToolNames, allowProviderOwnedThinkingReplay, }); - // OpenAI's fc_* pairing downgrade needs the raw call_id|fc_id separator intact, - // but displaced tool results must first be repaired back next to their - // assistant turn so the downgrade can rewrite both sides consistently. + // OpenAI Responses rejects orphan/missing function_call_output items. Upstream + // Codex repairs those gaps with "aborted"; keep that before the fc_* downgrade + // so both call and result ids are rewritten together. Covered by unit replay + // tests plus live OpenAI/Codex and generic replay-repair model tests. const openAIRepairedToolCalls = isOpenAIResponsesApi && policy.repairToolUseResultPairing ? sanitizeToolUseResultPairing(sanitizedToolCalls, { erroredAssistantResultPolicy: "drop", + // Match upstream Codex history normalization for OpenAI Responses: + // missing function_call_output entries are model-visible "aborted". + missingToolResultText: "aborted", }) : sanitizedToolCalls; const openAISafeToolCalls = isOpenAIResponsesApi @@ -517,6 +521,9 @@ export async function sanitizeSessionHistory(params: { allowedToolNames: params.allowedToolNames, }) : openAISafeToolCalls; + // Gemini/Anthropic-class providers also require tool results to stay adjacent + // to their assistant tool calls. They do not use Codex's "aborted" text, but + // the same ordering repair is live-tested with Gemini 3 Flash. const repairedTools = !isOpenAIResponsesApi && policy.repairToolUseResultPairing ? sanitizeToolUseResultPairing(sanitizedToolIds, { diff --git a/src/agents/pi-embedded-runner/run/attempt.tool-call-normalization.test.ts b/src/agents/pi-embedded-runner/run/attempt.tool-call-normalization.test.ts index 6620dda0d89..c605c7e4b1d 100644 --- a/src/agents/pi-embedded-runner/run/attempt.tool-call-normalization.test.ts +++ b/src/agents/pi-embedded-runner/run/attempt.tool-call-normalization.test.ts @@ -61,6 +61,65 @@ describe("sanitizeReplayToolCallIdsForStream", () => { ]); }); + it("synthesizes missing tool results after strict id sanitization", () => { + const rawId = "call_function_av7cbkigmk7x1"; + const out = sanitizeReplayToolCallIdsForStream({ + messages: [ + { + role: "assistant", + content: [ + { type: "toolUse", id: rawId, name: "read", input: { path: "." } }, + { type: "toolUse", id: "call_missing", name: "exec", input: { cmd: "true" } }, + ], + } as never, + { + role: "toolResult", + toolCallId: rawId, + toolUseId: rawId, + toolName: "read", + content: [{ type: "text", text: "ok" }], + isError: false, + } as never, + ], + mode: "strict", + repairToolUseResultPairing: true, + }); + + expect(out.map((message) => message.role)).toEqual(["assistant", "toolResult", "toolResult"]); + expect((out[0] as Extract).content).toMatchObject([ + { type: "toolUse", id: "callfunctionav7cbkigmk7x1", name: "read" }, + { type: "toolUse", id: "callmissing", name: "exec" }, + ]); + expect(out[1]).toMatchObject({ + role: "toolResult", + toolCallId: "callfunctionav7cbkigmk7x1", + toolUseId: "callfunctionav7cbkigmk7x1", + }); + expect(out[2]).toMatchObject({ + role: "toolResult", + toolCallId: "callmissing", + isError: true, + }); + }); + + it("synthesizes missing tool results when repair is enabled", () => { + const out = sanitizeReplayToolCallIdsForStream({ + messages: [ + { + role: "assistant", + content: [{ type: "toolUse", id: "call_missing", name: "exec", input: { cmd: "true" } }], + } as never, + ], + mode: "strict", + repairToolUseResultPairing: true, + }); + + expect(out).toMatchObject([ + { role: "assistant" }, + { role: "toolResult", toolCallId: "callmissing", isError: true }, + ]); + }); + it("keeps real tool results for aborted assistant spans", () => { const rawId = "call_function_av7cbkigmk7x1"; const out = sanitizeReplayToolCallIdsForStream({ diff --git a/src/agents/pi-embedded-runner/run/attempt.ts b/src/agents/pi-embedded-runner/run/attempt.ts index 5bb325ed5c0..c2bc4162b86 100644 --- a/src/agents/pi-embedded-runner/run/attempt.ts +++ b/src/agents/pi-embedded-runner/run/attempt.ts @@ -1193,6 +1193,12 @@ export async function runEmbeddedAttempt( contextWindowTokens: params.contextTokenBudget, inputProvenance: params.inputProvenance, allowSyntheticToolResults: transcriptPolicy.allowSyntheticToolResults, + missingToolResultText: + params.model.api === "openai-responses" || + params.model.api === "azure-openai-responses" || + params.model.api === "openai-codex-responses" + ? "aborted" + : undefined, allowedToolNames, }); trackSessionManagerAccess(params.sessionFile); @@ -1840,6 +1846,7 @@ export async function runEmbeddedAttempt( const limited = transcriptPolicy.repairToolUseResultPairing ? sanitizeToolUseResultPairing(truncated, { erroredAssistantResultPolicy: "drop", + ...(isOpenAIResponsesApi ? { missingToolResultText: "aborted" } : {}), }) : truncated; cacheTrace?.recordStage("session:limited", { messages: limited }); diff --git a/src/agents/session-tool-result-guard-wrapper.ts b/src/agents/session-tool-result-guard-wrapper.ts index cdfd499d90a..79939e7ab96 100644 --- a/src/agents/session-tool-result-guard-wrapper.ts +++ b/src/agents/session-tool-result-guard-wrapper.ts @@ -29,6 +29,7 @@ export function guardSessionManager( contextWindowTokens?: number; inputProvenance?: InputProvenance; allowSyntheticToolResults?: boolean; + missingToolResultText?: string; allowedToolNames?: Iterable; }, ): GuardedSessionManager { @@ -75,6 +76,7 @@ export function guardSessionManager( applyInputProvenanceToUserMessage(message, opts?.inputProvenance), transformToolResultForPersistence: transform, allowSyntheticToolResults: opts?.allowSyntheticToolResults, + missingToolResultText: opts?.missingToolResultText, allowedToolNames: opts?.allowedToolNames, beforeMessageWriteHook: beforeMessageWrite, maxToolResultChars: diff --git a/src/agents/session-tool-result-guard.test.ts b/src/agents/session-tool-result-guard.test.ts index 93fa78083a9..82c1fc5311a 100644 --- a/src/agents/session-tool-result-guard.test.ts +++ b/src/agents/session-tool-result-guard.test.ts @@ -111,6 +111,18 @@ describe("installSessionToolResultGuard", () => { expectPersistedRoles(sm, ["assistant", "toolResult"]); }); + it("uses configured text for synthetic tool results", () => { + const sm = SessionManager.inMemory(); + const guard = installSessionToolResultGuard(sm, { + missingToolResultText: "aborted", + }); + + sm.appendMessage(toolCallMessage); + guard.flushPendingToolResults(); + + expect(getToolResultText(getPersistedMessages(sm))).toBe("aborted"); + }); + it("clears pending tool calls without inserting synthetic tool results", () => { const sm = SessionManager.inMemory(); const guard = installSessionToolResultGuard(sm); diff --git a/src/agents/session-tool-result-guard.ts b/src/agents/session-tool-result-guard.ts index 242e26ac0eb..c3cb82314c5 100644 --- a/src/agents/session-tool-result-guard.ts +++ b/src/agents/session-tool-result-guard.ts @@ -90,6 +90,7 @@ export function installSessionToolResultGuard( * Defaults to true. */ allowSyntheticToolResults?: boolean; + missingToolResultText?: string; /** * Optional set/list of tool names accepted for assistant toolCall/toolUse blocks. * When set, tool calls with unknown names are dropped before persistence. @@ -127,6 +128,7 @@ export function installSessionToolResultGuard( }; const allowSyntheticToolResults = opts?.allowSyntheticToolResults ?? true; + const missingToolResultText = opts?.missingToolResultText; const beforeWrite = opts?.beforeMessageWriteHook; const maxToolResultChars = resolveMaxToolResultChars(opts); @@ -154,7 +156,11 @@ export function installSessionToolResultGuard( } if (allowSyntheticToolResults) { for (const [id, name] of pendingState.entries()) { - const synthetic = makeMissingToolResult({ toolCallId: id, toolName: name }); + const synthetic = makeMissingToolResult({ + toolCallId: id, + toolName: name, + text: missingToolResultText, + }); const flushed = applyBeforeWriteHook( persistToolResult(persistMessage(synthetic), { toolCallId: id, diff --git a/src/agents/session-transcript-repair.test.ts b/src/agents/session-transcript-repair.test.ts index d95cef17564..ecab4cc2485 100644 --- a/src/agents/session-transcript-repair.test.ts +++ b/src/agents/session-transcript-repair.test.ts @@ -76,6 +76,68 @@ describe("sanitizeToolUseResultPairing", () => { expect(out[3]?.role).toBe("user"); }); + it("uses custom text for synthesized missing tool results", () => { + const input = castAgentMessages([ + { + role: "assistant", + content: [{ type: "toolCall", id: "call_1", name: "read", arguments: {} }], + }, + { role: "user", content: "user message that should come after tool use" }, + ]); + + const result = repairToolUseResultPairing(input, { + missingToolResultText: "aborted", + }); + + expect(result.added).toHaveLength(1); + expect(result.messages.map((m) => m.role)).toEqual(["assistant", "toolResult", "user"]); + expect(result.added[0]?.content).toEqual([{ type: "text", text: "aborted" }]); + }); + + it("keeps matched parallel tool results and synthesizes only missing siblings", () => { + const input = castAgentMessages([ + { + role: "assistant", + content: [ + { type: "text", text: "checking" }, + { type: "toolCall", id: "call_1", name: "read", arguments: {} }, + { type: "toolCall", id: "call_2", name: "exec", arguments: {} }, + { type: "toolCall", id: "call_3", name: "write", arguments: {} }, + ], + }, + { role: "user", content: "user message that should come after tool use" }, + { + role: "toolResult", + toolCallId: "call_2", + toolName: "exec", + content: [{ type: "text", text: "ok" }], + isError: false, + }, + ]); + + const result = repairToolUseResultPairing(input, { + missingToolResultText: "aborted", + }); + + expect(result.added.map((message) => message.toolCallId)).toEqual(["call_1", "call_3"]); + expect(result.messages.map((m) => m.role)).toEqual([ + "assistant", + "toolResult", + "toolResult", + "toolResult", + "user", + ]); + expect(getAssistantToolCallBlocks(result.messages)).toMatchObject([ + { id: "call_1", name: "read" }, + { id: "call_2", name: "exec" }, + { id: "call_3", name: "write" }, + ]); + expect((result.messages[1] as { toolCallId?: string }).toolCallId).toBe("call_1"); + expect((result.messages[2] as { toolCallId?: string }).toolCallId).toBe("call_2"); + expect((result.messages[3] as { toolCallId?: string }).toolCallId).toBe("call_3"); + expect(JSON.stringify(result.added)).not.toContain("missing tool result"); + }); + it("repairs blank tool result names from matching tool calls", () => { const input = castAgentMessages([ { @@ -248,9 +310,8 @@ describe("sanitizeToolUseResultPairing", () => { }); expect(result.droppedOrphanCount).toBe(0); - expect(result.messages).toHaveLength(2); - expect(result.messages[0]?.role).toBe("assistant"); - expect(result.messages[1]?.role).toBe("user"); + expect(result.messages).toHaveLength(1); + expect(result.messages[0]?.role).toBe("user"); expect(result.added).toHaveLength(0); }); }); diff --git a/src/agents/session-transcript-repair.ts b/src/agents/session-transcript-repair.ts index 2260a29b1f0..5e651603e80 100644 --- a/src/agents/session-transcript-repair.ts +++ b/src/agents/session-transcript-repair.ts @@ -175,6 +175,12 @@ function isReplaySafeThinkingAssistantTurn( function makeMissingToolResult(params: { toolCallId: string; toolName?: string; + // OpenAI Responses/Codex replay should match upstream Codex's "aborted" + // function_call_output normalization; live coverage in + // openai-reasoning-compat.live.test.ts and tool-replay-repair.live.test.ts + // sends this repaired history to real models. Other providers keep the older, + // explicit OpenClaw diagnostic text unless the caller opts in. + text?: string; }): Extract { return { role: "toolResult", @@ -183,7 +189,9 @@ function makeMissingToolResult(params: { content: [ { type: "text", - text: "[openclaw] missing tool result in session history; inserted synthetic error result for transcript repair.", + text: + params.text ?? + "[openclaw] missing tool result in session history; inserted synthetic error result for transcript repair.", }, ], isError: true, @@ -232,6 +240,7 @@ export type ErroredAssistantResultPolicy = "preserve" | "drop"; export type ToolUseResultPairingOptions = { erroredAssistantResultPolicy?: ErroredAssistantResultPolicy; + missingToolResultText?: string; }; export function stripToolResultDetails(messages: AgentMessage[]): AgentMessage[] { @@ -529,8 +538,8 @@ export function repairToolUseResultPairing( // tool calls in the same turn after malformed siblings are dropped. const stopReason = (assistant as { stopReason?: string }).stopReason; if (stopReason === "error" || stopReason === "aborted") { - out.push(msg); if (!shouldDropErroredAssistantResults(options)) { + out.push(msg); for (const toolCall of toolCalls) { const result = spanResultsById.get(toolCall.id); if (!result) { @@ -540,6 +549,8 @@ export function repairToolUseResultPairing( } } else if (spanResultsById.size > 0) { changed = true; + } else { + changed = true; } for (const rem of remainder) { out.push(rem); @@ -551,6 +562,8 @@ export function repairToolUseResultPairing( out.push(msg); if (spanResultsById.size > 0 && remainder.length > 0) { + // Preserve real late-arriving results before synthesizing missing siblings; + // otherwise parallel tool replay can replace useful output with repair noise. moved = true; changed = true; } @@ -563,6 +576,7 @@ export function repairToolUseResultPairing( const missing = makeMissingToolResult({ toolCallId: call.id, toolName: call.name, + text: options?.missingToolResultText, }); added.push(missing); changed = true; diff --git a/src/agents/subagent-list.test.ts b/src/agents/subagent-list.test.ts index 6a2816540d2..111bd826502 100644 --- a/src/agents/subagent-list.test.ts +++ b/src/agents/subagent-list.test.ts @@ -115,9 +115,52 @@ describe("buildSubagentList", () => { }); expect(list.active[0]?.status).toBe("active (waiting on 1 child)"); + expect(list.active[0]?.childSessions).toEqual([ + "agent:main:subagent:orchestrator-ended:subagent:child", + ]); expect(list.recent).toEqual([]); }); + it("omits old ended descendants from child session summaries", () => { + const now = Date.now(); + const parentRun = { + runId: "run-parent-active-old-child", + childSessionKey: "agent:main:subagent:parent-active-old-child", + requesterSessionKey: "agent:main:main", + requesterDisplayKey: "main", + task: "parent active", + cleanup: "keep", + createdAt: now - 120_000, + startedAt: now - 120_000, + } satisfies SubagentRunRecord; + addSubagentRunForTests(parentRun); + addSubagentRunForTests({ + runId: "run-old-ended-child-summary", + childSessionKey: `${parentRun.childSessionKey}:subagent:old-ended-child`, + requesterSessionKey: parentRun.childSessionKey, + requesterDisplayKey: "subagent:parent-active-old-child", + task: "old ended child", + cleanup: "keep", + createdAt: now - 60 * 60_000, + startedAt: now - 59 * 60_000, + endedAt: now - 31 * 60_000, + outcome: { status: "ok" }, + }); + const cfg = { + commands: { text: true }, + channels: { whatsapp: { allowFrom: ["*"] } }, + } as OpenClawConfig; + + const list = buildSubagentList({ + cfg, + runs: [parentRun], + recentMinutes: 30, + taskMaxChars: 110, + }); + + expect(list.active[0]?.childSessions).toBeUndefined(); + }); + it("formats io and prompt/cache usage from session entries", async () => { const run = { runId: "run-usage", diff --git a/src/agents/subagent-list.ts b/src/agents/subagent-list.ts index 06dae768ff9..81112948661 100644 --- a/src/agents/subagent-list.ts +++ b/src/agents/subagent-list.ts @@ -13,14 +13,21 @@ import { } from "../shared/subagents-format.js"; import { resolveModelDisplayName, resolveModelDisplayRef } from "./model-selection-display.js"; import { subagentRuns } from "./subagent-registry-memory.js"; -import { countPendingDescendantRunsFromRuns } from "./subagent-registry-queries.js"; +import { + countActiveDescendantRunsFromRuns, + countPendingDescendantRunsFromRuns, +} from "./subagent-registry-queries.js"; import { getSubagentSessionRuntimeMs, getSubagentSessionStartedAt, } from "./subagent-registry-read.js"; import { getSubagentRunsSnapshotForRead } from "./subagent-registry-state.js"; import type { SubagentRunRecord } from "./subagent-registry.types.js"; -import { hasSubagentRunEnded, isLiveUnendedSubagentRun } from "./subagent-run-liveness.js"; +import { + hasSubagentRunEnded, + isLiveUnendedSubagentRun, + shouldKeepSubagentRunChildLink, +} from "./subagent-run-liveness.js"; export type SubagentListItem = { index: number; @@ -80,7 +87,11 @@ export function resolveSessionEntryForKey(params: { }; } -export function buildLatestSubagentRunIndex(runs: Map) { +export function buildLatestSubagentRunIndex( + runs: Map, + options?: { now?: number }, +) { + const now = options?.now ?? Date.now(); const latestByChildSessionKey = new Map(); for (const entry of runs.values()) { const childSessionKey = entry.childSessionKey?.trim(); @@ -100,6 +111,14 @@ export function buildLatestSubagentRunIndex(runs: Map if (!controllerSessionKey) { continue; } + if ( + !shouldKeepSubagentRunChildLink(entry, { + activeDescendants: countActiveDescendantRunsFromRuns(runs, childSessionKey), + now, + }) + ) { + continue; + } const existing = childSessionsByController.get(controllerSessionKey); if (existing) { existing.push(childSessionKey); diff --git a/src/agents/subagent-registry.persistence.test.ts b/src/agents/subagent-registry.persistence.test.ts index eabd2493c8e..cbe97c164ca 100644 --- a/src/agents/subagent-registry.persistence.test.ts +++ b/src/agents/subagent-registry.persistence.test.ts @@ -166,7 +166,7 @@ describe("subagent registry persistence", () => { const waitForRegistryWork = async (predicate: () => boolean | Promise) => { await vi.waitFor(async () => expect(await predicate()).toBe(true), { interval: 1, - timeout: 1_000, + timeout: 5_000, }); }; diff --git a/src/agents/subagent-run-liveness.test.ts b/src/agents/subagent-run-liveness.test.ts index 78376d14584..4bed1d2aa5e 100644 --- a/src/agents/subagent-run-liveness.test.ts +++ b/src/agents/subagent-run-liveness.test.ts @@ -1,8 +1,10 @@ import { describe, expect, it, vi } from "vitest"; import { isLiveUnendedSubagentRun, + RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS, isStaleUnendedSubagentRun, STALE_UNENDED_SUBAGENT_RUN_MS, + shouldKeepSubagentRunChildLink, } from "./subagent-run-liveness.js"; describe("subagent run liveness", () => { @@ -72,4 +74,43 @@ describe("subagent run liveness", () => { vi.useRealTimers(); } }); + + it("keeps child links only while live, recently ended, or waiting on descendants", () => { + expect(shouldKeepSubagentRunChildLink({ createdAt: now - 60_000 }, { now })).toBe(true); + expect( + shouldKeepSubagentRunChildLink( + { + createdAt: now - RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS - 60_000, + endedAt: now - RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS + 1, + }, + { now }, + ), + ).toBe(true); + expect( + shouldKeepSubagentRunChildLink( + { + createdAt: now - RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS - 60_000, + endedAt: now - RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS - 1, + }, + { now }, + ), + ).toBe(false); + expect( + shouldKeepSubagentRunChildLink( + { + createdAt: now - RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS - 60_000, + endedAt: now - RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS - 1, + }, + { activeDescendants: 1, now }, + ), + ).toBe(true); + expect( + shouldKeepSubagentRunChildLink( + { + createdAt: now - STALE_UNENDED_SUBAGENT_RUN_MS - 1, + }, + { now }, + ), + ).toBe(false); + }); }); diff --git a/src/agents/subagent-run-liveness.ts b/src/agents/subagent-run-liveness.ts index 70a1de5fb41..2243472eb98 100644 --- a/src/agents/subagent-run-liveness.ts +++ b/src/agents/subagent-run-liveness.ts @@ -2,10 +2,13 @@ import type { SubagentRunRecord } from "./subagent-registry.types.js"; import { getSubagentSessionStartedAt } from "./subagent-session-metrics.js"; export const STALE_UNENDED_SUBAGENT_RUN_MS = 2 * 60 * 60 * 1_000; +export const RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS = 30 * 60 * 1_000; const EXPLICIT_TIMEOUT_STALE_GRACE_MS = 60_000; const MIN_REALISTIC_RUN_TIMESTAMP_MS = Date.UTC(2020, 0, 1); -export function hasSubagentRunEnded(entry: Pick): boolean { +export function hasSubagentRunEnded>( + entry: T, +): entry is T & { endedAt: number } { return typeof entry.endedAt === "number" && Number.isFinite(entry.endedAt); } @@ -50,3 +53,32 @@ export function isLiveUnendedSubagentRun( ): boolean { return !hasSubagentRunEnded(entry) && !isStaleUnendedSubagentRun(entry, now); } + +export function isRecentlyEndedSubagentRun( + entry: Pick, + now = Date.now(), + recentMs = RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS, +): boolean { + if (!hasSubagentRunEnded(entry)) { + return false; + } + return now - entry.endedAt <= recentMs; +} + +export function shouldKeepSubagentRunChildLink( + entry: Pick< + SubagentRunRecord, + "createdAt" | "startedAt" | "sessionStartedAt" | "endedAt" | "runTimeoutSeconds" + >, + options?: { + activeDescendants?: number; + now?: number; + }, +): boolean { + const now = options?.now ?? Date.now(); + return ( + isLiveUnendedSubagentRun(entry, now) || + (options?.activeDescendants ?? 0) > 0 || + isRecentlyEndedSubagentRun(entry, now) + ); +} diff --git a/src/agents/tool-replay-repair.live.test.ts b/src/agents/tool-replay-repair.live.test.ts new file mode 100644 index 00000000000..ebde652c777 --- /dev/null +++ b/src/agents/tool-replay-repair.live.test.ts @@ -0,0 +1,386 @@ +import type { AgentMessage } from "@mariozechner/pi-agent-core"; +import { completeSimple, type Api, type Context, type Model } from "@mariozechner/pi-ai"; +import { SessionManager } from "@mariozechner/pi-coding-agent"; +import { Type } from "typebox"; +import { describe, expect, it } from "vitest"; +import { loadConfig } from "../config/config.js"; +import { resolveOpenClawAgentDir } from "./agent-paths.js"; +import { isLiveProfileKeyModeEnabled, isLiveTestEnabled } from "./live-test-helpers.js"; +import { getApiKeyForModel, requireApiKey } from "./model-auth.js"; +import { ensureOpenClawModelsJson } from "./models-config.js"; +import { sanitizeSessionHistory } from "./pi-embedded-runner/replay-history.js"; +import { discoverAuthStorage, discoverModels } from "./pi-model-discovery.js"; +import { transformTransportMessages } from "./transport-message-transform.js"; + +const LIVE = isLiveTestEnabled(); +const REQUIRE_PROFILE_KEYS = isLiveProfileKeyModeEnabled(); +const LIVE_CREDENTIAL_PRECEDENCE = REQUIRE_PROFILE_KEYS ? "profile-first" : "env-first"; +const DEFAULT_TARGET_MODEL_REFS = "openai-codex/gpt-5.5,google/gemini-3-flash-preview"; +const TARGET_MODEL_REFS = parseTargetModelRefs( + process.env.OPENCLAW_LIVE_TOOL_REPLAY_REPAIR_MODELS ?? DEFAULT_TARGET_MODEL_REFS, +); +const describeLive = LIVE ? describe : describe.skip; + +type TargetModelRef = { + ref: string; + provider: string; + modelId: string; +}; + +function parseTargetModelRefs(raw: string | undefined): TargetModelRef[] { + return (raw ?? "") + .split(",") + .map((item) => item.trim()) + .filter(Boolean) + .map((ref) => { + const [provider, ...rest] = ref.split("/"); + const modelId = rest.join("/").trim(); + if (!provider?.trim() || !modelId) { + throw new Error( + `Invalid OPENCLAW_LIVE_TOOL_REPLAY_REPAIR_MODELS entry: ${JSON.stringify(ref)}`, + ); + } + return { ref, provider: provider.trim(), modelId }; + }); +} + +function logProgress(message: string): void { + process.stderr.write(`[live] ${message}\n`); +} + +async function completeSimpleWithTimeout( + model: Model, + context: Parameters>[1], + options: Parameters>[2], + timeoutMs: number, +): Promise>>> { + const controller = new AbortController(); + const abortTimer = setTimeout(() => { + controller.abort(); + }, timeoutMs); + abortTimer.unref?.(); + try { + return await Promise.race([ + completeSimple(model, context, { + ...options, + signal: controller.signal, + }), + new Promise((_, reject) => { + const hardTimer = setTimeout(() => { + reject(new Error(`model call timed out after ${timeoutMs}ms`)); + }, timeoutMs); + hardTimer.unref?.(); + }), + ]); + } finally { + clearTimeout(abortTimer); + } +} + +function isOpenAIResponsesFamily(api: string): boolean { + return ( + api === "openai-responses" || + api === "openai-codex-responses" || + api === "azure-openai-responses" + ); +} + +function buildReplayMessages(model: Model): AgentMessage[] { + const now = Date.now(); + // Gemini source metadata deliberately simulates a model switch from a + // provider-owned transcript. That forces the same id sanitization and replay + // repair path that failed in real session replays, not just the happy path for + // a same-provider synthetic fixture. + const source = + model.provider === "google" + ? { + api: "google-gemini-cli", + provider: "google-antigravity", + model: "claude-sonnet-4-20250514", + } + : { + api: model.api, + provider: model.provider, + model: model.id, + }; + + return [ + { + role: "user", + content: "Use noop.", + timestamp: now, + }, + { + role: "assistant", + provider: source.provider, + api: source.api, + model: source.model, + stopReason: "toolUse", + timestamp: now + 1, + content: [ + { type: "toolCall", id: "call_keep", name: "noop", arguments: {} }, + { type: "toolCall", id: "call_missing_a", name: "noop", arguments: {} }, + { type: "toolCall", id: "call_missing_b", name: "noop", arguments: {} }, + ], + }, + { + role: "user", + content: "Reply with exactly: replay repair ok.", + timestamp: now + 2, + }, + { + role: "toolResult", + toolCallId: "call_keep", + toolName: "noop", + content: [{ type: "text", text: "ok" }], + isError: false, + timestamp: now + 3, + }, + ] as unknown as AgentMessage[]; +} + +function buildAbortedTransportMessages(model: Model): Context["messages"] { + const now = Date.now(); + return [ + { + role: "assistant", + provider: model.provider, + api: model.api, + model: model.id, + stopReason: "aborted", + timestamp: now, + content: [{ type: "toolCall", id: "call_transport_aborted", name: "noop", arguments: {} }], + }, + { + role: "user", + content: "Reply with exactly: transport replay ok.", + timestamp: now + 1, + }, + ] as Context["messages"]; +} + +function syntheticToolResultText(message: AgentMessage): string | undefined { + if (message.role !== "toolResult") { + return undefined; + } + const first = message.content[0] as { type?: unknown; text?: unknown } | undefined; + return first?.type === "text" && typeof first.text === "string" ? first.text : undefined; +} + +function assistantToolCallIds(message: AgentMessage): string[] { + if (message.role !== "assistant") { + return []; + } + return message.content.filter((block) => block.type === "toolCall").map((block) => block.id); +} + +function isKnownLiveBlocker(errorMessage: string): boolean { + return ( + /not supported when using codex with a chatgpt account/i.test(errorMessage) || + /hit your chatgpt usage limit/i.test(errorMessage) + ); +} + +describeLive("tool replay repair live", () => { + for (const target of TARGET_MODEL_REFS) { + it( + `accepts repaired displaced and missing tool results with ${target.ref}`, + async () => { + const cfg = loadConfig(); + await ensureOpenClawModelsJson(cfg); + + const agentDir = resolveOpenClawAgentDir(); + const authStorage = discoverAuthStorage(agentDir); + const modelRegistry = discoverModels(authStorage, agentDir); + const model = modelRegistry.find(target.provider, target.modelId) as Model | null; + + if (!model) { + logProgress(`[tool-replay-repair] model missing from registry: ${target.ref}`); + return; + } + + let apiKeyInfo; + try { + apiKeyInfo = await getApiKeyForModel({ + model, + cfg, + credentialPrecedence: LIVE_CREDENTIAL_PRECEDENCE, + }); + } catch (error) { + logProgress(`[tool-replay-repair] skip ${target.ref} (${String(error)})`); + return; + } + + if (REQUIRE_PROFILE_KEYS && !apiKeyInfo.source.startsWith("profile:")) { + logProgress( + `[tool-replay-repair] skip ${target.ref} (non-profile credential source: ${apiKeyInfo.source})`, + ); + return; + } + + logProgress(`[tool-replay-repair] target=${target.ref} auth source=${apiKeyInfo.source}`); + const sanitized = await sanitizeSessionHistory({ + messages: buildReplayMessages(model), + modelApi: model.api, + provider: model.provider, + modelId: model.id, + sessionManager: SessionManager.inMemory(), + sessionId: `tool-replay-repair-live-${target.provider}-${target.modelId}`, + }); + + expect(sanitized.map((message) => message.role)).toEqual([ + "user", + "assistant", + "toolResult", + "toolResult", + "toolResult", + "user", + ]); + const assistantMessage = sanitized[1]; + expect(assistantMessage?.role).toBe("assistant"); + expect( + sanitized.slice(2, 5).map((message) => (message as { toolCallId?: string }).toolCallId), + ).toEqual(assistantToolCallIds(assistantMessage)); + + // These assertions are the model-visible contract: OpenAI Responses + // gets Codex-compatible "aborted" outputs, while Gemini proves the + // generic repair does not leak OpenAI wording into other providers. + const insertedTexts = sanitized.slice(3, 5).map(syntheticToolResultText); + if (isOpenAIResponsesFamily(model.api)) { + expect(insertedTexts).toEqual(["aborted", "aborted"]); + } else { + expect(insertedTexts).not.toContain("aborted"); + } + + // Sending the repaired transcript to the real model is the live proof: + // providers reject malformed tool-call adjacency before generation, so + // any non-error response here validates the repair shape end to end. + const response = await completeSimpleWithTimeout( + model, + { + systemPrompt: "You are a concise assistant. Follow the user's instruction exactly.", + messages: sanitized as never, + tools: [ + { + name: "noop", + description: "Return ok.", + parameters: Type.Object({}, { additionalProperties: false }), + }, + ], + }, + { + apiKey: requireApiKey(apiKeyInfo, model.provider), + reasoning: "low", + maxTokens: 96, + }, + 120_000, + ); + + const text = response.content + .filter((block) => block.type === "text") + .map((block) => block.text.trim()) + .join(" ") + .trim(); + const errorMessage = + typeof (response as { errorMessage?: unknown }).errorMessage === "string" + ? ((response as { errorMessage?: string }).errorMessage ?? "") + : ""; + if (errorMessage && isKnownLiveBlocker(errorMessage)) { + logProgress(`[tool-replay-repair] skip ${target.ref} (${errorMessage})`); + return; + } + + expect(response.stopReason).not.toBe("error"); + if (text.length > 0) { + expect(text).toMatch(/^replay repair ok\.?$/i); + } + }, + 3 * 60 * 1000, + ); + + it( + `accepts transport replay after dropping aborted assistant tool calls with ${target.ref}`, + async () => { + const cfg = loadConfig(); + await ensureOpenClawModelsJson(cfg); + + const agentDir = resolveOpenClawAgentDir(); + const authStorage = discoverAuthStorage(agentDir); + const modelRegistry = discoverModels(authStorage, agentDir); + const model = modelRegistry.find(target.provider, target.modelId) as Model | null; + + if (!model) { + logProgress(`[tool-replay-repair] model missing from registry: ${target.ref}`); + return; + } + + let apiKeyInfo; + try { + apiKeyInfo = await getApiKeyForModel({ + model, + cfg, + credentialPrecedence: LIVE_CREDENTIAL_PRECEDENCE, + }); + } catch (error) { + logProgress(`[tool-replay-repair] skip ${target.ref} (${String(error)})`); + return; + } + + if (REQUIRE_PROFILE_KEYS && !apiKeyInfo.source.startsWith("profile:")) { + logProgress( + `[tool-replay-repair] skip ${target.ref} (non-profile credential source: ${apiKeyInfo.source})`, + ); + return; + } + + const transformed = transformTransportMessages(buildAbortedTransportMessages(model), model); + expect(transformed.map((message) => message.role)).toEqual(["user"]); + expect(JSON.stringify(transformed)).not.toContain("call_transport_aborted"); + + // This is the transport replay regression proof: providers reject + // assistant(tool_call)->user replays without a matching result, so the + // dropped transcript must still be accepted by real model APIs. + const response = await completeSimpleWithTimeout( + model, + { + systemPrompt: "You are a concise assistant. Follow the user's instruction exactly.", + messages: transformed as never, + tools: [ + { + name: "noop", + description: "Return ok.", + parameters: Type.Object({}, { additionalProperties: false }), + }, + ], + }, + { + apiKey: requireApiKey(apiKeyInfo, model.provider), + reasoning: "low", + maxTokens: 96, + }, + 120_000, + ); + + const text = response.content + .filter((block) => block.type === "text") + .map((block) => block.text.trim()) + .join(" ") + .trim(); + const errorMessage = + typeof (response as { errorMessage?: unknown }).errorMessage === "string" + ? ((response as { errorMessage?: string }).errorMessage ?? "") + : ""; + if (errorMessage && isKnownLiveBlocker(errorMessage)) { + logProgress(`[tool-replay-repair] skip ${target.ref} (${errorMessage})`); + return; + } + + expect(response.stopReason).not.toBe("error"); + if (text.length > 0) { + expect(text).toMatch(/^transport replay ok\.?$/i); + } + }, + 3 * 60 * 1000, + ); + } +}); diff --git a/src/agents/transport-message-transform.test.ts b/src/agents/transport-message-transform.test.ts index f2f5b66ce5b..f185ed7cc95 100644 --- a/src/agents/transport-message-transform.test.ts +++ b/src/agents/transport-message-transform.test.ts @@ -9,20 +9,21 @@ function makeModel(api: Api, provider: string, id: string): Model { function assistantToolCall( id: string, name = "read", + stopReason: Extract["stopReason"] = "toolUse", ): Extract { return { role: "assistant", provider: "openai", api: "openai-responses", model: "gpt-5.4", - stopReason: "toolUse", + stopReason, timestamp: Date.now(), content: [{ type: "toolCall", id, name, arguments: {} }], } as Extract; } describe("transformTransportMessages synthetic tool-result policy", () => { - it("does not synthesize missing tool results for OpenAI-compatible transports", () => { + it("synthesizes Codex-style aborted tool results for OpenAI Responses transports", () => { const messages: Context["messages"] = [ assistantToolCall("call_openai_1"), { role: "user", content: "continue", timestamp: Date.now() }, @@ -33,7 +34,166 @@ describe("transformTransportMessages synthetic tool-result policy", () => { makeModel("openai-responses", "openai", "gpt-5.4"), ); - expect(result.map((msg) => msg.role)).toEqual(["assistant", "user"]); + expect(result.map((msg) => msg.role)).toEqual(["assistant", "toolResult", "user"]); + expect(result[1]).toMatchObject({ + role: "toolResult", + toolCallId: "call_openai_1", + isError: true, + content: [{ type: "text", text: "aborted" }], + }); + }); + + it("preserves real OpenAI transport results and aborts missing parallel siblings", () => { + const messages: Context["messages"] = [ + { + ...assistantToolCall("call_keep"), + content: [ + { type: "toolCall", id: "call_keep", name: "read", arguments: {} }, + { type: "toolCall", id: "call_missing", name: "exec", arguments: {} }, + ], + }, + { + role: "toolResult", + toolCallId: "call_keep", + toolName: "read", + content: [{ type: "text", text: "ok" }], + isError: false, + timestamp: Date.now(), + }, + { role: "user", content: "continue", timestamp: Date.now() }, + ]; + + const result = transformTransportMessages( + messages, + makeModel("openclaw-openai-responses-transport" as Api, "openai", "gpt-5.4"), + ); + + expect(result.map((msg) => msg.role)).toEqual([ + "assistant", + "toolResult", + "toolResult", + "user", + ]); + expect(result.slice(1, 3)).toMatchObject([ + { role: "toolResult", toolCallId: "call_keep", content: [{ type: "text", text: "ok" }] }, + { + role: "toolResult", + toolCallId: "call_missing", + content: [{ type: "text", text: "aborted" }], + }, + ]); + }); + + it("moves displaced OpenAI transport results before synthesizing missing siblings", () => { + const messages: Context["messages"] = [ + { + ...assistantToolCall("call_keep"), + content: [ + { type: "toolCall", id: "call_keep", name: "read", arguments: {} }, + { type: "toolCall", id: "call_missing", name: "exec", arguments: {} }, + ], + }, + { role: "user", content: "continue", timestamp: Date.now() }, + { + role: "toolResult", + toolCallId: "call_keep", + toolName: "read", + content: [{ type: "text", text: "late ok" }], + isError: false, + timestamp: Date.now(), + }, + ]; + + const result = transformTransportMessages( + messages, + makeModel("openai-responses", "openai", "gpt-5.4"), + ); + + expect(result.map((msg) => msg.role)).toEqual([ + "assistant", + "toolResult", + "toolResult", + "user", + ]); + expect(result.slice(1, 3)).toMatchObject([ + { role: "toolResult", toolCallId: "call_keep", content: [{ type: "text", text: "late ok" }] }, + { + role: "toolResult", + toolCallId: "call_missing", + content: [{ type: "text", text: "aborted" }], + }, + ]); + }); + + it("drops aborted OpenAI transport assistant tool calls before replay", () => { + const messages: Context["messages"] = [ + assistantToolCall("call_aborted", "exec", "aborted"), + { role: "user", content: "retry after abort", timestamp: Date.now() }, + ]; + + const result = transformTransportMessages( + messages, + makeModel("openai-responses", "openai", "gpt-5.4"), + ); + + expect(result.map((msg) => msg.role)).toEqual(["user"]); + expect(JSON.stringify(result)).not.toContain("call_aborted"); + }); + + it("drops text-only aborted and errored transport assistant turns before replay", () => { + const messages: Context["messages"] = [ + { + role: "assistant", + provider: "openai", + api: "openai-responses", + model: "gpt-5.4", + stopReason: "aborted", + timestamp: Date.now(), + content: [{ type: "text", text: "partial aborted output" }], + } as Extract, + { + role: "assistant", + provider: "openai", + api: "openai-responses", + model: "gpt-5.4", + stopReason: "error", + timestamp: Date.now(), + content: [{ type: "text", text: "partial error output" }], + } as Extract, + { role: "user", content: "retry after failed text turns", timestamp: Date.now() }, + ]; + + const result = transformTransportMessages( + messages, + makeModel("openai-responses", "openai", "gpt-5.4"), + ); + + expect(result.map((msg) => msg.role)).toEqual(["user"]); + expect(JSON.stringify(result)).not.toContain("partial aborted output"); + expect(JSON.stringify(result)).not.toContain("partial error output"); + }); + + it("drops errored Anthropic transport assistant tool calls and matching results before replay", () => { + const messages: Context["messages"] = [ + assistantToolCall("call_error", "exec", "error"), + { + role: "toolResult", + toolCallId: "call_error", + toolName: "exec", + content: [{ type: "text", text: "partial" }], + isError: true, + timestamp: Date.now(), + }, + { role: "user", content: "retry after error", timestamp: Date.now() }, + ]; + + const result = transformTransportMessages( + messages, + makeModel("anthropic-messages", "anthropic", "claude-opus-4-6"), + ); + + expect(result.map((msg) => msg.role)).toEqual(["user"]); + expect(JSON.stringify(result)).not.toContain("call_error"); }); it("still synthesizes missing tool results for Anthropic transports", () => { @@ -72,6 +232,10 @@ describe("transformTransportMessages synthetic tool-result policy", () => { makeModel("openclaw-google-generative-ai-transport" as Api, "google", "gemini-2.5-pro"), ); expect(googleAlias.map((msg) => msg.role)).toEqual(["assistant", "toolResult", "user"]); + expect(googleAlias[1]).toMatchObject({ + role: "toolResult", + content: [{ type: "text", text: "No result provided" }], + }); const bedrockCanonical = transformTransportMessages( messages, diff --git a/src/agents/transport-message-transform.ts b/src/agents/transport-message-transform.ts index 2262d014f3b..20d47f7bc70 100644 --- a/src/agents/transport-message-transform.ts +++ b/src/agents/transport-message-transform.ts @@ -1,4 +1,5 @@ import type { Api, Context, Model } from "@mariozechner/pi-ai"; +import { repairToolUseResultPairing } from "./session-transcript-repair.js"; const SYNTHETIC_TOOL_RESULT_APIS = new Set([ "anthropic-messages", @@ -6,31 +7,34 @@ const SYNTHETIC_TOOL_RESULT_APIS = new Set([ "bedrock-converse-stream", "google-generative-ai", "openclaw-google-generative-ai-transport", + "openai-responses", + "openai-codex-responses", + "azure-openai-responses", + "openclaw-openai-responses-transport", + "openclaw-azure-openai-responses-transport", ]); -type PendingToolCall = { id: string; name: string }; +// "aborted" is an OpenAI Responses-family convention from upstream Codex +// history normalization. Gemini/Anthropic transports use their own text while +// still needing synthetic results to satisfy provider turn-shape contracts; +// tool-replay-repair.live.test.ts exercises both paths against real models. +const CODEX_STYLE_ABORTED_OUTPUT_APIS = new Set([ + "openai-responses", + "openai-codex-responses", + "azure-openai-responses", + "openclaw-openai-responses-transport", + "openclaw-azure-openai-responses-transport", +]); function defaultAllowSyntheticToolResults(modelApi: Api): boolean { return SYNTHETIC_TOOL_RESULT_APIS.has(modelApi); } -function appendMissingToolResults( - result: Context["messages"], - pendingToolCalls: PendingToolCall[], - existingToolResultIds: ReadonlySet, -): void { - for (const toolCall of pendingToolCalls) { - if (!existingToolResultIds.has(toolCall.id)) { - result.push({ - role: "toolResult", - toolCallId: toolCall.id, - toolName: toolCall.name, - content: [{ type: "text", text: "No result provided" }], - isError: true, - timestamp: Date.now(), - }); - } +function isFailedAssistantTurn(message: Context["messages"][number]): boolean { + if (message.role !== "assistant") { + return false; } + return message.stopReason === "error" || message.stopReason === "aborted"; } export function transformTransportMessages( @@ -43,6 +47,9 @@ export function transformTransportMessages( ) => string, ): Context["messages"] { const allowSyntheticToolResults = defaultAllowSyntheticToolResults(model.api); + const syntheticToolResultText = CODEX_STYLE_ABORTED_OUTPUT_APIS.has(model.api) + ? "aborted" + : "No result provided"; const toolCallIdMap = new Map(); const transformed = messages.map((msg) => { if (msg.role === "user") { @@ -102,42 +109,21 @@ export function transformTransportMessages( } return { ...msg, content }; }); + // Preserve the old transport replay filter: failed streamed turns can contain + // partial text, partial tool calls, or both, and strict providers can treat + // them as valid assistant context on retry unless we drop the whole turn. + const replayable = transformed.filter((msg) => !isFailedAssistantTurn(msg)); - const result: Context["messages"] = []; - let pendingToolCalls: PendingToolCall[] = []; - let existingToolResultIds = new Set(); - for (const msg of transformed) { - if (msg.role === "assistant") { - if (allowSyntheticToolResults && pendingToolCalls.length > 0) { - appendMissingToolResults(result, pendingToolCalls, existingToolResultIds); - } - pendingToolCalls = []; - existingToolResultIds = new Set(); - if (msg.stopReason === "error" || msg.stopReason === "aborted") { - continue; - } - const toolCalls = msg.content.filter( - (block): block is Extract<(typeof msg.content)[number], { type: "toolCall" }> => - block.type === "toolCall", - ); - if (toolCalls.length > 0) { - pendingToolCalls = toolCalls.map((block) => ({ id: block.id, name: block.name })); - existingToolResultIds = new Set(); - } - result.push(msg); - continue; - } - if (msg.role === "toolResult") { - existingToolResultIds.add(msg.toolCallId); - result.push(msg); - continue; - } - if (allowSyntheticToolResults && pendingToolCalls.length > 0) { - appendMissingToolResults(result, pendingToolCalls, existingToolResultIds); - } - pendingToolCalls = []; - existingToolResultIds = new Set(); - result.push(msg); + if (!allowSyntheticToolResults) { + return replayable; } - return result; + + // PI's local transform can synthesize missing results, but it does not move + // displaced real results back before an intervening user turn. Shared repair + // handles both, while preserving the previous transport behavior of dropping + // aborted/error assistant tool-call turns before replaying strict providers. + return repairToolUseResultPairing(replayable, { + erroredAssistantResultPolicy: "drop", + missingToolResultText: syntheticToolResultText, + }).messages as Context["messages"]; } diff --git a/src/auto-reply/dispatch.test.ts b/src/auto-reply/dispatch.test.ts index ef1f5bb0d99..484f2a640a9 100644 --- a/src/auto-reply/dispatch.test.ts +++ b/src/auto-reply/dispatch.test.ts @@ -1,4 +1,4 @@ -import { describe, expect, it, vi } from "vitest"; +import { beforeEach, describe, expect, it, vi } from "vitest"; import type { OpenClawConfig } from "../config/config.js"; import type { ReplyDispatcher } from "./reply/reply-dispatcher.js"; import { buildTestCtx } from "./reply/test-ctx.js"; @@ -6,12 +6,19 @@ import { buildTestCtx } from "./reply/test-ctx.js"; type DispatchReplyFromConfigFn = typeof import("./reply/dispatch-from-config.js").dispatchReplyFromConfig; type FinalizeInboundContextFn = typeof import("./reply/inbound-context.js").finalizeInboundContext; +type DeriveInboundMessageHookContextFn = + typeof import("../hooks/message-hook-mappers.js").deriveInboundMessageHookContext; +type GetGlobalHookRunnerFn = typeof import("../plugins/hook-runner-global.js").getGlobalHookRunner; +type CreateReplyDispatcherFn = typeof import("./reply/reply-dispatcher.js").createReplyDispatcher; type CreateReplyDispatcherWithTypingFn = typeof import("./reply/reply-dispatcher.js").createReplyDispatcherWithTyping; const hoisted = vi.hoisted(() => ({ dispatchReplyFromConfigMock: vi.fn(), finalizeInboundContextMock: vi.fn((ctx: unknown, _opts?: unknown) => ctx), + deriveInboundMessageHookContextMock: vi.fn(), + getGlobalHookRunnerMock: vi.fn(), + createReplyDispatcherMock: vi.fn(), createReplyDispatcherWithTypingMock: vi.fn(), })); @@ -25,12 +32,33 @@ vi.mock("./reply/inbound-context.js", () => ({ hoisted.finalizeInboundContextMock(...args), })); +vi.mock("../hooks/message-hook-mappers.js", () => ({ + deriveInboundMessageHookContext: (...args: Parameters) => + hoisted.deriveInboundMessageHookContextMock(...args), + toPluginMessageContext: (canonical: { + channelId?: string; + accountId?: string; + conversationId?: string; + }) => ({ + channelId: canonical.channelId, + accountId: canonical.accountId, + conversationId: canonical.conversationId, + }), +})); + +vi.mock("../plugins/hook-runner-global.js", () => ({ + getGlobalHookRunner: (...args: Parameters) => + hoisted.getGlobalHookRunnerMock(...args), +})); + vi.mock("./reply/reply-dispatcher.js", async () => { const actual = await vi.importActual( "./reply/reply-dispatcher.js", ); return { ...actual, + createReplyDispatcher: (...args: Parameters) => + hoisted.createReplyDispatcherMock(...args), createReplyDispatcherWithTyping: (...args: Parameters) => hoisted.createReplyDispatcherWithTypingMock(...args), }; @@ -38,6 +66,7 @@ vi.mock("./reply/reply-dispatcher.js", async () => { const { dispatchInboundMessage, + dispatchInboundMessageWithDispatcher, dispatchInboundMessageWithBufferedDispatcher, withReplyDispatcher, } = await import("./dispatch.js"); @@ -59,6 +88,22 @@ function createDispatcher(record: string[]): ReplyDispatcher { } describe("withReplyDispatcher", () => { + beforeEach(() => { + vi.clearAllMocks(); + hoisted.finalizeInboundContextMock.mockImplementation((ctx: unknown) => ctx); + hoisted.deriveInboundMessageHookContextMock.mockReturnValue({ + channelId: "threads", + accountId: "acct-1", + conversationId: "conv-1", + isGroup: false, + to: "thread:1", + }); + hoisted.getGlobalHookRunnerMock.mockReturnValue({ + hasHooks: vi.fn(() => false), + runMessageSending: vi.fn(async () => undefined), + }); + }); + it("dispatchInboundMessage owns dispatcher lifecycle", async () => { const order: string[] = []; const dispatcher = { @@ -168,6 +213,76 @@ describe("withReplyDispatcher", () => { expect(typing.markDispatchIdle).toHaveBeenCalled(); }); + it("runs message_sending hooks before inbound dispatcher delivery", async () => { + const runMessageSending = vi.fn(async () => ({ content: "sanitized reply" })); + hoisted.getGlobalHookRunnerMock.mockReturnValue({ + hasHooks: vi.fn((hookName?: string) => hookName === "message_sending"), + runMessageSending, + }); + hoisted.createReplyDispatcherMock.mockReturnValueOnce(createDispatcher([])); + hoisted.dispatchReplyFromConfigMock.mockResolvedValueOnce({ text: "ok" }); + + await dispatchInboundMessageWithDispatcher({ + ctx: buildTestCtx({ + From: "whatsapp:+15551234567", + To: "whatsapp:+15557654321", + OriginatingTo: "whatsapp:+15551234567", + }), + cfg: {} as OpenClawConfig, + dispatcherOptions: { + deliver: async () => undefined, + }, + replyResolver: async () => ({ text: "ok" }), + }); + + const dispatcherOptions = hoisted.createReplyDispatcherMock.mock.calls[0]?.[0]; + expect(dispatcherOptions?.beforeDeliver).toEqual(expect.any(Function)); + + const payload = await dispatcherOptions.beforeDeliver( + { text: "original reply" }, + { kind: "final" }, + ); + + expect(payload).toEqual({ text: "sanitized reply" }); + expect(runMessageSending).toHaveBeenCalledWith( + { content: "original reply", to: "whatsapp:+15551234567" }, + { + channelId: "threads", + accountId: "acct-1", + conversationId: "conv-1", + }, + ); + }); + + it("reconciles queuedFinal and counts after dispatcher-side cancellation", async () => { + const dispatcher = { + sendToolResult: () => true, + sendBlockReply: () => true, + sendFinalReply: () => true, + getQueuedCounts: () => ({ tool: 0, block: 0, final: 0 }), + getCancelledCounts: () => ({ tool: 0, block: 0, final: 1 }), + getFailedCounts: () => ({ tool: 0, block: 0, final: 0 }), + markComplete: () => undefined, + waitForIdle: async () => undefined, + } satisfies ReplyDispatcher; + hoisted.dispatchReplyFromConfigMock.mockResolvedValueOnce({ + queuedFinal: true, + counts: { tool: 0, block: 0, final: 1 }, + }); + + const result = await dispatchInboundMessage({ + ctx: buildTestCtx(), + cfg: {} as OpenClawConfig, + dispatcher, + replyResolver: async () => ({ text: "ok" }), + }); + + expect(result).toEqual({ + queuedFinal: false, + counts: { tool: 0, block: 0, final: 0 }, + }); + }); + it("uses CommandTargetSessionKey for silent-reply policy on native command turns", async () => { hoisted.createReplyDispatcherWithTypingMock.mockReturnValueOnce({ dispatcher: createDispatcher([]), diff --git a/src/auto-reply/dispatch.ts b/src/auto-reply/dispatch.ts index b7bd73c0eb3..6b9a72e523e 100644 --- a/src/auto-reply/dispatch.ts +++ b/src/auto-reply/dispatch.ts @@ -1,5 +1,10 @@ import { normalizeChatType } from "../channels/chat-type.js"; import type { OpenClawConfig } from "../config/types.openclaw.js"; +import { + deriveInboundMessageHookContext, + toPluginMessageContext, +} from "../hooks/message-hook-mappers.js"; +import { getGlobalHookRunner } from "../plugins/hook-runner-global.js"; import type { SilentReplyConversationType } from "../shared/silent-reply-policy.js"; import { withReplyDispatcher } from "./dispatch-dispatcher.js"; import { dispatchReplyFromConfig } from "./reply/dispatch-from-config.js"; @@ -9,12 +14,13 @@ import { finalizeInboundContext } from "./reply/inbound-context.js"; import { createReplyDispatcher, createReplyDispatcherWithTyping, + type ReplyDispatchBeforeDeliver, type ReplyDispatcherOptions, type ReplyDispatcherWithTypingOptions, } from "./reply/reply-dispatcher.js"; import type { ReplyDispatcher } from "./reply/reply-dispatcher.types.js"; import type { FinalizedMsgContext, MsgContext } from "./templating.js"; -import type { GetReplyOptions } from "./types.js"; +import type { GetReplyOptions, ReplyPayload } from "./types.js"; function resolveDispatcherSilentReplyContext( ctx: MsgContext | FinalizedMsgContext, @@ -44,9 +50,74 @@ function resolveDispatcherSilentReplyContext( }; } +function resolveInboundReplyHookTarget( + finalized: FinalizedMsgContext, + hookCtx: ReturnType, +): string { + if (typeof finalized.OriginatingTo === "string" && finalized.OriginatingTo.trim()) { + return finalized.OriginatingTo; + } + if (hookCtx.isGroup) { + return hookCtx.conversationId ?? hookCtx.to ?? hookCtx.from; + } + return hookCtx.from || hookCtx.conversationId || hookCtx.to || ""; +} + +function buildMessageSendingBeforeDeliver( + ctx: MsgContext | FinalizedMsgContext, +): ReplyDispatchBeforeDeliver | undefined { + const hookRunner = getGlobalHookRunner(); + if (!hookRunner?.hasHooks("message_sending")) { + return undefined; + } + + const finalized = finalizeInboundContext(ctx); + const hookCtx = deriveInboundMessageHookContext(finalized); + const replyTarget = resolveInboundReplyHookTarget(finalized, hookCtx); + + return async (payload: ReplyPayload): Promise => { + if (!payload.text) { + return payload; + } + + const result = await hookRunner.runMessageSending( + { content: payload.text, to: replyTarget }, + toPluginMessageContext(hookCtx), + ); + + if (result?.cancel) { + return null; + } + if (result?.content != null) { + return { ...payload, text: result.content }; + } + return payload; + }; +} + export type DispatchInboundResult = DispatchFromConfigResult; export { withReplyDispatcher } from "./dispatch-dispatcher.js"; +function finalizeDispatchResult( + result: DispatchFromConfigResult, + dispatcher: ReplyDispatcher, +): DispatchFromConfigResult { + const cancelledCounts = dispatcher.getCancelledCounts?.(); + if (!cancelledCounts) { + return result; + } + + const counts = { + tool: Math.max(0, result.counts.tool - cancelledCounts.tool), + block: Math.max(0, result.counts.block - cancelledCounts.block), + final: Math.max(0, result.counts.final - cancelledCounts.final), + }; + return { + queuedFinal: result.queuedFinal && counts.final > 0, + counts, + }; +} + export async function dispatchInboundMessage(params: { ctx: MsgContext | FinalizedMsgContext; cfg: OpenClawConfig; @@ -55,7 +126,7 @@ export async function dispatchInboundMessage(params: { replyResolver?: GetReplyFromConfig; }): Promise { const finalized = finalizeInboundContext(params.ctx); - return await withReplyDispatcher({ + const result = await withReplyDispatcher({ dispatcher: params.dispatcher, run: () => dispatchReplyFromConfig({ @@ -66,6 +137,7 @@ export async function dispatchInboundMessage(params: { replyResolver: params.replyResolver, }), }); + return finalizeDispatchResult(result, params.dispatcher); } export async function dispatchInboundMessageWithBufferedDispatcher(params: { @@ -76,9 +148,12 @@ export async function dispatchInboundMessageWithBufferedDispatcher(params: { replyResolver?: GetReplyFromConfig; }): Promise { const silentReplyContext = resolveDispatcherSilentReplyContext(params.ctx, params.cfg); + const beforeDeliver = + params.dispatcherOptions.beforeDeliver ?? buildMessageSendingBeforeDeliver(params.ctx); const { dispatcher, replyOptions, markDispatchIdle, markRunComplete } = createReplyDispatcherWithTyping({ ...params.dispatcherOptions, + beforeDeliver, silentReplyContext: params.dispatcherOptions.silentReplyContext ?? silentReplyContext, }); try { @@ -108,6 +183,8 @@ export async function dispatchInboundMessageWithDispatcher(params: { const silentReplyContext = resolveDispatcherSilentReplyContext(params.ctx, params.cfg); const dispatcher = createReplyDispatcher({ ...params.dispatcherOptions, + beforeDeliver: + params.dispatcherOptions.beforeDeliver ?? buildMessageSendingBeforeDeliver(params.ctx), silentReplyContext: params.dispatcherOptions.silentReplyContext ?? silentReplyContext, }); return await dispatchInboundMessage({ diff --git a/src/auto-reply/reply/agent-runner-execution.test.ts b/src/auto-reply/reply/agent-runner-execution.test.ts index e664e205997..8ef3fa8bdf9 100644 --- a/src/auto-reply/reply/agent-runner-execution.test.ts +++ b/src/auto-reply/reply/agent-runner-execution.test.ts @@ -691,6 +691,7 @@ describe("runAgentTurnWithFallback", () => { didStream: vi.fn(() => false), isAborted: vi.fn(() => false), hasSentPayload: vi.fn(() => false), + getSentMediaUrls: vi.fn(() => []), }; state.runWithModelFallbackMock.mockImplementationOnce(async (params: FallbackRunnerParams) => { const result = { payloads: [], meta: {} }; diff --git a/src/auto-reply/reply/agent-runner-payloads.test.ts b/src/auto-reply/reply/agent-runner-payloads.test.ts index 79cd98d6bea..c29fa559ddd 100644 --- a/src/auto-reply/reply/agent-runner-payloads.test.ts +++ b/src/auto-reply/reply/agent-runner-payloads.test.ts @@ -201,6 +201,77 @@ describe("buildReplyPayloads media filter integration", () => { await expectSameTargetRepliesSuppressed({ provider: "lark", to: "ou_abc123" }); }); + it("strips media already sent by the block pipeline after normalizing both paths", async () => { + const normalizeMediaPaths = async (payload: { mediaUrl?: string; mediaUrls?: string[] }) => { + const rewrite = (value?: string) => + value === "file:///tmp/voice.ogg" ? "file:///tmp/outbound/voice.ogg" : value; + return { + ...payload, + mediaUrl: rewrite(payload.mediaUrl), + mediaUrls: payload.mediaUrls?.map((value) => rewrite(value) ?? value), + }; + }; + const pipeline: Parameters[0]["blockReplyPipeline"] = { + didStream: () => false, + isAborted: () => false, + hasSentPayload: () => false, + enqueue: () => {}, + flush: async () => {}, + stop: () => {}, + hasBuffered: () => false, + getSentMediaUrls: () => ["file:///tmp/voice.ogg"], + }; + + const { replyPayloads } = await buildReplyPayloads({ + ...baseParams, + blockStreamingEnabled: true, + blockReplyPipeline: pipeline, + normalizeMediaPaths, + payloads: [{ text: "caption", mediaUrl: "file:///tmp/voice.ogg" }], + }); + + expect(replyPayloads).toHaveLength(1); + expect(replyPayloads[0]).toMatchObject({ + text: "caption", + mediaUrl: undefined, + mediaUrls: undefined, + }); + }); + + it("suppresses already-sent text plus media before stripping block-sent media", async () => { + const sentKey = JSON.stringify({ + text: "caption", + mediaList: ["file:///tmp/outbound/voice.ogg"], + }); + const pipeline: Parameters[0]["blockReplyPipeline"] = { + didStream: () => false, + isAborted: () => false, + hasSentPayload: (payload) => + JSON.stringify({ + text: (payload.text ?? "").trim(), + mediaList: [ + ...(payload.mediaUrl ? [payload.mediaUrl] : []), + ...(payload.mediaUrls ?? []), + ], + }) === sentKey, + enqueue: () => {}, + flush: async () => {}, + stop: () => {}, + hasBuffered: () => false, + getSentMediaUrls: () => ["file:///tmp/outbound/voice.ogg"], + }; + + const { replyPayloads } = await buildReplyPayloads({ + ...baseParams, + blockStreamingEnabled: true, + blockReplyPipeline: pipeline, + normalizeMediaPaths: async (payload) => payload, + payloads: [{ text: "caption", mediaUrl: "file:///tmp/outbound/voice.ogg" }], + }); + + expect(replyPayloads).toHaveLength(0); + }); + it("drops all final payloads when block pipeline streamed successfully", async () => { const pipeline: Parameters[0]["blockReplyPipeline"] = { didStream: () => true, @@ -210,6 +281,7 @@ describe("buildReplyPayloads media filter integration", () => { flush: async () => {}, stop: () => {}, hasBuffered: () => false, + getSentMediaUrls: () => [], }; // shouldDropFinalPayloads short-circuits to [] when the pipeline streamed // without aborting, so hasSentPayload is never reached. @@ -233,6 +305,7 @@ describe("buildReplyPayloads media filter integration", () => { flush: async () => {}, stop: () => {}, hasBuffered: () => false, + getSentMediaUrls: () => [], }; const { replyPayloads } = await buildReplyPayloads({ diff --git a/src/auto-reply/reply/agent-runner-payloads.ts b/src/auto-reply/reply/agent-runner-payloads.ts index e1777261af3..1f6e7d2e000 100644 --- a/src/auto-reply/reply/agent-runner-payloads.ts +++ b/src/auto-reply/reply/agent-runner-payloads.ts @@ -47,11 +47,11 @@ async function normalizeReplyPayloadMedia(params: { } async function normalizeSentMediaUrlsForDedupe(params: { - sentMediaUrls: string[]; + sentMediaUrls: readonly string[]; normalizeMediaPaths?: (payload: ReplyPayload) => Promise; }): Promise { if (params.sentMediaUrls.length === 0 || !params.normalizeMediaPaths) { - return params.sentMediaUrls; + return [...params.sentMediaUrls]; } const normalizedUrls: string[] = []; @@ -222,8 +222,7 @@ export async function buildReplyPayloads(params: { : mediaFilteredPayloads; const isDirectlySentBlockPayload = (payload: ReplyPayload) => Boolean(params.directlySentBlockKeys?.has(createBlockReplyContentKey(payload))); - // Filter out payloads already sent via pipeline or directly during tool flush. - const filteredPayloads = shouldDropFinalPayloads + const contentSuppressedPayloads = shouldDropFinalPayloads ? dedupedPayloads.filter((payload) => payload.isError) : params.blockStreamingEnabled ? dedupedPayloads.filter( @@ -236,6 +235,21 @@ export async function buildReplyPayloads(params: { (payload) => !params.directlySentBlockKeys!.has(createBlockReplyContentKey(payload)), ) : dedupedPayloads; + const blockSentMediaUrls = params.blockStreamingEnabled + ? await normalizeSentMediaUrlsForDedupe({ + sentMediaUrls: params.blockReplyPipeline?.getSentMediaUrls() ?? [], + normalizeMediaPaths: params.normalizeMediaPaths, + }) + : []; + const filteredPayloads = + blockSentMediaUrls.length > 0 + ? ( + dedupeRuntime ?? (await loadReplyPayloadsDedupeRuntime()) + ).filterMessagingToolMediaDuplicates({ + payloads: contentSuppressedPayloads, + sentMediaUrls: blockSentMediaUrls, + }) + : contentSuppressedPayloads; const replyPayloads = suppressMessagingToolReplies ? [] : filteredPayloads; return { diff --git a/src/auto-reply/reply/before-deliver.test.ts b/src/auto-reply/reply/before-deliver.test.ts new file mode 100644 index 00000000000..02948a7df8d --- /dev/null +++ b/src/auto-reply/reply/before-deliver.test.ts @@ -0,0 +1,68 @@ +import { describe, expect, it } from "vitest"; +import type { ReplyPayload } from "../types.js"; +import { createReplyDispatcher } from "./reply-dispatcher.js"; + +describe("beforeDeliver in reply dispatcher", () => { + it("cancels delivery when beforeDeliver returns null", async () => { + const delivered: string[] = []; + + const dispatcher = createReplyDispatcher({ + deliver: async (payload) => { + delivered.push(payload.text ?? ""); + }, + beforeDeliver: async (payload: ReplyPayload) => { + if (payload.text?.includes("blocked")) { + return null; + } + return payload; + }, + }); + + dispatcher.sendFinalReply({ text: "blocked reply" }); + dispatcher.sendFinalReply({ text: "safe reply" }); + dispatcher.markComplete(); + await dispatcher.waitForIdle(); + + expect(delivered).toEqual(["safe reply"]); + expect(dispatcher.getQueuedCounts()).toEqual({ tool: 0, block: 0, final: 2 }); + expect(dispatcher.getCancelledCounts?.()).toEqual({ tool: 0, block: 0, final: 1 }); + }); + + it("allows modifying payload in beforeDeliver", async () => { + const delivered: string[] = []; + + const dispatcher = createReplyDispatcher({ + deliver: async (payload) => { + delivered.push(payload.text ?? ""); + }, + beforeDeliver: async (payload: ReplyPayload) => { + if (payload.text?.includes("error")) { + return { ...payload, text: "replaced" }; + } + return payload; + }, + }); + + dispatcher.sendFinalReply({ text: "some error occurred" }); + dispatcher.markComplete(); + await dispatcher.waitForIdle(); + + expect(delivered).toEqual(["replaced"]); + }); + + it("delivers normally without beforeDeliver", async () => { + const delivered: string[] = []; + + const dispatcher = createReplyDispatcher({ + deliver: async (payload) => { + delivered.push(payload.text ?? ""); + }, + }); + + dispatcher.sendFinalReply({ text: "plain reply" }); + dispatcher.markComplete(); + await dispatcher.waitForIdle(); + + expect(delivered).toEqual(["plain reply"]); + }); +}); diff --git a/src/auto-reply/reply/block-reply-pipeline.test.ts b/src/auto-reply/reply/block-reply-pipeline.test.ts index a7bc76b58b1..72f2d48cfa8 100644 --- a/src/auto-reply/reply/block-reply-pipeline.test.ts +++ b/src/auto-reply/reply/block-reply-pipeline.test.ts @@ -78,6 +78,38 @@ describe("createBlockReplyPipeline dedup with threading", () => { expect(pipeline.hasSentPayload({ text: "response text", replyToId: "other-id" })).toBe(true); }); + it("tracks media URLs delivered via block replies", async () => { + const pipeline = createBlockReplyPipeline({ + onBlockReply: async () => {}, + timeoutMs: 5000, + }); + + expect(pipeline.getSentMediaUrls()).toEqual([]); + + pipeline.enqueue({ text: "caption", mediaUrl: "file:///a.ogg" }); + pipeline.enqueue({ mediaUrls: ["file:///b.ogg", "file:///c.ogg"] }); + await pipeline.flush({ force: true }); + + expect(pipeline.getSentMediaUrls()).toEqual([ + "file:///a.ogg", + "file:///b.ogg", + "file:///c.ogg", + ]); + }); + + it("does not track media when text-only blocks are delivered", async () => { + const pipeline = createBlockReplyPipeline({ + onBlockReply: async () => {}, + timeoutMs: 5000, + }); + + pipeline.enqueue({ text: "hello" }); + pipeline.enqueue({ text: "world" }); + await pipeline.flush({ force: true }); + + expect(pipeline.getSentMediaUrls()).toEqual([]); + }); + it("does not coalesce logical assistant blocks across assistantMessageIndex boundaries", async () => { const sent: string[] = []; const pipeline = createBlockReplyPipeline({ diff --git a/src/auto-reply/reply/block-reply-pipeline.ts b/src/auto-reply/reply/block-reply-pipeline.ts index 1898d5d8443..1531826e8cb 100644 --- a/src/auto-reply/reply/block-reply-pipeline.ts +++ b/src/auto-reply/reply/block-reply-pipeline.ts @@ -13,6 +13,7 @@ export type BlockReplyPipeline = { didStream: () => boolean; isAborted: () => boolean; hasSentPayload: (payload: ReplyPayload) => boolean; + getSentMediaUrls: () => readonly string[]; }; export type BlockReplyBuffer = { @@ -86,6 +87,7 @@ export function createBlockReplyPipeline(params: { const { onBlockReply, timeoutMs, coalescing, buffer } = params; const sentKeys = new Set(); const sentContentKeys = new Set(); + const sentMediaUrls = new Set(); const pendingKeys = new Set(); const seenKeys = new Set(); const bufferedKeys = new Set(); @@ -149,6 +151,9 @@ export function createBlockReplyPipeline(params: { sentKeys.add(payloadKey); sentContentKeys.add(contentKey); const reply = resolveSendableOutboundReplyParts(payload); + for (const mediaUrl of reply.mediaUrls) { + sentMediaUrls.add(mediaUrl); + } if (!reply.hasMedia && reply.trimmedText) { streamedTextFragments.push(reply.trimmedText); } @@ -284,5 +289,6 @@ export function createBlockReplyPipeline(params: { const normalize = (text: string) => text.replace(/\s+/g, ""); return normalize(streamedTextFragments.join("")) === normalize(reply.trimmedText); }, + getSentMediaUrls: () => Array.from(sentMediaUrls), }; } diff --git a/src/auto-reply/reply/reply-dispatcher.ts b/src/auto-reply/reply/reply-dispatcher.ts index 0bee71b169c..9fad32e668e 100644 --- a/src/auto-reply/reply/reply-dispatcher.ts +++ b/src/auto-reply/reply/reply-dispatcher.ts @@ -31,6 +31,11 @@ type ReplyDispatchDeliverer = ( info: { kind: ReplyDispatchKind }, ) => Promise; +export type ReplyDispatchBeforeDeliver = ( + payload: ReplyPayload, + info: { kind: ReplyDispatchKind }, +) => Promise | ReplyPayload | null; + const DEFAULT_HUMAN_DELAY_MIN_MS = 800; const DEFAULT_HUMAN_DELAY_MAX_MS = 2500; const silentReplyLogger = createSubsystemLogger("silent-reply/dispatcher"); @@ -73,6 +78,7 @@ export type ReplyDispatcherOptions = { onSkip?: ReplyDispatchSkipHandler; /** Human-like delay between block replies for natural rhythm. */ humanDelay?: HumanDelayConfig; + beforeDeliver?: ReplyDispatchBeforeDeliver; }; export type ReplyDispatcherWithTypingOptions = Omit & { @@ -190,6 +196,11 @@ export function createReplyDispatcher(options: ReplyDispatcherOptions): ReplyDis block: 0, final: 0, }; + const cancelledCounts: Record = { + tool: 0, + block: 0, + final: 0, + }; // Register this dispatcher globally for gateway restart coordination. const { unregister } = registerDispatcher({ @@ -242,9 +253,15 @@ export function createReplyDispatcher(options: ReplyDispatcherOptions): ReplyDis await sleep(delayMs); } } - // Safe: deliver is called inside an async .then() callback, so even a synchronous - // throw becomes a rejection that flows through .catch()/.finally(), ensuring cleanup. - await options.deliver(normalized, { kind }); + let deliverPayload: ReplyPayload | null = normalized; + if (options.beforeDeliver) { + deliverPayload = await options.beforeDeliver(normalized, { kind }); + if (!deliverPayload) { + cancelledCounts[kind] += 1; + return; + } + } + await options.deliver(deliverPayload, { kind }); }) .catch((err) => { failedCounts[kind] += 1; @@ -294,6 +311,7 @@ export function createReplyDispatcher(options: ReplyDispatcherOptions): ReplyDis sendFinalReply: (payload) => enqueue("final", payload), waitForIdle: () => sendChain, getQueuedCounts: () => ({ ...queuedCounts }), + getCancelledCounts: () => ({ ...cancelledCounts }), getFailedCounts: () => ({ ...failedCounts }), markComplete, }; diff --git a/src/auto-reply/reply/reply-dispatcher.types.ts b/src/auto-reply/reply/reply-dispatcher.types.ts index 7a4a9766f19..cd482e258ab 100644 --- a/src/auto-reply/reply/reply-dispatcher.types.ts +++ b/src/auto-reply/reply/reply-dispatcher.types.ts @@ -8,6 +8,7 @@ export type ReplyDispatcher = { sendFinalReply: (payload: ReplyPayload) => boolean; waitForIdle: () => Promise; getQueuedCounts: () => Record; + getCancelledCounts?: () => Record; getFailedCounts: () => Record; markComplete: () => void; }; diff --git a/src/commands/doctor/shared/legacy-config-migrate.provider-shapes.test.ts b/src/commands/doctor/shared/legacy-config-migrate.provider-shapes.test.ts index ae56943baf3..a6999f1ae9b 100644 --- a/src/commands/doctor/shared/legacy-config-migrate.provider-shapes.test.ts +++ b/src/commands/doctor/shared/legacy-config-migrate.provider-shapes.test.ts @@ -48,6 +48,41 @@ describe("legacy migrate provider-shaped config", () => { }); }); + it("moves legacy edge provider aliases into microsoft tts config", () => { + const res = migrateLegacyConfig({ + messages: { + tts: { + provider: "edge", + providers: { + edge: { + voice: "en-US-AvaNeural", + rate: "+8%", + }, + microsoft: { + lang: "en-US", + rate: "+4%", + }, + }, + }, + }, + }); + + expect(res.changes).toContain('Moved messages.tts.provider "edge" → "microsoft".'); + expect(res.changes).toContain( + "Moved messages.tts.providers.edge → messages.tts.providers.microsoft.", + ); + expect(res.config?.messages?.tts).toEqual({ + provider: "microsoft", + providers: { + microsoft: { + lang: "en-US", + rate: "+4%", + voice: "en-US-AvaNeural", + }, + }, + }); + }); + it("moves plugins.entries.voice-call.config.tts. keys into providers", () => { const res = migrateLegacyConfig({ plugins: { @@ -86,6 +121,47 @@ describe("legacy migrate provider-shaped config", () => { }); }); + it("moves voice-call legacy edge provider aliases into microsoft tts config", () => { + const res = migrateLegacyConfig({ + plugins: { + entries: { + "voice-call": { + config: { + tts: { + provider: "edge", + providers: { + edge: { + voice: "en-US-AvaNeural", + }, + }, + }, + }, + }, + }, + }, + }); + + expect(res.changes).toContain( + 'Moved plugins.entries.voice-call.config.tts.provider "edge" → "microsoft".', + ); + expect(res.changes).toContain( + "Moved plugins.entries.voice-call.config.tts.providers.edge → plugins.entries.voice-call.config.tts.providers.microsoft.", + ); + const voiceCallTts = ( + res.config?.plugins?.entries as + | Record } }> + | undefined + )?.["voice-call"]?.config?.tts; + expect(voiceCallTts).toEqual({ + provider: "microsoft", + providers: { + microsoft: { + voice: "en-US-AvaNeural", + }, + }, + }); + }); + it("does not migrate legacy tts provider keys for unknown plugin ids", () => { const res = migrateLegacyConfig({ plugins: { diff --git a/src/commands/doctor/shared/legacy-config-migrations.runtime.tts.ts b/src/commands/doctor/shared/legacy-config-migrations.runtime.tts.ts index 8877f3f098f..d3531b34fc2 100644 --- a/src/commands/doctor/shared/legacy-config-migrations.runtime.tts.ts +++ b/src/commands/doctor/shared/legacy-config-migrations.runtime.tts.ts @@ -10,12 +10,23 @@ import { isBlockedObjectKey } from "../../../config/prototype-keys.js"; const LEGACY_TTS_PROVIDER_KEYS = ["openai", "elevenlabs", "microsoft", "edge"] as const; const LEGACY_TTS_PLUGIN_IDS = new Set(["voice-call"]); +function isLegacyEdgeProviderId(value: unknown): boolean { + return typeof value === "string" && value.trim().toLowerCase() === "edge"; +} + function hasLegacyTtsProviderKeys(value: unknown): boolean { const tts = getRecord(value); if (!tts) { return false; } - return LEGACY_TTS_PROVIDER_KEYS.some((key) => Object.prototype.hasOwnProperty.call(tts, key)); + if (isLegacyEdgeProviderId(tts.provider)) { + return true; + } + if (LEGACY_TTS_PROVIDER_KEYS.some((key) => Object.prototype.hasOwnProperty.call(tts, key))) { + return true; + } + const providers = getRecord(tts.providers); + return Boolean(providers && Object.prototype.hasOwnProperty.call(providers, "edge")); } function hasLegacyPluginEntryTtsProviderKeys(value: unknown): boolean { @@ -57,6 +68,24 @@ function mergeLegacyTtsProviderConfig( return true; } +function mergeLegacyTtsProviderAliasConfig( + tts: Record, + aliasKey: string, + providerId: string, +): boolean { + const providers = getRecord(tts.providers); + const aliasValue = getRecord(providers?.[aliasKey]); + if (!providers || !aliasValue) { + return false; + } + const existing = getRecord(providers[providerId]) ?? {}; + const merged = structuredClone(existing); + mergeMissing(merged, aliasValue); + providers[providerId] = merged; + delete providers[aliasKey]; + return true; +} + function migrateLegacyTtsConfig( tts: Record | null | undefined, pathLabel: string, @@ -65,9 +94,14 @@ function migrateLegacyTtsConfig( if (!tts) { return; } + if (isLegacyEdgeProviderId(tts.provider)) { + tts.provider = "microsoft"; + changes.push(`Moved ${pathLabel}.provider "edge" → "microsoft".`); + } const movedOpenAI = mergeLegacyTtsProviderConfig(tts, "openai", "openai"); const movedElevenLabs = mergeLegacyTtsProviderConfig(tts, "elevenlabs", "elevenlabs"); const movedMicrosoft = mergeLegacyTtsProviderConfig(tts, "microsoft", "microsoft"); + const movedProviderEdge = mergeLegacyTtsProviderAliasConfig(tts, "edge", "microsoft"); const movedEdge = mergeLegacyTtsProviderConfig(tts, "edge", "microsoft"); if (movedOpenAI) { @@ -79,6 +113,9 @@ function migrateLegacyTtsConfig( if (movedMicrosoft) { changes.push(`Moved ${pathLabel}.microsoft → ${pathLabel}.providers.microsoft.`); } + if (movedProviderEdge) { + changes.push(`Moved ${pathLabel}.providers.edge → ${pathLabel}.providers.microsoft.`); + } if (movedEdge) { changes.push(`Moved ${pathLabel}.edge → ${pathLabel}.providers.microsoft.`); } @@ -88,13 +125,13 @@ const LEGACY_TTS_RULES: LegacyConfigRule[] = [ { path: ["messages", "tts"], message: - 'messages.tts. keys (openai/elevenlabs/microsoft/edge) are legacy; use messages.tts.providers.. Run "openclaw doctor --fix".', + 'messages.tts legacy provider aliases/keys are legacy; use provider: "microsoft" and messages.tts.providers.. Run "openclaw doctor --fix".', match: (value) => hasLegacyTtsProviderKeys(value), }, { path: ["plugins", "entries"], message: - 'plugins.entries.voice-call.config.tts. keys (openai/elevenlabs/microsoft/edge) are legacy; use plugins.entries.voice-call.config.tts.providers.. Run "openclaw doctor --fix".', + 'plugins.entries.voice-call.config.tts legacy provider aliases/keys are legacy; use provider: "microsoft" and plugins.entries.voice-call.config.tts.providers.. Run "openclaw doctor --fix".', match: (value) => hasLegacyPluginEntryTtsProviderKeys(value), }, ]; diff --git a/src/config/schema.base.generated.ts b/src/config/schema.base.generated.ts index a843427a647..5c40b3fc3dd 100644 --- a/src/config/schema.base.generated.ts +++ b/src/config/schema.base.generated.ts @@ -750,6 +750,9 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = { description: "Per-profile headless override for locally launched browser instances. Use this when one profile should stay headless without forcing browser.headless for every other profile.", }, + executablePath: { + type: "string", + }, attachOnly: { type: "boolean", title: "Browser Profile Attach-only Mode", diff --git a/src/config/types.browser.ts b/src/config/types.browser.ts index 5834cce9b73..562ab7108c7 100644 --- a/src/config/types.browser.ts +++ b/src/config/types.browser.ts @@ -9,6 +9,8 @@ export type BrowserProfileConfig = { driver?: "openclaw" | "clawd" | "existing-session"; /** If true, launch this profile in headless mode. Falls back to browser.headless. */ headless?: boolean; + /** Browser executable path for this profile. Falls back to browser.executablePath. */ + executablePath?: string; /** If true, never launch a browser for this profile; only attach. Falls back to browser.attachOnly. */ attachOnly?: boolean; /** Profile color (hex). Auto-assigned at creation. */ diff --git a/src/config/zod-schema.ts b/src/config/zod-schema.ts index 66dd10dd346..d183d02cff3 100644 --- a/src/config/zod-schema.ts +++ b/src/config/zod-schema.ts @@ -410,6 +410,7 @@ export const OpenClawSchema = z .union([z.literal("openclaw"), z.literal("clawd"), z.literal("existing-session")]) .optional(), headless: z.boolean().optional(), + executablePath: z.string().optional(), attachOnly: z.boolean().optional(), color: HexColorSchema, }) diff --git a/src/gateway/method-scopes.test.ts b/src/gateway/method-scopes.test.ts index afba94d8f8d..f64f6d94cca 100644 --- a/src/gateway/method-scopes.test.ts +++ b/src/gateway/method-scopes.test.ts @@ -11,7 +11,10 @@ import { coreGatewayHandlers } from "./server-methods.js"; const RESERVED_ADMIN_PLUGIN_METHOD = "config.plugin.inspect"; -function setPluginGatewayMethodScope(method: string, scope: "operator.read" | "operator.write") { +function setPluginGatewayMethodScope( + method: string, + scope: "operator.read" | "operator.write" | "operator.admin", +) { const registry = createEmptyPluginRegistry(); registry.gatewayMethodScopes = { [method]: scope, @@ -54,12 +57,12 @@ describe("method scope resolution", () => { it("reads plugin-registered gateway method scopes from the active plugin registry", () => { const registry = createEmptyPluginRegistry(); registry.gatewayMethodScopes = { - "browser.request": "operator.write", + "browser.request": "operator.admin", }; setActivePluginRegistry(registry); expect(resolveLeastPrivilegeOperatorScopesForMethod("browser.request")).toEqual([ - "operator.write", + "operator.admin", ]); }); @@ -89,6 +92,18 @@ describe("operator scope authorization", () => { }); }); + it("requires admin for browser.request", () => { + setPluginGatewayMethodScope("browser.request", "operator.admin"); + + expect(authorizeOperatorScopesForMethod("browser.request", ["operator.write"])).toEqual({ + allowed: false, + missingScope: "operator.admin", + }); + expect(authorizeOperatorScopesForMethod("browser.request", ["operator.admin"])).toEqual({ + allowed: true, + }); + }); + it("requires pairing scope for node pairing approvals", () => { expect(authorizeOperatorScopesForMethod("node.pair.approve", ["operator.pairing"])).toEqual({ allowed: true, diff --git a/src/gateway/session-utils.subagent.test.ts b/src/gateway/session-utils.subagent.test.ts index a8a427216a9..feba73bb0db 100644 --- a/src/gateway/session-utils.subagent.test.ts +++ b/src/gateway/session-utils.subagent.test.ts @@ -726,6 +726,184 @@ describe("listSessionsFromStore subagent metadata", () => { expect(result.sessions.map((session) => session.key)).toEqual(["agent:main:dashboard:child"]); }); + test("does not reattach stale terminal store-only child links", () => { + resetSubagentRegistryForTests({ persist: false }); + const now = Date.now(); + const staleAt = now - 2 * 60 * 60_000; + const store: Record = { + "agent:main:main": { + sessionId: "sess-main", + updatedAt: now, + } as SessionEntry, + "agent:claude:acp:done-child": { + sessionId: "sess-done-child", + updatedAt: staleAt, + spawnedBy: "agent:main:main", + status: "done", + endedAt: staleAt, + } as SessionEntry, + }; + + const all = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: {}, + }); + const main = all.sessions.find((session) => session.key === "agent:main:main"); + expect(main?.childSessions).toBeUndefined(); + + const filtered = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: { + spawnedBy: "agent:main:main", + }, + }); + expect(filtered.sessions.map((session) => session.key)).toEqual([]); + }); + + test("does not reattach stale orphan store-only child links without lifecycle fields", () => { + resetSubagentRegistryForTests({ persist: false }); + const now = Date.now(); + const staleAt = now - 2 * 60 * 60_000; + const store: Record = { + "agent:main:main": { + sessionId: "sess-main", + updatedAt: now, + } as SessionEntry, + "agent:main:subagent:orphan": { + sessionId: "sess-orphan", + updatedAt: staleAt, + parentSessionKey: "agent:main:main", + } as SessionEntry, + }; + + const all = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: {}, + }); + const main = all.sessions.find((session) => session.key === "agent:main:main"); + expect(main?.childSessions).toBeUndefined(); + + const filtered = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: { + spawnedBy: "agent:main:main", + }, + }); + expect(filtered.sessions.map((session) => session.key)).toEqual([]); + }); + + test("does not keep old ended registry runs attached as child sessions", () => { + const now = Date.now(); + const store: Record = { + "agent:main:main": { + sessionId: "sess-main", + updatedAt: now, + } as SessionEntry, + "agent:main:subagent:old-ended": { + sessionId: "sess-old-ended", + updatedAt: now - 60 * 60_000, + spawnedBy: "agent:main:main", + } as SessionEntry, + }; + + addSubagentRunForTests({ + runId: "run-old-ended", + childSessionKey: "agent:main:subagent:old-ended", + controllerSessionKey: "agent:main:main", + requesterSessionKey: "agent:main:main", + requesterDisplayKey: "main", + task: "old ended task", + cleanup: "keep", + createdAt: now - 60 * 60_000, + startedAt: now - 59 * 60_000, + endedAt: now - 31 * 60_000, + outcome: { status: "ok" }, + }); + + const all = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: {}, + }); + const main = all.sessions.find((session) => session.key === "agent:main:main"); + expect(main?.childSessions).toBeUndefined(); + + const filtered = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: { + spawnedBy: "agent:main:main", + }, + }); + expect(filtered.sessions.map((session) => session.key)).toEqual([]); + }); + + test("keeps ended parents attached while live descendants are still running", () => { + const now = Date.now(); + const parentKey = "agent:main:subagent:ended-parent"; + const childKey = "agent:main:subagent:ended-parent:subagent:live-child"; + const store: Record = { + "agent:main:main": { + sessionId: "sess-main", + updatedAt: now, + } as SessionEntry, + [parentKey]: { + sessionId: "sess-ended-parent", + updatedAt: now - 31 * 60_000, + spawnedBy: "agent:main:main", + } as SessionEntry, + [childKey]: { + sessionId: "sess-live-child", + updatedAt: now, + spawnedBy: parentKey, + } as SessionEntry, + }; + + addSubagentRunForTests({ + runId: "run-ended-parent", + childSessionKey: parentKey, + controllerSessionKey: "agent:main:main", + requesterSessionKey: "agent:main:main", + requesterDisplayKey: "main", + task: "ended parent task", + cleanup: "keep", + createdAt: now - 60 * 60_000, + startedAt: now - 59 * 60_000, + endedAt: now - 31 * 60_000, + outcome: { status: "ok" }, + }); + addSubagentRunForTests({ + runId: "run-live-child", + childSessionKey: childKey, + controllerSessionKey: parentKey, + requesterSessionKey: parentKey, + requesterDisplayKey: "ended-parent", + task: "live child task", + cleanup: "keep", + createdAt: now - 1_000, + startedAt: now - 900, + }); + + const result = listSessionsFromStore({ + cfg, + storePath: "/tmp/sessions.json", + store, + opts: {}, + }); + const main = result.sessions.find((session) => session.key === "agent:main:main"); + expect(main?.childSessions).toEqual([parentKey]); + }); + test("falls back to persisted subagent timing after run archival", () => { const now = Date.now(); const store: Record = { diff --git a/src/gateway/session-utils.ts b/src/gateway/session-utils.ts index cdbc342e928..33b2198b014 100644 --- a/src/gateway/session-utils.ts +++ b/src/gateway/session-utils.ts @@ -19,12 +19,17 @@ import { resolvePersistedSelectedModelRef, } from "../agents/model-selection.js"; import { + countActiveDescendantRuns, getSessionDisplaySubagentRunByChildSessionKey, getSubagentSessionRuntimeMs, getSubagentSessionStartedAt, listSubagentRunsForController, resolveSubagentSessionStatus, } from "../agents/subagent-registry-read.js"; +import { + RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS, + shouldKeepSubagentRunChildLink, +} from "../agents/subagent-run-liveness.js"; import { listThinkingLevelOptions, resolveThinkingDefaultForModel, @@ -81,6 +86,7 @@ import type { GatewayAgentRow, GatewaySessionRow, GatewaySessionsDefaults, + SessionRunStatus, SessionsListResult, } from "./session-utils.types.js"; @@ -291,9 +297,36 @@ function resolveEstimatedSessionCostUsd(params: { return resolveNonNegativeNumber(estimated); } +const STALE_STORE_ONLY_CHILD_LINK_MS = 60 * 60 * 1_000; + +function isFinitePositiveTimestamp(value: unknown): value is number { + return typeof value === "number" && Number.isFinite(value) && value > 0; +} + +function isTerminalSessionStatus(status: unknown): status is Exclude { + return status === "done" || status === "failed" || status === "killed" || status === "timeout"; +} + +function shouldKeepStoreOnlyChildLink(entry: SessionEntry, now: number): boolean { + if (isTerminalSessionStatus(entry.status) || isFinitePositiveTimestamp(entry.endedAt)) { + const endedAt = isFinitePositiveTimestamp(entry.endedAt) ? entry.endedAt : entry.updatedAt; + return ( + isFinitePositiveTimestamp(endedAt) && now - endedAt <= RECENT_ENDED_SUBAGENT_CHILD_SESSION_MS + ); + } + if (entry.status === "running" || isFinitePositiveTimestamp(entry.startedAt)) { + return true; + } + return ( + isFinitePositiveTimestamp(entry.updatedAt) && + now - entry.updatedAt <= STALE_STORE_ONLY_CHILD_LINK_MS + ); +} + function resolveChildSessionKeys( controllerSessionKey: string, store: Record, + now = Date.now(), ): string[] | undefined { const childSessionKeys = new Set(); for (const entry of listSubagentRunsForController(controllerSessionKey)) { @@ -302,12 +335,23 @@ function resolveChildSessionKeys( continue; } const latest = getSessionDisplaySubagentRunByChildSessionKey(childSessionKey); + if (!latest) { + continue; + } const latestControllerSessionKey = normalizeOptionalString(latest?.controllerSessionKey) || normalizeOptionalString(latest?.requesterSessionKey); if (latestControllerSessionKey !== controllerSessionKey) { continue; } + if ( + !shouldKeepSubagentRunChildLink(latest, { + activeDescendants: countActiveDescendantRuns(childSessionKey), + now, + }) + ) { + continue; + } childSessionKeys.add(childSessionKey); } for (const [key, entry] of Object.entries(store)) { @@ -327,6 +371,16 @@ function resolveChildSessionKeys( if (latestControllerSessionKey !== controllerSessionKey) { continue; } + if ( + !shouldKeepSubagentRunChildLink(latest, { + activeDescendants: countActiveDescendantRuns(key), + now, + }) + ) { + continue; + } + } else if (!shouldKeepStoreOnlyChildLink(entry, now)) { + continue; } childSessionKeys.add(key); } @@ -1262,7 +1316,7 @@ export function buildGatewaySessionRow(params: { typeof totalTokens === "number" && Number.isFinite(totalTokens) && totalTokens > 0 ? true : transcriptUsage?.totalTokensFresh === true; - const childSessions = resolveChildSessionKeys(key, store); + const childSessions = resolveChildSessionKeys(key, store, now); const latestCompactionCheckpoint = resolveLatestCompactionCheckpoint(entry); const estimatedCostUsd = resolveEstimatedSessionCostUsd({ @@ -1445,9 +1499,18 @@ export function listSessionsFromStore(params: { const latestControllerSessionKey = normalizeOptionalString(latest.controllerSessionKey) || normalizeOptionalString(latest.requesterSessionKey); - return latestControllerSessionKey === spawnedBy; + return ( + latestControllerSessionKey === spawnedBy && + shouldKeepSubagentRunChildLink(latest, { + activeDescendants: countActiveDescendantRuns(key), + now, + }) + ); } - return entry?.spawnedBy === spawnedBy || entry?.parentSessionKey === spawnedBy; + return ( + shouldKeepStoreOnlyChildLink(entry, now) && + (entry?.spawnedBy === spawnedBy || entry?.parentSessionKey === spawnedBy) + ); }) .filter(([, entry]) => { if (!label) { diff --git a/src/gateway/tools-invoke-http.test.ts b/src/gateway/tools-invoke-http.test.ts index 4a56a92006a..45e6caf757c 100644 --- a/src/gateway/tools-invoke-http.test.ts +++ b/src/gateway/tools-invoke-http.test.ts @@ -127,6 +127,11 @@ vi.mock("../agents/openclaw-tools.js", () => { parameters: { type: "object", properties: {} }, execute: async () => ({ ok: true, result: "nodes" }), }, + { + name: "browser", + parameters: { type: "object", properties: {} }, + execute: async () => ({ ok: true, result: "browser" }), + }, { name: "owner_only_test", ownerOnly: true, @@ -181,7 +186,7 @@ vi.mock("../agents/openclaw-tools.js", () => { return { createOpenClawTools: (ctx: Record) => { lastCreateOpenClawToolsContext = ctx; - return tools; + return ctx.disablePluginTools ? tools.filter((tool) => tool.name !== "browser") : tools; }, }; }); @@ -878,4 +883,17 @@ describe("POST /tools/invoke", () => { expect(nodesRes.status).toBe(404); expect(nodesAdminRes.status).toBe(404); }); + + it("falls back to plugin-backed tools when a cataloged core tool has no core implementation", async () => { + setMainAllowedTools({ allow: ["browser"] }); + + const res = await invokeToolAuthed({ + tool: "browser", + sessionKey: "main", + }); + + const body = await expectOkInvokeResponse(res); + expect(body.result).toEqual({ ok: true, result: "browser" }); + expect(lastCreateOpenClawToolsContext?.disablePluginTools).toBe(false); + }); }); diff --git a/src/gateway/tools-invoke-http.ts b/src/gateway/tools-invoke-http.ts index e79964ad27a..5b10d5dd0c5 100644 --- a/src/gateway/tools-invoke-http.ts +++ b/src/gateway/tools-invoke-http.ts @@ -226,19 +226,25 @@ export async function handleToolsInvokeHttpRequest( // with the correct owner context and channel-action gates (e.g. Matrix set-profile) // work correctly for both owner and non-owner callers. const senderIsOwner = resolveOpenAiCompatibleHttpSenderIsOwner(req, requestAuth); - const { agentId, tools } = resolveGatewayScopedTools({ - cfg, - sessionKey, - messageProvider: messageChannel ?? undefined, - accountId, - agentTo, - agentThreadId, - allowGatewaySubagentBinding: true, - allowMediaInvokeCommands: true, - surface: "http", - disablePluginTools: isKnownCoreToolId(toolName), - senderIsOwner, - }); + const resolveTools = (disablePluginTools: boolean) => + resolveGatewayScopedTools({ + cfg, + sessionKey, + messageProvider: messageChannel ?? undefined, + accountId, + agentTo, + agentThreadId, + allowGatewaySubagentBinding: true, + allowMediaInvokeCommands: true, + surface: "http", + disablePluginTools, + senderIsOwner, + }); + const knownCoreTool = isKnownCoreToolId(toolName); + let { agentId, tools } = resolveTools(knownCoreTool); + if (knownCoreTool && !tools.some((candidate) => candidate.name === toolName)) { + ({ agentId, tools } = resolveTools(false)); + } const gatewayFiltered = applyOwnerOnlyToolPolicy(tools, senderIsOwner); const tool = gatewayFiltered.find((t) => t.name === toolName); diff --git a/src/image-generation/provider-registry.allowlist.test.ts b/src/image-generation/provider-registry.allowlist.test.ts index 2466f39137b..d444cb8d07c 100644 --- a/src/image-generation/provider-registry.allowlist.test.ts +++ b/src/image-generation/provider-registry.allowlist.test.ts @@ -26,6 +26,7 @@ describe("image-generation provider registry allowlist fallback", () => { expect(getImageGenerationProvider("openai", cfg as OpenClawConfig)).toBeUndefined(); expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ config: compatConfig, + activate: false, }); }); }); diff --git a/src/media-understanding/provider-registry.allowlist.test.ts b/src/media-understanding/provider-registry.allowlist.test.ts index ca315b0bbad..3238b572f88 100644 --- a/src/media-understanding/provider-registry.allowlist.test.ts +++ b/src/media-understanding/provider-registry.allowlist.test.ts @@ -27,6 +27,7 @@ describe("media-understanding provider registry allowlist fallback", () => { expect(getMediaUnderstandingProvider("openai", registry)).toBeUndefined(); expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ config: compatConfig, + activate: false, }); }); }); diff --git a/src/plugins/capability-provider-runtime.test.ts b/src/plugins/capability-provider-runtime.test.ts index e4bdc05169b..1ee6c7f6100 100644 --- a/src/plugins/capability-provider-runtime.test.ts +++ b/src/plugins/capability-provider-runtime.test.ts @@ -82,6 +82,7 @@ function expectBundledCompatLoadPath(params: { }); expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ config: params.enablementCompat, + activate: false, }); } @@ -203,7 +204,36 @@ describe("resolvePluginCapabilityProviders", () => { expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith(); }); - it("keeps active capability providers when cfg compat has no extra providers", () => { + it("uses active non-speech capability providers even when cfg is passed", () => { + const active = createEmptyPluginRegistry(); + active.mediaUnderstandingProviders.push({ + pluginId: "deepgram", + pluginName: "Deepgram", + source: "test", + provider: { + id: "deepgram", + capabilities: ["audio"], + }, + } as never); + mocks.resolveRuntimePluginRegistry.mockReturnValue(active); + + const providers = resolvePluginCapabilityProviders({ + key: "mediaUnderstandingProviders", + cfg: { + tools: { + media: { + models: [{ provider: "deepgram" }], + }, + }, + } as OpenClawConfig, + }); + + expectResolvedCapabilityProviderIds(providers, ["deepgram"]); + expect(mocks.loadPluginManifestRegistry).not.toHaveBeenCalled(); + expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith(); + }); + + it("keeps active speech providers when cfg requests an active provider alias", () => { const active = createEmptyPluginRegistry(); active.speechProviders.push({ pluginId: "microsoft", @@ -222,18 +252,59 @@ describe("resolvePluginCapabilityProviders", () => { }), }, } as never); - mocks.resolveRuntimePluginRegistry.mockImplementation((params?: unknown) => - params === undefined ? active : createEmptyPluginRegistry(), - ); + mocks.resolveRuntimePluginRegistry.mockReturnValue(active); const providers = resolvePluginCapabilityProviders({ key: "speechProviders", - cfg: { messages: { tts: { provider: "edge" } } } as OpenClawConfig, + cfg: { + plugins: { entries: { microsoft: { enabled: true } } }, + messages: { tts: { provider: "edge" } }, + } as OpenClawConfig, }); expectResolvedCapabilityProviderIds(providers, ["microsoft"]); + expect(mocks.loadPluginManifestRegistry).not.toHaveBeenCalled(); expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith(); - expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ + }); + + it("keeps active capability providers when cfg has no explicit plugin config", () => { + const active = createEmptyPluginRegistry(); + active.speechProviders.push({ + pluginId: "acme", + pluginName: "acme", + source: "test", + provider: { + id: "acme", + label: "acme", + isConfigured: () => true, + synthesize: async () => ({ + audioBuffer: Buffer.from("x"), + outputFormat: "mp3", + voiceCompatible: false, + fileExtension: ".mp3", + }), + }, + } as never); + mocks.loadPluginManifestRegistry.mockReturnValue({ + plugins: [ + { + id: "microsoft", + origin: "bundled", + contracts: { speechProviders: ["microsoft"] }, + }, + ] as never, + diagnostics: [], + }); + mocks.resolveRuntimePluginRegistry.mockReturnValue(active); + + const providers = resolvePluginCapabilityProviders({ + key: "speechProviders", + cfg: { messages: { tts: { provider: "acme" } } } as OpenClawConfig, + }); + + expectResolvedCapabilityProviderIds(providers, ["acme"]); + expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith(); + expect(mocks.resolveRuntimePluginRegistry).not.toHaveBeenCalledWith({ config: expect.anything(), }); }); @@ -304,9 +375,94 @@ describe("resolvePluginCapabilityProviders", () => { allow: ["openai", "microsoft"], }), }), + activate: false, }); }); + it("does not merge unrelated bundled capability providers when cfg requests one provider", () => { + const active = createEmptyPluginRegistry(); + active.speechProviders.push({ + pluginId: "openai", + pluginName: "openai", + source: "test", + provider: { + id: "openai", + label: "openai", + isConfigured: () => true, + synthesize: async () => ({ + audioBuffer: Buffer.from("x"), + outputFormat: "mp3", + voiceCompatible: false, + fileExtension: ".mp3", + }), + }, + } as never); + const loaded = createEmptyPluginRegistry(); + loaded.speechProviders.push( + { + pluginId: "microsoft", + pluginName: "microsoft", + source: "test", + provider: { + id: "microsoft", + label: "microsoft", + aliases: ["edge"], + isConfigured: () => true, + synthesize: async () => ({ + audioBuffer: Buffer.from("x"), + outputFormat: "mp3", + voiceCompatible: false, + fileExtension: ".mp3", + }), + }, + } as never, + { + pluginId: "elevenlabs", + pluginName: "elevenlabs", + source: "test", + provider: { + id: "elevenlabs", + label: "elevenlabs", + isConfigured: () => true, + synthesize: async () => ({ + audioBuffer: Buffer.from("x"), + outputFormat: "mp3", + voiceCompatible: false, + fileExtension: ".mp3", + }), + }, + } as never, + ); + mocks.loadPluginManifestRegistry.mockReturnValue({ + plugins: [ + { + id: "microsoft", + origin: "bundled", + contracts: { speechProviders: ["microsoft"] }, + }, + { + id: "elevenlabs", + origin: "bundled", + contracts: { speechProviders: ["elevenlabs"] }, + }, + ] as never, + diagnostics: [], + }); + mocks.resolveRuntimePluginRegistry.mockImplementation((params?: unknown) => + params === undefined ? active : loaded, + ); + + const providers = resolvePluginCapabilityProviders({ + key: "speechProviders", + cfg: { + plugins: { allow: ["openai", "microsoft", "elevenlabs"] }, + messages: { tts: { provider: "edge" } }, + } as OpenClawConfig, + }); + + expectResolvedCapabilityProviderIds(providers, ["openai", "microsoft"]); + }); + it.each([ ["memoryEmbeddingProviders", "memoryEmbeddingProviders"], ["speechProviders", "speechProviders"], @@ -339,6 +495,7 @@ describe("resolvePluginCapabilityProviders", () => { expectNoResolvedCapabilityProviders(providers); expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ config: expect.anything(), + activate: false, }); }); @@ -379,7 +536,10 @@ describe("resolvePluginCapabilityProviders", () => { config: undefined, env: process.env, }); - expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ config: compatConfig }); + expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ + config: compatConfig, + activate: false, + }); }); it("loads only the bundled owner plugin for a targeted provider lookup", () => { @@ -443,6 +603,7 @@ describe("resolvePluginCapabilityProviders", () => { }); expect(mocks.resolveRuntimePluginRegistry).toHaveBeenCalledWith({ config: enablementCompat, + activate: false, }); }); }); diff --git a/src/plugins/capability-provider-runtime.ts b/src/plugins/capability-provider-runtime.ts index 27fd2596d1a..599a6f60024 100644 --- a/src/plugins/capability-provider-runtime.ts +++ b/src/plugins/capability-provider-runtime.ts @@ -4,6 +4,7 @@ import { withBundledPluginEnablementCompat, withBundledPluginVitestCompat, } from "./bundled-compat.js"; +import { hasExplicitPluginConfig } from "./config-policy.js"; import { resolveRuntimePluginRegistry } from "./loader.js"; import { loadPluginManifestRegistry } from "./manifest-registry.js"; import type { PluginRegistry } from "./registry-types.js"; @@ -122,6 +123,81 @@ function mergeCapabilityProviders( return [...merged.values(), ...unnamed]; } +function addObjectKeys(target: Set, value: unknown): void { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return; + } + for (const key of Object.keys(value)) { + const normalized = key.trim().toLowerCase(); + if (normalized) { + target.add(normalized); + } + } +} + +function addStringValue(target: Set, value: unknown): void { + if (typeof value !== "string") { + return; + } + const normalized = value.trim().toLowerCase(); + if (normalized) { + target.add(normalized); + } +} + +function collectRequestedSpeechProviderIds(cfg: OpenClawConfig | undefined): Set { + const requested = new Set(); + const tts = + typeof cfg?.messages?.tts === "object" && cfg.messages.tts !== null + ? (cfg.messages.tts as Record) + : undefined; + addStringValue(requested, tts?.provider); + addObjectKeys(requested, tts?.providers); + addObjectKeys(requested, cfg?.models?.providers); + return requested; +} + +function removeActiveProviderIds(requested: Set, entries: readonly unknown[]): void { + for (const entry of entries as Array<{ provider: { id?: unknown; aliases?: unknown } }>) { + const provider = entry.provider as { id?: unknown; aliases?: unknown }; + if (typeof provider.id === "string") { + requested.delete(provider.id.toLowerCase()); + } + if (Array.isArray(provider.aliases)) { + for (const alias of provider.aliases) { + if (typeof alias === "string") { + requested.delete(alias.toLowerCase()); + } + } + } + } +} + +function filterLoadedProvidersForRequestedConfig(params: { + key: K; + requested: Set; + entries: PluginRegistry[K]; +}): PluginRegistry[K] { + if (params.key !== "speechProviders") { + return [] as unknown as PluginRegistry[K]; + } + if (params.requested.size === 0) { + return [] as unknown as PluginRegistry[K]; + } + return params.entries.filter((entry) => { + const provider = entry.provider as { id?: unknown; aliases?: unknown }; + if (typeof provider.id === "string" && params.requested.has(provider.id.toLowerCase())) { + return true; + } + if (Array.isArray(provider.aliases)) { + return provider.aliases.some( + (alias) => typeof alias === "string" && params.requested.has(alias.toLowerCase()), + ); + } + return false; + }) as PluginRegistry[K]; +} + export function resolvePluginCapabilityProvider(params: { key: K; providerId: string; @@ -147,7 +223,8 @@ export function resolvePluginCapabilityProvider[] { const activeRegistry = resolveRuntimePluginRegistry(); const activeProviders = activeRegistry?.[params.key] ?? []; - if (activeProviders.length > 0 && params.key !== "memoryEmbeddingProviders" && !params.cfg) { + if ( + activeProviders.length > 0 && + params.key !== "memoryEmbeddingProviders" && + params.key !== "speechProviders" && + !hasExplicitPluginConfig(params.cfg?.plugins) + ) { return activeProviders.map((entry) => entry.provider) as CapabilityProviderForKey[]; } + if (activeProviders.length > 0 && params.key === "speechProviders" && !params.cfg) { + return activeProviders.map((entry) => entry.provider) as CapabilityProviderForKey[]; + } + const missingRequestedSpeechProviders = + activeProviders.length > 0 && params.key === "speechProviders" + ? collectRequestedSpeechProviderIds(params.cfg) + : undefined; + if (missingRequestedSpeechProviders) { + removeActiveProviderIds(missingRequestedSpeechProviders, activeProviders); + if (missingRequestedSpeechProviders.size === 0) { + return activeProviders.map((entry) => entry.provider) as CapabilityProviderForKey[]; + } + } const compatConfig = resolveCapabilityProviderConfig({ key: params.key, cfg: params.cfg }); - const loadOptions = compatConfig === undefined ? undefined : { config: compatConfig }; + const loadOptions = + compatConfig === undefined ? undefined : { config: compatConfig, activate: false }; const registry = resolveRuntimePluginRegistry(loadOptions); const loadedProviders = registry?.[params.key] ?? []; if (params.key !== "memoryEmbeddingProviders") { - return mergeCapabilityProviders(activeProviders, loadedProviders); + const mergeLoadedProviders = + activeProviders.length > 0 + ? filterLoadedProvidersForRequestedConfig({ + key: params.key, + requested: missingRequestedSpeechProviders ?? new Set(), + entries: loadedProviders, + }) + : loadedProviders; + return mergeCapabilityProviders(activeProviders, mergeLoadedProviders); } return mergeCapabilityProviders(activeProviders, loadedProviders); } diff --git a/src/tts/provider-registry.test.ts b/src/tts/provider-registry.test.ts index f8e122cb5b6..3ccba8c5760 100644 --- a/src/tts/provider-registry.test.ts +++ b/src/tts/provider-registry.test.ts @@ -123,6 +123,7 @@ describe("speech provider registry", () => { }, }, }, + activate: false, }); }); diff --git a/src/tts/provider-registry.ts b/src/tts/provider-registry.ts index c145727047c..f43a746fd87 100644 --- a/src/tts/provider-registry.ts +++ b/src/tts/provider-registry.ts @@ -1,5 +1,8 @@ import type { OpenClawConfig } from "../config/types.js"; -import { resolvePluginCapabilityProviders } from "../plugins/capability-provider-runtime.js"; +import { + resolvePluginCapabilityProvider, + resolvePluginCapabilityProviders, +} from "../plugins/capability-provider-runtime.js"; import { buildCapabilityProviderMaps, normalizeCapabilityProviderId, @@ -39,7 +42,13 @@ export function getSpeechProvider( if (!normalized) { return undefined; } - return buildProviderMaps(cfg).aliases.get(normalized); + return ( + resolvePluginCapabilityProvider({ + key: "speechProviders", + providerId: normalized, + cfg, + }) ?? buildProviderMaps(cfg).aliases.get(normalized) + ); } export function canonicalizeSpeechProviderId( diff --git a/test/helpers/browser-bundled-plugin-fixture.ts b/test/helpers/browser-bundled-plugin-fixture.ts index 16eccd37299..ce333d44d7d 100644 --- a/test/helpers/browser-bundled-plugin-fixture.ts +++ b/test/helpers/browser-bundled-plugin-fixture.ts @@ -43,7 +43,7 @@ const BROWSER_FIXTURE_ENTRY = `module.exports = { program.command("browser"); }, { commands: ["browser"] }); api.registerGatewayMethod("browser.request", async () => ({ ok: true }), { - scope: "operator.write", + scope: "operator.admin", }); api.registerService({ id: "browser-control", diff --git a/test/helpers/plugins/tts-contract-suites.ts b/test/helpers/plugins/tts-contract-suites.ts index 35990fccda1..342791fb29e 100644 --- a/test/helpers/plugins/tts-contract-suites.ts +++ b/test/helpers/plugins/tts-contract-suites.ts @@ -5,7 +5,7 @@ import { createEmptyPluginRegistry } from "../../../src/plugins/registry-empty.j import { setActivePluginRegistry } from "../../../src/plugins/runtime.js"; import type { SpeechProviderPlugin } from "../../../src/plugins/types.js"; import { resolveWorkspacePackagePublicModuleUrl } from "../../../src/test-utils/bundled-plugin-public-surface.js"; -import { withEnv } from "../../../src/test-utils/env.js"; +import { withEnv, withEnvAsync } from "../../../src/test-utils/env.js"; import type { ResolvedTtsConfig } from "../../../src/tts/tts-types.js"; type TtsRuntimeModule = typeof import("../../../src/tts/tts.js"); @@ -36,6 +36,41 @@ let getResolvedSpeechProviderConfig: TtsRuntimeModule["_test"]["getResolvedSpeec let formatTtsProviderError: TtsRuntimeModule["_test"]["formatTtsProviderError"]; let sanitizeTtsErrorForLog: TtsRuntimeModule["_test"]["sanitizeTtsErrorForLog"]; +const SPEECH_PROVIDER_ENV_KEYS = [ + "ELEVENLABS_API_KEY", + "GEMINI_API_KEY", + "GOOGLE_API_KEY", + "GRADIUM_API_KEY", + "MINIMAX_API_KEY", + "OPENAI_API_KEY", + "VYDRA_API_KEY", + "XAI_API_KEY", + "XI_API_KEY", +] as const; + +function isolatedSpeechProviderEnv( + overrides: Record = {}, +): Record { + return { + ...Object.fromEntries(SPEECH_PROVIDER_ENV_KEYS.map((key) => [key, undefined])), + ...overrides, + }; +} + +function withIsolatedSpeechProviderEnv( + overrides: Record, + fn: () => T, +): T { + return withEnv(isolatedSpeechProviderEnv(overrides), fn); +} + +async function withIsolatedSpeechProviderEnvAsync( + overrides: Record, + fn: () => Promise, +): Promise { + return await withEnvAsync(isolatedSpeechProviderEnv(overrides), fn); +} + vi.mock("@mariozechner/pi-ai", () => { const getApiProvider = vi.fn(() => undefined); return { @@ -670,7 +705,7 @@ export function describeTtsConfigContract() { expected: "microsoft", }, ] as const)("selects provider based on available API keys: $name", (testCase) => { - withEnv(testCase.env, () => { + withIsolatedSpeechProviderEnv(testCase.env, () => { const config = { auto: "off", mode: "final", @@ -693,7 +728,7 @@ export function describeTtsConfigContract() { }); it("passes cfg into auto-selection so model-provider Google keys can configure TTS", () => { - withEnv( + withIsolatedSpeechProviderEnv( { OPENAI_API_KEY: undefined, ELEVENLABS_API_KEY: undefined, @@ -974,133 +1009,137 @@ export function describeTtsProviderRuntimeContract() { describe("fallback readiness errors", () => { it("continues synthesize fallback when primary readiness checks throw", async () => { - const throwingPrimary: SpeechProviderPlugin = { - id: "openai", - label: "OpenAI", - autoSelectOrder: 10, - resolveConfig: () => ({}), - isConfigured: () => { - throw new Error("Authorization: Bearer sk-readiness-throw-token-1234567890\nboom"); - }, - synthesize: async () => { - throw new Error("unexpected synthesize call"); - }, - }; - const fallback: SpeechProviderPlugin = { - id: "microsoft", - label: "Microsoft", - autoSelectOrder: 20, - resolveConfig: () => ({}), - isConfigured: () => true, - synthesize: async () => ({ - audioBuffer: createAudioBuffer(2), - outputFormat: "mp3", - fileExtension: ".mp3", - voiceCompatible: true, - }), - }; - const registry = createEmptyPluginRegistry(); - registry.speechProviders = [ - { pluginId: "openai", provider: throwingPrimary, source: "test" }, - { pluginId: "microsoft", provider: fallback, source: "test" }, - ]; - setActivePluginRegistry(registry); + await withIsolatedSpeechProviderEnvAsync({}, async () => { + const throwingPrimary: SpeechProviderPlugin = { + id: "openai", + label: "OpenAI", + autoSelectOrder: 10, + resolveConfig: () => ({}), + isConfigured: () => { + throw new Error("Authorization: Bearer sk-readiness-throw-token-1234567890\nboom"); + }, + synthesize: async () => { + throw new Error("unexpected synthesize call"); + }, + }; + const fallback: SpeechProviderPlugin = { + id: "microsoft", + label: "Microsoft", + autoSelectOrder: 20, + resolveConfig: () => ({}), + isConfigured: () => true, + synthesize: async () => ({ + audioBuffer: createAudioBuffer(2), + outputFormat: "mp3", + fileExtension: ".mp3", + voiceCompatible: true, + }), + }; + const registry = createEmptyPluginRegistry(); + registry.speechProviders = [ + { pluginId: "openai", provider: throwingPrimary, source: "test" }, + { pluginId: "microsoft", provider: fallback, source: "test" }, + ]; + setActivePluginRegistry(registry); - const result = await ttsRuntime.synthesizeSpeech({ - text: "hello fallback", - cfg: { - messages: { - tts: { - provider: "openai", + const result = await ttsRuntime.synthesizeSpeech({ + text: "hello fallback", + cfg: { + messages: { + tts: { + provider: "openai", + }, }, }, - }, - }); + }); - expect(result.success).toBe(true); - if (!result.success) { - throw new Error("expected fallback synthesis success"); - } - expect(result.provider).toBe("microsoft"); - expect(result.fallbackFrom).toBe("openai"); - expect(result.attemptedProviders).toEqual(["openai", "microsoft"]); - expect(result.attempts?.[0]).toMatchObject({ - provider: "openai", - outcome: "failed", - reasonCode: "provider_error", - }); - expect(result.attempts?.[1]).toMatchObject({ - provider: "microsoft", - outcome: "success", - reasonCode: "success", + expect(result.success).toBe(true); + if (!result.success) { + throw new Error("expected fallback synthesis success"); + } + expect(result.provider).toBe("microsoft"); + expect(result.fallbackFrom).toBe("openai"); + expect(result.attemptedProviders).toEqual(["openai", "microsoft"]); + expect(result.attempts?.[0]).toMatchObject({ + provider: "openai", + outcome: "failed", + reasonCode: "provider_error", + }); + expect(result.attempts?.[1]).toMatchObject({ + provider: "microsoft", + outcome: "success", + reasonCode: "success", + }); }); }); it("continues telephony fallback when primary readiness checks throw", async () => { - const throwingPrimary: SpeechProviderPlugin = { - id: "primary-throws", - label: "PrimaryThrows", - autoSelectOrder: 10, - resolveConfig: () => ({}), - isConfigured: () => { - throw new Error("Authorization: Bearer sk-telephony-throw-token-1234567890\tboom"); - }, - synthesize: async () => { - throw new Error("unexpected synthesize call"); - }, - }; - const fallback: SpeechProviderPlugin = { - id: "microsoft", - label: "Microsoft", - autoSelectOrder: 20, - resolveConfig: () => ({}), - isConfigured: () => true, - synthesize: async () => ({ - audioBuffer: createAudioBuffer(2), - outputFormat: "mp3", - fileExtension: ".mp3", - voiceCompatible: true, - }), - synthesizeTelephony: async () => ({ - audioBuffer: createAudioBuffer(2), - outputFormat: "mp3", - sampleRate: 24000, - }), - }; - const registry = createEmptyPluginRegistry(); - registry.speechProviders = [ - { pluginId: "primary-throws", provider: throwingPrimary, source: "test" }, - { pluginId: "microsoft", provider: fallback, source: "test" }, - ]; - setActivePluginRegistry(registry); + await withIsolatedSpeechProviderEnvAsync({}, async () => { + const throwingPrimary: SpeechProviderPlugin = { + id: "primary-throws", + label: "PrimaryThrows", + autoSelectOrder: 10, + resolveConfig: () => ({}), + isConfigured: () => { + throw new Error("Authorization: Bearer sk-telephony-throw-token-1234567890\tboom"); + }, + synthesize: async () => { + throw new Error("unexpected synthesize call"); + }, + }; + const fallback: SpeechProviderPlugin = { + id: "microsoft", + label: "Microsoft", + autoSelectOrder: 20, + resolveConfig: () => ({}), + isConfigured: () => true, + synthesize: async () => ({ + audioBuffer: createAudioBuffer(2), + outputFormat: "mp3", + fileExtension: ".mp3", + voiceCompatible: true, + }), + synthesizeTelephony: async () => ({ + audioBuffer: createAudioBuffer(2), + outputFormat: "mp3", + sampleRate: 24000, + }), + }; + const registry = createEmptyPluginRegistry(); + registry.speechProviders = [ + { pluginId: "primary-throws", provider: throwingPrimary, source: "test" }, + { pluginId: "microsoft", provider: fallback, source: "test" }, + ]; + setActivePluginRegistry(registry); - const result = await ttsRuntime.textToSpeechTelephony({ - text: "hello telephony fallback", - cfg: { - messages: { - tts: { - provider: "primary-throws", + const result = await ttsRuntime.textToSpeechTelephony({ + text: "hello telephony fallback", + cfg: { + messages: { + tts: { + provider: "primary-throws", + }, }, }, - }, - }); + }); - expect(result.success).toBe(true); - if (!result.success) { - throw new Error("expected telephony fallback success"); - } - expect(result.provider).toBe("microsoft"); - expect(result.fallbackFrom).toBe("primary-throws"); - expect(result.attemptedProviders).toEqual(["primary-throws", "microsoft"]); - expect(result.attempts?.[0]).toMatchObject({ - provider: "primary-throws", - outcome: "failed", - reasonCode: "provider_error", - }); - expect(result.attempts?.[1]).toMatchObject({ - provider: "microsoft", - outcome: "success", - reasonCode: "success", + expect(result.success).toBe(true); + if (!result.success) { + throw new Error("expected telephony fallback success"); + } + expect(result.provider).toBe("microsoft"); + expect(result.fallbackFrom).toBe("primary-throws"); + expect(result.attemptedProviders).toEqual(["primary-throws", "microsoft"]); + expect(result.attempts?.[0]).toMatchObject({ + provider: "primary-throws", + outcome: "failed", + reasonCode: "provider_error", + }); + expect(result.attempts?.[1]).toMatchObject({ + provider: "microsoft", + outcome: "success", + reasonCode: "success", + }); }); });