* refactor(sessions): migrate runtime storage to sqlite
* test(sessions): fix sqlite CI regressions
* test(sessions): align remaining sqlite fixtures
* fix(codex): require sqlite trajectory recorder
* test(sessions): align orphan recovery sqlite fixture
* test(sessions): align sqlite rebase fixtures
* fix(sessions): finish current-main integration of the sqlite flip
Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.
* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs
Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.
* feat(sessions): keep archived transcripts by default with zstd cold storage
Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.
* fix(state): converge agent DB migration lineages and bound database growth
Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.
Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.
* fix(codex): resolve the store path for thread-history import via the SDK
The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.
* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors
A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.
* fix(sessions): complete sqlite conflict translations
* test(sqlite): align hardening checks with maintenance
* test(sessions): inspect compressed transcript archives
* fix(tests): await session seeds and drop unused helpers flagged by CI lint
The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.
* test(sessions): type legacy proof events directly
* test(sessions): align hardening contracts
* perf(sessions): read usage transcript sizes from SQL aggregates
Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.
* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir
Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.
* test(agents): seed harness admission through sqlite
* fix(sqlite): close agent db on pragma setup failure
* fix(doctor): compact and retrofit incremental auto-vacuum after session import
The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat: add metadata-only message audit events
* chore(protocol): restore generated swift models and config baseline after rebase
* fix(state): gate agent-db ownership check before schema version
* fix(cli): sync audit command description into the root-help catalog
* fix(audit): require destination proof before classifying outbound messages as direct
* refactor(audit): drop dead migration guard and add contract comments
* docs(gateway): add dedicated audit history page and cross-links
* test(e2e): enable direct-mode message audit in the telegram proof SUT config
* test(channels): expect declared conversationKind in durable delivery session context
* fix(audit): record the routing channel id for inbound rows from plugin channels
* fix(audit): match channel-prefixed delivery targets in the destination route gate
* fix(audit): validate explicit target kind against the destination route kind
* fix(audit): use the canonical target-prefix grammar in the destination route gate and fail closed on foreign migration tables
* fix(audit): normalize nested provider and kind target prefixes in the destination gate
* fix(audit): strip registered provider aliases and the direct kind prefix in the destination gate
* chore(docs): refresh config baseline after rebase
* feat(sessions): durable session state events with parent invalidation
Parents no longer act on stale child-session state: a durable, typed
signal log (session_state_events + per-agent heads) records direct human
messages to children, child spawn/terminal outcomes, goal changes, and
compaction at the seams where those facts are created. A frozen-watermark
cursor protocol (session_watch_cursors) delivers one coalesced system
notice to the parent through the existing system-event + heartbeat wake
idiom, acknowledged at the shared generic drain, with a deferred startup
sweep for restart recovery. session_status gains stateVersion and
changesSince (with exact pruned-history gap signaling); sessions_list
rows gain stateVersion.
Closes#104565
* chore: regenerate docs map and prompt snapshots for session_status changesSince
* feat: node-hosted plugins — dynamic tools, MCP servers, and skills
Nodes become declarative plugin hosts:
- node.pluginTools.update: node hosts publish plugin-registered agent tool
descriptors; gateway materializes them as agent tools executing via
node.invoke under the node command allowlist, with tools.effective
invalidation and node online/offline removal.
- Trusted paired-node descriptors: no gateway-side plugin registration
required; gateway.nodes.pluginTools.enabled off-switch (default on);
description/count caps; deterministic node-prefixed collision names.
- Declarative node-hosted MCP: nodeHost.mcp.servers (McpServerConfig shape)
starts MCP clients on the node host, publishes tools as pluginId node-mcp,
executes via built-in mcp.tools.call.v1 with per-layer timeouts, failure
isolation, and orphan-safe shutdown. No re-pairing when servers change.
- Node-hosted skills: node.skills.update publishes ~/.openclaw/skills
content (64 skills/64KB/512KB caps both sides); gateway merges them into
the skills snapshot while connected and exec host=node is available, with
node:// locators, node-prefixed collisions, disabled command dispatch,
and gateway.nodes.skills.enabled + nodeHost.skills.enabled switches.
- Security: node-supplied pluginIds cannot satisfy pluginId-scoped tool
allowlists unless gateway-registered; reserved node-mcp id requires the
core MCP descriptor shape; protocol registry kept out of public
plugin-sdk dts.
- E2E: pond harness proves publication, MCP round-trip, skills locator, and
disconnect/reconnect for all three surfaces.
* style: format node-plugin-tools test
* fix(skills): keep status loader unfiltered when eligibility is passed
skills.status started passing eligibility for the node-skill merge, which
flipped loadWorkspaceSkillEntries into filtered mode and dropped disabled
skills from status reports (QA plugin-lifecycle-hot-reload timeout). Status
now merges node skills explicitly around an unfiltered load. Also: regen
docs_map for new node docs sections; add the intentional node-host MCP
onclose suppression to the lint-suppression allowlist.
One RUN_STALE_TAKEOVER_MS constant and one resolveRunStaleThresholdMs in
diagnostic-run-activity.ts replace the hand-paired constants and duplicated
blocked-tool-floor derivations in the reply and embedded run registries.
Thresholds are byte-identical before and after; zero behavior change.
Part of #104219 (seam 4).
* feat(agents): derive a provider-declared default utility model when unset
When agents.defaults.utilityModel is not set, utility tasks (titles, progress
narration) now use the primary provider's declared small model
(modelCatalog.providers.<id>.defaultUtilityModel: OpenAI -> gpt-5.6-luna,
Anthropic -> claude-haiku-4-5). Auth is inherent because derivation follows
the agent's primary provider. Setting utilityModel to an empty string
disables utility routing entirely; narration turns on automatically when a
default resolves and stays off otherwise.
Formatting/lint/tests verified on Testbox (oxfmt, oxlint, plugins:inventory,
docs:map, import-cycles, check:test-types, 4 test files).
* fix(ai): drop the temperature parameter for models that reject it
The GPT-5.6 family 400s on temperature via the Responses API (live-verified:
gpt-5.6-luna/-terra reject it; gpt-5.5 and gpt-5.4-mini/nano accept it).
supportsOpenAITemperature gates all three OpenAI payload builders, with a
catalog compat override (compat.supportsTemperature) declared for the 5.6
family in the openai manifest. Without this, utility tasks (titles,
narration) would fail once gpt-5.6-luna becomes the derived default.
Gates on Testbox: oxfmt, oxlint, plugins:inventory, import-cycles,
check:test-types, 6 test files. Live proof on Testbox: gpt-5.6-luna and
claude-haiku-4-5 one-shot completions succeed with temperature requested.
* fix(agents): carry the primary model's auth profile onto the derived utility default
A primary like openai/gpt-5.5@work previously reached utility tasks via the
profiled fallback; the derived default shares the provider, so its trailing
auth profile carries over instead of silently switching to default
credentials (Codex review). Gates on Testbox: oxfmt, check:test-types, tests.
* docs: realign the manifest provider-fields table after adding defaultUtilityModel
* feat(ui): add plugin catalog management
* feat(gateway): add plugins.uninstall and richer plugin catalog metadata
Adds a plugins.uninstall gateway method (operator.admin, control-plane write)
backed by a lock-guarded uninstallManagedPlugin that mirrors the CLI flow:
config cleanup, install-record removal, managed file deletion, and registry
refresh. Bundled plugins stay disable-only. Catalog entries now carry a
manifest-derived category and a removable flag; ClawHub search results expose
download counts and verification tiers.
* feat(ui): redesign plugins page with inventory, store shelves, and cover art
Rebuilds /settings/plugins around three tabs: Installed (category-grouped
inventory with overview stats, state filters, uninstall for external plugins,
and inline MCP server management through the shared config seam), Discover
(featured/official shelves plus one-click MCP connectors and curated ClawHub
searches), and ClawHub (search with download counts and verification badges).
Every catalog entry renders bundled cover art or a deterministic gradient
monogram tile - no more empty boxes. Artwork generated with Codex CLI, shipped
as 512px WebP under ui/public/plugin-art.
* chore(ui): regenerate locale bundles for plugins manager strings
* docs: describe plugins manager tabs, uninstall, and MCP connectors
* fix(plugins): human catalog labels and un-pinned hosted fallback ids
listManagedPlugins now prefers manifest names over registry package-name
backfill, falls back to channel catalog labels and blurbs, and stops pinning
expectedPluginId when a hosted feed entry only exposes its package name
(which rejected every legitimate install of that package). Found via live
gateway testing against ClawHub.
* fix(ui): send minimal RFC 7396 merge patches for MCP server edits
config.patch merges rather than replaces, so key removal needs an explicit
null; sending the full config back made MCP server removal a no-op. Found
via live gateway testing.
* fix(ui): write explicit MCP transports for URL servers
The MCP runtime defaults URL-only servers to SSE, so streamable HTTP
endpoints saved by the add form or connector templates would fail at
connect time. Connector templates now declare their transport and the
add form infers streamable-http unless the URL follows the /sse
convention. Flagged by autoreview against the transport resolver.
* test(ui): wait for deferred plugin requests before resolving in e2e
* feat(ui): plugins detail view, action menus, and unified ClawHub search
Reworks the plugins page from PR #103176 feedback: merges the ClawHub tab into
Discover (typing searches ClawHub inline and appends a quiet From ClawHub
section, with Browse ClawHub demoted to a header text link), makes every row and
store card open a plugin detail overlay (hero art, primary enable/install
action, metadata table), and replaces enable/disable switches with a state chip
plus an overflow menu (Enable/Disable, Remove for external plugins, View
details) matching the ChatGPT-store install+menu pattern. MCP rows use the same
menu; refresh is now icon-only.
* chore(ui): regenerate locale bundles for plugins UI iteration
* feat(ui): vetted, grouped connector catalog for the plugins store
Expands Connect your world to 28 connectors organized into use-case shelves
(Work & productivity, Coding & infrastructure, Home & media, Everyday life).
Every entry passed a three-stage subagent review: official-docs verification
plus live endpoint probes for MCP servers, ClawHub result-quality and
malware/typosquat screening for curated searches, and an adversarial pass
that dynamically registered OAuth clients to prove one-click viability.
That review removed Figma (registration allowlisted, 403) and Atlassian
(OAuth issuer-mismatch bug upstream), downgraded GitHub to PAT-based setup
(no dynamic client registration upstream), fixed Linear (/sse retired) and
Home Assistant (/api/mcp, streamable HTTP) endpoints, retargeted poisoned or
dead searches (youtube, finance, hue dropped; calendar -> google calendar;
stocks replaces finance), and added Todoist, Airtable, Canva, Stripe,
Context7, DeepWiki, Hugging Face one-click MCP servers plus Jira, PDF,
transcription, Kubernetes, Reddit, maps, translation, and notes searches.
Keyless servers get a ready-to-use success message; new cover art included.
* chore(ui): regenerate locale bundles for connector groups
* fix(plugins): suppress hosted catalog rows once their package is installed
Hosted feed entries without a declared runtime id fall back to their package
name as catalog id, which never matches the installed runtime id, so the
Discover shelf kept offering an already-installed package. Installed package
names now also suppress official rows. Flagged by autoreview.
* fix(plugins): pin declared runtime ids and surface connector errors in place
The runtime-id pin now keys off explicitly declared catalog ids (plugin,
channel, or provider) instead of string-comparing against the package name,
so declared ids that equal their package name stay enforced while entry-id
fallbacks stay unpinned. Connector add failures on Discover now render on the
triggering card instead of the Installed tab's MCP section. Both flagged by
autoreview; regression tests included.
* feat(ui): full inventory artwork, pulse header, and two-column plugin list
Every bundled plugin now ships distinctive cover art (113 new Codex CLI
illustrations; 172 total, ~2.1MB WebP), so inventory rows and detail views
never fall back to monogram tiles. The four stat cards give way to a compact
inventory pulse: a segmented enabled/disabled/issues meter whose legend and
counts live inside the filter chips. Inventory, MCP, and search rows flow
into two columns when the panel is wide enough.
* chore(ui): regenerate locale bundles for pulse header
* fix(ui): omit stdio args from the MCP server row target
Stdio MCP args routinely carry tokens, and the inventory is visible to
read-only operators; mirror the config page and show only the command.
Flagged by autoreview; regression test included.
* fix(merge): point crestodian setup at relocated plugin commit/refresh modules
* fix(merge): add bootstrapToken to plugins page test gateway harness
* fix(plugins): name catalog install-action branches so Swift emits the union
* fix(ui): satisfy strict lint on plugins page form parsing and mocks
* chore(build): regen docs map, raise plugin-sdk declaration budget for new protocol surface
* fix(ui): type the plugins page patch mock with its real call signature
In messages.visibleReplies "message_tool" sessions, a successful agent turn that produced a substantive private final without calling message(action=send) previously left the user with silence and only an operator log. The gateway now enqueues one protected front-of-queue retry prompting the model to deliver the reply, and falls back to a sanitized visible diagnostic when the retry cannot be enqueued or also strands. Queue overflow protection is unified with the in-flight-aware drop policy (skip in-flight or protected items, reject when nothing is droppable), rejected overflow no longer refreshes the drain debounce, heartbeat turns are excluded from recovery, and recovery retries no longer share the client turn's queued-turn lifecycle.
Fixes#85714
Thanks to Eva (@100yenadmin) for the contribution.
* feat(channels): narrated progress drafts + activity receipt on the final answer
Progress mode replaces raw tool lines with short utility-model narration of
what the agent is doing (streaming.progress.narration, default on, requires
an explicit utilityModel). On Discord the final answer now carries the
-# activity receipt and the working draft is deleted once the answer lands,
so busy channels keep no orphaned tool log above the reply.
Formatting verified remotely via Testbox oxfmt --check (hook bypassed:
no node_modules in this worktree).
* fix(channels): keep narration toggle independent of channel-default stream mode
Discord resolves its own progress default, so the resolver must not re-derive
mode with the generic partial fallback (narration was off for unset config).
* fix(auto-reply): honor status-only command text in narration model input
streaming.progress.commandText: "status" hides raw exec/bash text from the
channel draft; narration input now mirrors that policy so the utility model
never receives more command detail than the draft shows (Codex review P2).
Formatting verified remotely via Testbox oxfmt --check.
* fix(auto-reply): share the draft's command-tool set for narration and regen channel metadata
Reuse isCommandToolName (exec|shell|bash) so narration's commandText policy
matches the draft formatter exactly, and regenerate bundled channel config
metadata for the new streaming.progress.narration key (Codex review round 2).
Gates verified on Testbox: config:channels:check, oxfmt --check, 4 test shards.
* fix(channels): clear stale narration when the narrator stops mid-turn
An empty narration update now falls the draft back to raw tool lines, and the
narrator emits that clear when it disables after consecutive failures or the
per-turn cap, so drafts never pin stale status text (Codex review round 3).
Gates verified on Testbox: oxfmt --check + 4 test shards.
* chore(config): regen bundled channel metadata after rebase onto main
* chore: CI fixups — lint nits, test harness types, docs map, SDK surface budget
Two deliberate public SDK additions (resolveChannelStreamingProgressNarration,
isCommandToolName via the streaming wildcard re-export) bump the pinned
public-surface budgets to current counts (exports 10488, callable 5235).
Gates verified on Testbox: oxfmt, targeted oxlint, docs:map:check,
config:channels:check, check:test-types, 5 test shards.
* fix(agents): keep model fallback turn-local instead of persisting over user pins
* fix(telegram): use live config snapshots per operation
Co-authored-by: Ayaan Zaidi <hi@obviy.us>
* test(telegram): fix config snapshot type coverage
---------
Co-authored-by: Ayaan Zaidi <hi@obviy.us>
* fix(mcp): redact secrets in /mcp show replies
Prevent owner /mcp show from dumping raw headers/env credentials into
group chats. Mark MCP env values sensitive, restore redaction sentinels
on set so show→set cannot overwrite real secrets.
Fixes#103053
* test(mcp): e2e redaction of /mcp show secrets on live gateway
Spin up an ephemeral gateway process, chat.send /mcp show with live
credential-bearing config, and prove secrets never leave the reply path.
Also prove show→set sentinel restore on disk via chat and CLI.
* fix(queue): prevent applyQueueDropPolicy from selecting in-flight items as overflow victims
When a burst of inbound messages hits the followup queue cap while the
head item is mid-delivery, applyQueueDropPolicy can select that same
in-flight item as an overflow victim. With the default
dropPolicy: "summarize", the in-flight item ends up recorded in the
overflow summary as dropped even though it is still being delivered,
producing a contradictory record where the same message is both
answered and reported as unanswered-due-to-overflow.
The fix introduces an optional `inFlight` Set parameter to
applyQueueDropPolicy and drainNextQueueItem. The followup queue state
now owns a shared inFlight set that is:
- Populated by drainNextQueueItem during the await run(next) window
- Populated by the collect-merge drain path for activeGroupItems
- Passed to applyQueueDropPolicy in enqueueFollowupRun
The drop policy now computes an effective queue length that excludes
in-flight items, and skips them when selecting splice victims.
Fixes#103246
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(queue): align pending depth with active deliveries
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(usage-bar): cap warnedTemplateOverrides warn-once dedupe cache
Replace unbounded warnedTemplateOverrides Set with createDedupeCache(maxSize=256)
for consistency with the bounded fileCache Map (MAX_CACHED_TEMPLATE_FILES=64)
already present in the same file.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(usage-bar): bound invalid-template warnings
Co-authored-by: ZengWen-DT <ceng.wen@xydigit.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(agent-runner-memory): use truncateUtf16Safe for memory flush error truncation
Replace raw UTF-16 slices in buildMemoryFlushErrorPayload and
truncateMemoryFlushErrorMessage with truncateUtf16Safe to prevent
surrogate pair splitting in error messages shown to users.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* test(auto-reply): cover UTF-16-safe memory errors
* test(auto-reply): tighten UTF-16 memory error coverage
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* test: shorten loopback token fixtures below secret-scanner thresholds
* feat: carry gateway client capabilities through the CLI loopback backend
CLI-backed model runs now transport the originating client's declared
capabilities along the existing per-field loopback contract: RunCliAgentParams
gains clientCaps, prepare emits OPENCLAW_MCP_CLIENT_CAPS, the loopback header
template forwards x-openclaw-client-caps, the request context parses and
normalizes it (grant-authenticated callers keep ignoring spoofable headers),
and the loopback tool cache keys on a stable caps serialization so capless and
capped requests never share tool lists. Capability-gated tools such as
show_widget now work on CLI backends; the CLI session binding hash includes
caps, consistent with messageProvider. Cron and command-attempt runs stay
capless (fail closed).
Fixes#102577
Adds client-capability-gated tool availability: gateway clients declare
capabilities at connect (new inline-widgets cap), chat.send stamps them into
the run context, and every tool assembly path (embedded runner, queued
followups, Codex app-server harness, plugin-only construction plans) drops
tools whose requiredClientCaps the originating client did not declare. The
Canvas plugin ships the first such tool, show_widget: agents pass SVG or an
HTML fragment plus a title; the plugin hosts it as a bounded, retention-scoped
Canvas document and returns the existing canvas preview handle, which web chat
renders as a sandboxed iframe fitted to the widget's reported content height.
Widget frames never get allow-same-origin (per-preview sandbox ceiling,
including the sidebar path) and the Canvas host serves widget documents with a
CSP sandbox header so direct navigation runs in an opaque origin. Verified
live end-to-end on a Testbox with gpt-5.5 (screenshots on the PR). CLI-backed
model backends do not carry client caps yet and stay fail-closed (#102577).
Closes#101790
Session-scoped tool-result projection freeze (shrink-only, stable
fallback identities for ambiguous tool ids) and a sent-watermark so
late-resolving inbound media appends a new user turn instead of
rewriting an already-sent slot. Stops Anthropic prompt-cache tail
invalidation on busy/media-heavy embedded sessions.
