Commit Graph

301 Commits

Author SHA1 Message Date
Josh Lehman
0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00
Peter Steinberger
49b5b862ac feat: node-hosted plugins — dynamic tools, MCP servers, and skills (#90431)
* feat: node-hosted plugins — dynamic tools, MCP servers, and skills

Nodes become declarative plugin hosts:
- node.pluginTools.update: node hosts publish plugin-registered agent tool
  descriptors; gateway materializes them as agent tools executing via
  node.invoke under the node command allowlist, with tools.effective
  invalidation and node online/offline removal.
- Trusted paired-node descriptors: no gateway-side plugin registration
  required; gateway.nodes.pluginTools.enabled off-switch (default on);
  description/count caps; deterministic node-prefixed collision names.
- Declarative node-hosted MCP: nodeHost.mcp.servers (McpServerConfig shape)
  starts MCP clients on the node host, publishes tools as pluginId node-mcp,
  executes via built-in mcp.tools.call.v1 with per-layer timeouts, failure
  isolation, and orphan-safe shutdown. No re-pairing when servers change.
- Node-hosted skills: node.skills.update publishes ~/.openclaw/skills
  content (64 skills/64KB/512KB caps both sides); gateway merges them into
  the skills snapshot while connected and exec host=node is available, with
  node:// locators, node-prefixed collisions, disabled command dispatch,
  and gateway.nodes.skills.enabled + nodeHost.skills.enabled switches.
- Security: node-supplied pluginIds cannot satisfy pluginId-scoped tool
  allowlists unless gateway-registered; reserved node-mcp id requires the
  core MCP descriptor shape; protocol registry kept out of public
  plugin-sdk dts.
- E2E: pond harness proves publication, MCP round-trip, skills locator, and
  disconnect/reconnect for all three surfaces.

* style: format node-plugin-tools test

* fix(skills): keep status loader unfiltered when eligibility is passed

skills.status started passing eligibility for the node-skill merge, which
flipped loadWorkspaceSkillEntries into filtered mode and dropped disabled
skills from status reports (QA plugin-lifecycle-hot-reload timeout). Status
now merges node skills explicitly around an unfiltered load. Also: regen
docs_map for new node docs sections; add the intentional node-host MCP
onclose suppression to the lint-suppression allowlist.
2026-07-11 10:16:34 -07:00
Peter Steinberger
2f9cb92006 fix(tasks): show registry restore errors in console (#103976) 2026-07-11 00:10:14 +01:00
Bek
91ac7ca700 fix(tasks): repair legacy delivery statuses (#103946)
* fix(tasks): repair legacy delivery statuses

* docs(changelog): note task state migration repair

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-10 23:49:22 +01:00
Peter Steinberger
1bcc4c5e70 feat(gateway): add cooperative host suspension (#103618)
* feat(gateway): add cooperative suspension preparation

* style: satisfy suspension lint checks

* test(gateway): reset work admission between shared suites

* fix(gateway): reject upgrades during suspension

* fix(gateway): preserve admitted work during suspension

* test(gateway): isolate suspension and restart state

* fix(gateway): close suspension false-ready gaps

* refactor(protocol): slim suspension declaration graph

* refactor(plugin-sdk): sever protocol registry edges

* fix(gateway): preserve admitted restart follow-ups

* fix(gateway): make suspension recovery fail closed

* fix(protocol): keep validation formatter re-export only

* test(gateway): simplify deferred fixture type

* style(gateway): clarify suspension entry name

* fix(gateway): retain detached work admission
2026-07-10 20:24:53 +01:00
Darren2030
b8f365a82b fix(tasks): skip discarded registry sort (#102202)
Preserve the cloned task snapshot and deterministic activity ordering while avoiding the createdAt sort that tasks.list immediately replaces.

Closes #101716

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-10 03:12:35 +01:00
Peter Lee
cab8040b14 fix(cron): preserve cron context in session entry for async completion wakes (#101078)
* fix(cron): preserve cron context in session entry for async completion wakes (#99919)

Persist bootstrapContextRunKind on the session entry after cron agent
runs, and restore provider/model/thinking/runKind from the session
entry into directAgentParams when an async completion wake (e.g. media
generation) resumes a cron session.

Before this fix, async completion wakes lost the original cron run
context and fell back to account defaults, causing multi-step cron tasks
to silently dead-end after the first async media generation call.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cron): authorize model override on trusted in-process dispatch for cron context restoration

* fix(cron): persist bootstrapContextRunKind on base session entry for async completion wakes

* fix(cron): route async completion wake to base cron session key instead of ephemeral run key

When the cron run-key fallback fires, record the base cron key as
effectiveSessionKey so deliverDirectSubagentAnnounce can route the
agent call to the persisted session row with its transcript and task
context instead of starting a fresh turn (#99919).

* fix(cron): add bootstrapContextRunKind to session entry slot keys and declare loadRequesterSessionEntry return type

* fix(cron): rebase onto upstream main, restore upstream agent-command behavior while keeping bootstrapContextRunKind persistence

* fix(cron): persist bootstrapContextRunKind from a pre-mutation snapshot to produce a real delta

* fix(cron): move pre-mutation snapshot inside guard so initialEntry is non-optional

* fix(cron): preserve async media continuation context

Co-authored-by: Peter Lee <li.xialong@xydigit.com>

* fix(cron): retain continuation after base cleanup

* test(gateway): complete agent scope mock

* test(gateway): preserve agent scope exports

* test(cron): persist mocked session mutations

* test(cron): model persisted session fixtures

* fix(agents): snapshot cron continuation retries

* fix(cron): harden async continuation settlement

* test(cron): control continuation recovery clock

* fix(gateway): narrow continuation model row

* chore(changelog): defer cron continuation note

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-10 03:07:30 +01:00
wm0018
7de325d060 fix: keep task title truncation UTF-16 safe in restart/diagnostic output (#101934)
* fix: keep task title truncation UTF-16 safe in restart/diagnostic output

Three private copies of formatTaskBlocker share the same pattern —
task.title.slice(0, 80) — which can split surrogate pairs in task
titles, producing broken U+FFFD in restart blocker diagnostics,
gateway reload logs, and CLI run-loop output.

Replace raw slice with truncateUtf16Safe in all three locations.

Relevant files:
- gateway/server-reload-handlers.ts:270
- infra/restart-coordinator.ts:73
- cli/gateway-cli/run-loop.ts:569

* refactor(tasks): centralize restart blocker formatting

* fix(tasks): keep restart formatter side-effect free

* refactor(tasks): tighten restart blocker ownership

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-09 07:10:25 +01:00
Vincent Koc
f565138ddc refactor(deadcode): localize core helpers (#101869) 2026-07-07 13:16:57 -07:00
Vincent Koc
b88f80940a refactor: localize internal implementation symbols (#101831) 2026-07-07 10:23:36 -07:00
Peter Steinberger
a9582a1bb6 fix: keep bounded text truncation UTF-16 safe (#101654)
* fix: keep bounded text UTF-16 safe

Co-authored-by: wm0018 <wu.min5@xydigit.com>
Co-authored-by: 廖石荣 0668000909 <liao.shirong@xydigit.com>
Co-authored-by: 0668000787 <ma.weibin@xydigit.com>

* chore: keep UTF-16 release note in PR body

---------

Co-authored-by: wm0018 <wu.min5@xydigit.com>
Co-authored-by: 廖石荣 0668000909 <liao.shirong@xydigit.com>
Co-authored-by: 0668000787 <ma.weibin@xydigit.com>
2026-07-07 13:44:32 +01:00
Peter Steinberger
f53346944d feat: correlate native search outcomes in audit history (#98704)
* feat: correlate native search outcomes in audit history

Metadata-only audit ledger for agent runs and tool actions in the shared
state DB: stable event identity, closed action/status/error vocabularies,
one-way-hashed tool-call ids, never-inferred terminal outcomes for native
web-search (explicit completed/failed only; otherwise unknown), bounded
retention, audit.list gateway RPC and openclaw audit CLI. Squashed from
the 82-commit audit stack for replay onto current main.

* feat(audit): add audit.enabled config gate (default on)

The metadata-only audit ledger records by default: an audit trail enabled
only after an incident cannot explain the incident, and the rows are
strictly less sensitive than the transcripts every install already
stores. audit.enabled=false stops new writes at the gateway subscription
seam; audit.list and openclaw audit keep serving existing records until
they expire. Documented in the configuration reference, protocol page,
and CLI reference.

* fix: repair full-matrix CI findings after rebase

- break the dynamic-tools/dynamic-tool-execution import cycle by
  extracting resolveCodexToolAbortTerminalReason into a leaf module
- restore main's session-worktree protocol exports lost in the
  index.ts auto-merge
- register the audit event writer worker as a knip entry point
- docs table formatting; subagent wait-cancellation test scoped to its
  audit intent (outcome + timing) and advanced past main's new
  lifecycle-timeout retry grace
2026-07-06 12:30:12 +01:00
Vincent Koc
17777b3a9f refactor: remove proven dead code across runtime surfaces (#100823)
* refactor: remove unused internal helpers

* refactor(ui): remove unused compatibility helpers

* refactor(android): remove unused talk cleanup shim

* refactor(android): remove orphaned legacy UI

* refactor: remove unused private exports

* refactor(docs-i18n): remove dead matcher and satisfy lint
2026-07-06 03:31:19 -07:00
Masato Hoshino
153ee2abba fix(subagents): killed subagent runs stay running in the task list (#99806)
* fix(subagents): reconcile killed task outcomes

Co-authored-by: masatohoshino <g515hoshino@gmail.com>

* fix(subagents): retain reconciliation marker narrowing

* fix(subagents): honor generations in latest-run views

* test(subagents): align reconciliation fixtures

* fix(subagents): keep steer recovery best effort

* style(tasks): use type-only runtime contract import

* refactor(subagents): keep generation ordering leaf-only

* fix(subagents): sanitize persisted task owner ids

* fix(subagents): preserve failed replay lifecycle reason

* fix(agents): clear killed lifecycle timeout grace

* fix(tasks): preserve canonical subagent outcomes

* fix(agents): continue requester-wide cancellation

* fix(agents): fence yield revival races

* test(agents): mock detached task lookup

* fix: fence subagent cleanup deletion

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-05 02:17:10 -07:00
Dallin Romney
dc3b5df684 refactor(types): remove redundant local aliases (#100061)
* refactor(models): remove redundant generation model ref alias

* refactor(types): remove redundant local aliases

* refactor(types): preserve shipped model ref type
2026-07-04 16:34:47 -07:00
Dallin Romney
b98c11a46e refactor(shared): consolidate gateway and stateful runtime lazy loaders (#99296) 2026-07-02 18:43:14 -07:00
Ayaan Zaidi
bc7f9a36ec fix(cron): detach session-targeted cron runs (#98755)
* fix(cron): detach session-targeted runs
* fix(tasks): cancel stale childless cron rows

---------

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-07-01 14:04:25 -07:00
Andi Liao
be94853de0 fix(tasks): recover terminal lost cron rows (#86088)
* fix(tasks): recover terminal lost cron rows

* fix(tasks): require backing-session lost recovery signal

* fix(tasks): satisfy has-own lint rule

---------

Co-authored-by: Altay <altay@hey.com>
2026-06-29 15:32:47 +03:00
aliahnaf2013-max
f527def0a9 fix(tasks): harden ACP task cancellation (#97352)
* fix(tasks): harden acp task cancellation

* test(gateway): cover acp task cancellation handler
2026-06-28 12:27:49 -07:00
Vincent Koc
273eed4c51 fix(harness): recover Copilot native subagent tasks 2026-06-23 22:13:59 +08:00
Vincent Koc
0529281430 refactor(sqlite): share numeric column decoding 2026-06-23 03:38:18 +08:00
Vincent Koc
d9a38130b1 chore(deadcode): remove test-only task mutation wrappers 2026-06-22 16:24:57 +08:00
Vincent Koc
1711d0123c chore(deadcode): remove task registry test-only queries 2026-06-22 15:56:48 +08:00
Vincent Koc
5c0b99ae2b chore(deadcode): remove unused task flow retry path 2026-06-21 01:00:42 +08:00
Vincent Koc
d29c3a5d6f refactor(cron): drop duplicate active-job reset alias 2026-06-19 21:15:22 +08:00
Vincent Koc
29e44f5eba refactor(tasks): drop duplicate maintenance stop alias 2026-06-19 20:56:56 +08:00
Vincent Koc
ccdec2e294 test(tasks): pass temp root into registry helper 2026-06-19 08:55:56 +02:00
JC
5697ab810e fix(tasks): deliver ACP completions to bound Discord threads (#89279)
Summary:
- The branch adds a bounded task-registry predicate and tests so successful delegated ACP parent-review comple ... with a Discord channel target and threadId send the parent-review terminal message directly to that thread.
- PR surface: Source +24, Tests +142. Total +166 across 2 files.
- Reproducibility: yes. at source level. Current main queues successful ACP parent-review completions through  ... annel/group owner keys, and the linked canonical issue includes matching Discord thread-bound ACP evidence.

Automerge notes:
- No ClawSweeper repair was needed after automerge opt-in.

Validation:
- ClawSweeper review passed for head 04ad66b23d.
- Required merge gates passed before the squash merge.

Prepared head SHA: 04ad66b23d
Review: https://github.com/openclaw/openclaw/pull/89279#issuecomment-4597994374

Co-authored-by: anyech <anyech@gmail.com>
2026-06-19 06:31:42 +00:00
Vincent Koc
2e27a37791 refactor(runtime): remove unused helper exports 2026-06-17 20:31:20 +08:00
Vincent Koc
0842cb71eb refactor(runtime): hide default constants 2026-06-17 18:20:05 +08:00
Vincent Koc
71645bb8a3 refactor(tasks): trim registry helper exports 2026-06-17 16:37:22 +08:00
Vincent Koc
745b011632 refactor(tasks): hide flow audit internals 2026-06-17 16:34:14 +08:00
liuhao1024
e567986355 perf(tasks): memoize reconcileInspectableTasks for same-tick calls (fixes #73531) (#93607)
Merged via squash.

Prepared head SHA: f010d09db9
Co-authored-by: liuhao1024 <11816344+liuhao1024@users.noreply.github.com>
Co-authored-by: vincentkoc <25068+vincentkoc@users.noreply.github.com>
Reviewed-by: @vincentkoc
2026-06-16 20:31:51 +08:00
Peter Steinberger
1fef20c96b fix(tasks): preserve requester agent attribution 2026-06-15 07:09:01 -07:00
Alix-007
77d6ad6f65 fix(tasks): honor explicit agentId in gateway tasks.list and repair test helper signature
Two code-review findings. (1) gateway taskMatchesAgent fell through to a requester/owner/child session-key scan even when the task had an explicit agentId, so a worker subagent task owned by agent:main:main also matched agentId:main; make explicit task.agentId authoritative and keep the session-key fallback only for legacy records without an agentId, with a gateway tasks.list regression. (2) the cross-agent attribution test passed async (root) to the zero-arg withTaskRegistryTempDir helper (TS2345/TS7006); drop the unused parameter and redundant env assignment.
2026-06-15 07:09:01 -07:00
Alix-007
206ac169e6 fix: attribute spawned task runs to child agent 2026-06-15 07:09:01 -07:00
ghitafilali
b2c5e790b4 fix: restart gateway after isolated cron setup timeout
Restart the gateway after isolated cron setup timeouts and harden stale cron-task finalization around restart/reload boundaries.

Verification: focused cron/tasks/CLI regression suites, gateway filesystem/session regression suite, test typecheck, core lint shards, git diff --check, autoreview, Blacksmith Testbox changed gate tbx_01kv2srgbex71w9ce5rwv2wtr4, and clean GitHub PR checks on 13d06b5d6f.
2026-06-14 18:26:27 +08:00
Shakker
9082233a43 fix: unblock timed cron cancellation 2026-06-09 01:22:29 +01:00
Shakker
93313c95a5 fix: preserve cron timeout terminal state 2026-06-09 01:22:29 +01:00
Shakker
372f85d368 fix: avoid cron cancel runtime cycle 2026-06-09 01:22:29 +01:00
ai-hpc
c3cdd4971b fix(cron): cancel active cron task runs 2026-06-09 01:22:29 +01:00
Shakker
cb17c84410 test: let flow registry helper own state env 2026-06-05 01:51:27 +01:00
Shakker
f57adba400 test: snapshot task executor env 2026-06-05 01:51:27 +01:00
Shakker
9f6ed16a6d test: snapshot task flow maintenance env 2026-06-05 01:51:27 +01:00
Shakker
99a838fac4 test: snapshot task registry store env 2026-06-05 01:51:27 +01:00
Shakker
064182aff8 test: snapshot task flow audit env 2026-06-05 01:51:26 +01:00
Shakker
0f9bb59b73 test: snapshot task owner state env 2026-06-05 01:51:26 +01:00
Vincent Koc
563dac5989 test(core): remove stale unused test bindings 2026-06-04 17:03:40 -07:00
Shakker
840eaf9c19 test: centralize task flow state env 2026-06-04 23:06:33 +01:00
Shakker
4a6dc1b830 test: scope task registry env setup 2026-06-04 21:06:04 +01:00