Commit Graph

82 Commits

Author SHA1 Message Date
Vincent Koc
5e453f3c72 fix(ci): restore repo e2e and beta plugin compatibility (#105287) 2026-07-12 13:43:25 +02:00
Josh Lehman
0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00
Vincent Koc
34be976c6d refactor(plugins): remove status dependency barrel 2026-06-18 01:49:46 +08:00
Peter Steinberger
a6d084113a docs: document plugin install helpers 2026-06-04 04:15:33 -04:00
Vincent Koc
d65b3a68aa perf(cli): keep plugins JSON list on snapshot path 2026-05-30 13:00:31 +02:00
Peter Steinberger
bb46b79d3c refactor: internalize OpenClaw agent runtime (#85341)
* refactor: extract agent core package

Introduce packages/agent-core as the OpenClaw-owned home for reusable agent loop, harness, session, prompt, and runtime dependency contracts.

* refactor: extract shared llm runtime

Move provider model registries, stream wrappers, OAuth helpers, and LLM utilities into src/llm with plugin-sdk barrels instead of depending on the old embedded runtime layout.

* refactor: remove pi runtime internals

Rename remaining Pi-shaped agent surfaces to OpenClaw agent runtime names, delete obsolete Pi docs and package graph checks, and add the third-party notice for incorporated code.

* refactor: tighten agent session runtime

Make agent-core/runtime dependencies explicit, consolidate compaction and session transcript helpers, and move model/session helpers behind OpenClaw-owned contracts.

* refactor: remove static model and pi auth paths

Drop static model catalogs and Pi auth bridges, move model/provider facts to manifest-owned runtime contracts, and harden internal embedded-agent utilities.

* refactor: remove legacy provider compat paths

* docs: remove agent parity notes

* fix: skip provider wildcard metadata parsing

* refactor: share session extension sdk loading

* refactor: inline acpx proxy error formatter

* refactor: fold edit recovery into edit tool

* fix: accept extension batch separator

* test: align startup provider plugin expectations

* fix: restore provider-scoped release discovery

* test: align static asset packaging expectations

* fix: run static provider catalogs during scoped discovery

* fix: add provider entry catalogs for scoped live discovery

* fix: load lightweight provider catalog entries

* fix: refresh provider-scoped plugin metadata

* fix: keep provider catalog entries on release live path

* fix: keep static manifest models in release live checks

* fix: harden release model discovery

* fix: reduce OpenAI live cache probe reasoning

* fix: disable OpenAI cache probe reasoning

* ci: extend OpenAI gateway live timeout

* fix: extend live gateway model budget

* fix: stabilize release validation regressions

* fix: honor provider aliases in model rows

* fix: stabilize release validation lanes

* fix: stabilize release memory qa

* ci: stabilize release validation lanes

* ci: prefer ipv4 for live docker node calls

* fix: restore shared tool-call stream wrapper

* ci: remove legacy pi test shard alias

* fix: clean up embedded agent test drift

* fix: stabilize runtime alias status

* fix: clean up embedded agent ci drift

* fix: restore release ci invariants

* fix: clean up post-rebase runtime drift

* fix: restore release ci checks

* fix: restore release ci after rebase

* fix: remove stale pi runtime path

* test: align compaction runtime expectations

* test: update plugin prerelease expectations

* fix: handle claude live tool approvals

* fix: stabilize release validation gates

* fix: finish agent runtime import

* test: finish post-rebase agent runtime mocks

* fix: keep codex compaction native

* fix: stabilize codex app-server hook tests

* test: isolate codex diagnostic active run

* test: remove codex diagnostic completion race

# Conflicts:
#	extensions/codex/src/app-server/run-attempt.test.ts

* ci: fix full release manifest performance run id

* refactor: narrow llm plugin sdk boundary

* chore: drop generated google boundary stamps

* fix: repair rebase fallout

* fix: clean up rebased runtime references

* fix: decode codex jwt payloads as base64url

* fix: preserve shipped pi runtime alias

* fix: add scoped sdk virtual modules

* fix: decode llm codex oauth jwt as base64url

* fix: avoid stale vertex adc negative cache

* fix: harden tool arg decoding and codeql path

* fix: keep vertex adc negative checks live

* refactor: consolidate codex jwt and edit helpers

* fix: await codex oauth node runtime imports

* fix: preserve sdk tool and notice contracts

* fix: preserve shipped compat config boundaries

* fix: align codex oauth callback host

* fix: terminate agent-core loop streams on failure

* fix: keep codex oauth callback alive during fallback

* ci: include session tools in critical codeql scans

* fix: keep Cloudflare Anthropic provider auth header

* docs: redirect legacy pi runtime pages

* fix: honor bundled web provider compat discovery

* fix: protect session output spill files

* fix: keep legacy agent dir env blocked

* fix: contain auto-discovered skill symlinks

* fix: harden agent core sdk proxy surfaces

* fix: restore approval reaction sdk compat

* fix: keep live docker runs bounded

* fix: keep codex oauth redirect host aligned

* fix: resolve post-rebase agent runtime drift

* fix: redact anthropic oauth parse failures

* fix: preserve responses strict tool shaping

* fix: repair agent runtime rebase cleanup

* docs: redirect retired parity pages

* fix: bound auto-discovered resources to roots

* fix: repair post-rebase agent test drift

* fix: preserve bundled provider allowlist migration

* fix: preserve manifest-owned provider aliases

* fix: declare photon image dependency

* fix: keep provider headers out of proxy body

* fix: preserve shipped env aliases

* fix: refresh control ui i18n generated state

* fix: quote read fallback paths

* fix: preview edits through configured backend

* test: satisfy core test typecheck

* fix: preserve ZAI usage auth fallback

* test: repair codex diagnostic test

* fix: repair agent runtime rebase drift

* test: finish embedded runner import rename

* fix: repair agent runtime rebase integrations

* test: align compaction oauth fallback expectations

* fix: allow sdk-auth session models

* fix: update doctor tool schema import

* fix: preserve bedrock plugin region

* fix: stream harmony-like prose immediately

* ci: include session runtime in codeql shards

* fix: repair latest rebase integrations

* fix: honor explicit codex websocket transport

* fix: keep openai-compatible credentials provider-scoped

* fix: refresh sdk api baseline after rebase

* fix: route cli runtime aliases through openclaw harness

* test: rename stale harness mock expectation

* test: rename embedded agent overflow calls

* test: clean embedded auth test wording

* test: use openclaw stream types in deepinfra cache test

* fix: refresh sdk api baseline on latest main

* fix: honor bundled discovery compat allowlists

* fix: refresh sdk api baseline after latest rebase

* fix: remove stale rebase imports

* test: rename stale model catalog mock

* test: mock renamed doctor runtime modules

* fix: map canonical kimi env auth

* fix: use internal model registry in bench script

* fix: migrate deepinfra provider catalog entry

* fix: enforce builtin tool suppression

* fix: route compaction auth and proxy payloads safely

* refactor: prune unused llm registry leftovers

* test: update codex hooks session import

* test: fix model picker ci coverage

* test: align model picker auth mock types
2026-05-27 19:24:04 +01:00
Mariano
c9d4f7e35c Deprecate memory-specific embedding provider registration (#85072)
Merged via squash.

Prepared head SHA: 661eb99066
Co-authored-by: mbelinky <132747814+mbelinky@users.noreply.github.com>
Co-authored-by: mbelinky <132747814+mbelinky@users.noreply.github.com>
Reviewed-by: @mbelinky
2026-05-27 15:24:17 +02:00
Peter Steinberger
cac0b2db18 refactor: move transcripts into core
Move meeting notes into core transcripts, remove the bundled meeting-notes plugin/API, and require explicit transcripts.enabled before exposing the recording-capable tool.
2026-05-26 14:51:11 +01:00
Peter Steinberger
07694c639d feat: add meeting notes plugin
Adds source-only external meeting notes plugin, SDK source-provider contract, CLI access, date-sharded storage, and Discord voice source.
2026-05-23 10:38:09 +01:00
Bob
ae4806ed9a feat(plugins): add embedding provider contract (#84947)
Summary:
- Merged feat(plugins): add embedding provider contract after ClawSweeper review.

Automerge notes:
- PR branch already contained follow-up commit before automerge: chore(plugins): refresh embedding provider sdk baseline
- PR branch already contained follow-up commit before automerge: docs(plugins): document embedding provider contract
- PR branch already contained follow-up commit before automerge: fix(plugins): restore embedding providers after snapshot loads
- PR branch already contained follow-up commit before automerge: fix(plugins): resolve embedding providers from manifests
- PR branch already contained follow-up commit before automerge: fix(plugin-sdk): keep embedding provider registry mutators internal
- PR branch already contained follow-up commit before automerge: chore(plugin-sdk): refresh embedding provider API baseline

Validation:
- ClawSweeper review passed for head 41ebd66ab4.
- Required merge gates passed before the squash merge.

Prepared head SHA: 41ebd66ab4
Review: https://github.com/openclaw/openclaw/pull/84947#issuecomment-4514762026

Co-authored-by: Bob <dutifulbob@gmail.com>
Co-authored-by: Mariano Belinky <mbelinky@gmail.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: osolmaz
Co-authored-by: osolmaz <2453968+osolmaz@users.noreply.github.com>
2026-05-22 03:36:51 +00:00
Peter Steinberger
93b9223bee fix(plugins): tolerate legacy inspect reports 2026-05-15 12:25:31 +01:00
Peter Steinberger
db3c4ba8d3 refactor(gateway): collapse method metadata shims 2026-05-15 12:25:31 +01:00
Vincent Koc
1d34564de9 fix(plugins): expose hook timeout overrides 2026-05-03 12:21:59 -07:00
Peter Steinberger
4a3ad3963b feat(plugins): report dependency install state 2026-05-02 19:58:19 +01:00
Peter Steinberger
ebb45a8a28 refactor: unify plugin metadata snapshot callers 2026-05-02 07:51:17 +01:00
Peter Steinberger
7ac23eeeb5 refactor: drop legacy implicit startup sidecar fallback 2026-05-01 22:58:18 +01:00
Peter Steinberger
ed8f50f240 refactor: simplify plugin dependency handling
Simplify plugin installation and runtime loading around package-manager-owned dependencies, with Jiti reserved for local/TS fallback paths.

Also scans npm plugin install roots so hoisted transitive dependencies are covered by dependency denylist and node_modules symlink checks.
2026-05-01 21:32:22 +01:00
Peter Steinberger
f6fea7770d fix(release): repair packaged plugin startup metadata 2026-05-01 19:44:28 +01:00
Vincent Koc
575854c096 fix(plugins): reuse cold inspect registry snapshots (#75620)
Summary:
- The PR reuses a request-scoped cold manifest registry/runtime context across plugin status and inspect report paths, threads that context through provider/setup/metadata helpers, adds targeted coverage, and adds a changelog entry.

ClawSweeper fixups:
- Included follow-up commit: fix(plugins): preserve setup auto-enable lookup

Validation:
- ClawSweeper review passed for head 4d8e8e2d24.
- Required merge gates passed before the squash merge.

Prepared head SHA: 4d8e8e2d24
Review: https://github.com/openclaw/openclaw/pull/75620#issuecomment-4359143053

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-05-01 16:00:47 +00:00
Peter Steinberger
250376f885 fix: simplify bundled runtime dependency repair (#75183)
Summary:
- Merged fix: simplify bundled runtime dependency repair after ClawSweeper review.

ClawSweeper fixups:
- Included follow-up commit: fix: verify cached bundled runtime roots
- Included follow-up commit: refactor: simplify plugin runtime startup paths
- Included follow-up commit: refactor: trim plugin startup policy helpers
- Included follow-up commit: refactor: trust package manager runtime deps materialization
- Included follow-up commit: fix: narrow channel runtime deps skip policy
- Included follow-up commit: refactor: defer startup plugin runtime deps
- Ran the ClawSweeper repair loop before final review.

Validation:
- ClawSweeper review passed for head 04dc566534.
- Required merge gates passed before the squash merge.

Prepared head SHA: 04dc566534
Review: https://github.com/openclaw/openclaw/pull/75183#issuecomment-4358383786

Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: Shakker <shakkerdroid@gmail.com>
Co-authored-by: clawsweeper-repair <clawsweeper-repair@users.noreply.github.com>
2026-05-01 07:49:02 +00:00
Shakker
a48ffda7f7 chore: trace plugin lifecycle phases 2026-04-28 18:03:01 +01:00
Shakker
7727e102a5 fix: scope plugin inspect runtime loading 2026-04-28 11:25:09 +01:00
Shakker
d062f8130b feat: warn on implicit startup plugin compatibility 2026-04-28 06:18:45 +01:00
EVA
1adaa28dc8 [plugin sdk] Add generic plugin host-hook contracts (#72287)
Merged via squash.

Prepared head SHA: 68e5f2ce19
Co-authored-by: 100yenadmin <239388517+100yenadmin@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-27 17:07:02 -07:00
Peter Steinberger
7f3f108521 refactor(config): migrate plugin config access 2026-04-27 12:35:58 +01:00
Vincent Koc
1fc5b2b703 feat(migrations): add plugin-owned Hermes import
* feat: add migration providers

* feat: offer Hermes migration during onboarding

* feat(hermes): map imported config surfaces

* feat(onboard): require fresh migration imports

* docs(cli): clarify Hermes import coverage

* chore(migrations): rename Hermes importer package

* chore(migrations): rewire Hermes importer id

* fix(migrations): redact migration JSON details

* fix(hermes): use provider runtime for config imports

* test(hermes): cover missing source planning

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-04-27 00:34:29 -07:00
Vincent Koc
eed7b13b62 fix(doctor): scope bundled runtime deps to active plugins 2026-04-26 18:17:56 -07:00
Shakker
b5e4e2f257 Revert "fix(plugins): persist registry contribution metadata"
This reverts commit 1ee5654220.
2026-04-26 08:11:09 +01:00
Vincent Koc
1ee5654220 fix(plugins): persist registry contribution metadata 2026-04-26 00:03:21 -07:00
Shakker
3ea20d1413 fix: harden cold plugin metadata paths 2026-04-26 05:48:10 +01:00
Shakker
4e3b860e60 refactor: scope plugin capabilities to manifests 2026-04-26 02:31:02 +01:00
Vincent Koc
5859dcd298 feat(plugins): list from registry snapshot 2026-04-25 10:35:34 -07:00
Vincent Koc
86dc820560 feat(plugins): add compatibility registry 2026-04-24 22:15:41 -07:00
Vincent Koc
cb4fc58547 feat(plugins): move Bonjour discovery into bundled plugin
* fix(deps): detect constant dynamic imports in ownership audit

* feat(plugins): move bonjour discovery into bundled plugin

* test(plugins): remove moved bonjour core tests

* fix(plugins): harden bonjour disable and console restore

* fix(plugins): split gateway discovery ids from services

* fix(plugins): harden bonjour advertiser shutdown

* fix(plugins): clean up bonjour split lint
2026-04-23 23:29:51 -07:00
Vincent Koc
51f9f94cc3 fix(hooks): harden cli transcript loading (#70786) 2026-04-23 14:25:27 -07:00
Gustavo Madeira Santana
24db09a19b fix(cli): keep channel status checks off plugin runtimes (#69479)
Merged via squash.

Prepared head SHA: 63f6e416a9
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
2026-04-21 13:53:08 -04:00
Pavan Kumar Gondhi
31160dc069 fix(agents): enforce subagent envelope inheritance on ACP child sessions [AI-assisted] (#69383)
* fix: address issue

* fix: address review feedback

* fix: finalize issue changes

* fix: address PR review feedback

* address build faiure

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback
2026-04-21 17:25:25 +05:30
Peter Steinberger
4fa961d4f1 refactor(lint): enable map spread rule 2026-04-18 20:37:12 +01:00
Peter Steinberger
30cbfa3457 test: slim plugin shape contracts 2026-04-18 01:36:15 +01:00
Brian
143c1e81a2 fix(plugins): treat context-engine plugins as capabilities in status/inspect (#58766)
Merged via squash.

Prepared head SHA: 23269d2db5
Co-authored-by: zhuisDEV <95547369+zhuisDEV@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-13 07:32:24 -07:00
Vincent Koc
74e7b8d47b fix(cycles): bulk extract leaf type surfaces 2026-04-11 13:26:50 +01:00
Peter Steinberger
44ec4d05de feat: add pluggable agent harness registry 2026-04-10 21:22:16 +01:00
Vincent Koc
7bb61a07db fix(check): repair plugin and secret type drift 2026-04-06 15:36:42 +01:00
Peter Steinberger
58f4099a4f refactor: share plugin runtime load context 2026-04-06 15:26:32 +01:00
Peter Steinberger
c39f061003 Revert "refactor(cli): remove bundled cli text providers"
This reverts commit 05d351c430.
2026-04-06 13:40:41 +01:00
Peter Steinberger
8cb85ff85f refactor: harden plugin metadata and bundled channel entry seams 2026-04-06 00:15:38 +01:00
Peter Steinberger
84e76f7cce refactor(cli): remove stale cli provider leftovers 2026-04-05 19:11:34 +01:00
Peter Steinberger
05d351c430 refactor(cli): remove bundled cli text providers 2026-04-05 18:46:36 +01:00
Peter Steinberger
a23ab9b906 refactor: move voice-call realtime providers into extensions 2026-04-04 12:07:23 +09:00
Vincent Koc
bb3f17fc02 refactor(plugins): drop generic status report alias (#59700) 2026-04-02 22:59:25 +09:00