Commit Graph

36 Commits

Author SHA1 Message Date
Peter Steinberger
bb46b79d3c refactor: internalize OpenClaw agent runtime (#85341)
* refactor: extract agent core package

Introduce packages/agent-core as the OpenClaw-owned home for reusable agent loop, harness, session, prompt, and runtime dependency contracts.

* refactor: extract shared llm runtime

Move provider model registries, stream wrappers, OAuth helpers, and LLM utilities into src/llm with plugin-sdk barrels instead of depending on the old embedded runtime layout.

* refactor: remove pi runtime internals

Rename remaining Pi-shaped agent surfaces to OpenClaw agent runtime names, delete obsolete Pi docs and package graph checks, and add the third-party notice for incorporated code.

* refactor: tighten agent session runtime

Make agent-core/runtime dependencies explicit, consolidate compaction and session transcript helpers, and move model/session helpers behind OpenClaw-owned contracts.

* refactor: remove static model and pi auth paths

Drop static model catalogs and Pi auth bridges, move model/provider facts to manifest-owned runtime contracts, and harden internal embedded-agent utilities.

* refactor: remove legacy provider compat paths

* docs: remove agent parity notes

* fix: skip provider wildcard metadata parsing

* refactor: share session extension sdk loading

* refactor: inline acpx proxy error formatter

* refactor: fold edit recovery into edit tool

* fix: accept extension batch separator

* test: align startup provider plugin expectations

* fix: restore provider-scoped release discovery

* test: align static asset packaging expectations

* fix: run static provider catalogs during scoped discovery

* fix: add provider entry catalogs for scoped live discovery

* fix: load lightweight provider catalog entries

* fix: refresh provider-scoped plugin metadata

* fix: keep provider catalog entries on release live path

* fix: keep static manifest models in release live checks

* fix: harden release model discovery

* fix: reduce OpenAI live cache probe reasoning

* fix: disable OpenAI cache probe reasoning

* ci: extend OpenAI gateway live timeout

* fix: extend live gateway model budget

* fix: stabilize release validation regressions

* fix: honor provider aliases in model rows

* fix: stabilize release validation lanes

* fix: stabilize release memory qa

* ci: stabilize release validation lanes

* ci: prefer ipv4 for live docker node calls

* fix: restore shared tool-call stream wrapper

* ci: remove legacy pi test shard alias

* fix: clean up embedded agent test drift

* fix: stabilize runtime alias status

* fix: clean up embedded agent ci drift

* fix: restore release ci invariants

* fix: clean up post-rebase runtime drift

* fix: restore release ci checks

* fix: restore release ci after rebase

* fix: remove stale pi runtime path

* test: align compaction runtime expectations

* test: update plugin prerelease expectations

* fix: handle claude live tool approvals

* fix: stabilize release validation gates

* fix: finish agent runtime import

* test: finish post-rebase agent runtime mocks

* fix: keep codex compaction native

* fix: stabilize codex app-server hook tests

* test: isolate codex diagnostic active run

* test: remove codex diagnostic completion race

# Conflicts:
#	extensions/codex/src/app-server/run-attempt.test.ts

* ci: fix full release manifest performance run id

* refactor: narrow llm plugin sdk boundary

* chore: drop generated google boundary stamps

* fix: repair rebase fallout

* fix: clean up rebased runtime references

* fix: decode codex jwt payloads as base64url

* fix: preserve shipped pi runtime alias

* fix: add scoped sdk virtual modules

* fix: decode llm codex oauth jwt as base64url

* fix: avoid stale vertex adc negative cache

* fix: harden tool arg decoding and codeql path

* fix: keep vertex adc negative checks live

* refactor: consolidate codex jwt and edit helpers

* fix: await codex oauth node runtime imports

* fix: preserve sdk tool and notice contracts

* fix: preserve shipped compat config boundaries

* fix: align codex oauth callback host

* fix: terminate agent-core loop streams on failure

* fix: keep codex oauth callback alive during fallback

* ci: include session tools in critical codeql scans

* fix: keep Cloudflare Anthropic provider auth header

* docs: redirect legacy pi runtime pages

* fix: honor bundled web provider compat discovery

* fix: protect session output spill files

* fix: keep legacy agent dir env blocked

* fix: contain auto-discovered skill symlinks

* fix: harden agent core sdk proxy surfaces

* fix: restore approval reaction sdk compat

* fix: keep live docker runs bounded

* fix: keep codex oauth redirect host aligned

* fix: resolve post-rebase agent runtime drift

* fix: redact anthropic oauth parse failures

* fix: preserve responses strict tool shaping

* fix: repair agent runtime rebase cleanup

* docs: redirect retired parity pages

* fix: bound auto-discovered resources to roots

* fix: repair post-rebase agent test drift

* fix: preserve bundled provider allowlist migration

* fix: preserve manifest-owned provider aliases

* fix: declare photon image dependency

* fix: keep provider headers out of proxy body

* fix: preserve shipped env aliases

* fix: refresh control ui i18n generated state

* fix: quote read fallback paths

* fix: preview edits through configured backend

* test: satisfy core test typecheck

* fix: preserve ZAI usage auth fallback

* test: repair codex diagnostic test

* fix: repair agent runtime rebase drift

* test: finish embedded runner import rename

* fix: repair agent runtime rebase integrations

* test: align compaction oauth fallback expectations

* fix: allow sdk-auth session models

* fix: update doctor tool schema import

* fix: preserve bedrock plugin region

* fix: stream harmony-like prose immediately

* ci: include session runtime in codeql shards

* fix: repair latest rebase integrations

* fix: honor explicit codex websocket transport

* fix: keep openai-compatible credentials provider-scoped

* fix: refresh sdk api baseline after rebase

* fix: route cli runtime aliases through openclaw harness

* test: rename stale harness mock expectation

* test: rename embedded agent overflow calls

* test: clean embedded auth test wording

* test: use openclaw stream types in deepinfra cache test

* fix: refresh sdk api baseline on latest main

* fix: honor bundled discovery compat allowlists

* fix: refresh sdk api baseline after latest rebase

* fix: remove stale rebase imports

* test: rename stale model catalog mock

* test: mock renamed doctor runtime modules

* fix: map canonical kimi env auth

* fix: use internal model registry in bench script

* fix: migrate deepinfra provider catalog entry

* fix: enforce builtin tool suppression

* fix: route compaction auth and proxy payloads safely

* refactor: prune unused llm registry leftovers

* test: update codex hooks session import

* test: fix model picker ci coverage

* test: align model picker auth mock types
2026-05-27 19:24:04 +01:00
Peter Steinberger
77d9ac30bb refactor: reuse shared coercion helpers (#86419)
* refactor: share talk event metric extraction

* refactor: reuse shared coercion helpers

* refactor: reuse shared primitive guards

* refactor: reuse shared record guard

* refactor: reuse shared primitive helpers

* refactor: reuse shared string guards

* refactor: reuse shared non-empty string guard

* refactor: share plugin primitive coercion helpers

* refactor: reuse plugin coercion helpers

* refactor: reuse plugin coercion helpers in more plugins

* refactor: reuse channel coercion helpers

* refactor: reuse monitor coercion helpers

* refactor: reuse provider coercion helpers

* refactor: reuse core coercion helpers

* refactor: reuse runtime coercion helpers

* refactor: reuse helper coercion in codex paths

* refactor: reuse helper coercion in runtime paths

* refactor: reuse codex app-server coercion helpers

* refactor: reuse codex record helpers

* refactor: reuse migration and qa record helpers

* refactor: reuse feishu and core helper guards

* refactor: reuse browser and policy coercion helpers

* refactor: reuse memory wiki record helper

* refactor: share boolean coercion helpers

* refactor: reuse finite number coercion

* refactor: reuse trimmed string list helpers

* refactor: reuse string list normalization

* refactor: reuse remaining string list helpers

* refactor: reuse string entry normalizer

* refactor: share sorted string helpers

* refactor: share string list normalization

* test: preserve command registry browser imports

* refactor: reuse trimmed list helpers

* refactor: reuse string dedupe helpers

* refactor: reuse local dedupe helpers

* refactor: reuse more string dedupe helpers

* refactor: reuse command string dedupe helpers

* refactor: dedupe memory path lists with helper

* refactor: expose string dedupe helpers to plugins

* refactor: reuse core string dedupe helpers

* refactor: reuse shared unique value helpers

* refactor: reuse unique helpers in agent utilities

* refactor: reuse unique helpers in config plumbing

* refactor: reuse unique helpers in extensions

* refactor: reuse unique helpers in core utilities

* refactor: reuse unique helpers in qa plugins

* refactor: reuse unique helpers in memory plugins

* refactor: reuse unique helpers in channel plugins

* refactor: reuse unique helpers in core tails

* refactor: reuse unique helper in comfy workflow

* refactor: reuse unique helpers in test utilities

* refactor: expose unique value helper to plugins

* refactor: reuse unique helpers for numeric lists

* refactor: replace index dedupe filters

* refactor: reuse string entry normalization

* refactor: reuse string normalization in plugin helpers

* refactor: reuse string normalization in extension helpers

* refactor: reuse string normalization in channel parsers

* refactor: reuse string normalization in memory search

* refactor: reuse string normalization in provider parsers

* refactor: reuse string normalization in qa helpers

* refactor: reuse string normalization in infra parsers

* refactor: reuse string normalization in messaging parsers

* refactor: reuse string normalization in core parsers

* refactor: reuse string normalization in extension parsers

* refactor: reuse string normalization in remaining parsers

* refactor: reuse string normalization in final parser spots

* refactor: reuse string normalization in qa media helpers

* refactor: reuse normalization in provider and media lists

* refactor: reuse normalization for remaining set filters

* refactor: reuse normalization in policy allowlists

* refactor: reuse normalization in session and owner lists

* refactor: centralize primitive string lists

* refactor: reuse lowercase entry helpers

* refactor: reuse sorted string helpers

* refactor: reuse unique trimmed helpers

* refactor: reuse string normalization helpers

* refactor: reuse catalog string helpers

* refactor: reuse remaining string helpers

* refactor: simplify remaining list normalization

* refactor: reuse codex auth order normalization

* chore: refresh plugin sdk api baseline

* fix: make shared string sorting deterministic

* chore: refresh plugin sdk api baseline

* fix: align host env security ordering
2026-05-25 21:20:41 +01:00
Dallin Romney
8ccb11cbfc perf(plugins,gateway): thread metadata snapshot + discovery through hot paths + plugin owner fixes (#84649)
* perf(plugins): thread metadata snapshot and discovery through hot paths

With the snapshot memo now actually hitting, route the snapshot's
manifestRegistry and discovery through the helper chains that already
had fast paths for them. Eliminates redundant per-call rebuilds at
two big amplifiers.

- Provider resolve paths (resolvePluginProviders /
  isPluginProvidersLoadInFlight / resolveOwningPluginIdsForProvider /
  resolveExternalAuthProfilesWithPlugins) self-service a snapshot once
  at the public entry, then thread it as a separate required arg
  through resolvePluginProviderLoadBase,
  resolveExplicitProviderOwnerPluginIds, and the setup/runtime load
  state helpers. Inner reads change from
  'params.pluginMetadataSnapshot?.x' to 'snapshot.x', no more
  enrichedParams clone. loadPluginManifestRegistryForInstalledIndex
  fires drop ~685 -> ~10 per cold start.

- Bundled-channel / auto-enable chain accepts an optional
  PluginDiscoveryResult. discoverOpenClawPlugins is fired once during
  snapshot building (resolveInstalledPluginIndexRegistry already
  produced it internally; now bubbled up through
  loadInstalledPluginIndexWithDiscovery, PluginRegistrySnapshotResult,
  and onto PluginMetadataSnapshot.discovery). load-context reads
  metadataSnapshot.discovery and passes it through
  applyPluginAutoEnable, so the bundled-channel cascade
  (collectConfiguredChannelIds, listBundledChannelIdsWith*,
  listPotentialConfiguredChannelPresenceSignals) short-circuits
  instead of each leaf re-firing discovery. Persisted-cache path is
  unchanged: no discovery on the snapshot, downstream chain handles
  its own fallback (pre-PR behavior on that path).

* test(plugins): isolate snapshot memo across tests that mock manifest registry

The snapshot memo is now process-scoped and effective (~98% hit rate).
Three test files were depending on cache misses (because the broken
cache returned them) — each test would set up its own
loadPluginManifestRegistry mock and expect a fresh derive. With the
cache fixed, an earlier test's mocked registry now leaks into later
tests in the same file.

- io.write-config.test.ts: afterEach now clears the snapshot memo so
  the 'demo' plugin mocked in the first test does not survive into
  'keeps shipped plugin install config records when index migration
  fails', which expects an empty registry to surface the 'plugin not
  found: demo' warning.

- gateway/model-pricing-cache.ts: resetGatewayModelPricingCacheForTest
  also clears the memo. Tests in model-pricing-cache.test.ts assert
  loadPluginManifestRegistryForInstalledIndex was called; the memo
  hit otherwise skips the call.

- providers.test.ts: vi.doMock loadPluginMetadataSnapshot to wrap the
  existing loadPluginManifestRegistryMock fixture. The plumbing
  commit added an auto-fetch fall-through in
  resolveOwningPluginIdsForProvider; without the mock, providers
  tests hit real disk reads and return empty registries (which is
  what surfaced as 9 unrelated-looking failures in the prior CI
  run).

* fix(plugins): preserve setup.cliBackends owner matching in provider scan

resolveOwningPluginIdsForProvider now also checks plugin.setup?.cliBackends.
The pre-PR no-registry fallback used resolvePluginContributionOwners which
includes both top-level cliBackends and setup.cliBackends; the PR's manifest
scan replacement was missing the setup case.

* fix(plugins): inherit active registry workspaceDir before loading metadata snapshot

isPluginProvidersLoadInFlight and resolvePluginProviders now resolve
env and workspaceDir once at the entry point (falling back to
getActivePluginRegistryWorkspaceDir) and pass them into both
loadPluginMetadataSnapshot and resolvePluginProviderLoadBase. Pre-fix
the snapshot used params.workspaceDir raw while the load base inherited
the active workspace, so workspace-scoped provider plugins could be
absent from the snapshot manifest registry even though owner resolution
expected them.

Regression test asserts the snapshot mock receives the active
workspaceDir when the caller omits it.

* perf(gateway): thread discovery into applyPluginAutoEnable call sites

Every gateway applyPluginAutoEnable call now passes the snapshot's
PluginDiscoveryResult so the bundled-channel cascade (collectConfiguredChannelIds
→ listBundledChannelIdsWith* → listPotentialConfiguredChannelPresenceSignals)
short-circuits instead of each leaf re-firing discovery.

Startup-time sites pull discovery from the snapshot/lookup-table they already
hold:
- server-plugin-bootstrap.ts (pluginLookUpTable)
- server-startup-plugins.ts (pluginMetadataSnapshot)
- server-startup-config.ts (pluginMetadataSnapshot)
- server-plugins.ts (pluginLookUpTable, both call sites)

Per-RPC sites (server.impl getRuntimeConfig callback, server-methods/channels
status + start handlers, server-methods/send) source discovery via
getCurrentPluginMetadataSnapshot using the runtime config to validate
compatibility. Falls through to the original slow path when the snapshot is
absent or incompatible.
2026-05-24 13:44:03 -07:00
Dallin Romney
dca9cecaee perf(plugins): thread install records through plugin load options (#85026)
Adds installRecords to PluginLoadOptions and PluginRuntimeLoadContext so
callers that already hold a PluginMetadataSnapshot can pass the snapshot's
in-memory records instead of forcing each downstream loader to re-read
installs.json. resolvePluginRuntimeLoadContext extracts the records from
the snapshot via extractPluginInstallRecordsFromInstalledPluginIndex,
buildPluginRuntimeLoadOptionsFromValues forwards them, and the setup +
runtime provider load paths in providers.runtime.ts pass them through
from params.pluginMetadataSnapshot. resolvePluginLoadCacheContext uses
the threaded records (falling back to the sync read) and
loader-provenance now uses params.installRecords ?? sync-read instead of
always reading and overlaying.
2026-05-21 15:24:31 -07:00
Josh Avant
f29bcff4da fix(models): reuse plugin metadata snapshot (#83033)
* fix(models): reuse plugin metadata snapshot

* docs: add models performance changelog

* test: satisfy models metadata fixture types
2026-05-17 04:51:59 -05:00
Peter Steinberger
91055b9a37 fix(release): keep provider registry fallback behind loader 2026-05-02 14:44:17 +01:00
Peter Steinberger
efa15001d6 fix(release): handle missing plugin registry fallback 2026-05-02 14:34:13 +01:00
Peter Steinberger
81e1deade2 fix(release): restore plugin runtime loading 2026-05-02 14:28:55 +01:00
DmitryPogodaev
8283c5d6cc perf(plugins): reuse startup runtime registry
Reuse the startup runtime plugin registry across provider/tool helper paths while preserving standalone CLI/MCP fallback loading.

Includes follow-up fixes for migration/provider/tool registry bootstrap and regression coverage for compatible registry reuse.

Co-authored-by: DmitryPogodaev <pogodaev.dm@gmail.com>
2026-05-02 13:44:49 +01:00
Peter Steinberger
ed8f50f240 refactor: simplify plugin dependency handling
Simplify plugin installation and runtime loading around package-manager-owned dependencies, with Jiti reserved for local/TS fallback paths.

Also scans npm plugin install roots so hoisted transitive dependencies are covered by dependency denylist and node_modules symlink checks.
2026-05-01 21:32:22 +01:00
Peter Steinberger
29ed5266bf fix: keep runtime deps repair out of hot paths 2026-05-01 09:26:45 +01:00
Peter Steinberger
a3e0674261 fix(ollama): harden native provider routing 2026-04-27 01:02:13 +01:00
Peter Steinberger
8ba9c9098a fix(agents): avoid provider startup scans 2026-04-26 11:11:37 +01:00
Peter Steinberger
0c46e8000e fix(plugins): cache discovery registration snapshots
Co-authored-by: junpei.o <14040213+livingghost@users.noreply.github.com>
Co-authored-by: Yoshiaki Okuyama <okuyam2y@gmail.com>
Co-authored-by: Shion Eria <shioneria@foxmail.com>
Co-authored-by: Billy Shih <1472300+bbshih@users.noreply.github.com>
2026-04-24 23:55:29 +01:00
Peter Steinberger
4fa961d4f1 refactor(lint): enable map spread rule 2026-04-18 20:37:12 +01:00
Vincent Koc
6a189eec0b fix(plugins): centralize explicit plugin scope handling (#65298)
* fix(plugins): centralize explicit plugin scope handling

* fix(plugins): preserve explicit empty web scopes

* fix(plugins): preserve runtime web provider scopes without config

* fix(plugins): preserve web provider runtime filtering

* fix(plugins): preserve scoped web runtime fallback

* fix(plugins): harden plugin scope normalization
2026-04-12 16:16:37 +01:00
Vincent Koc
12db6dfc8d feat(plugins): narrow explicit provider loads from manifests (#65259)
* feat(plugins): narrow explicit provider loads from manifests

* fix(plugins): preserve setup trust filtering for explicit owners

* fix(plugins): respect runtime owner trust and disablement

* fix(plugins): preserve provider owner policy bounds
2026-04-12 10:52:03 +01:00
Pavan Kumar Gondhi
2d97eae53e fix(plugins): prevent untrusted workspace plugins from hijacking bundled provider auth choices [AI] (#62368)
* fix: address issue

* fix: address review feedback

* docs(changelog): add onboarding auth-choice guard entry

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

* fix: address PR review feedback

---------

Co-authored-by: Devin Robison <drobison@nvidia.com>
2026-04-08 23:08:14 +05:30
Peter Steinberger
7d9a6b5572 fix(plugins): detect reentrant plugin loads 2026-04-06 18:05:33 +01:00
Peter Steinberger
725cbcc362 fix(plugins): narrow provider hook reentry guard 2026-04-06 17:57:49 +01:00
Peter Steinberger
58f4099a4f refactor: share plugin runtime load context 2026-04-06 15:26:32 +01:00
Vincent Koc
f1f8fd5970 fix(plugins): drop dead runtime helper 2026-04-05 09:59:50 +01:00
Peter Steinberger
647fc7bfec refactor(plugins): unify explicit provider ownership loading 2026-04-05 09:38:04 +01:00
Jamil Zakirov
ffb5b99114 fix: propagate workspaceDir to snapshot plugin loads (#61138)
* plugins: include resolved workspaceDir in provider hook cache keys

resolveProviderPluginsForHooks, resolveProviderPluginsForCatalogHooks, and
resolveProviderRuntimePlugin used the raw params.workspaceDir for cache keys
and plugin-id discovery while resolvePluginProviders already fell back to
the active registry workspace. Resolve workspaceDir once at the top of each
function so cache keys, candidate filtering, and loading all use the same
workspace root.

* fix(plugins): inherit runtime workspace for snapshot loads

* test(gateway): stub runtime registry seam

* fix(plugins): restore workspace fallback after rebase

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-04-05 09:03:54 +01:00
Peter Steinberger
d655a8bc76 feat: add Fireworks provider and simplify plugin setup loading 2026-04-05 07:43:14 +01:00
Peter Steinberger
fff7e610df feat(plugins): auto-load provider plugins from model support 2026-04-04 04:52:25 +01:00
Peter Steinberger
41ce3269f4 refactor(plugins): split activation snapshot and compat flow 2026-04-04 00:42:11 +09:00
Peter Steinberger
8b6c224554 refactor(plugins): share activation context for provider runtimes 2026-04-04 00:29:09 +09:00
Vincent Koc
d34bca3ce6 fix(plugins): reuse runtime registry for provider resolution (#59856)
* fix(plugins): reuse runtime registry for provider resolution

* test(plugins): align provider runtime helper names
2026-04-03 03:40:24 +09:00
Vincent Koc
9823833383 fix(plugins): preserve activation provenance (#59641)
* fix(plugins): preserve activation provenance

* fix(gateway): preserve activation reason metadata

* fix(plugins): harden activation state policy
2026-04-02 20:57:14 +09:00
Peter Steinberger
7d000088a4 fix(ci): use process env for provider compat fallback 2026-03-28 08:10:47 +00:00
Tak Hoffman
09e35e69b2 fix(regression): auto-enable provider runtime loads 2026-03-27 22:53:32 -05:00
Peter Steinberger
43ba3ab6b5 refactor: scope provider runtime to enabled provider plugins 2026-03-28 01:40:30 +00:00
Peter Steinberger
d73dbb6753 fix: restore provider auth and build checks 2026-03-27 20:20:31 +00:00
Peter Steinberger
546a1aad98 refactor: replace plugin-sdk dist env hacks with loader option 2026-03-27 13:46:17 +00:00
Peter Steinberger
4235fb16d6 perf: split provider metadata from runtime loading 2026-03-22 21:52:42 +00:00