Commit Graph

25 Commits

Author SHA1 Message Date
Josh Lehman
e46e32b98c feat: expose prompt-cache runtime context to context engines (#62179)
* Context engine: plumb prompt cache runtime context

Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it.

Regeneration-Prompt: |
  Expose OpenClaw prompt-cache telemetry to context engines in a narrow,
  additive way without changing compaction policy. Keep the public change on
  the OpenClaw side only: add a typed promptCache payload to the context-engine
  runtime context, thread it into afterTurn, and also into compact where the
  existing run loop already has the data cheaply available.

  Use OpenClaw's resolved cache retention, not raw config. Use last-call usage
  for the new payload, not accumulated retry or tool-loop totals. Reuse the
  existing prompt-cache observability result and tracked change causes instead
  of inventing a new heuristic. If cache-touch metadata is already available
  from the cache-TTL bookkeeping, include it; do not invent expiry timestamps
  for providers where OpenClaw cannot know them confidently.

  Keep the interface backward-compatible for engines that ignore the new field.
  Add focused tests around the existing attempt/context-engine helpers and the
  compaction runtime-context propagation path rather than broad new integration
  coverage.

* Agents: fix prompt-cache afterTurn usage

Regeneration-Prompt: |
  Fix PR #62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well.

* Agents: document prompt-cache context

* Agents: address prompt-cache review feedback

* Doctor: drop unused isRecord import
2026-04-07 09:29:57 -07:00
Vincent Koc
2988203a5e feat(context-engine): add memory prompt helper 2026-04-07 08:56:41 +01:00
Peter Steinberger
27d4992eef Tests: mock context-engine compact runtime seam 2026-04-07 12:29:33 +08:00
Peter Steinberger
ee63fdb056 test(ci): harden context engine runtime bridge test 2026-04-03 21:12:56 +01:00
Tak Hoffman
f5643544c2 Reduce lint suppressions in core tests and runtime 2026-03-27 02:11:26 -05:00
Peter Steinberger
2fc017788c test: reduce remaining clone seams 2026-03-26 20:01:01 +00:00
Peter Steinberger
9428b38452 refactor: consolidate core runtime state helpers 2026-03-22 18:09:45 +00:00
Peter Steinberger
4229ffe2b9 perf: reduce runtime and test startup overhead 2026-03-21 20:18:16 +00:00
Danh Doan
e78129a4d9 feat(context-engine): pass incoming prompt to assemble (#50848)
Merged via squash.

Prepared head SHA: 282dc9264d
Co-authored-by: danhdoan <12591333+danhdoan@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-20 17:03:21 -07:00
Josh Lehman
751d5b7849 feat: add context engine transcript maintenance (#51191)
Merged via squash.

Prepared head SHA: b42a3c28b4
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-20 16:28:27 -07:00
John Scianna
5607da90d5 feat: pass modelId to context engine assemble() (#47437)
Merged via squash.

Prepared head SHA: d708ddb222
Co-authored-by: jscianna <9017016+jscianna@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-20 08:05:02 -07:00
Josh Lehman
7f0f8dd268 feat: expose context-engine compaction delegate helper (#49061)
* ContextEngine: add runtime compaction delegate helper

* plugin-sdk: expose compaction delegate through compat

* docs: clarify delegated plugin compaction

* docs: use scoped compaction delegate import
2026-03-17 22:54:18 -07:00
Peter Steinberger
8a9dee9ac8 test: trim redundant context engine assertions 2026-03-17 16:21:58 +00:00
F_ool
094a0cc412 fix(context-engine): preserve legacy plugin sessionKey interop (#44779)
Merged via squash.

Prepared head SHA: e04c6fb47d
Co-authored-by: hhhhao28 <112874572+hhhhao28@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-17 09:14:14 -07:00
Vincent Koc
7931f06c00 Plugins: harden context engine ownership 2026-03-15 13:51:15 -07:00
Vincent Koc
b795ba1d02 Merge branch 'main' of https://github.com/openclaw/openclaw
* 'main' of https://github.com/openclaw/openclaw:
  Plugins: reserve context engine ownership (#47595)
  fix(release): block oversized npm packs that regress low-memory startup (#46850)
  Scripts: rebuild on extension and tsdown config changes (#47571)
  Docs: move release runbook to maintainer repo (#47532)
  docs(zalo): document current Marketplace bot behavior (openclaw#47552)
2026-03-15 13:42:21 -07:00
Vincent Koc
85dd0ab2f8 Plugins: reserve context engine ownership (#47595)
* Plugins: reserve context engine ownership

* Update src/context-engine/registry.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-15 13:33:37 -07:00
Vincent Koc
51631e5797 Plugins: reserve context engine ownership 2026-03-15 12:27:29 -07:00
Josh Lehman
50cc375c11 feat(context-engine): plumb sessionKey into all ContextEngine methods (#44157)
Merged via squash.

Prepared head SHA: 0b341f6f4c
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-12 12:43:36 -07:00
rabsef-bicrym
ff47876e61 fix: carry observed overflow token counts into compaction (#40357)
Merged via squash.

Prepared head SHA: b99eed4329
Co-authored-by: rabsef-bicrym <52549148+rabsef-bicrym@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-12 06:58:42 -07:00
Daniel Reis
fbf5d56366 test(context-engine): add bundle chunk isolation tests for registry (#40460)
Merged via squash.

Prepared head SHA: 44622abfbc
Co-authored-by: dsantoreis <220753637+dsantoreis@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-09 08:15:35 -07:00
Josh Lehman
4bfa800cc7 fix: share context engine registry across bundled chunks (#40115)
Merged via squash.

Prepared head SHA: 6af4820b7d
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-08 11:56:01 -07:00
Peter Steinberger
11be305609 refactor: neutralize context engine runtime bridge 2026-03-08 18:40:15 +00:00
Vincent Koc
063b9aabe2 fix: xxxxx 2026-03-06 19:46:38 -05:00
Josh Lehman
fee91fefce feature(context): extend plugin system to support custom context management (#22201)
* feat(context-engine): add ContextEngine interface and registry

Introduce the pluggable ContextEngine abstraction that allows external
plugins to register custom context management strategies.

- ContextEngine interface with lifecycle methods: bootstrap, ingest,
  ingestBatch, afterTurn, assemble, compact, prepareSubagentSpawn,
  onSubagentEnded, dispose
- Module-level singleton registry with registerContextEngine() and
  resolveContextEngine() (config-driven slot selection)
- LegacyContextEngine: pass-through implementation wrapping existing
  compaction behavior for 100% backward compatibility
- ensureContextEnginesInitialized() guard for safe one-time registration
- 19 tests covering contract, registry, resolution, and legacy parity

* feat(plugins): add context-engine slot and registerContextEngine API

Wire the ContextEngine abstraction into the plugin system so external
plugins can register context engines via the standard plugin API.

- Add 'context-engine' to PluginKind union type
- Add 'contextEngine' slot to PluginSlotsConfig (default: 'legacy')
- Wire registerContextEngine() through OpenClawPluginApi
- Export ContextEngine types from plugin-sdk for external consumers
- Restore proper slot-based resolution in registry

* feat(context-engine): wire ContextEngine into agent run lifecycle

Integrate the ContextEngine abstraction into the core agent run path:

- Resolve context engine once per run (reused across retries)
- Bootstrap: hydrate canonical store from session file on first run
- Assemble: route context assembly through pluggable engine
- Auto-compaction guard: disable built-in auto-compaction when
  the engine declares ownsCompaction (prevents double-compaction)
- AfterTurn: post-turn lifecycle hook for ingest + background
  compaction decisions
- Overflow compaction: route through contextEngine.compact()
- Dispose: clean up engine resources in finally block
- Notify context engine on subagent lifecycle events

Legacy engine: all lifecycle methods are pass-through/no-op, preserving
100% backward compatibility for users without a context engine plugin.

* feat(plugins): add scoped subagent methods and gateway request scope

Expose runtime.subagent.{run, waitForRun, getSession, deleteSession}
so external plugins can spawn sub-agent sessions without raw gateway
dispatch access.

Uses AsyncLocalStorage request-scope bridge to dispatch internally via
handleGatewayRequest with a synthetic operator client. Methods are only
available during gateway request handling.

- Symbol.for-backed global singleton for cross-module-reload safety
- Fallback gateway context for non-WS dispatch paths (Telegram/WhatsApp)
- Set gateway request scope for all handlers, not just plugin handlers
- 3 staleness tests for fallback context hardening

* feat(context-engine): route /compact and sessions.get through context engine

Wire the /compact command and sessions.get handler through the pluggable
ContextEngine interface.

- Thread tokenBudget and force parameters to context engine compact
- Route /compact through contextEngine.compact() when registered
- Wire sessions.get as runtime alias for plugin subagent dispatch
- Add .pebbles/ to .gitignore

* style: format with oxfmt 0.33.0

Fix duplicate import (ControlUiRootState in server.impl.ts) and
import ordering across all changed files.

* fix: update extension test mocks for context-engine types

Add missing subagent property to bluebubbles PluginRuntime mock.
Add missing registerContextEngine to lobster OpenClawPluginApi mock.

* fix(subagents): keep deferred delete cleanup retryable

* style: format run attempt for CI

* fix(rebase): remove duplicate embedded-run imports

* test: add missing gateway context mock export

* fix: pass resolved auth profile into afterTurn compaction

Ensure the embedded runner forwards resolved auth profile context into
legacy context-engine compaction params on the normal afterTurn path,
matching overflow compaction behavior. This allows downstream LCM
summarization to use the intended provider auth/profile consistently.

Also fix strict TS typing in external-link token dedupe and align an
attempt unit test reasoningLevel value with the current ReasoningLevel
enum.

Regeneration-Prompt: |
  We were debugging context-engine compaction where downstream summary
  calls were missing the right auth/profile context in normal afterTurn
  flow, while overflow compaction already propagated it. Preserve current
  behavior and keep changes additive: thread the resolved authProfileId
  through run -> attempt -> legacy compaction param builder without
  broad refactors.

  Add tests that prove the auth profile is included in afterTurn legacy
  params and that overflow compaction still passes it through run
  attempts. Keep existing APIs stable, and only adjust small type issues
  needed for strict compilation.

* fix: remove duplicate imports from rebase

* feat: add context-engine system prompt additions

* fix(rebase): dedupe attempt import declarations

* test: fix fetch mock typing in ollama autodiscovery

* fix(test): add registerContextEngine to diffs extension mock APIs

* test(windows): use path.delimiter in ios-team-id fixture PATH

* test(cron): add model formatting and precedence edge case tests

Covers:
- Provider/model string splitting (whitespace, nested paths, empty segments)
- Provider normalization (casing, aliases like bedrock→amazon-bedrock)
- Anthropic model alias normalization (opus-4.5→claude-opus-4-5)
- Precedence: job payload > session override > config default
- Sequential runs with different providers (CI flake regression pattern)
- forceNew session preserving stored model overrides
- Whitespace/empty model string edge cases
- Config model as string vs object format

* test(cron): fix model formatting test config types

* test(phone-control): add registerContextEngine to mock API

* fix: re-export ChannelKind from config-reload-plan

* fix: add subagent mock to plugin-runtime-mock test util

* docs: add changelog fragment for context engine PR #22201
2026-03-06 05:31:59 -08:00