Commit Graph

102 Commits

Author SHA1 Message Date
Peter Steinberger
e0beea97aa perf: speed up focused tests 2026-04-25 08:26:28 +01:00
Alex Fries
66e66f19c6 feat(memory-core): expose hybrid search component scores
Expose raw `vectorScore` and `textScore` alongside the combined hybrid memory search `score`.

- Preserve vector/text component scores from `mergeHybridResults` output.
- Add optional component-score fields to both memory host SDK type surfaces.
- Extend hybrid merge tests for vector-only, text-only, and overlapping result cases.
- Document that component scores remain raw retrieval diagnostics while temporal decay/MMR only adjust or reorder the combined ranking `score`.

Closes #68166.

Maintainer verification:
- `pnpm test extensions/memory-core/src/memory/hybrid.test.ts`
- `pnpm check:changed`
- Fresh GitHub checks passed.

Co-authored-by: Alex Fries <alex@engramlabs.io>
2026-04-25 01:46:03 -05:00
Michiel van den Donker
2c716f5677 fix: enforce memory search session visibility (#70761) (thanks @nefainl)
* [EV-001] memory-core: filter memory_search session hits by visibility

- Move session visibility + listSpawnedSessionKeys to plugin-sdk; sync test
  hook with sessions-resolution __testing.setDepsForTest
- Extract loadCombinedSessionStoreForGateway to config/sessions; re-export
  from gateway session-utils
- Add session-transcript-hit stem resolver for builtin + QMD paths
- Post-filter memory_search results before citations/recall; fail closed when
  requester session key missing; optional corpus=sessions
- Tests: stem extraction, visibility filter smoke, existing suites green

* chore: sync plugin-sdk exports for session-transcript-hit and session-visibility

Run pnpm plugin-sdk:sync-exports so package.json exports match
scripts/lib/plugin-sdk-entrypoints.json. Fixes contract tests and
lint:plugins:plugin-sdk-subpaths-exported for memory-core imports.

* fix(EV-001): cross-agent session memory hits + hoist combined store load

- resolveTranscriptStemToSessionKeys: stop filtering by requester agentId so
  keys from other agents reach createSessionVisibilityGuard (a2a + visibility=all).
- Re-export loadCombinedSessionStoreForGateway from session-transcript-hit;
  filterMemorySearchHitsBySessionVisibility loads the combined store once per pass.
- Drop unused agentId from filter params; extend tests (Greptile/Codex review).

* fix(memory_search): honor corpus=sessions before maxResults cap

Pass sources into MemoryIndexManager.search so FTS/vector queries add
source IN (...) before ranking and top-N slice (Codex: non-session hits
could fill the window).

QMD path: oversample fetch limit for single-source recall, filter by
source, then diversify/clamp to the requested maxResults.

Wire corpus=sessions from tools; extend MemorySearchManager opts and
wrappers.

* fix(memory_search): apply corpus=memory source filter like sessions

Pass sources: ["memory"] into manager.search so maxResults applies only
within the memory index; post-filter for defense in depth. Document
corpus=memory in the tool description.

* fix: scope qmd session memory search

* fix: enforce memory search session visibility (#70761) (thanks @nefainl)

---------

Co-authored-by: NefAI <info@nefai.nl>
Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-04-25 09:30:21 +05:30
Patrick Erichsen
aca92b2906 memory/dreaming: decouple managed cron from heartbeat (#70737)
* Revert "fix(memory/dreaming): surface blocked status when heartbeat is disabled for main (#69875)"

This reverts commit 529577e045.

Making way for the dreaming-vs-heartbeat decoupling from Josh's
josh/dreaming-isolated-cron-fix branch, which moves the managed dreaming
cron to isolated agent turns (sessionTarget: "isolated") so dreaming no
longer requires heartbeat to fire. Once the cron no longer rides the
heartbeat path, the blocked-reason observability has nothing left to
report — removing it cleanly here before the cherry-picks land.

* openclaw-3ba.1: move managed dreaming cron to isolated agent turns

* openclaw-46d: claim cron runs before embedded attempts

* openclaw-575: disable managed dreaming cron delivery

* openclaw-575: accept wrapped dreaming cron tokens

* openclaw-ccd: filter cron and wrapper transcript noise from dreaming corpus

* openclaw-cd9: filter archived, cron, and heartbeat transcript noise from dreaming corpus

* openclaw-cd9: suppress role-label reflection tags in rem dreaming

* openclaw-b49: stop narrative timeouts from blocking dreaming cron

* openclaw-b49: keep managed dreaming cron out of diary subagents

* openclaw-ff9: restore cron dream diary generation without serial waits

* openclaw-ff9: run dreaming narratives with lightweight isolated subagent lanes

* openclaw-ff9: detach cron dream diary generation from run completion

* openclaw-ff9: defer cron diary task startup until after cron completion

* doctor/cron: migrate stale managed dreaming jobs to isolated agent turns

After the dreaming cron moved off the heartbeat path to sessionTarget:
"isolated" + payload.kind: "agentTurn" (see the preceding memory-core
changes), users with existing ~/.openclaw/cron/jobs.json entries in the
old sessionTarget: "main" + payload.kind: "systemEvent" shape still
carry stale jobs until the gateway restart reconcile rewrites them.

Add a dreaming-specific cron migration to the existing
maybeRepairLegacyCronStore doctor path so "openclaw doctor" (and
"openclaw doctor --fix") rewrites those jobs without needing a gateway
restart. Match lives in a new doctor-cron-dreaming-payload-migration
helper alongside the existing legacy-delivery and store-migration files.

The matching uses the memory-core managed-job name and description tag
plus the short-term-promotion payload token. Constants are mirrored
from extensions/memory-core/src/dreaming.ts and commented so a future
rename in memory-core is a visible drift point here too.

* memory/dreaming: tighten cron-token match to known wrapper, not substring

The previous match relaxed the line check from 'trimmed line equals token'
to 'line contains token anywhere as a substring' to accept the
`[cron:<id>] <token>` wrapper that isolated-cron turns add. Substring
matching also let any user message embedding the token mid-sentence
trigger the dream-promotion hook, and was flagged by both Greptile and
Aisle on PR #70737.

Replace it with strip-the-known-prefix-then-exact-match: keep the
`[cron:<id>]` wrapper case working, reject every other variant. Add
focused unit coverage that the bare token, the wrapped token, and bare
multiline cases match while embedded / code-fenced / arbitrarily-wrapped
variants do not.

* memory/dreaming: drop assistant followup only on assistant-side signals

Per PR #70737 review (aisle-research-bot, Medium): the previous logic
suppressed the next assistant message whenever the prior user message
matched a 'generated prompt' pattern (`[cron:...]`,
`System (untrusted): ...`, heartbeat prompts, exec-completion events).
Real users can type those same patterns, which let a user exfiltrate
real assistant replies from the dreaming corpus by prefixing their own
prompt — the assistant's reply would be silently dropped.

Remove the cross-message coupling. Assistant-side machinery (silent
replies, system wrappers) is already dropped by sanitizeSessionText,
which is the right layer for that filter. Add an explicit assistant-side
HEARTBEAT_TOKEN check to keep the legitimate `HEARTBEAT_OK` ack drop
working without depending on the prior user message. Add a regression
test exercising the spoofing scenario.

* doctor/cron: assert mirrored dreaming constants stay in sync

Per PR #70737 review (greptile-apps): the doctor migration mirrors three
constants (MANAGED_DREAMING_CRON_NAME, MANAGED_DREAMING_CRON_TAG,
DREAMING_SYSTEM_EVENT_TEXT) from extensions/memory-core/src/dreaming.ts.
A future rename in either file would silently break the migration.

Add a vitest unit that reads both files and asserts the literals match.
Manually verified the assertion fires with a clear error when one side
diverges. Adds no runtime cost; sits in the regular test pipeline.

* fix(memory): stabilize dreaming CI checks

* memory/dreaming: skip eager narrative session cleanup when detached

Per PR #70737 review (chatgpt-codex-connector, P2): runDreamingSweepPhases
called deleteNarrativeSessionBestEffort synchronously right after each
phase. Once narrative generation moved to detached mode (queued via
queueMicrotask), the eager cleanup races the writer: the session is
deleted before the queued subagent run reads it, silently dropping cron
diary entries.

Skip the eager cleanup branch when params.detachNarratives is true.
generateAndAppendDreamNarrative still runs its own deleteSession in the
finally{} block, so the cleanup intent is preserved without the race.
Heartbeat-driven (non-detached) runs keep the original eager-cleanup
behavior.

* fix(plugin-sdk): restore heartbeat-summary re-export

Per PR #70737 review (chatgpt-codex-connector, P1): the revert of
PR #69875 dropped the `heartbeat-summary` re-export from
`openclaw/plugin-sdk/infra-runtime`. That subpath shipped publicly two
days earlier, so removing it is technically a breaking change to a
public SDK surface — third-party plugins importing
`isHeartbeatEnabledForAgent` / `resolveHeartbeatIntervalMs` from this
path would fail with no replacement contract introduced.

Restore the re-export. Costs nothing to keep; the helpers are already
public via `../infra/heartbeat-summary.ts`. SDK additions are by
default backwards-compatible (CLAUDE.md), so removing within days of
introduction violates that intent.

* changelog: note dreaming decoupling from heartbeat

Refs PR #70737.

---------

Co-authored-by: Josh Lehman <josh@martian.engineering>
2026-04-23 22:23:19 -07:00
aalekh-sarvam
d40dd9088e feat(memory): configurable local embedding contextSize (default 4096) (#70544)
node-llama-cpp defaults contextSize to "auto", which on large embedding
models like Qwen3-Embedding-8B (trained context 40,960) inflates gateway
VRAM from ~8.8 GB to ~32 GB and causes OOM on single-GPU hosts that share
the gateway with an LLM runtime.

Expose memorySearch.local.contextSize in openclaw.json (number | "auto"),
default to 4096 which comfortably covers typical memory-search chunks
(128–512 tokens) while keeping non-weight VRAM bounded.

Closes #69667.
2026-04-23 14:21:53 -07:00
Peter Steinberger
5ebef46183 perf: speed up hot test paths 2026-04-23 18:17:09 +01:00
Peter Steinberger
deb1364dfb refactor: centralize root memory file policy 2026-04-23 16:40:41 +01:00
Mariano
10a9acbf29 fix: keep root memory uppercase (#70621)
Thanks @mbelinky.
2026-04-23 16:10:36 +01:00
Peter Steinberger
b2472d6560 build: migrate schema deps to typebox 2026-04-23 04:59:42 +01:00
Bek
1acb094579 fix: wrap oversized session lines before JSONL write (#64494)
updates the real session-export path so pathological transcript messages no longer become a single toxic export line for downstream indexing.
2026-04-21 18:18:22 -04:00
Peter Steinberger
66665eea6d test: use synthetic status session fixtures 2026-04-21 01:40:29 +01:00
Peter Steinberger
da5a6b68bd refactor: share ssrf base url policy 2026-04-20 23:15:58 +01:00
Peter Steinberger
e0c01bf956 perf: trim hot path allocations 2026-04-20 23:13:20 +01:00
Peter Steinberger
59657913fd perf(test): prune duplicate memory host tests 2026-04-20 20:05:37 +01:00
Peter Steinberger
88de927a0c perf(test): dedupe memory host mirror tests 2026-04-20 20:02:19 +01:00
Peter Steinberger
f006678f3c refactor: share balanced json extraction 2026-04-20 14:40:21 +01:00
Peter Steinberger
faae8e08b3 test: share qmd multi-agent config fixture 2026-04-19 02:43:52 +01:00
Peter Steinberger
805481c176 perf: narrow bonjour and sqlite runtime type surfaces 2026-04-19 02:03:19 +01:00
Peter Steinberger
a0919685be perf: narrow local llama type surface 2026-04-18 23:38:04 +01:00
Peter Steinberger
0195da6b0e refactor: cache optional runtime imports 2026-04-18 20:45:26 +01:00
Peter Steinberger
2b7b5774b6 chore: remove dead code 2026-04-18 20:27:42 +01:00
Peter Steinberger
5f075d3d49 test: reuse session file fixture root 2026-04-17 19:57:19 +01:00
Peter Steinberger
dadcfb574f test: trim surrogate chunk fixtures 2026-04-17 19:38:53 +01:00
Peter Steinberger
77e6e4cf87 refactor: move memory embeddings into provider plugins 2026-04-17 02:57:18 +01:00
Peter Steinberger
3a59eddd07 test: reduce hotspot fixture overhead 2026-04-17 02:37:00 +01:00
Peter Steinberger
dbc8179f31 test: narrow hotspot mocks 2026-04-17 01:53:16 +01:00
Peter Steinberger
cd330f5f98 test: isolate gemini embedding request helpers 2026-04-17 01:46:47 +01:00
Peter Steinberger
fd48dfa68f test: trim memory and mcp hotspots 2026-04-17 01:39:39 +01:00
Peter Steinberger
2e08c77582 test: slim provider registry mocks 2026-04-17 01:29:12 +01:00
Peter Steinberger
59b98334f6 test: narrow hotspot boundaries 2026-04-17 01:10:48 +01:00
Peter Steinberger
ce1be0f43d test: speed gemini embedding tests 2026-04-17 00:00:41 +01:00
Peter Steinberger
b2cae7f12a test: trim duplicate memory hotspot coverage 2026-04-16 23:15:38 +01:00
Peter Steinberger
d59604b15e test: speed up hotspot boundaries 2026-04-16 22:55:30 +01:00
Peter Steinberger
63e53fbf2e test: trim duplicate hotspot coverage 2026-04-16 22:19:32 +01:00
Peter Steinberger
372c0051ba test: speed up slow import-boundary tests 2026-04-16 21:14:17 +01:00
zqchris
82e349a48a memory: strip inbound metadata envelopes from user messages in session corpus (#66548)
Merged via squash.

Prepared head SHA: 98562b2a84
Co-authored-by: zqchris <4436110+zqchris@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-16 11:15:44 -07:00
mjamiv
8c392f0019 fix(dreaming): default storage.mode to "separate" so phase blocks stop polluting daily memory files (#66412)
Merged via squash.

Prepared head SHA: 4b1c8ac4ec
Co-authored-by: mjamiv <142179942+mjamiv@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-15 13:49:18 -07:00
Josh Lehman
a1b01f0281 fix(memory-core): skip dreaming transcript ingestion via session store (#67315)
Merged via squash.

Prepared head SHA: 87c09b2a75
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-15 13:09:07 -07:00
Tak Hoffman
4f00b76925 fix(context-window): Tighten context limits and bound memory excerpts (#67277)
* Tighten context limits and bound memory excerpts

* Align startup context defaults in config docs

* Align qmd memory_get bounds with shared limits

* Preserve qmd partial memory reads

* Fix shared memory read type import

* Add changelog entry for context bounds
2026-04-15 13:06:02 -05:00
Pengfei Ni
88d3620a85 feat(github-copilot): add embedding provider for memory search (#61718)
Merged via squash.

Prepared head SHA: 05a78ce7f2
Co-authored-by: feiskyer <676637+feiskyer@users.noreply.github.com>
Co-authored-by: vincentkoc <25068+vincentkoc@users.noreply.github.com>
Reviewed-by: @vincentkoc
2026-04-15 10:39:28 +01:00
Gustavo Madeira Santana
0c4e0d7030 memory: block dreaming self-ingestion (#66852)
Merged via squash.

Prepared head SHA: 4742656a0d
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
2026-04-14 20:29:12 -04:00
Vincent Koc
8820a43818 fix(memory): preserve embedding proxy provider prefixes (#66452)
* fix(memory): preserve embedding proxy provider prefixes

* docs(changelog): fix embeddings entry

* Update CHANGELOG.md
2026-04-14 11:05:07 +01:00
Peter Steinberger
2e1b5407dd fix: preserve qmd command paths 2026-04-13 10:51:32 +01:00
Rugved Somwanshi
0cfb83edfa feat: LM Studio Integration (#53248)
* Feat: LM Studio Integration

* Format

* Support usage in streaming true

Fix token count

* Add custom window check

* Drop max tokens fallback

* tweak docs

Update generated

* Avoid error if stale header does not resolve

* Fix test

* Fix test

* Fix rebase issues

Trim code

* Fix tests

Drop keyless

Fixes

* Fix linter issues in tests

* Update generated artifacts

* Do not have fatal header resoltuion for discovery

* Do the same for API key as well

* fix: honor lmstudio preload runtime auth

* fix: clear stale lmstudio header auth

* fix: lazy-load lmstudio runtime facade

* fix: preserve lmstudio shared synthetic auth

* fix: clear stale lmstudio header auth in discovery

* fix: prefer lmstudio header auth for discovery

* fix: honor lmstudio header auth in warmup paths

* fix: clear stale lmstudio profile auth

* fix: ignore lmstudio env auth on header migration

* fix: use local lmstudio setup seam

* fix: resolve lmstudio rebase fallout

---------

Co-authored-by: Frank Yang <frank.ekn@gmail.com>
2026-04-13 15:22:44 +08:00
Peter Steinberger
067f27f6a2 fix: normalize stale qmd binary paths 2026-04-12 19:08:59 +01:00
Peter Steinberger
15b86ac6d0 fix: narrow qmd defaults and clawblocker memory 2026-04-12 18:52:06 +01:00
Tak Hoffman
847739d82c Fix dreaming replay, repair polluted artifacts, and gate wiki tabs (#65138)
* fix(active-memory): preserve parent channel context for recall runs

* fix(active-memory): keep recall runs on the resolved channel

* fix(active-memory): prefer resolved recall channel over wrapper hints

* fix(active-memory): trust explicit recall channel hints

* fix(active-memory): rank recall channel fallbacks by trust

* Fix dreaming replay and recovery flows

* fix: prevent dreaming event loss and diary write races

* chore: add changelog entry for memory fixes

* fix: harden dreaming repair and diary writes

* fix: harden dreaming artifact archive naming
2026-04-12 00:25:11 -05:00
Vincent Koc
a45c4bebc5 test(memory): share backend config collection helpers 2026-04-12 05:16:46 +01:00
Tak Hoffman
885209ed03 feat: default active memory QMD recall to search (#65068)
* feat(active-memory): default QMD recall to search

* feat(active-memory): surface search debug telemetry

* fix(active-memory): avoid forking qmd managers
2026-04-11 20:56:21 -05:00
Vincent Koc
74e7b8d47b fix(cycles): bulk extract leaf type surfaces 2026-04-11 13:26:50 +01:00