Commit Graph

360 Commits

Author SHA1 Message Date
Dallin Romney
8c5f45fc36 refactor(shared): consolidate provider and utility lazy loaders (#98749)
* refactor(shared): consolidate provider lazy loaders

* refactor(shared): leave enforcement to follow-up
2026-07-02 21:16:26 -07:00
Yzx
39898fe91b test(memory): cover live manager after CLI reindex (#96094) 2026-07-01 01:59:20 -07:00
zw-xysk
742f0c9f82 fix(memory): detect unindexed session transcripts in status mode (fixes #97814) (#97857)
* fix(memory): detect unindexed session transcripts in status mode (fixes #97814)

The status-purpose MemoryIndexManager init skips ensureSessionStartupCatchup(),
so openclaw memory status reports dirty=false while unindexed session files
exist on disk. This is a false-clean state: the operator sees no backlog,
but memory search silently misses unindexed transcripts.

Fix: run markSessionStartupCatchupDirtyFiles() in status mode. It checks
for on-disk session files without corresponding memory_index_sources rows
and sets sessionsDirty=true if found, without triggering a full sync.

The full catchup+sync runs on the next non-transient manager cycle (CLI
sync or normal runtime init).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): await status dirty detection before status() reads the flag (fixes #97814)

The status-purpose MemoryIndexManager init skips ensureSessionStartupCatchup(),
so openclaw memory status reports dirty=false while unindexed session files
exist on disk. This is a false-clean state: the operator sees no backlog,
but memory search silently misses unindexed transcripts.

Fix: move status-mode markSessionStartupCatchupDirtyFiles() from the
constructor (fire-and-forget via void) into the create() factory method
where it is awaited. This guarantees sessionsDirty is set before any
caller reads manager.status(). The detection does NOT trigger a sync,
so status remains lightweight.

Review: verified manually — unit tests 21/21 pass, compilation 0 errors.

* test(memory): add regression test for status-purpose dirty detection (fixes #97814)

Adds a regression test that exercises the actual purpose:'status' manager
flow: create a session file without index row, create status manager, verify
status().dirty is true. This addresses the ClawSweeper P1 finding requesting
a test that exercises the real status path, not only the protected helper.

Also fixes the timing issue: move dirty detection from constructor (void)
to create() factory (await), guaranteeing sessionsDirty is set before any
caller reads manager.status().

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-30 17:47:43 -07:00
Brian Potter
cd31cbcd17 fix(memory-core): don't run the LLM reranker in vsearch/search modes (#88887)
vsearch/search are documented as vector/lexical-only modes, but
runQmdSearchViaMcporter omitted `rerank` from the query tool callArgs,
so QMD's query tool (which defaults rerank:true) ran the LLM reranker
anyway. Pass rerank:false for those modes; full "query" mode keeps it.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 18:11:03 -07:00
Bek
9636bea901 perf(memory): add QMD search diagnostics and runtime cache (#96655) 2026-06-26 16:16:12 -04:00
Shakker
1cd6f81a46 fix: close memory test failure paths 2026-06-26 14:05:07 +01:00
Shakker
338e119533 fix: preserve watcher test state 2026-06-26 13:05:44 +01:00
Shakker
e4f63577d0 test: scope manager reindex state 2026-06-26 12:56:21 +01:00
Shakker
94d93d4c85 fix: retain qmd test environment state 2026-06-26 12:48:24 +01:00
Shakker
7718e25b2a test: preserve memory startup env state 2026-06-26 12:40:55 +01:00
Shakker
8079aa62a2 fix: isolate memory index test state 2026-06-26 12:32:39 +01:00
Peter Steinberger
73dd758310 fix(memory): abort orphaned qmd search processes 2026-06-23 13:31:46 -07:00
Alix-007
eadd69b44c fix(memory): abort orphaned qmd subprocess on the mcporter search path too
The initial fix threaded the abort signal through the direct qmd
(runQmd/runQmdSearch) path, but the mcporter / QMD 1.1+ daemon search path
(runQmdSearchViaMcporter, runMcporterAcrossCollections) never received it, so
a grouped/mcporter search left its subprocess running on abort.

Thread the search signal through QmdMcporterSearchParams,
QmdMcporterAcrossCollectionsParams, all four mcporter call sites in search(),
and runMcporter, down to the shared runCliCommand spawn (which already
SIGKILLs the child on abort). Guard runQmdSearchViaMcporter on an
already-aborted signal so the multi-collection loop stops spawning. Reuses the
existing abort mechanism; no new machinery. Adds mcporter-path regression tests.
2026-06-23 13:31:46 -07:00
Alix-007
2a021f3b9b fix(memory): thread qmd search abort signal through grouped collection search
memory_search timeout cancellation only reached single-group direct qmd
searches. Multi-collection or mixed memory/session configs route through
runQueryAcrossCollectionGroups, which still called runQmdSearch without the
caller signal, so an aborted memory_search left the grouped qmd child running
until the qmd command timeout instead of being killed promptly.

Thread searchSignal through the grouped search path and its unsupported-option
fallback, and add a grouped multi-collection abort regression asserting the
spawned qmd child is SIGKILLed when the caller signal aborts.
2026-06-23 13:31:46 -07:00
Alix-007
78184ea7e4 fix(memory): abort orphaned qmd search subprocess when memory_search times out
PR #91742 wired memory_search's 15s deadline AbortSignal through the builtin
memory manager but missed the QMD backend behind the same
MemorySearchManager.search interface. With QMD, the tool returns "timed out
after 15s" to the agent while the spawned qmd query/search subprocess keeps
running for the full qmd command timeout (memory.qmd.limits.timeoutMs, whose
embed-heavy default was raised to 600s in #87572), leaving orphaned
embedding/search work running after the agent already moved on.

Add optional AbortSignal support to runCliCommand: an aborting signal kills the
spawned child immediately and rejects with the abort reason, funneled through a
single settle() guard so abort/timeout/error/close cannot double-settle. Thread
the search signal through QmdMemoryManager.search -> runQmdSearch -> runQmd ->
runCliCommand for the default direct-qmd subprocess path (including the query
fallback), and fast-fail search() when the signal is already aborted.
2026-06-23 13:31:46 -07:00
Josh Lehman
c24d266b2d refactor: use accessor-backed transcript corpus for memory (#96162)
* refactor: ratchet memory transcript corpus access

* test: use narrow runtime config snapshot import

* test: update plugin sdk surface budgets

* refactor: split memory transcript corpus module
2026-06-23 12:37:44 -07:00
Josh Lehman
475252453b refactor: add transcript update identity contract (#89912) 2026-06-23 05:52:08 -07:00
Josh Lehman
d3781cc4b8 refactor: add memory and QMD session identity mapping (#95087) 2026-06-22 06:28:54 -07:00
Vincent Koc
0ea39a2276 chore(deadcode): remove inert memory provider bootstrap 2026-06-22 02:30:56 +08:00
Vincent Koc
c372f6ef0b fix(plugins): keep extension tests on public boundaries 2026-06-21 15:39:51 +08:00
Vincent Koc
8797564254 chore(deadcode): share deferred test helper 2026-06-21 13:53:24 +08:00
Vincent Koc
ba34052a0e chore(deadcode): share memory vector blob encoding 2026-06-21 08:23:16 +08:00
Vincent Koc
7cc66b5175 refactor(channels): drop unused bootstrap exports 2026-06-19 15:32:30 +08:00
Vincent Koc
a9cc2697d3 test(memory): align metadata repair expectation 2026-06-19 01:20:06 +08:00
Vincent Koc
6e6bd5633f fix(memory): rebuild migrated search indexes 2026-06-19 01:20:06 +08:00
Josh Lehman
9cfd1cd287 fix(memory): key index sources by path and source 2026-06-19 01:20:06 +08:00
Josh Lehman
21163a704b fix(memory): make file index publication atomic 2026-06-19 01:20:06 +08:00
Vincent Koc
cc80128222 test(memory): type legacy vector migration fixture 2026-06-19 01:20:06 +08:00
Vincent Koc
7485dd6492 fix(sqlite): migrate legacy memory and proxy state 2026-06-19 01:20:06 +08:00
Vincent Koc
88bc08c124 refactor(memory): canonicalize agent database tables 2026-06-19 01:20:06 +08:00
Vincent Koc
acdcf37f87 chore(memory): simplify publish error 2026-06-19 01:20:06 +08:00
Vincent Koc
d64db0fc3d fix(memory): load vector extension before publish 2026-06-19 01:20:06 +08:00
Vincent Koc
25e68e9fa7 fix(memory): reject stale reindex publishes 2026-06-19 01:20:06 +08:00
Vincent Koc
1db7c41bf2 fix(sdk): keep sqlite lifecycle helpers private 2026-06-19 01:20:06 +08:00
Vincent Koc
a7fada4b61 fix(memory): invalidate stale shadow indexes 2026-06-19 01:20:06 +08:00
Vincent Koc
8b7269d197 fix(sqlite): preserve migrations and reindex safety 2026-06-19 01:20:06 +08:00
Vincent Koc
f324f7e281 refactor(memory): use the per-agent sqlite database 2026-06-19 01:20:06 +08:00
Vincent Koc
58b77e787d refactor(extensions): remove unused helper exports 2026-06-18 15:05:35 +08:00
liuhao1024
b3a1472875 fix(memory): await search-sync before returning results to prevent stale index (fixes #52115) (#93791)
* fix(memory): await search-sync before returning results to prevent stale index

When the gateway process has been running for a while, memory_search
returns stale results because startAsyncSearchSync fires off the index
sync as a background task (void ... .catch()) without waiting for it
to complete. Search results are then read from the old index state.

Change startAsyncSearchSync from sync/fire-and-forget to async/await
so that the index is synced before search results are returned. This
ensures memory_search reflects the current filesystem state, matching
the behavior of the CLI  command which creates
a fresh manager each time.

Fixes #52115

* test(memory): prove search waits for dirty sync

* test(memory): align search with synchronous sync

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
2026-06-17 09:04:24 +08:00
Andy Ye
6aff1e8f9e fix(memory): report skipped QMD embedding probe (#93473)
Merged via squash.

Prepared head SHA: eea1ba563b
Co-authored-by: TurboTheTurtle <35905412+TurboTheTurtle@users.noreply.github.com>
Co-authored-by: vincentkoc <25068+vincentkoc@users.noreply.github.com>
Reviewed-by: @vincentkoc
2026-06-16 14:34:01 +08:00
openclaw-clownfish[bot]
e1d3f12d7f fix(memory): use per-keyword FTS search in hybrid mode #39484 (#73976)
Co-authored-by: openclaw-clownfish[bot] <280122609+openclaw-clownfish[bot]@users.noreply.github.com>
2026-06-16 14:25:02 +08:00
Vincent Koc
9b6bed7a75 fix(memory): release reindex lock after failed init 2026-06-16 04:04:19 +02:00
Vincent Koc
f87d194b8b fix(memory): prevent peer-write loss during reindex 2026-06-16 04:04:19 +02:00
Vincent Koc
ea346f4361 fix(sqlite): close databases after failed initialization 2026-06-16 03:00:23 +02:00
Alix-007
0d50ec77de fix(memory): swap rollback-journal sidecar during atomic reindex (#93295)
The atomic reindex file ops hardcoded the WAL sidecar pair (-wal/-shm)
when moving, removing, and backing up index files. NFS-backed memory
stores run SQLite under journal_mode=DELETE, which produces a
rollback-journal (-journal) sidecar instead. As a result an index swap
left the previous targets stale -journal next to the freshly published
2026-06-16 07:37:17 +08:00
xydt-tanshanshan
85a635368e fix(memory): prevent empty expected model in memory index identity
Guard memory index identity resolution against empty or whitespace provider models by falling back to fts-only, and use fts-only as the fallback source model when an adapter fallback cannot resolve a model.

This prevents empty expectedModel mismatch reasons that can leave memory search dirty while preserving registered adapter default-model resolution.

Refs #90787
2026-06-15 23:56:39 +08:00
openclaw-clownfish[bot]
93b7e3d717 fix(memory-core): safely refresh qmd index during collection repair
Squash merge ProjectClownfish replacement PR #92910.\n\nSource PR credit: thanks @imadal1n for the original index rewrite direction in #68590.
2026-06-15 23:34:01 +08:00
Alix-007
018d279468 fix(memory): clean rollback-journal reindex temp sidecar on NFS stores
cleanupAgedMemoryReindexTempFiles only removed WAL sidecars (-wal/-shm) of orphaned reindex temp DBs. On NFS-backed stores configureMemorySqliteWalMaintenance -> requireRollbackJournalMode forces journal_mode=DELETE, so the reindex temp DB uses a rollback journal; a hard crash leaves an orphaned .tmp-<uuid>-journal that leaked forever (cleanup neither deleted nor even discovered it). Add -journal to both the delete set (memoryIndexFileSuffixes) and the discovery set (reindexTempEntrySuffixes), with regression tests for the temp-plus-journal and stranded-journal cases.
2026-06-15 05:49:12 -07:00
Vincent Koc
dc46a67e80 fix(memory-core): remove redundant provider key assertion 2026-06-15 11:05:07 +08:00
mushuiyu_xydt
44e6caff54 fix(memory): accept local default model path migration (#92954)
* fix(memory): accept local default model path migration

Treat the official local default embedding model's hf URI and downloaded GGUF path identities as equivalent so upgraded local memory indexes do not pause solely on path-format changes.

* fix(memory): satisfy local identity lint

Avoid filtered array tail access in the local model filename helper while preserving the same compatibility behavior.

* fix(memory): preserve local embedding identity aliases

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
2026-06-15 09:29:42 +08:00