Commit Graph

563 Commits

Author SHA1 Message Date
mushuiyu_xydt
3a1a5c0dac fix(memory): preserve qmd startup errors (#92618) 2026-06-13 02:59:14 -05:00
Ben Newell
cdb55b3edb fix(memory-core): retry narrative message reads (#89091)
* fix(memory-core): retry narrative message reads

* fix(memory-core): wait longer for narrative text
2026-06-11 20:36:18 +09:00
Dream Hunter
8d72cb9401 fix(memory): abort orphaned embedding work when memory_search times out (#91742)
* fix(memory): abort orphaned embedding work when memory_search times out

memory_search raced its 15s deadline with Promise.race and returned a clean
timeout to the agent, but the underlying embedQueryWithRetry loop kept
retrying (3 attempts x 60s) against the embedding backend with no consumer.
Thread the tool-owned AbortSignal through manager.search ->
embedQueryWithRetry -> runEmbeddingOperationWithTimeout so the deadline
cancels in-flight embedding work, stops the retry loop, and skips
fallback-provider activation for an absent caller.

Fixes #91718

* fix(memory): let the deadline result win before aborting the search

Abort listeners dispatch synchronously, so an abort-aware search could
reject the raced task before the timeout promise resolved and replace the
stable 'memory_search timed out after 15s' result with a provider-wrapped
abort error. Resolve the timeout first, then abort.

* fix(memory): scope deadline abort to builtin embeddings

* fix(memory): preserve deadline signal across fallback

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-06-11 20:36:11 +09:00
xydt-tanshanshan
865fdab075 fix(memory): preserve live SQLite index during swaps
Fixes #91216.

Preserve the live memory SQLite index during atomic reindex swaps by publishing the POSIX main DB with an overwrite rename, keeping target WAL/SHM sidecars rollbackable until publish succeeds, and refusing no-create post-swap reopens that would otherwise auto-create an empty DB.

Verification:
- node scripts/run-vitest.mjs extensions/memory-core/src/memory/manager.atomic-reindex.test.ts extensions/memory-core/src/memory/manager-db-probe.test.ts extensions/memory-core/src/memory/manager.self-heal-missing-identity.test.ts extensions/memory-core/src/memory/manager-reindex-state.test.ts extensions/memory-core/src/memory/manager.fts-only-reindex.test.ts extensions/memory-core/src/memory/manager.readonly-recovery.test.ts
- git diff --check origin/main...HEAD
- node scripts/run-oxlint.mjs extensions/memory-core/src/memory/manager-atomic-reindex.ts extensions/memory-core/src/memory/manager.atomic-reindex.test.ts extensions/memory-core/src/memory/manager-db.ts extensions/memory-core/src/memory/manager-db-probe.test.ts extensions/memory-core/src/memory/manager-sync-ops.ts
- .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main
- GitHub CI on 60df2b4178
2026-06-11 19:15:20 +09:00
Vincent Koc
7b0b6290b5 test: stabilize full Mac regression suite 2026-06-11 11:10:00 +09:00
Vincent Koc
f03e3372b3 test(memory): type self-heal identity assertions 2026-06-11 10:10:30 +09:00
xydt-tanshanshan
1a2eb74b9d fix(memory): self-heal missing index identity by initializing provider during sync (#91897)
* [AI] fix(memory): self-heal missing index identity by initializing provider during sync

When a gateway's memory index identity becomes "missing" with chunks
already indexed, canRebuildMissingIdentity stays false because
this.provider is null (async provider init hasn completed yet), so
needsMissingIdentityReindex is false, and the sync loop bails out
with dirty=true forever — the gateway never self-heals.

Now, when indexIdentity.status is "missing" and a provider is
configured but this.provider is null, runSync() calls
ensureProviderInitialized() first, then re-evaluates the identity
state. If the provider becomes available,
canRebuildMissingIdentity flips to true, unlocking the self-heal
reindex path.

Refs #91167

* [AI] fix(memory): allow FTS-only self-heal when chunks are all FTS-only and provider unavailable

When a gateway's memory index identity is 'missing' with chunks already
indexed, canRebuildMissingIdentity stays false if the embedding provider
is unavailable, causing the sync loop to bail out with dirty=true forever.

The previous approach (calling ensureProviderInitialized inside runSync)
was redundant because the public sync() method already initializes the
provider before runSyncWithReadonlyRecovery.

The real fix: when every existing chunk has model='fts-only', rebuilding
the index as FTS-only is safe — no semantic data is lost. So
canRebuildMissingIdentity should also be true when hasOnlyFtsChunks,
even if the provider is unavailable.

Also adds hasSemanticChunks() helper to detect whether any chunks have
a non-fts-only model.

Non-forced test: seeds FTS-only chunks with no meta, syncs without
force, verifies identity transitions from 'missing' to 'valid'.

Refs #91167

* [AI] fix(memory): gate hasSemanticChunks scan to missing-identity path only

Only compute hasOnlyFtsChunks when identity is missing, chunks exist,
and the provider is unavailable. This avoids scanning the chunks table
for model classification on every ordinary sync.

* test(memory): protect semantic index self-heal

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-06-11 09:53:01 +09:00
AMARA
f049477dd4 fix(memory): write dream fallback without subagent runtime (#90121) 2026-06-11 08:29:22 +09:00
zengLingbiao
15498f88fb fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes #92017) (#92020)
* fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes #92017)

* fix: add SQLite-only regression tests for dreaming ingestion audit (fixes #92017)
2026-06-11 08:26:50 +09:00
Vincent Koc
797aab5ab7 test(memory): create rem preview source fixture 2026-06-11 03:52:40 +09:00
Vincent Koc
dbcae5b78d fix(memory): keep ignored-name QMD roots watchable
Fix QMD watcher ignore handling for explicitly configured roots whose directory names are normally ignored, and prefer the most-specific configured watch root for overlapping collections.

Validated with focused QMD/tooling tests, full core support boundary tests, green CI, and ClawSweeper re-review.
2026-06-11 00:12:15 +09:00
Andy Ye
9408380ae7 fix(memory-core): keep qmd json search one-shot (#91837) 2026-06-10 16:30:15 +09:00
Vincent Koc
7b7e8f6e88 test(memory): stop asserting private sync return 2026-06-10 15:23:58 +09:00
Vincent Koc
92418fc9da fix(memory-core): filter stale recall entries in REM harness preview 2026-06-10 14:35:44 +09:00
mushuiyu_xydt
a36e05050a fix #88009: [Feature]: batched memory embedding should batch over files (#89138)
Merged via squash.

Prepared head SHA: 66d362a56d
Co-authored-by: mushuiyu886 <266724580+mushuiyu886@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-06-09 06:38:30 -04:00
Onur Solmaz
3137110167 fix(memory): move local llama.cpp runtime to provider plugin
* fix(memory): move local llama.cpp runtime to provider plugin

* chore: ignore llama cpp dynamic dependency

* test: remove invalid local provider alias fixture

* chore: refresh llama cpp shrinkwrap

* chore: drop stale memory embedding defaults facade
2026-06-09 14:30:35 +08:00
Vincent Koc
329fa44d23 fix(memory-core): write deep sleep summaries to dreams 2026-06-09 11:31:31 +09:00
Vincent Koc
20604f7a8f test(memory-core): seed dreaming session store 2026-06-09 01:02:22 +02:00
Sasan Sotoodehfar
22bda60cbe fix(memory): rebind qmd collections when a collection root changes
ensureCollections() never rebound a managed collection after its root
path changed (e.g. an agent workspace repoint): listCollectionsBestEffort()
read `qmd collection list`, whose output carries no filesystem path, so
listed.path was always undefined and shouldRebindCollection took its
defensive branch and skipped the rebind. The collection stayed pinned to
the stale root and recall kept resolving the old location.

Enrich the listed collections with their path via `qmd collection show`
so path-change detection works and the rebind fires.

Closes #91251

(cherry picked from commit 4a225011e9)
2026-06-09 01:10:19 +09:00
Vincent Koc
d46dc39b18 fix(memory): rebuild missing index metadata safely
Gateway/background sync now repairs missing memory index metadata with the existing full reindex path when the configured embedding provider is available, while preserving dirty/paused state instead of downgrading semantic chunks when embeddings are unavailable.

Fixes #90338
2026-06-08 23:58:10 +09:00
rudi193-cmd
e3ef136bca fix(memory): keep FTS keyword search model agnostic
Make lexical FTS/LIKE search ignore embedding model identity so exact keyword recall survives provider/model changes. Vector search remains model-scoped, and refreshed or stale FTS rows are cleaned by path/source with live-chunk filtering to prevent old orphan rows from surfacing.

Fixes #48300
2026-06-08 23:38:46 +09:00
张贵萍0668001030
3e0f7e4931 fix(memory-core): preserve sqlite-vec table on unsafe reindex 2026-06-08 23:29:42 +09:00
Peter Steinberger
08ae0e6d29 refactor: store Zalo hosted media in plugin state
Move Zalo hosted outbound media metadata and expiry into plugin state, add SDK chunked hosted media storage, and keep CI/type/lint gates green after rebase.
2026-06-06 22:56:48 -07:00
Peter Steinberger
3f5e001844 fix: store memory-core dreams state in sqlite (#91056) 2026-06-06 18:38:45 -07:00
Onur Solmaz
0aea58ab66 fix(memory): fail fast when embeddings provider is unavailable
Fixes #89691.

Memory search now treats explicitly configured non-local embedding providers as required. When that provider is unavailable, search and sync surface an unavailable memory-search result instead of silently returning FTS-only recall.

Unset/default/local/none-style paths keep FTS fallback so existing workflows do not lose keyword recall entirely. The fallback state is now surfaced in diagnostics/status instead of being hidden.

Maintainer merge note: current CI still has unrelated baseline boundary failures in extensions/google/google.live.test.ts and extensions/minimax/minimax.live.test.ts. This PR does not touch those files; the PR-specific memory, docs, lint, type, security, and ClawSweeper checks were reviewed before merge.
2026-06-06 20:39:38 +08:00
clawsweeper[bot]
50aaf1f9b6 fix(memory): resolve adapter default model in plain status identity check (#90816)
Summary:
- This PR updates memory-core index identity resolution to treat an empty configured model as the embedding adapter default and adds a regression test for plain memory status.
- PR surface: Source +5, Tests +33. Total +38 across 2 files.
- Reproducibility: yes. from source and inherited proof: current main compares identity against an unresolved empty model in the plain status path, and the source PR shows the before/after CLI behavior on the same index.

Automerge notes:
- PR branch already contained follow-up commit before automerge: fix(memory): resolve adapter default model in plain status identity c…

Validation:
- ClawSweeper review passed for head 9741437564.
- Required merge gates passed before the squash merge.

Prepared head SHA: 9741437564
Review: https://github.com/openclaw/openclaw/pull/90816#issuecomment-4637058847

Co-authored-by: 宇宙熊Yzx <53250620+849261680@users.noreply.github.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
2026-06-06 02:34:29 +00:00
Onur Solmaz
0dbf17471b feat(memory): support qmd query rerank toggle
Add memory.qmd.rerank as an opt-out for QMD query reranking when searchMode is query.

When set to false, direct QMD query calls pass --no-rerank and the mcporter unified query tool receives rerank:false. Search and vsearch modes keep their existing behavior.

Refs #61834.
2026-06-05 11:18:57 +08:00
Peter Steinberger
6868cde4d4 docs: document large extension sources 2026-06-04 21:40:44 -04:00
Dallin Romney
f0237caf27 fix(memory): warn after startup watcher pressure check (#89244)
* fix(config): warn only for large memory watch paths

* fix(config): align memory watch warning scan roots

* fix(config): import memory search config type

* fix(config): align memory watch warning sources

* fix(config): match memory source fallback

* fix(memory): warn from runtime watcher pressure

* fix(memory): warn after watcher fallback pressure

* fix(memory): keep watcher pressure warning startup scoped

* fix(memory): delay startup watcher pressure sample

* chore(memory): clarify watcher pressure comments

* test(memory): remove redundant watcher assertion

* refactor(memory): share watcher pressure warning policy

* refactor(memory): trim watcher pressure duplication

* refactor(memory): flatten watcher pressure calls

* docs(readme): use contributor-facing community wording

* Revert "docs(readme): use contributor-facing community wording"

This reverts commit 796bc03913.
2026-06-03 11:57:35 -07:00
Vincent Koc
6c7644268f fix(test): stabilize ARM extension timer tests 2026-06-02 07:53:25 -07:00
Vincent Koc
36c1a3e006 fix(memory): avoid optional vector status dereference 2026-06-02 14:53:35 +02:00
Vincent Koc
212eaead01 fix(memory): force provider-none indexes to FTS-only vectors 2026-06-02 14:53:35 +02:00
Vincent Koc
0b7c94a5e1 fix(memory): initialize provider-none lifecycle during sync 2026-06-02 14:53:35 +02:00
Vincent Koc
0b61add479 fix(memory): report provider-none probes as FTS-only 2026-06-02 14:53:35 +02:00
Vincent Koc
5a4f868de0 fix(memory): scope provider-none FTS bypass 2026-06-02 14:53:34 +02:00
Vincent Koc
cd0af35e5c fix(memory): keep FTS-only sync offline 2026-06-02 14:53:34 +02:00
clawsweeper[bot]
abc3fa0396 fix(memory-core): keep startup cron retries quiet (#89075)
Summary:
- The branch adds a memory-core `startup_retry` reconciliation mode and regression tests for quiet startup retries, retry-window exhaustion, and live-config retry semantics.
- PR surface: Source +9, Tests +114. Total +123 across 2 files.
- Reproducibility: yes. from source: current main routes the first startup retry through runtime reconciliatio ... st expects the warn-level `cron service unavailable` log. I did not execute tests in this read-only review.

Automerge notes:
- Ran the ClawSweeper repair loop before final review.
- Included post-review commit in the final squash: fix(memory-core): keep startup cron retries quiet

Validation:
- ClawSweeper review passed for head 7220f940d0.
- Required merge gates passed before the squash merge.

Prepared head SHA: 7220f940d0
Review: https://github.com/openclaw/openclaw/pull/89075#issuecomment-4592446250

Co-authored-by: bennewell35 <newelljben@gmail.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
2026-06-02 09:09:52 +00:00
Onur Solmaz
a4b4fed412 fix(memory): validate memory index identity
* docs: add memory index identity plan

* fix(memory): validate memory index identity

* fix(memory): align status index identity with vector probe

* fix(memory): fail closed on stale fts-only search

* fix(memory): clear sessions-only identity reindex dirty state

* fix(memory): gate targeted session sync by index identity

* fix(memory): clear resolved index identity dirtiness

* fix(memory): block search on missing index identity

* fix(memory): preserve dirty events during identity reindex

* fix(memory): resolve provider aliases for index identity

* fix(memory): report missing identity states accurately

* fix(memory): mark missing session index identity dirty

* test(memory): expose provider alias resolver in mocks

* chore(memory): remove scratch implementation plan

* fix(memory): avoid automatic full reindex on provider cutover

* docs(memory): plan no-schema cutover repair

* fix(memory): pause vector search on index identity mismatch

* fix(memory): freeze dirty identity sync writes

* fix(memory): skip paused-index search retry

* test(memory): keep retry tests on same provider identity

* fix(memory): surface paused index recall

* chore(memory): remove scratch plan from pr

* fix(memory): preserve paused session dirtiness

* fix(memory): make paused recall warning explicit

* docs(memory): document explicit index repair
2026-06-02 14:22:25 +08:00
Dallin Romney
0904f3e553 revert: undo gateway memory watch warning (#89246) 2026-06-01 15:32:42 -07:00
Peter Steinberger
65d7fa2420 fix(memory): reattach Linux watchers on directory rename
(cherry picked from commit 0db7781514)
2026-06-01 23:15:00 +01:00
Dallin Romney
2405bbcbaf fix(memory): warn on gateway watcher FD risk (#89185)
* fix(memory): default gateway memory watch off

* fix(memory): warn on gateway watcher fd risk

* fix(config): avoid warning helper narrowing

* fix(config): remove redundant warning boolean cast

* docs(memory): clarify watcher default wording

* docs(memory): simplify watcher warning copy

* fix(config): scope watcher warning to local gateway
2026-06-01 14:23:25 -07:00
Dallin Romney
5a67c5c556 fix(memory-core): reduce Linux watcher fan-out (#89188)
* fix(memory-core): reduce Linux watcher fan-out

* fix(memory-core): satisfy watcher type and lint checks

* fix(memory-core): harden Linux watcher subtree races
2026-06-01 12:54:30 -07:00
NianJiu
5a55135146 fix(memory): retry transient FileProvider-backed reads (#85351) 2026-06-01 12:40:20 -07:00
Peter Steinberger
ec2455a842 test(memory): drive timeout tests with explicit fake clocks
(cherry picked from commit d75eea53c9)
2026-06-01 13:12:07 +01:00
Peter Steinberger
7d9fae5b3a fix(memory): keep embedding timeout watchdog active
(cherry picked from commit 591f310869)
2026-06-01 12:29:27 +01:00
Peter Steinberger
4e57546a87 test(memory): isolate qmd timer state in prerelease shard 2026-06-01 06:03:43 +01:00
Firas Alswihry
70c59f59b2 feat(dreaming): score candidates with shadow trial results
Add report-only memory-core dreaming shadow trial scoring and ranking helpers. Keep rank lookup keyed by durable candidate keys and document the advisory behavior. Thanks @iFiras-Max1.
2026-05-31 23:40:20 -04:00
Peter Steinberger
ebcdb637bb perf(memory-core): defer embedding engine startup imports 2026-06-01 04:22:22 +01:00
Ted Li
c002887223 fix(memory): rehydrate daily list promotions
* fix(memory): rehydrate daily list promotions

* fix(memory): preserve multi-line daily list promotions

* fix(memory): preserve daily list promotion context

* fix(memory): rehydrate capped daily list promotions

* test(memory): cover capped daily list promotion

* test(agents): update model selection mocks

* ci: ignore lazy three dependency

* fix(memory): skip heading-only rehydration

* fix(memory): preserve list rehydration mode

* fix(memory): match capped renamed heading bodies

* fix(memory): avoid duplicate tail heading matches

* fix(microsoft-foundry): satisfy provider lint

* perf(memory): precompute promotion heading context

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-05-31 22:08:35 -04:00
Peter Steinberger
27dde7a4d6 chore(lint): enable stricter error rules 2026-06-01 01:12:21 +01:00