Commit Graph

24 Commits

Author SHA1 Message Date
Vignesh Natarajan
1a0c3bf400 Memory: fix FTS-only branch compile on rebased main 2026-03-29 00:09:33 -07:00
Vignesh Natarajan
598f539be5 Memory: keep FTS-only indexing on reindex (#42714) 2026-03-29 00:06:49 -07:00
opriz
41c30f0c59 fix: populate FTS-only memory search without provider (#56473) (thanks @opriz)
* fix(memory): build FTS index when no embedding provider is available

* fix(memory): trigger full reindex on provider→FTS-only transition

* fix(memory): return FTS-only keyword hits at default threshold

* fix: keep FTS-only memory hits at default threshold (#56473) (thanks @opriz)

---------

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-03-29 12:22:35 +05:30
Tak Hoffman
3ce48aff66 Memory: add configurable FTS5 tokenizer for CJK text support (openclaw#56707)
Verified:
- pnpm build
- pnpm check
- pnpm test -- extensions/memory-core/src/memory/manager-search.test.ts packages/memory-host-sdk/src/host/query-expansion.test.ts
- pnpm test -- extensions/memory-core/src/memory/index.test.ts -t "reindexes when extraPaths change"
- pnpm test -- src/config/schema.base.generated.test.ts
- pnpm test -- src/media-understanding/image.test.ts
- pnpm test

Co-authored-by: Mitsuyuki Osabe <24588751+carrotRakko@users.noreply.github.com>
2026-03-28 20:53:29 -05:00
Vignesh Natarajan
19e52a1ba2 fix(memory/qmd): honor embedInterval independent of update interval 2026-03-28 18:05:05 -07:00
Vignesh Natarajan
4e74e7e26c fix(memory): resolve slugified qmd search paths (#50313) 2026-03-28 16:26:38 -07:00
buyitsydney
4b69c6d3f1 fix(memory): add CJK/Kana/Hangul support to MMR tokenize() for diversity detection
The tokenize() function only matched [a-z0-9_]+ patterns, returning an
empty set for CJK-only text. This made Jaccard similarity always 0 (or
always 1 for two empty sets) for CJK content, effectively disabling MMR
diversity detection.

Add support for:
- CJK Unified Ideographs (U+4E00–U+9FFF, U+3400–U+4DBF)
- Hiragana (U+3040–U+309F) and Katakana (U+30A0–U+30FF)
- Hangul Syllables (U+AC00–U+D7AF) and Jamo (U+1100–U+11FF)

Characters are extracted as unigrams, and bigrams are generated only
from characters that are adjacent in the original text (no spurious
bigrams across ASCII boundaries).

Fixes #28000
2026-03-28 09:19:52 +05:30
Peter Steinberger
ef1784d264 refactor: move bundled plugin policy into manifests 2026-03-27 16:40:27 +00:00
Tak Hoffman
f5643544c2 Reduce lint suppressions in core tests and runtime 2026-03-27 02:11:26 -05:00
Peter Steinberger
a3e73daa6b refactor: remove ollama legacy shims 2026-03-27 06:38:23 +00:00
Peter Steinberger
2f979e9be0 test: fix memory-core host type import 2026-03-27 05:38:58 +00:00
Peter Steinberger
4c27c90fc2 refactor: finish moving provider runtime into extensions 2026-03-27 05:38:58 +00:00
Peter Steinberger
64bf80d4d5 refactor: move provider runtime into extensions 2026-03-27 05:38:58 +00:00
Peter Steinberger
eebce9e9c7 refactor: move memory host into sdk package 2026-03-27 04:12:04 +00:00
Peter Steinberger
a9b982c954 refactor: remove memory-core engine barrel 2026-03-27 03:35:00 +00:00
Peter Steinberger
f6de4cd766 refactor: remove memory-core runtime barrel 2026-03-27 02:54:23 +00:00
Peter Steinberger
bd6c7969ea refactor: extract memory host sdk package 2026-03-27 02:49:33 +00:00
Peter Steinberger
7a35bca2ec refactor: make memory embedding adapters generic 2026-03-27 02:02:24 +00:00
Peter Steinberger
dbf78de7c6 refactor: move memory engine behind plugin adapters 2026-03-27 00:47:01 +00:00
Peter Steinberger
cad83db8b2 refactor: move memory engine into memory plugin 2026-03-26 23:20:35 +00:00
Peter Steinberger
b96fccadb9 refactor: clean memory plugin host boundary 2026-03-26 23:02:24 +00:00
Peter Steinberger
0f5a77d058 refactor: route memory runtime through memory plugin 2026-03-26 22:30:47 +00:00
Peter Steinberger
d0ce2d1044 refactor: split memory-core plugin helpers 2026-03-26 22:06:06 +00:00
Peter Steinberger
3d0050c306 refactor: add memory-core extension sources 2026-03-26 22:00:13 +00:00