Commit Graph

372 Commits

Author SHA1 Message Date
Peter Steinberger
b7d6291bfd fix(memory): preserve Unicode in QMD snippets (#103996) 2026-07-11 01:28:29 +01:00
llagy007
56fb460b83 fix(memory): keep batch failure attempts finite (#103117)
* fix(memory): keep batch failure attempts finite

* fix(memory): own batch retry attempt accounting

Co-authored-by: llagy007 <0668001470@xydigit.com>

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-10 08:03:23 +01:00
Vincent Koc
6765eb0166 refactor(memory-core): localize internal declarations (#101969) 2026-07-07 19:31:25 -07:00
Vincent Koc
b88f80940a refactor: localize internal implementation symbols (#101831) 2026-07-07 10:23:36 -07:00
Vincent Koc
407443264c refactor: localize internal reply and plugin types (#101666) 2026-07-07 05:51:47 -07:00
Youssef Hemimy
333a77fde3 fix(memory-core): clamp widen-fallback kNN k to sqlite-vec 4096 limit (#96157)
* fix(memory-core): clamp widen-fallback kNN k to sqlite-vec 4096 limit

The widen fallback in searchVector re-runs the kNN with k = vectorCount
(the full vector table size), unclamped. sqlite-vec hard-caps `k` at 4096,
so once a memory index exceeds 4096 chunks any query that triggers the
fallback fails with "k value in knn query too large".

Clamp the widen re-query to MAX_VECTOR_KNN_K (4096). 4096 oversampled
candidates is far more than enough to feed top-N hybrid ranking, so there
is no meaningful recall loss.

* test(memory-core): cover widen-fallback kNN clamp to sqlite-vec 4096 ceiling

Add two regression tests for the searchVector widen-fallback clamp:
- asserts a >4096-vector widen re-query caps k at 4096 (out-of-range k
  would raise sqlite-vec "k out of range" and kill the whole search)
- asserts a sub-ceiling widen (30) passes through untouched, proving the
  clamp is a ceiling and not a fixed value

Uses a prepare-mock on the KNN/COUNT statements rather than materializing
4097+ real vectors plus the native extension; the assertion is purely on
the k bind handed to the widen re-query.

* fix(memory-core): preserve filtered recall beyond KNN cap

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-07-06 16:53:34 -07:00
xydt-tanshanshan
738654d20c fix(memory): add batch completed log after embedding batch finishes (#94732)
* [AI] fix(memory): add batch completed and batch failed logs for embedding ops

The embedBatchWithRetry function logged 'batch start' but never logged
'batch completed' or 'batch failed' after the embedding batch call,
leaving operators with no post-request feedback. When batches hang or
time out, only 'batch start' appears in logs with no diagnostic signal.

Add 'batch completed' log after runEmbeddingOperationWithTimeout success
and 'batch failed' log in the catch handler. This is an observability
improvement, not a functional fix for the underlying hang (#93312).

Related to #93312

* [AI] fix(memory): use formatErrorMessage for embedding batch error log

Replace String(err) with formatErrorMessage(err) in the batch failed
catch handler to redact sensitive provider error text (e.g. API keys
and tokens embedded in error messages) before logging.

Related to #94732

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-07-06 03:10:51 -07:00
Vincent Koc
c9abc75884 refactor: remove dead code and consolidate repeated paths (#100650)
* refactor(search): reuse provider setup finalizer

* refactor(android): remove unused helpers

* refactor(apps): remove unused Swift declarations

* refactor(diagnostics): reuse private reply delivery

* refactor(memory): reuse raw short-term store writer

* refactor(logbook): reuse batch row mapping

* refactor(scripts): centralize oxlint format policy

* refactor(memory): share qmd cache context hashing

* refactor(doctor): share auth issue classification

* chore(i18n): refresh native source inventory
2026-07-06 02:24:37 -07:00
Miorbnli
7d939b6456 fix(memory-core): guard qmd mcporter JSON.parse against non-JSON stdout (#98381)
* fix(memory-core): guard qmd mcporter JSON.parse against non-JSON stdout

runQmdSearchViaMcporter parsed mcporter subprocess stdout with JSON.parse
outside the runMcporter try/catch (qmd-manager.ts:2722). A non-JSON stdout
(daemon warning bleeding onto stdout, output truncated by maxOutputChars, CLI
killed early, or flag mismatch) threw a raw SyntaxError that propagated
uncaught out of runQmdSearchViaMcporter, surfacing in agent logs as a
context-free SyntaxError with no hint of the actual mcporter failure.

Wrap JSON.parse in try/catch and throw a typed Error carrying a stdout snippet
(matching the guard pattern already used in parseListedCollections in this
file, and the recent matrix #97973 / sms #97999 / signal #98073 /
telegram-ingress #98372 JSON.parse guard series).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory-core): preserve cause in qmd mcporter JSON.parse guard

Add { cause: err } to the re-thrown Error to satisfy the preserve-caught-error
lint rule; the original SyntaxError is now chained, improving diagnosability.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory-core): redact raw stdout from qmd mcporter error (security-boundary)

ClawSweeper flagged that the previous error message exposed raw mcporter
stdout (first 200 chars) before session visibility filtering, which could
leak sensitive content. Drop the stdout preview from the thrown message;
keep the original SyntaxError as `cause` for diagnostics so the parse-failure
reason is still reachable without surfacing unfiltered subprocess output.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory-core): keep qmd mcporter error message generic (no raw stdout leak)

The SyntaxError thrown by JSON.parse embeds a snippet of the raw input in its
message (e.g. Unexpected token '<', then the raw bytes). Including that
SyntaxError message in the thrown Error would surface unfiltered mcporter
stdout before session visibility filtering. Drop the parse-error message from
the thrown Error; keep the original SyntaxError as cause for developer
diagnostics only.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory-core): keep qmd mcporter cause generic (no raw stdout leak via formatErrorMessage)

formatErrorMessage walks the .cause chain into the user-visible path.
Keeping the JSON.parse SyntaxError on .cause leaked its embedded raw
stdout snippet through formatErrorMessage even with a generic message.
Give the cause a generic message too; the raw snippet no longer reaches
the user.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-06 06:57:41 +00:00
Peter Steinberger
b22c36f112 fix: land ten small reliability fixes (#100399)
* fix(cron): reject sub-millisecond durations

* fix(skill-workshop): preserve proposal terminal newline

Preserve proposal_content exactly at the agent tool boundary and make
renderProposalMarkdown defensively emit a terminal newline.

Add focused regressions for the tool write path and markdown renderer.

* fix(skill-workshop): reject blank raw proposal content

* fix: treat empty-string optional integer tool params as unset

Optional positive-integer tool params (e.g. Telegram replyTo/threadId)
threw ToolInputError when a tool-calling model populated them with an
empty-string or whitespace-only default. Those defaults carry no value,
so readPositiveIntegerParam/readNonNegativeIntegerParam now treat a
blank string as unset (undefined) instead of throwing, while still
rejecting genuinely invalid present values (0, "42.5", "-3"). This
prevents silent message-delivery failures when models emit empty
routing-param defaults. Adds unit tests covering blank vs invalid.

* fix(gateway): log start session persistence failures

The gateway session-lifecycle "start" event persistence
(persistGatewaySessionLifecycleEvent, which surfaces write failures via
requireWriteSuccess) was fired as void ...catch(() => undefined), swallowing
the rejection with zero logging. A failed start-marker write silently dropped
the run's start record from restart-recovery accounting, with no
operator-visible trace.

The sibling terminal-phase catch already logs this since #97839; the start
path was the unfixed sibling. Mirror that fix: log the swallowed start-phase
persistence failure with the same redacted message shape via formatForLog,
keeping the fire-and-forget semantics unchanged. Adds a focused regression
test asserting the log fires on start-persist rejection.

* fix(memory): report close-time pending work failures

* fix(shared): return "" from sliceUtf16Safe when end <= start, matching native .slice

sliceUtf16Safe silently swapped reversed bounds (to < from) instead of
returning "" like String.prototype.slice, creating a subtle footgun for
callers with dynamic start/end pairs.

Caller scan across src/, extensions/, and packages/ confirmed no production
code relies on the old swap behavior — all callers use (text, 0, N) or
(text, -N) forms only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(plugins): require plugin manifest in npm verifier

* fix(android): propagate CancellationException in CameraHandler catch blocks

Catch (err: Throwable) swallows kotlinx.coroutines.CancellationException,
breaking structured concurrency when the coroutine scope is cancelled
during camera operations (handleList/handleSnap/handleClip).

Add CancellationException rethrow before each Throwable catch to match
the existing pattern used in GatewaySession and TalkModeManager.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(android): propagate CancellationException past GatewaySession invoke boundary

* chore: sync native i18n inventory after gateway session line shift

* fix(agents): prevent native hook relay bridge race condition on renew and registration

Remove synchronous bridge record write after server.listen() that races
before the TCP server binds, and guard renew handler with server.listening
check to prevent stale relay registrations.

Closes #98650

* fix: harden small reliability fixes

Co-authored-by: cxbAsDev <cxbAsDev@users.noreply.github.com>

* docs(agents): explain buffered LSP spawn failures

* docs(agents): clarify LSP spawn timeout invariant

---------

Co-authored-by: qingminlong <qing.minlong@xydigit.com>
Co-authored-by: anyech <anyech@gmail.com>
Co-authored-by: snotty <snotty@users.noreply.github.com>
Co-authored-by: masatohoshino <g515hoshino@gmail.com>
Co-authored-by: lin-hongkuan <lin-hongkuan@users.noreply.github.com>
Co-authored-by: simon-w <weng.qimeng@xydigit.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: 宇宙熊Yzx <53250620+849261680@users.noreply.github.com>
Co-authored-by: xialonglee <li.xialong@xydigit.com>
Co-authored-by: nankingjing <1079826437@qq.com>
Co-authored-by: cxbAsDev <cxbAsDev@users.noreply.github.com>
2026-07-05 11:12:55 -07:00
Peter Steinberger
eafe2a8d0b refactor: consolidate duplicated plugin state and doctor migration plumbing onto SDK seams (#99850)
* refactor(plugin-sdk): add createPersistentDedupeCache and migrate channel presence caches

* refactor(matrix): adopt SDK approval reaction target store

* refactor(plugin-sdk): share doctor legacy-state migration fs helpers

* refactor(memory-core): dedupe qmd cache entry envelope validation

* chore(plugin-sdk): pin surface budgets for shared dedupe and doctor helpers

* test(matrix): use future approval expiry fixtures for reaction targets

* test(matrix): use future approval expiry fixtures for reaction targets
2026-07-04 01:51:03 -07:00
Dallin Romney
c7aca4f029 refactor: consolidate async timing helpers (#99721) 2026-07-03 17:58:15 -07:00
Dallin Romney
8c5f45fc36 refactor(shared): consolidate provider and utility lazy loaders (#98749)
* refactor(shared): consolidate provider lazy loaders

* refactor(shared): leave enforcement to follow-up
2026-07-02 21:16:26 -07:00
Yzx
39898fe91b test(memory): cover live manager after CLI reindex (#96094) 2026-07-01 01:59:20 -07:00
zw-xysk
742f0c9f82 fix(memory): detect unindexed session transcripts in status mode (fixes #97814) (#97857)
* fix(memory): detect unindexed session transcripts in status mode (fixes #97814)

The status-purpose MemoryIndexManager init skips ensureSessionStartupCatchup(),
so openclaw memory status reports dirty=false while unindexed session files
exist on disk. This is a false-clean state: the operator sees no backlog,
but memory search silently misses unindexed transcripts.

Fix: run markSessionStartupCatchupDirtyFiles() in status mode. It checks
for on-disk session files without corresponding memory_index_sources rows
and sets sessionsDirty=true if found, without triggering a full sync.

The full catchup+sync runs on the next non-transient manager cycle (CLI
sync or normal runtime init).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): await status dirty detection before status() reads the flag (fixes #97814)

The status-purpose MemoryIndexManager init skips ensureSessionStartupCatchup(),
so openclaw memory status reports dirty=false while unindexed session files
exist on disk. This is a false-clean state: the operator sees no backlog,
but memory search silently misses unindexed transcripts.

Fix: move status-mode markSessionStartupCatchupDirtyFiles() from the
constructor (fire-and-forget via void) into the create() factory method
where it is awaited. This guarantees sessionsDirty is set before any
caller reads manager.status(). The detection does NOT trigger a sync,
so status remains lightweight.

Review: verified manually — unit tests 21/21 pass, compilation 0 errors.

* test(memory): add regression test for status-purpose dirty detection (fixes #97814)

Adds a regression test that exercises the actual purpose:'status' manager
flow: create a session file without index row, create status manager, verify
status().dirty is true. This addresses the ClawSweeper P1 finding requesting
a test that exercises the real status path, not only the protected helper.

Also fixes the timing issue: move dirty detection from constructor (void)
to create() factory (await), guaranteeing sessionsDirty is set before any
caller reads manager.status().

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-30 17:47:43 -07:00
Brian Potter
cd31cbcd17 fix(memory-core): don't run the LLM reranker in vsearch/search modes (#88887)
vsearch/search are documented as vector/lexical-only modes, but
runQmdSearchViaMcporter omitted `rerank` from the query tool callArgs,
so QMD's query tool (which defaults rerank:true) ran the LLM reranker
anyway. Pass rerank:false for those modes; full "query" mode keeps it.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 18:11:03 -07:00
Bek
9636bea901 perf(memory): add QMD search diagnostics and runtime cache (#96655) 2026-06-26 16:16:12 -04:00
Shakker
1cd6f81a46 fix: close memory test failure paths 2026-06-26 14:05:07 +01:00
Shakker
338e119533 fix: preserve watcher test state 2026-06-26 13:05:44 +01:00
Shakker
e4f63577d0 test: scope manager reindex state 2026-06-26 12:56:21 +01:00
Shakker
94d93d4c85 fix: retain qmd test environment state 2026-06-26 12:48:24 +01:00
Shakker
7718e25b2a test: preserve memory startup env state 2026-06-26 12:40:55 +01:00
Shakker
8079aa62a2 fix: isolate memory index test state 2026-06-26 12:32:39 +01:00
Peter Steinberger
73dd758310 fix(memory): abort orphaned qmd search processes 2026-06-23 13:31:46 -07:00
Alix-007
eadd69b44c fix(memory): abort orphaned qmd subprocess on the mcporter search path too
The initial fix threaded the abort signal through the direct qmd
(runQmd/runQmdSearch) path, but the mcporter / QMD 1.1+ daemon search path
(runQmdSearchViaMcporter, runMcporterAcrossCollections) never received it, so
a grouped/mcporter search left its subprocess running on abort.

Thread the search signal through QmdMcporterSearchParams,
QmdMcporterAcrossCollectionsParams, all four mcporter call sites in search(),
and runMcporter, down to the shared runCliCommand spawn (which already
SIGKILLs the child on abort). Guard runQmdSearchViaMcporter on an
already-aborted signal so the multi-collection loop stops spawning. Reuses the
existing abort mechanism; no new machinery. Adds mcporter-path regression tests.
2026-06-23 13:31:46 -07:00
Alix-007
2a021f3b9b fix(memory): thread qmd search abort signal through grouped collection search
memory_search timeout cancellation only reached single-group direct qmd
searches. Multi-collection or mixed memory/session configs route through
runQueryAcrossCollectionGroups, which still called runQmdSearch without the
caller signal, so an aborted memory_search left the grouped qmd child running
until the qmd command timeout instead of being killed promptly.

Thread searchSignal through the grouped search path and its unsupported-option
fallback, and add a grouped multi-collection abort regression asserting the
spawned qmd child is SIGKILLed when the caller signal aborts.
2026-06-23 13:31:46 -07:00
Alix-007
78184ea7e4 fix(memory): abort orphaned qmd search subprocess when memory_search times out
PR #91742 wired memory_search's 15s deadline AbortSignal through the builtin
memory manager but missed the QMD backend behind the same
MemorySearchManager.search interface. With QMD, the tool returns "timed out
after 15s" to the agent while the spawned qmd query/search subprocess keeps
running for the full qmd command timeout (memory.qmd.limits.timeoutMs, whose
embed-heavy default was raised to 600s in #87572), leaving orphaned
embedding/search work running after the agent already moved on.

Add optional AbortSignal support to runCliCommand: an aborting signal kills the
spawned child immediately and rejects with the abort reason, funneled through a
single settle() guard so abort/timeout/error/close cannot double-settle. Thread
the search signal through QmdMemoryManager.search -> runQmdSearch -> runQmd ->
runCliCommand for the default direct-qmd subprocess path (including the query
fallback), and fast-fail search() when the signal is already aborted.
2026-06-23 13:31:46 -07:00
Josh Lehman
c24d266b2d refactor: use accessor-backed transcript corpus for memory (#96162)
* refactor: ratchet memory transcript corpus access

* test: use narrow runtime config snapshot import

* test: update plugin sdk surface budgets

* refactor: split memory transcript corpus module
2026-06-23 12:37:44 -07:00
Josh Lehman
475252453b refactor: add transcript update identity contract (#89912) 2026-06-23 05:52:08 -07:00
Josh Lehman
d3781cc4b8 refactor: add memory and QMD session identity mapping (#95087) 2026-06-22 06:28:54 -07:00
Vincent Koc
0ea39a2276 chore(deadcode): remove inert memory provider bootstrap 2026-06-22 02:30:56 +08:00
Vincent Koc
c372f6ef0b fix(plugins): keep extension tests on public boundaries 2026-06-21 15:39:51 +08:00
Vincent Koc
8797564254 chore(deadcode): share deferred test helper 2026-06-21 13:53:24 +08:00
Vincent Koc
ba34052a0e chore(deadcode): share memory vector blob encoding 2026-06-21 08:23:16 +08:00
Vincent Koc
7cc66b5175 refactor(channels): drop unused bootstrap exports 2026-06-19 15:32:30 +08:00
Vincent Koc
a9cc2697d3 test(memory): align metadata repair expectation 2026-06-19 01:20:06 +08:00
Vincent Koc
6e6bd5633f fix(memory): rebuild migrated search indexes 2026-06-19 01:20:06 +08:00
Josh Lehman
9cfd1cd287 fix(memory): key index sources by path and source 2026-06-19 01:20:06 +08:00
Josh Lehman
21163a704b fix(memory): make file index publication atomic 2026-06-19 01:20:06 +08:00
Vincent Koc
cc80128222 test(memory): type legacy vector migration fixture 2026-06-19 01:20:06 +08:00
Vincent Koc
7485dd6492 fix(sqlite): migrate legacy memory and proxy state 2026-06-19 01:20:06 +08:00
Vincent Koc
88bc08c124 refactor(memory): canonicalize agent database tables 2026-06-19 01:20:06 +08:00
Vincent Koc
acdcf37f87 chore(memory): simplify publish error 2026-06-19 01:20:06 +08:00
Vincent Koc
d64db0fc3d fix(memory): load vector extension before publish 2026-06-19 01:20:06 +08:00
Vincent Koc
25e68e9fa7 fix(memory): reject stale reindex publishes 2026-06-19 01:20:06 +08:00
Vincent Koc
1db7c41bf2 fix(sdk): keep sqlite lifecycle helpers private 2026-06-19 01:20:06 +08:00
Vincent Koc
a7fada4b61 fix(memory): invalidate stale shadow indexes 2026-06-19 01:20:06 +08:00
Vincent Koc
8b7269d197 fix(sqlite): preserve migrations and reindex safety 2026-06-19 01:20:06 +08:00
Vincent Koc
f324f7e281 refactor(memory): use the per-agent sqlite database 2026-06-19 01:20:06 +08:00
Vincent Koc
58b77e787d refactor(extensions): remove unused helper exports 2026-06-18 15:05:35 +08:00