Commit Graph

292 Commits

Author SHA1 Message Date
zengLingbiao
a172857acc fix(memory-host-sdk): decode JSON response bodies with fatal UTF-8 validation (#111279) 2026-08-01 07:08:04 -07:00
Peter Steinberger
597699401a refactor(memory): use sqlite transcript identities (#117334) 2026-08-01 04:38:40 -07:00
Peter Steinberger
618f3298c7 fix(memory): reject malformed direct and batch embedding vectors (#117200)
Co-authored-by: Peter Steinberger <steipete@macos.shared>
2026-07-31 21:38:31 -07:00
Jason (Json)
cf927f7b1b fix(memory): keep rollbacks compatible after recall metadata upgrade (#116003)
* fix(memory): keep agent database rollbacks compatible

* fix(memory): satisfy rollback migration guards
2026-07-29 11:19:22 -06:00
Peter Steinberger
216f45af8e fix(memory): preserve Windows session ownership (#115851)
* fix(memory): preserve Windows session ownership

* test(memory): cover custom and nested Windows transcripts

* fix(memory): retain nested and custom Windows transcripts

* fix(memory): isolate agent-local Windows transcript stores

* style(memory): format Windows ownership regression coverage

* fix(memory): validate physical transcript ownership boundaries

* fix(memory): bind external session paths to configured agent stores

* fix(memory): reject unowned shared transcript store roots

* fix(memory): restrict transcript archives to agent-owned roots

* refactor(memory): preserve canonical transcript corpus contracts

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 07:11:24 -04:00
loong
7bdd1f646b fix(memory-host-sdk): resolve stable execPath for worker fork to survive Homebrew Node upgrades (#99318)
* fix(memory): survive Homebrew Node upgrades

Co-authored-by: 袁龙辉0668001277 <yuan.longhui@xydigit.com>

* test(whatsapp): isolate last-route coverage

* fix(deepinfra): preserve offline model compatibility

* test(memory): isolate migration cleanup lifecycle

* test: stabilize aggregate extension gates

* ci: retrigger pull request workflow

* test: preserve inherited Node options

* test(cli): tolerate cold hosted startup

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 06:49:02 -04:00
Peter Steinberger
20668eed03 fix(memory): quarantine writes from tainted agent turns (#115818)
* feat(memory): taint writes after network tools

* fix(memory): keep provenance mutations recoverable

* fix(memory): roll back failed provenance writes

* fix(memory): serialize provenance mutations

* docs(memory): explain flush provenance boundary

* refactor(memory): share provenance mutation wrapper

* docs(memory): clarify flush provenance fallback

* fix(memory): canonicalize provenance paths
2026-07-29 06:02:06 -04:00
Peter Steinberger
4232126bba chore: update dependencies across workspace (#115677)
* chore(deps): update dependencies

* fix(deps): restore CI compatibility
2026-07-29 05:16:42 -04:00
Peter Steinberger
b148150281 fix(memory): reconcile derived FTS schemas (#115775)
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 04:54:50 -04:00
Peter Steinberger
327432831a fix(memory): reject malformed QMD result entries (#115705)
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 04:36:06 -04:00
Peter Steinberger
1b6ad8da8f fix(memory): avoid phantom continuation pages (#115722)
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 04:24:19 -04:00
Peter Steinberger
5692baafc3 fix(memory): exclude annotation carriers from recall text (#115719) 2026-07-29 04:03:12 -04:00
Peter Steinberger
263d1763d7 fix(memory): fully disable derived FTS indexes (#115685)
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 04:01:25 -04:00
Peter Steinberger
d41a56fca1 feat(memory): isolate curated entries by project (#115438)
* feat(memory): scope recall to active projects

* feat(memory): preserve project scope in promotion

* docs(memory): explain project-scoped recall

* fix(memory): chunk curated files per entry

* refactor(memory): narrow project scope internals

* fix(memory): exclude annotations from concept tags

* fix(memory): tighten project isolation boundaries

* fix(memory): preserve project path casing

* fix(memory): prioritize project bootstrap candidates

* docs: refresh memory architecture map

* test(memory): cover project scope schema repair

* fix(memory): complete project isolation paths

* fix(memory): close project ranking checks

* fix(memory): reinitialize search provider before ranking

* fix(memory): tolerate missing bootstrap paths
2026-07-28 23:55:49 -04:00
Peter Steinberger
5922dd4380 fix(memory): fall back to keyword search when embeddings cannot start (#115397)
* fix(memory): degrade bootstrap failures to keyword search

* docs(memory): clarify recovery status projection

* fix(memory): handle bootstrap embedding failures

* fix(memory): trust resolved provider diagnostics

* fix(memory): keep degraded syncs keyword-only

* fix(memory): confirm embedding recovery before semantic mode

* fix(memory): restore semantic readiness after recovery
2026-07-28 21:01:44 -04:00
Peter Steinberger
f0bf74bad4 feat(memory): backfill retained session transcripts (#115162)
* feat(memory): enumerate retained session transcripts

* feat(memory): add session transcript backfill

* test(memory): cover session backfill workflows

* docs(memory): document session transcript backfill

* fix(memory): preserve oldest session backfill ordering

* fix(memory): harden session archive backfill

* test(memory): prove owner-turn provenance gating

* fix(memory): persist session backfill progress

* fix(memory): preserve ranged backfill progress

* fix(memory): order per-file backfill candidates

* fix(memory): retain paged backfill diary entries

* fix(memory): preserve rem backfill diary history

* fix(memory): adapt backfill to retained session identity

* fix(memory): keep rem backfill preview-only

* fix(memory): satisfy backfill CI guards
2026-07-28 08:12:30 -04:00
Peter Steinberger
28630a9a65 feat(memory): provenance-gated memory with dreaming on by default (#114819)
* feat(memory): add provenance and recall metadata to the memory index

* feat(memory): provenance-gated promotion and capture hygiene

* feat(dreaming): LLM consolidation with deterministic gates, on by default

* feat(active-memory): deterministic recall lane with escalation default

* feat(memory): user model file and standing intents

* docs(memory): document the memory architecture

* fix(memory): live-QA fixes — metadata writers, provenance classes, intent scope, claim accumulation
2026-07-28 06:04:25 -04:00
Peter Steinberger
98b8a18b4c fix(node): harden reconnect and local inference cancellation (#115033)
* fix(node): harden reconnect and local inference cancellation

* fix(browser): refresh bundled gateway client

* fix(gateway): preserve restart after reconnect reset

* fix(node): preserve tracked remote command ownership
2026-07-28 04:18:51 -04:00
Peter Steinberger
81628fb323 fix(memory): preserve QMD results in noisy command output (#115023) 2026-07-28 04:09:28 -04:00
Peter Steinberger
0d7fb8eb39 refactor(fs): adopt fs-safe 0.5 core primitives (#113705)
* refactor(fs): unify exclusive file publication

* fix(fs): fence stale lock reclamation

* refactor(fs): bound wiki scans and secret reads

* chore(fs): finalize fs-safe 0.5 compatibility

* fix(fs): preserve publication ownership and legacy mode

* fix(fs): fail closed on unverifiable lock owners

* fix(fs): preserve concurrent backup publications

* refactor(fs): preserve ambiguous backup outputs

* fix(fs): preserve mixed-version lock coordination

* refactor(file-transfer): adopt fs-safe archive extraction

* refactor(fs): add bounded walk and secret seams

* refactor(auth): replace proper-lockfile with fs-safe

* fix(fs): honor Windows mode override casing

* refactor(snapshot): adopt fs-safe publication

* refactor(memory-wiki): adopt prunable root walks

* refactor(fleet): adopt bounded archive restore

* fix(fs): preserve post-publication ownership receipts

* refactor(fs): harvest final fs-safe primitives

* style(fs): clean harvest lint

* chore(plugin-sdk): refresh move helper API baseline

* refactor(snapshot): adopt native Windows ACL facts

* refactor(fs): adopt hardened atomic outputs

* fix(fs): scope lock reentrancy to logical owners

* chore(config): lower env var count budget

* fix(deps): adopt published fs-safe 0.5.0

* fix(ci): align SDK surface ratchets

* fix(ci): regenerate SDK API baseline after rebase

* fix(fs): preserve owner-scoped file lock nesting

* fix(ci): refresh SDK API baseline for file locks

* fix(fs): separate SQLite and file lock reentrancy

* fix(imessage): bound pinned attachment reads

* fix(agents): narrow session-key lock options

* fix(fs): preserve fs-safe 0.5 compatibility contracts

* fix(windows): retain private SQLite directory owner

* refactor(sqlite): centralize exclusive coordinator

* refactor(snapshot): isolate Windows ACL policy

* fix(windows): retain snapshot ACL inspector

* chore(config): realign env budget after rebase

* test(agents): accept canonical sandbox escape error

* docs(changelog): defer fs-safe release note
2026-07-28 03:41:47 -04:00
Peter Steinberger
4273ca9dbd refactor(sessions): remove file-era transcript runtime (#113233)
* refactor(sessions): keep helper transcripts in memory

* refactor(sessions): remove file-era transcript storage

* test(sessions): use SQLite identity in attempt persistence

* test(codex): isolate legacy transcript fixtures

* fix(sessions): preserve SQLite transcript identity

* fix(sessions): harden transcript lifecycle invariants

* fix(sessions): validate transcript identities

* fix(sessions): close identity compatibility gaps

* fix(sessions): preserve leaf and plugin identities

* fix(sessions): retain dispatch transcript targets

* fix(sessions): preserve active transcript context

* fix(sessions): isolate artifact accounting

* fix(sessions): bound SQLite usage accounting

* fix(sessions): retain bounded latest usage

* fix(sessions): align rebased transcript targets

* test(sessions): align accessor scope fixture

* fix(telegram): derive SQLite transcript identity

* refactor(sessions): remove file-era compaction residue

* chore(sessions): lower max-lines baseline

* fix(sessions): preserve structured transcript identity

* test(sessions): align doctor identity assertions

* fix(sessions): isolate default SDK database

* refactor(sessions): remove dead file-era exports

* fix(sessions): reconcile SQLite transcript identity

* fix(sessions): pass checkpoint identity explicitly

* test(sessions): make entry field probe explicit

* test(sessions): satisfy transcript cleanup lint

* test(sessions): align diagnostics identity proof

* fix(sessions): finish transcript runtime teardown

* fix(sessions): preserve transcript identity invariants

* fix(sessions): harden transcript compatibility edges

* fix(sessions): preserve checkpoint transcript anchors

* fix(sessions): preserve SQLite lifecycle invariants

* fix(sessions): retarget compaction successors

* test(sessions): preserve transcript fixture semantics

* feat(plugin-sdk): add command transcript targets

* fix(sessions): serialize transcript rewrites

* fix(sessions): validate legacy successor identity

* fix(sessions): normalize compaction ownership

* fix(sessions): validate successor identity before adoption

* fix(sessions): preserve plugin transcript ownership

* fix(sessions): carry transcript identity through commands

* fix(sessions): import legacy checkpoint artifacts into SQLite

* fix(sessions): preserve successor transcript ownership

* fix(sessions): align transcript consumers with target identity

* fix(sessions): scope transcript token estimates

* fix(sessions): retain agent identity across lifecycle hooks

* fix(sessions): resolve scoped SQLite targets

* fix(sessions): isolate lifecycle transcript targets

* fix(sessions): validate compaction agent ownership

* fix(sessions): preserve reset and cleanup lifecycle

* fix(sessions): serialize prompt cleanup lifecycle

* fix(sessions): remove stale lock import

* fix(sessions): preserve reset target context

* fix(sessions): fence prompt reload takeover

* fix(sessions): unblock abort and default lifecycle reads

* fix(sessions): validate legacy successor scope

* fix(sessions): reject metadata-only runtime rows

* fix(sessions): propagate custom transcript stores

* fix(sessions): preserve adopted retry targets

* fix(sessions): allow unkeyed usage reads

* fix(sessions): harden runtime target boundaries

* fix(sessions): serialize retry transcript writes

* fix(sessions): bound prompt reload disposal

* fix(sessions): complete retry marker identity

* fix(sessions): keep legacy marker identity minimal

* test(sessions): tighten teardown fixture types

* fix(sessions): preserve compatibility target identity

* test(sessions): persist post-checkpoint boundary turn

* test(sessions): align runtime store mock contracts

* style(sessions): simplify persisted identity guard

* fix(sessions): prefer complete typed targets

* fix(sessions): recover legacy marker targets

* test(sessions): align marker lookup fixture scope

* fix(sessions): validate partial transcript targets

* fix(sessions): reconcile partial transcript identities

* fix(sessions): canonicalize compatibility identities

* test(sessions): cover compatibility aliases

* fix(sessions): adopt legacy successor identity

* fix(sessions): preserve usage read identity

* fix(sessions): preserve partial marker compatibility

* fix(sessions): validate legacy successor mappings

* fix(sessions): reconcile marker store mappings

* fix(sessions): preserve legacy fallback identity

* fix(sessions): harden marker alias resolution

* fix(sessions): prefer verified successor aliases

* fix(sessions): resolve preferred marker aliases

* fix(sessions): serialize cleanup admission

* fix(sessions): align marker lookup scopes

* fix(codex): type marker alias summaries

* style(sessions): satisfy changed lint

* test(sessions): align structured target assertions

* fix(sessions): reconcile latest identity contracts

* fix(sessions): validate transcript identity boundaries

* docs(sessions): explain stable registry keys

* fix(sessions): harden compatibility target round trips

* fix(sessions): port usage identity to split modules

* test(sessions): align subagent transcript identity

* fix(sessions): finish transcript identity migration

* fix(agents): route subagent completion capture through transcript targets

* fix(agents): settle SQLite prompt handoff during cleanup

* chore: shrink max-lines baseline after teardown

* fix(sessions): port teardown across split runtime owners

* fix(sessions): carry transcript targets through split owners

* test(agents): use SQLite compaction target in abort coverage

* chore: retain unrelated max-lines suppressions

* chore: shrink max-lines baseline after main splits

* style(agents): const compaction checkpoint locals

* fix(sessions): harden SQLite teardown boundaries

* test(sessions): use typed metadata in predicate isolation fixture

* test(agents): cover malformed settlement rejections lint-safely

* fix(sessions): close remaining SQLite identity races

* fix(agents): fail closed on incomplete successor targets

* fix(sessions): preserve transcript identity fallbacks

* fix(agents): preserve session-key abort admission

* fix(trajectory): validate incomplete export targets

* test(sessions): drop retired pricing cache imports

* fix(sessions): validate partial transcript identities

* fix(sessions): close transcript identity edge cases

* fix(plugins): reserve retired transcript locator slot

* fix(sessions): scope transcript locks by target

* style(sessions): simplify SDK initialization error

* fix(sessions): preserve initialized transcript state

* fix(codex): verify mirrored history session keys

* fix(sessions): reject stale transcript ownership

* fix(sessions): anchor asynchronous transcript ownership

* fix(sessions): measure active transcript state

* fix(sessions): preserve scoped transcript compaction

* fix(sessions): harden transcript identity and lifecycle

* fix(sessions): resolve scoped command transcript stores

* fix(sessions): make transcript appends failure-atomic

* fix(sessions): enforce scoped transcript ownership

* fix(sessions): reject cross-owner transcript handoffs

* fix(sessions): fence cleanup transcript ownership

* fix(sessions): retire stale write ownership contexts

* fix(sessions): preserve pending session migration state

* fix(sessions): validate migrated transcript ownership

* fix(sessions): validate usage transcript targets

* fix(sessions): clear predecessor transcript metadata

* fix(sessions): align durable session event targets

* fix(sessions): fence late prompt handoffs

* fix(sessions): fence lifecycle transcript fallbacks

* fix(sessions): bound zero-length memory capture

* fix(sessions): preserve transcript teardown ownership

* fix(sessions): reject duplicate cleanup ownership

* fix(sessions): serialize runtime writes with sqlite leases

* fix(sessions): close sqlite teardown concurrency gaps

* fix(sessions): preserve nested lifecycle failures

* fix(sessions): canonicalize sqlite transcript ownership

* fix(sessions): settle disposed prompt handoffs

* fix(sessions): resolve canonical attempt lock targets

* test(sessions): align canonical target fixtures

* test(sessions): retire redundant jsonl parser coverage

* refactor(sessions): split active transcript cursors

* test(memory): retire legacy marker fixture

* fix(sessions): preserve canonical transcript access after rebase

* fix(sessions): fence prompt lease and return transcript targets

* fix(sessions): colocate transcript leases with target store

* fix(sessions): canonicalize transcript lease and worker targets

* fix(sessions): preserve plugin and fork identity markers

* fix(sessions): complete sqlite transcript target migration

* fix(sessions): integrate canonical followup identity

* fix(sessions): preserve bounded transcript topology

* fix(sessions): validate transcript identity boundaries

* fix(context): separate caller and successor targets

* test(sessions): split persistence compatibility coverage

* test(sessions): preserve fixture topology efficiently

* chore(sdk): refresh plugin api baseline

* test(agents): align compaction lock target mocks

* test(sessions): seed malformed transcript fixtures directly

* fix(agents): canonicalize transcript compatibility inputs

* fix(agents): type optional tool result ids

* test(ci): stabilize loaded process timing

* test(tui): wait for collect queue admission
2026-07-27 22:33:24 -04:00
Peter Steinberger
a1e0c9ea09 refactor(packages): DM-policy contract suite, dead routes, package folds (#114776)
* test(channels): share DM policy contract suite

* refactor(memory): remove dead host CLI routes

* refactor(acp): use canonical normalization helpers

* refactor(web): fold provider runtime into core

* refactor(packages): remove dead runtime exports

* chore(knip): drop retired ACP entry roots
2026-07-27 20:20:13 -04:00
Peter Steinberger
269bc5c89e fix(cli): preserve machine-readable stdout (#113654)
Co-authored-by: 1052326311 <65798732+1052326311@users.noreply.github.com>
2026-07-27 05:44:16 -04:00
Jesse Merhi
d7627d6f4c refactor(prompt): use plain inbound context labels and drop system-tag sanitizer (#112000)
* refactor(prompt): plain inbound context labels with a provenance marker

Replaces trust-worded inbound context labels ("(untrusted metadata)",
"(untrusted, for context)") with plain labels plus a fixed provenance
marker suffix appended to every OpenClaw-injected context header.

Detection keys on the marker, not label text, so strippers stay correct
across UI, TUI, replay, /trace segmentation, memory recall, and the Swift
chat preprocessor. Drops sanitizeInboundSystemTags in favor of the marker
boundary plus trusted system-prompt narration.

Renames the untrusted-named plugin SDK context identifiers to
channel-provenance names, keeping deprecated aliases registered for
removal after 2026-09-08.

Adds `openclaw doctor --fix` migrations that rewrite legacy inbound
labels in stored SQLite transcripts and purge legacy envelope-
contaminated LanceDB recall rows.

* fix(ci): resolve gate failures for plain inbound context labels

- doctor sqlite readers: open read-only connections via openNodeSqliteDatabase
  so the Kysely connection-boundary guardrail holds; unexport the now-internal
  transcript snapshot type (Knip unused-export gate).
- compat registry: split the record table into registry-records.ts and
  plugin-sdk-subpath-records.ts. The new compat record pushed registry.ts past
  the 700-line oxlint cap; suppressions are disallowed, so follow the existing
  sibling record-module pattern. Public exports and PluginCompatCode literals
  unchanged.
- acp-runtime test: assert current finalization behavior (newline normalization
  only). The bracket de-fang and System: rewrite it expected were removed with
  sanitizeInboundSystemTags; forged system lines are neutralized at the
  system-event queue, the single chokepoint feeding the System:-per-line render.
- regenerate docs_map and the plugin SDK API baseline manifest.

* fix(prompt): harden inbound context label migration and drop in-band sanitizer

Review follow-ups on the plain-label + provenance-marker change:

- Remove src/security/system-tags.ts. Rewriting inbound text to neutralize
  look-alike `System:`/`[System]` markers corrupted legitimate user text and is
  not a real injection boundary; role separation plus external-content wrapping
  is. Explicit product decision, recorded at the system-event queue.
- Narrow the LanceDB legacy-row purge so it cannot delete benign memories. It
  now requires a complete known legacy sentinel line, a legacy label followed by
  a fenced JSON body, or the complete legacy external-content header. The prior
  predicates matched ordinary prose such as `Notes (untrusted metadata):`, and
  deletion is irreversible.
- Make explicit-empty canonical ChannelStructuredContext win over the deprecated
  alias via a present/absent result instead of collapsing `[]` to undefined.
- Keep `\r?` in the active-memory doctor rule. It is the only rule spanning the
  header's line break, migrated assistant rows skip newline normalization, and
  without it the marked-header replace wins and the body strips to empty. Added
  a CRLF regression test.
- Fix stale comments that described removed behavior, and cover the Swift
  prose-block strip path.

Claude-Session: https://claude.ai/code/session_01WNzsPddQmxy9Y7jKD4wAxH
2026-07-27 11:27:53 +10:00
yt2102
6bd8e0387c fix(memory): close previous embedding provider before replacement (#113471)
* fix(memory): close previous embedding provider before replacement

* fix(memory): increase embedding worker close grace period for slow hosts

* fix(memory): clear this.provider after close in resetProviderInitializationForRetry

Co-authored-by: Sanjay Santhanam <notifications@github.com>

* fix(memory): serialize embedding provider replacement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): unify provider transition lifecycle

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): join worker shutdown lifecycle

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): block worker restart during close

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retain failed provider retirements

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): drain retirements on manager close

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): make worker close joinable

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): preserve sync before provider retirement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): separate worker exit from disposal errors

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): avoid shutdown admission gap

Co-authored-by: yt2102 <yt2102@qq.com>

* style(memory): format provider lifecycle fix

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): bound embedding worker termination

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): fail closed on fallback initialization errors

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retry primary after fallback creation failure

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retain disconnected embedding workers until exit

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): isolate shared fallback transition failures

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): serialize scoped manager retirement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retain failed global manager closes

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): serialize manager admission with teardown

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): serialize outer manager lifecycle

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): close request-scoped embedding providers

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retire qmd managers before replacement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): support synchronous embedding cleanup

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): isolate manager lifecycle scopes

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(cli): close request-scoped embedding providers

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): canonicalize manager lifecycle ownership

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retry primary after null fallback result

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): retain failed embedding provider closes

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): lease closable embedding providers

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): drain retained embedding providers

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): join lazy fallback teardown

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): drain embedding providers after HTTP close

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retain failed qmd candidate cleanup

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): aggregate retained qmd teardown failures

Co-authored-by: yt2102 <yt2102@qq.com>

* refactor(memory): keep qmd lifecycle policy unchanged

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retain failed worker construction clients

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(gateway): scope local embedding retirement by provider

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): drain provider generations before retirement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): drain admitted operations before teardown

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): preserve provider identity through vector search

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): lease provider generation through embedding

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): lease provider through index publication

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): keep sync harness hooks optional

Co-authored-by: yt2102 <yt2102@qq.com>

* refactor(memory): own generations in sync lifecycle

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): preserve query runtime across retirement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): pin FTS-only sync generations

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retry failed worker construction cleanup

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): drain admitted searches before closing

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): close manager and gateway admission races

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): serialize qmd wrapper replacement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): serialize failed qmd retirement

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): satisfy qmd lifecycle type and deadcode gates

Co-authored-by: yt2102 <yt2102@qq.com>

* test(memory): type qmd lifecycle doubles

Co-authored-by: yt2102 <yt2102@qq.com>

* style(gateway): clarify created embedding provider

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): drain availability probes before close

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): preserve cleanup ownership without blocking fallback

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retire lancedb embedding providers

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): preserve generic provider cleanup receiver

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): close lancedb CLI embeddings

Co-authored-by: yt2102 <yt2102@qq.com>

* test(memory): normalize abort rejection reason

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): retain lancedb provider retirements

Co-authored-by: yt2102 <yt2102@qq.com>

* fix(memory): drain lancedb embedding uses

Co-authored-by: yt2102 <yt2102@qq.com>

---------

Co-authored-by: Sanjay Santhanam <notifications@github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-26 03:00:22 -04:00
Peter Steinberger
b2f6ecc2a9 chore(compat): date the annotated deprecation families and expose removal-pending debt (#114002)
* chore(compat): date the annotated deprecation families and expose removal-pending debt

Adds 10 dated compat-registry records for shipped deprecated surfaces that
previously had @deprecated annotations only (removeAfter 2026-10-01 per the
approved 60-day window), extends the beta5 session-store record with the
package-root aliases, surfaces removal-pending records with due dates,
blockers, and readers in the plugin boundary report, and converts undated
cleanup obligations (CI timing job, zizmor excessive-permissions, staged
Codex CI auth, memory-host legacy state dir, agent-DB schema-ladder floors)
into dated markers visible to the weekly dated-TODO sweep. Metadata,
diagnostics, and docs only - no runtime behavior change.

* fix(compat): keep extension src test paths out of core string literals

The core import guardrail forbids extensions/*/src/ path shapes in core
production string literals; cite the package-root google-meet test and the
registry test for official-plugin-export-aliases instead.
2026-07-25 21:01:20 -07:00
Peter Steinberger
446816f751 test: table-drive followup scenarios, centralize sanitizer wiring, remove obsolete negatives (#113583)
* test(auto-reply): table-drive followup runner scenarios

* test(channels): centralize sanitizer wiring coverage

* test: remove obsolete negative coverage
2026-07-25 02:14:44 -07:00
Peter Steinberger
3018e8b78a refactor: deduplicate micro helpers onto canonical homes (#113534)
* refactor: deduplicate micro helpers

* fix(ai): keep hash prefix helper internal
2026-07-25 00:41:01 -07:00
Vincent Koc
b4598365d9 refactor(packages): reuse shared error coercion (#113529) 2026-07-25 14:59:59 +08:00
Peter Steinberger
bb657eec93 refactor(config): retire redundant settings (#113174)
* refactor(config): retire redundant settings

* style: apply current formatter

* chore: update plugin sdk baseline

* fix: keep Codex tool caps context-aware

* chore: remove stale imports

* test: align WhatsApp QA debounce config

* fix(config): clean up retired config checks

* fix(ci): align config cleanup checks
2026-07-23 17:45:02 -07:00
Peter Steinberger
fb4338b03c test(memory): clear embedding watchdog timers (#112887) 2026-07-23 03:43:09 -04:00
Vito Cappello
4ab686220f fix(memory): recover from same-file legacy index divergence (#110216)
* fix(memory): keep canonical rows when same-file legacy memory tables diverge

* fix(memory): keep canonical-owned sources' chunk sets coherent during legacy import

* fix(memory): import legacy chunks when canonical source has no chunks

The same-file legacy migration excluded a legacy chunk from import whenever the
canonical index already had a source row for its (path, source). That stranded a
file whose canonical source was registered but had no chunks yet (indexing
interrupted before chunks were written, or embedding pending/failed): the legacy
chunks were its only searchable content, and the matching source hash stops sync
from re-indexing, so the file went silently unsearchable.

Re-key the chunk-coherence exclusion on canonical chunk ownership instead:
snapshot the (path, source) pairs that already have canonical chunks before the
import and skip legacy chunks only for those. A source with a canonical row but
no chunks now imports its legacy chunks. The snapshot is taken pre-insert because
the exclusion predicate reads the chunks table the import writes to.

Add regressions: legacy chunks import for a chunk-less canonical source while a
chunk-owning source still drops its stale legacy chunk; and restore abort
coverage for the meta and chunks copy assertions (previously only files was
exercised).

* fix(memory): harden same-file legacy conflict recovery

* fix(memory): rebuild ambiguous partial legacy sources

* fix(memory): reconcile migrated derived indexes

* fix(memory): close migrated index ownership gaps

* test(memory): align migration expectations

* test(tooling): match routed test order

* test(memory): exercise vector reload cleanup

* test(memory): prove real vector reload cleanup

* fix(memory): make migrated indexes converge

---------

Co-authored-by: Serhii Leniv <leniv.tech@gmail.com>
Co-authored-by: VACInc <3279061+VACInc@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-22 00:27:01 -07:00
Peter Steinberger
edecdbd05e refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527)
* refactor(config): consolidate media model lists

* refactor(config): unify memory configuration

* refactor(config): consolidate TTS ownership

* refactor(config): move typing policy to agents

* refactor(config): retire product-level config surfaces

* refactor(config): share scoped tool policy type

* chore(config): refresh generated baselines

* fix(config): honor agent typing overrides

* fix(config): migrate sibling config consumers

* refactor(infra): keep base64url decoder private

* fix(config): strip invalid legacy TTS values

* chore(config): refresh rebased baseline hash

* fix(doctor): route legacy messages.tts.realtime voice to talk during tts move

* refactor(config): polish final layout names

* refactor(config): freeze retired tuning defaults

* feat(config): add fast mode default symmetry

* refactor(config): key agent entries by id

* docs(config): update final layout reference

* test(config): cover final layout migrations

* chore(config): refresh final layout baselines

* fix(config): align final layout runtime readers

* fix(config): align remaining readers

* fix(config): stabilize final layout migrations

* fix(config): finalize config projection proof

* fix(config): address final layout review

* docs(release): preserve historical config names

* fix(config): complete keyed agent migration

* fix(config): close final migration gaps

* fix(config): finish full-branch review

* fix(config): complete runtime secret detection

* fix(config): close final review findings

* fix(config): finish canonical docs and heartbeat migration

* fix(config): integrate latest main after rebase

* refactor(env): isolate test-only controls

* refactor(env): isolate build and development controls

* refactor(env): collapse process identity indirection

* refactor(env): remove duplicate config and temp aliases

* docs(env): define the operator-facing allowlist

* ci(env): ratchet production variable count

* fix(env): remove stale provider helper import

* fix(env): make ratchet sorting explicit

* test(env): keep test seam in dead-code audit

* test(env): cover ratchet growth and boundary; document surface budgets

* docs(config): document tier-eval consolidations

* docs(config): clarify speech preference ownership

* test(memory): align retired tuning fixtures

* refactor(memory): freeze engine heuristics

* refactor(config): apply tier-eval tranche

* refactor(tts): move persona shaping to providers

* refactor(compaction): move prompt policy to providers

* test(config): align hookified prompt fixtures

* chore(deadcode): classify test-only exports

* chore(github): remove unused spawn helper

* chore(deadcode): classify queue diagnostics

* chore(deadcode): remove unused lane snapshot export

* chore(plugin-sdk): ratchet consolidated surface

* fix(config): integrate latest main after rebase
2026-07-21 20:28:43 -07:00
Harjoth Khara
9405f2c5db fix(ollama): scope memory cache identity by endpoint (#97059)
Preserve the shipped default identity while hashing custom header identity without retaining raw values.

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-21 02:23:18 -07:00
Alix-007
ad3b8e0b6f fix(memory): bound qmd taskkill cleanup (#109265)
* fix(memory): bound qmd taskkill cleanup

* test(memory): satisfy qmd process lint
2026-07-18 06:11:14 -07:00
Peter Steinberger
01a9e1d398 feat(memory): default cross-conversation recall for personal installs (#110597)
* feat(memory): default private recall for personal installs

* fix(memory): repair remember-across CI checks
2026-07-18 12:16:49 +01:00
VectorPeak
badb5ae706 fix(memory): reject malformed embedding batch JSONL bytes (#110502)
Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
2026-07-18 09:16:19 +01:00
Dave Morin
4b3ee5e7eb feat: let agents remember across private conversations (#100140)
* feat(memory): remember across private conversations

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>

* chore(docs): regenerate config baseline

* fix(memory): restore recall configuration wiring

* fix(memory): scope recall transcript indexing

* test(memory): repair conversation recall fixtures

* test(memory): split session visibility coverage

* style(memory): format type imports

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-18 08:21:43 +01:00
Edward Abrams
8fa4867ab4 fix(memory): avoid rebuilding unchanged QMD session exports (#77158)
* perf(memory): cache qmd exports by corpus revision

Co-authored-by: Edward Abrams <zeroaltitude@gmail.com>

* test(memory): track qmd exporter temp dirs

* test(memory): use plugin SDK temp fixture

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-16 23:22:40 -07:00
Erick Kinnee
571f1dcef9 fix(dreaming): drop heartbeat assistant responses from dream corpus (#109403)
* fix(dreaming): drop heartbeat assistant responses from dream corpus

The dreaming ingestion pipeline filters heartbeat user messages
(containing [OpenClaw heartbeat poll]) via sanitizeSessionText, but the
paired assistant response is only dropped when it is the exact string
HEARTBEAT_OK. Local models frequently respond with natural-language
acknowledgments (e.g. "Heartbeat received. Main is active.") which pass
through the sanitizer unchanged and enter the dream corpus as
low-confidence (0.58) memory snippets.

Fix: track heartbeat user message drops in buildSessionEntry and skip
the immediately following assistant response. This cross-message coupling
is safe because the heartbeat prompt pattern is injected by the runtime
and cannot be spoofed by user input, unlike [cron:...] or
System (untrusted): ... patterns (see PR #70737).

Refs: #103720

* test(dreaming): add heartbeat assistant response filter test

* fix(dreaming): use provenance-based heartbeat detection instead of text matching

Replace text-based heartbeat detection (isGeneratedHeartbeatPromptMessage)
with provenance-based authentication. The heartbeat turn now carries
provenance: { kind: "heartbeat" } when persisted to the transcript,
and buildSessionEntry checks for this provenance instead of matching
user-spoofable text content.

Changes:
- src/sessions/input-provenance.ts: Add "heartbeat" to
  INPUT_PROVENANCE_KIND_VALUES
- src/auto-reply/reply/get-reply-run.ts: Attach heartbeat provenance
  to user turn input when isHeartbeat is true
- packages/memory-host-sdk/src/host/session-files.ts: Check
  message.provenance.kind === "heartbeat" instead of text matching
- packages/memory-host-sdk/src/host/session-files.test.ts: Update
  heartbeat test to include provenance; add lookalike regression test

This addresses the ClawSweeper review finding [P1] about user-spoofable
text matching. The cross-message coupling is now authenticated by
runtime provenance, not user-typed content.

Refs: #103720

* fix(dreaming): add targeted heartbeat-derived corpus repair

* fix(dreaming): fix TS return type for clearScopedLegacySessionIngestionJson catch

* fix(dreaming): match chunked SQLite seen-state keys by stored scope

* fix(dreaming): repair SQLite-backed session transcript lookup in doctor

The findHeartbeatContaminatedCorpusLines function could not read
SQLite-backed session transcripts. When a corpus ref had no .jsonl
extension (SQLite logical path format), it appended .jsonl, read an
empty file, and silently skipped — leaving pre-fix heartbeat
contamination intact for SQLite users.

Fix:
- Add loadTranscriptLinesFromSqlite helper that reads events via
  the canonical loadTranscriptEventsSync API
- Serialize all events to preserve original event positions for
  corpus #L<n> line-number references
- Fall back to SQLite reader when filesystem read fails on a
  non-.jsonl corpus ref path

Test: add SQLite-backed session regression test that seeds a
session, writes corpus refs without .jsonl extension, and
verifies both audit detection and targeted repair.

* fix(dreaming): address autoreview P1 and P2 for checkpoint clearing and early return

P1: Remove clearScopedSessionIngestionState call after heartbeat line
removal. Clearing the cursor causes the next ingestion to re-process the
transcript from the beginning, duplicating normal corpus lines that were
deliberately retained.

P2: Remove early return after heartbeat cleanup so the self-ingestion
narrative content check and archiveDiary request still run.

* fix(dreaming): resolve corpus ref session path without duplicate agent ID

The session path in corpus refs is already in the format
'sessions/main/abc.jsonl' (includes agent subdirectory). The
previous code prepended source.agentId again, creating a path
like 'agents/main/sessions/main/abc.jsonl' — which silently
returned empty in production workspaces.

Fix: use path.basename to extract just the filename from the
session path, then construct the transcript path correctly.

* refactor(dreaming): split repair utils into separate file to satisfy LOC ratchet

The dreaming-repair.ts file grew from 337 to 831 lines, exceeding the
500-line LOC ratchet limit. Split helper utilities into a new
dreaming-repair-utils.ts file:

- dreaming-repair.ts (365 lines): imports, types, public API functions
  (auditDreamingArtifacts, repairDreamingArtifacts)
- dreaming-repair-utils.ts (498 lines): all helper functions, constants,
  and internal types

* fix(ts): add missing type imports for return types in dreaming-repair.ts

* fix(deadcode): remove unused exports from dreaming-repair-utils.ts

* fix: resolve merge conflict marker in get-reply-run.ts

* fix: remove unnecessary export from INPUT_PROVENANCE_KIND_VALUES

* fix(dreaming): converge targeted repair path with wholesale archive+clear

When heartbeat contamination is found, archive the entire session-corpus directory and clear all SQLite checkpoints instead of doing targeted line-by-line rewrite. This ensures the cleaned corpus gets re-ingested under the new provenance-based forward filter.

* fix(dreaming): authenticate heartbeat transcript turns

Co-authored-by: Erick Kinnee <1707617+ekinnee@users.noreply.github.com>

---------

Co-authored-by: Erick Kinnee <erick@ekinnee.dev>
Co-authored-by: Erick Kinnee <ekinnee@gmail.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-16 22:00:21 -07:00
Peter Steinberger
30ae21fca7 fix: prevent SQLite lock stalls and schema drift (#108663)
* fix: harden SQLite state storage

* build: allow strict SQLite migration SQL

* test: align SQLite and model-default assertions
2026-07-16 01:45:06 -07:00
Peter Steinberger
bae9752c5a refactor(deadcode): enforce repository hard zero (#108641) 2026-07-15 22:40:00 -07:00
Peter Steinberger
ff8a015981 test: tighten additional async polling (#108450) 2026-07-15 13:14:35 -07:00
Jason (Json)
0644956830 fix(sqlite): compaction works with memory path search enabled (#107809)
* fix(sqlite): accept canonical memory path FTS triggers

* test(sqlite): reject drifted path FTS trigger

---------

Co-authored-by: Josh Lehman <josh@martian.engineering>
2026-07-14 16:25:37 -07:00
wahaha1223
7f36f49052 fix(memory): preserve UTF-8 when qmd output splits across pipe chunks (#107263)
* fix(memory): preserve UTF-8 across fragmented qmd output chunks

* fix(memory): decode qmd streams at UTF-8 boundary

Co-authored-by: wahaha1223 <0668001153@xydigit.com>

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-14 09:32:25 -07:00
Peter Steinberger
f81f9d8570 chore: enforce max-lines suppression ratchet (#107315)
* ci: enforce max-lines suppression ratchet

* chore: prune stale max-lines suppression

* fix: close max-lines ratchet enforcement gaps

* fix: harden max-lines ratchet checks

* fix(ci): satisfy max-lines ratchet checks

* style: format max-lines declarations

* fix(ci): match oxlint suppression grammar

* test: isolate max-lines git fixtures

* chore: prune resolved max-lines debt

* test: skip newline path fixture on Windows

* fix: harden max-lines suppression ratchet

* chore: refresh max-lines baseline

* fix: close max-lines ratchet bypasses

* fix: derive ratchet base from PR merge tree

* fix: support older Git in staged ratchet

* fix: align max-lines declarations and baseline

* chore: refresh max-lines baseline for current main

* fix: exclude generated wizard locales from max-lines

* chore: prune resolved max-lines debt
2026-07-14 09:27:02 -07:00
qingminlong
8ffe428bd8 fix(memory): accept leading-zero Content-Length on JSON responses (#105916)
* fix(memory): accept leading-zero content lengths

* test(memory): keep leading-zero size caps

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-13 11:20:06 -07:00
Peter Steinberger
2aa4ff0825 refactor: consolidate duplicate helper exports (#106016)
* refactor(normalization): consolidate string helpers

* refactor(normalization): consolidate agent ids

* refactor(paths): consolidate user path resolution

* refactor(gateway): preserve transcript helper facade

* fix(normalization): align helper build aliases

* fix(memory): keep helper import line neutral

* chore(plugin-sdk): align consolidated helper surface

* refactor(normalization): reuse lowercase string helper

* refactor(normalization): reuse record guard

* refactor(normalization): share surrogate boundary helper

* refactor(agents): share token estimate constant

* refactor(memory): distinguish standalone helper contracts

* refactor(normalization): share error cause formatter

* refactor(config): distinguish eligibility predicates

* refactor(plugins): share registry state symbol

* refactor(cli): reuse outbound dependency adapter

* refactor(cron): distinguish positive duration parser

* fix(gateway): keep transcript helper private

* fix(paths): preserve public helper status

* refactor(channels): reuse registry normalizer

* refactor(agents): reuse agent core runtime facade

* refactor(auth): reuse locked profile upsert

* refactor(records): distinguish trap-safe guard

* refactor(migrations): distinguish sync directory helper

* test(plugins): drop stale duration alias expectations

* fix(channels): remove stale registry import

* chore(plugin-sdk): refresh helper API baseline

* fix(memory): keep renamed helpers private

* fix(plugins): keep registry state key private

* fix(plugin-sdk): tighten wildcard surface budget

* chore(plugin-sdk): refresh rebased helper API baseline
2026-07-13 03:22:00 -07:00
Peter Steinberger
1136757174 refactor: consolidate bounded concurrency on p-limit and p-map (#106002)
* refactor: adopt p-limit and p-map for bounded concurrency

* chore: normalize lockfile ordering

* fix: preserve memory host concurrency export

* refactor: keep media apply within size budget

* fix: drain memory concurrency before failure

* fix: satisfy memory concurrency guards

* chore: scope memory host dead-code checks

* chore: lower concurrency LOC baselines

* fix: preserve concurrency API surfaces

* chore: reconcile concurrency LOC baseline

* test: keep memory concurrency gate portable

* style: simplify concurrency drain

* ci: refresh deadcode export baseline
2026-07-13 01:53:28 -07:00
Peter Steinberger
adf6b08ab4 refactor: consolidate duration parsing and formatting (#105988)
* refactor: consolidate duration handling

* docs: leave release notes to release workflow

* fix: preserve build timing precision

* ci: align plugin sdk surface budgets
2026-07-12 23:38:22 -07:00