* fix(dreaming): drop heartbeat assistant responses from dream corpus
The dreaming ingestion pipeline filters heartbeat user messages
(containing [OpenClaw heartbeat poll]) via sanitizeSessionText, but the
paired assistant response is only dropped when it is the exact string
HEARTBEAT_OK. Local models frequently respond with natural-language
acknowledgments (e.g. "Heartbeat received. Main is active.") which pass
through the sanitizer unchanged and enter the dream corpus as
low-confidence (0.58) memory snippets.
Fix: track heartbeat user message drops in buildSessionEntry and skip
the immediately following assistant response. This cross-message coupling
is safe because the heartbeat prompt pattern is injected by the runtime
and cannot be spoofed by user input, unlike [cron:...] or
System (untrusted): ... patterns (see PR #70737).
Refs: #103720
* test(dreaming): add heartbeat assistant response filter test
* fix(dreaming): use provenance-based heartbeat detection instead of text matching
Replace text-based heartbeat detection (isGeneratedHeartbeatPromptMessage)
with provenance-based authentication. The heartbeat turn now carries
provenance: { kind: "heartbeat" } when persisted to the transcript,
and buildSessionEntry checks for this provenance instead of matching
user-spoofable text content.
Changes:
- src/sessions/input-provenance.ts: Add "heartbeat" to
INPUT_PROVENANCE_KIND_VALUES
- src/auto-reply/reply/get-reply-run.ts: Attach heartbeat provenance
to user turn input when isHeartbeat is true
- packages/memory-host-sdk/src/host/session-files.ts: Check
message.provenance.kind === "heartbeat" instead of text matching
- packages/memory-host-sdk/src/host/session-files.test.ts: Update
heartbeat test to include provenance; add lookalike regression test
This addresses the ClawSweeper review finding [P1] about user-spoofable
text matching. The cross-message coupling is now authenticated by
runtime provenance, not user-typed content.
Refs: #103720
* fix(dreaming): add targeted heartbeat-derived corpus repair
* fix(dreaming): fix TS return type for clearScopedLegacySessionIngestionJson catch
* fix(dreaming): match chunked SQLite seen-state keys by stored scope
* fix(dreaming): repair SQLite-backed session transcript lookup in doctor
The findHeartbeatContaminatedCorpusLines function could not read
SQLite-backed session transcripts. When a corpus ref had no .jsonl
extension (SQLite logical path format), it appended .jsonl, read an
empty file, and silently skipped — leaving pre-fix heartbeat
contamination intact for SQLite users.
Fix:
- Add loadTranscriptLinesFromSqlite helper that reads events via
the canonical loadTranscriptEventsSync API
- Serialize all events to preserve original event positions for
corpus #L<n> line-number references
- Fall back to SQLite reader when filesystem read fails on a
non-.jsonl corpus ref path
Test: add SQLite-backed session regression test that seeds a
session, writes corpus refs without .jsonl extension, and
verifies both audit detection and targeted repair.
* fix(dreaming): address autoreview P1 and P2 for checkpoint clearing and early return
P1: Remove clearScopedSessionIngestionState call after heartbeat line
removal. Clearing the cursor causes the next ingestion to re-process the
transcript from the beginning, duplicating normal corpus lines that were
deliberately retained.
P2: Remove early return after heartbeat cleanup so the self-ingestion
narrative content check and archiveDiary request still run.
* fix(dreaming): resolve corpus ref session path without duplicate agent ID
The session path in corpus refs is already in the format
'sessions/main/abc.jsonl' (includes agent subdirectory). The
previous code prepended source.agentId again, creating a path
like 'agents/main/sessions/main/abc.jsonl' — which silently
returned empty in production workspaces.
Fix: use path.basename to extract just the filename from the
session path, then construct the transcript path correctly.
* refactor(dreaming): split repair utils into separate file to satisfy LOC ratchet
The dreaming-repair.ts file grew from 337 to 831 lines, exceeding the
500-line LOC ratchet limit. Split helper utilities into a new
dreaming-repair-utils.ts file:
- dreaming-repair.ts (365 lines): imports, types, public API functions
(auditDreamingArtifacts, repairDreamingArtifacts)
- dreaming-repair-utils.ts (498 lines): all helper functions, constants,
and internal types
* fix(ts): add missing type imports for return types in dreaming-repair.ts
* fix(deadcode): remove unused exports from dreaming-repair-utils.ts
* fix: resolve merge conflict marker in get-reply-run.ts
* fix: remove unnecessary export from INPUT_PROVENANCE_KIND_VALUES
* fix(dreaming): converge targeted repair path with wholesale archive+clear
When heartbeat contamination is found, archive the entire session-corpus directory and clear all SQLite checkpoints instead of doing targeted line-by-line rewrite. This ensures the cleaned corpus gets re-ingested under the new provenance-based forward filter.
* fix(dreaming): authenticate heartbeat transcript turns
Co-authored-by: Erick Kinnee <1707617+ekinnee@users.noreply.github.com>
---------
Co-authored-by: Erick Kinnee <erick@ekinnee.dev>
Co-authored-by: Erick Kinnee <ekinnee@gmail.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* refactor(sessions): migrate runtime storage to sqlite
* test(sessions): fix sqlite CI regressions
* test(sessions): align remaining sqlite fixtures
* fix(codex): require sqlite trajectory recorder
* test(sessions): align orphan recovery sqlite fixture
* test(sessions): align sqlite rebase fixtures
* fix(sessions): finish current-main integration of the sqlite flip
Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.
* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs
Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.
* feat(sessions): keep archived transcripts by default with zstd cold storage
Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.
* fix(state): converge agent DB migration lineages and bound database growth
Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.
Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.
* fix(codex): resolve the store path for thread-history import via the SDK
The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.
* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors
A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.
* fix(sessions): complete sqlite conflict translations
* test(sqlite): align hardening checks with maintenance
* test(sessions): inspect compressed transcript archives
* fix(tests): await session seeds and drop unused helpers flagged by CI lint
The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.
* test(sessions): type legacy proof events directly
* test(sessions): align hardening contracts
* perf(sessions): read usage transcript sizes from SQL aggregates
Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.
* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir
Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.
* test(agents): seed harness admission through sqlite
* fix(sqlite): close agent db on pragma setup failure
* fix(doctor): compact and retrofit incremental auto-vacuum after session import
The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat(tooling): add noUncheckedIndexedAccess strict-ratchet lane
* fix(packages): make indexed access explicit across ratchet packages
Burns down all 65 noUncheckedIndexedAccess errors in the seven ratchet
packages with behavior-identical restructuring (iteration, charAt/slice,
regex-group guards, validated hextet tuple). net-policy invariant
violations now throw instead of failing open. Adds
@openclaw/normalization-core/expect (expectDefined/first/last) with
subpath export and tests. memory-host-sdk fixes kept but the package
stays out of the lane: it re-exports core src/** so the flag would apply
transitively to all of core.
* fix(lint): keep .oxlintrc.json strict-JSON parseable for extension lint wrappers
* fix(memory-host-sdk): handle stdout/stderr stream errors in runCliCommand
Register error handlers on stdout and stderr streams in
runCliCommand() to prevent uncaught exceptions when a pipe
breaks (e.g. EPIPE after child process exit).
Without these listeners, Node.js throws an uncaught exception
that crashes the process.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory-host-sdk): extract listenForChildOutputErrors helper
Refactor the inline stream error handling into a reusable
listenForChildOutputErrors helper, matching the pattern
established in the merged #101370 (agent-core).
The helper registers error listeners on stdout/stderr via a
shared loop, keeping the pattern consistent across the codebase.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(memory-host-sdk): harden qmd stream failures
* fix(memory-host-sdk): harden qmd stream failures
* docs(changelog): note qmd stream hardening
* chore: keep release changelog out of contributor PR
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Prevent user-authored [cron:] text from clearing reset or deleted session memory while preserving structured cron-run opacity.
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
