Commit Graph

50 Commits

Author SHA1 Message Date
Josh Lehman
0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00
Peter Steinberger
6268eeef15 test: wire Control UI suite into CI, fix its broken tests, drop dead harness surface (#104361)
* test: wire Control UI suite into CI, fix its broken tests, drop dead harness surface

- fix 7 Control UI tests broken on main: module-mock factories are unreliable
  under isolate:false shared workers; use RealtimeTalkSession prototype spies
  and real jsdom storage instead (chat-realtime, chat-pull-requests)
- add gated checks-ui CI job (runUiTests changed-scope output): chromium
  provisioning, lint:ui:no-raw-window-open, pnpm --dir ui test; ~30s on a
  4vcpu runner, runs only when ui-affecting paths change
- add weekly node22-compat workflow exercising the supported lower Node bound
  with the same command set as the dispatch-only ci.yml job
- delete the unreachable channels entry in EXTENSION_TEST_CONFIG_ROUTES and
  the never-populated extensionRoutedChannelTestFiles override machinery
  (born empty in 2ccb5cff22); resolved globs are provably identical
- guard run-vitest.mjs hardcoded path maps with existence tests so renames
  cannot silently drop extended stall watchdogs
- remove unenforced coverage thresholds; test:coverage is informational, docs
  updated (reference/test, help/testing, plugins/sdk-testing)

Closes #104321

* test: pin OPENCLAW_CI_RUN_UI_TESTS in the manifest env expectation

* test(ui): capture the realtime start spy instead of referencing the unbound prototype method

* test(ui): raise ui vitest timeouts for real-browser layout tests on small CI runners
2026-07-11 03:04:11 -07:00
Vincent Koc
7ff3f2dbf9 refactor(deadcode): localize UI and script symbols (#101858) 2026-07-07 11:55:57 -07:00
Peter Steinberger
1f484a8dbd test: speed up and stabilize full suite 2026-07-05 08:00:23 -04:00
Shakker
65e12328aa feat: refactor the Control UI architecture
Refactor the Control UI around route-owned page lifecycle and state while preserving existing behavior and design.

Prepared head SHA: bd51b6fa76
Co-authored-by: Shakker <165377636+shakkernerd@users.noreply.github.com>
Reviewed-by: @shakkernerd
2026-07-04 23:19:38 +01:00
Peter Steinberger
ed098c6550 refactor(test): drop redundant linked-worktree bundled-plugins pin from run-vitest
The node_modules package-root boundary fix anchors bundled plugin
discovery to the running checkout in nested worktrees, so the wrapper
no longer needs to pin OPENCLAW_BUNDLED_PLUGINS_DIR for symlinked
node_modules layouts. Verified live in a .claude worktree: with the
boundary fix and no pin, vitest workers resolve the worktree's own
extensions in both no-node_modules and symlinked layouts, and linked
worktrees now follow the same canonical dist-preferred discovery as
normal checkouts. The trust env remains owned by test/setup.shared.ts.

Must land together with or after the boundary fix commit directly below.
2026-07-04 21:52:09 +01:00
Vincent Koc
bc754b3160 fix(ci): restore Vitest watchdog cleanup 2026-06-20 23:42:22 +08:00
Vincent Koc
730c7269ef fix(test): clean Vitest runner child groups on signal 2026-06-20 15:35:33 +02:00
Vincent Koc
3a1e49dbaa fix(test): preserve vitest force kill after idle timeout 2026-06-20 09:44:02 +02:00
Vincent Koc
6fc0a3a9bd fix(test): chunk broad script test routing 2026-06-20 01:32:13 +02:00
Vincent Koc
f7c3775140 fix(test): prefer local bundled plugins in linked Vitest worktrees 2026-06-19 14:00:32 +02:00
Vincent Koc
f5194c4647 fix(test): reject loose Vitest wrapper limits 2026-06-16 21:50:19 +02:00
Vincent Koc
45056a463a fix(test): extend watchdog for gateway core shard 2026-06-13 23:01:11 +08:00
Vincent Koc
27e24ca683 fix(test): extend watchdog for slow vitest shards 2026-06-13 21:37:57 +08:00
Vincent Koc
f1401b2cac perf(ci): isolate Docker tooling tests 2026-06-11 18:13:36 +09:00
Vincent Koc
286772e930 test: shorten vitest no-output heartbeat 2026-06-05 09:00:02 -07:00
Peter Steinberger
ff83d4d164 docs: document runner scripts 2026-06-04 23:52:06 -04:00
Vincent Koc
8e9a4e99f5 test(gateway): extend vitest idle watchdog 2026-06-04 06:49:43 -07:00
Vincent Koc
344e04b5d5 fix(testing): route source targets through test planner 2026-06-03 18:07:32 +02:00
Vincent Koc
8f6f2617ec test(vitest): extend full agentic watchdog 2026-06-03 07:35:39 -07:00
Vincent Koc
9dc1694eb7 test: lengthen ARM contracts shard watchdog 2026-06-03 05:05:35 -07:00
Vincent Koc
14f61d0637 fix(test): clean delegated vitest runners 2026-06-02 00:09:20 +02:00
Vincent Koc
fb17986af5 fix(ci): preserve hydrated Windows test deps 2026-06-01 06:08:10 +02:00
Vincent Koc
a753e6bc86 fix(test): extend e2e vitest watchdog 2026-05-31 11:50:18 +02:00
Peter Steinberger
4eba3e5d7d chore(lint): enable more readability rules 2026-05-31 07:38:33 +01:00
Peter Steinberger
a20b2dc740 refactor: extract web content core package (#88346)
Extract web-content shared runtime helpers into packages/web-content-core, move the focused tests with the new package, and split quiet CI shards so the node matrix no longer stalls past the no-output watchdog.\n\nVerification: node scripts/run-vitest.mjs test/scripts/ci-node-test-plan.test.ts test/scripts/run-vitest.test.ts src/infra/restart.test.ts src/infra/os-summary.test.ts src/infra/gateway-processes.test.ts src/infra/inline-option-token.test.ts src/infra/map-size.test.ts src/infra/machine-name.test.ts src/commands/doctor-whatsapp-responsiveness.test.ts; autoreview clean; manual CI https://github.com/openclaw/openclaw/actions/runs/26693962844; dependency guard https://github.com/openclaw/openclaw/actions/runs/26693959937. Admin merge used because optional Mantis Telegram Desktop proof was cancelled after blocking merge outside this PR's required proof.
2026-05-30 21:38:29 +01:00
Peter Steinberger
602364f1c7 ci: stabilize changed checks 2026-05-30 20:07:38 +01:00
Dallin Romney
7de025eacd fix: route explicit vitest files through project runner (#88127) 2026-05-29 20:38:52 -07:00
Vincent Koc
117af11a6f fix(test): route tooling vitest files narrowly 2026-05-30 05:32:13 +02:00
Vincent Koc
7615c3137d fix(test): fail explicit empty vitest runs 2026-05-27 09:41:53 +02:00
Vincent Koc
373b3bfe54 fix(test): explain missing vitest dependency 2026-05-26 18:33:50 +02:00
Vincent Koc
8685dbd547 fix(test): default Vitest stall watchdog 2026-05-26 16:17:13 +02:00
Vincent Koc
505aca9ef7 fix(test): reject missing explicit vitest files 2026-05-26 07:06:55 +02:00
Shakker
56a383cdfa fix: route explicit ui vitest targets narrowly 2026-05-25 05:17:45 +01:00
Shakker
777402eeb5 fix: route unit ui vitest targets narrowly 2026-05-25 05:17:45 +01:00
Vincent Koc
6d9b3887ea fix(test): suppress rolldown timing noise 2026-05-24 14:26:19 +02:00
Vincent Koc
814409a3b3 fix(test): keep local Vitest checks serialized 2026-04-25 03:07:27 -07:00
Vincent Koc
734748d4f4 fix(test): cap native worker pools for serial Vitest 2026-04-25 01:31:30 -07:00
Peter Steinberger
310b1eea4a test: stabilize full-suite flakes 2026-04-23 18:48:07 +01:00
Peter Steinberger
ebf351b138 fix(test): prevent Vitest shard stalls 2026-04-23 06:20:34 +01:00
Peter Steinberger
1d7d268a63 refactor: share duplicate script helpers 2026-04-18 21:12:54 +01:00
Vincent Koc
329bba6cca fix(test): include vitest lane labels in timeout logs 2026-04-12 04:39:25 +01:00
Vincent Koc
9dbbe740a4 fix(test): pass process kill into vitest watchdog signals 2026-04-12 04:37:26 +01:00
Vincent Koc
5df8412508 fix(test): extend no-output watchdog to sharded vitest runs 2026-04-12 04:34:45 +01:00
Vincent Koc
f466435529 fix(test): add opt-in vitest no-output watchdog 2026-04-12 04:32:03 +01:00
Vincent Koc
dbcb1f06ec fix(test): suppress vitest plugin timing noise 2026-04-07 10:54:20 +01:00
Vincent Koc
8301ddfa84 fix(test): clean up vitest child process groups 2026-04-06 18:10:44 +01:00
Peter Steinberger
9f7aaa8ad7 test: route vitest through node launcher 2026-04-05 13:31:15 +01:00
Peter Steinberger
074af3f40e test: speed up vitest launcher startup 2026-04-05 13:31:15 +01:00
Peter Steinberger
55812eaf14 fix: throttle vitest under local contention 2026-04-04 05:50:46 +01:00