perf(test): shard full vitest runs

This commit is contained in:
Peter Steinberger
2026-04-06 17:34:07 +01:00
parent a47cb0a3b3
commit 0335a8783c
20 changed files with 522 additions and 135 deletions

View File

@@ -46,7 +46,7 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost):
### Unit / integration (default)
- Command: `pnpm test`
- Config: native Vitest `projects` via `vitest.config.ts`
- Config: five sequential shard runs (`vitest.full-*.config.ts`) over the existing scoped Vitest projects
- Files: core/unit inventories under `src/**/*.test.ts`, `packages/**/*.test.ts`, `test/**/*.test.ts`, and the whitelisted `ui` node tests covered by `vitest.unit.config.ts`
- Scope:
- Pure unit tests
@@ -57,11 +57,13 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost):
- No real keys required
- Should be fast and stable
- Projects note:
- Untargeted `pnpm test` still uses the native Vitest root `projects` config.
- Untargeted `pnpm test` now runs five smaller shard configs (`core-unit`, `core-runtime`, `agentic`, `auto-reply`, `extensions`) instead of one giant native root-project process. This cuts peak RSS on loaded machines and avoids auto-reply/extension work starving unrelated suites.
- `pnpm test --watch` still uses the native root `vitest.config.ts` project graph, because a multi-shard watch loop is not practical.
- `pnpm test`, `pnpm test:watch`, and `pnpm test:perf:imports` route explicit file/directory targets through scoped lanes first, so `pnpm test extensions/discord/src/monitor/message-handler.preflight.test.ts` avoids paying the full root project startup tax.
- `pnpm test:changed` expands changed git paths into the same scoped lanes when the diff only touches routable source/test files; config/setup edits still fall back to the broad root-project rerun.
- Selected `plugin-sdk` and `commands` tests also route through dedicated light lanes that skip `test/setup-openclaw-runtime.ts`; stateful/runtime-heavy files stay on the existing lanes.
- Selected `plugin-sdk` and `commands` helper source files also map changed-mode runs to explicit sibling tests in those light lanes, so helper edits avoid rerunning the full heavy suite for that directory.
- `auto-reply` now has three dedicated buckets: top-level core helpers, top-level `reply.*` integration tests, and the `src/auto-reply/reply/**` subtree. This keeps the heaviest reply harness work off the cheap status/chunk/token tests.
- Embedded runner note:
- When you change message-tool discovery inputs or compaction runtime context,
keep both levels of coverage.
@@ -77,7 +79,7 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost):
- Base Vitest config now defaults to `threads`.
- The shared Vitest config also fixes `isolate: false` and uses the non-isolated runner across the root projects, e2e, and live configs.
- The root UI lane keeps its `jsdom` setup and optimizer, but now runs on the shared non-isolated runner too.
- `pnpm test` inherits the same `threads` + `isolate: false` defaults from the root `vitest.config.ts` projects config.
- Each `pnpm test` shard inherits the same `threads` + `isolate: false` defaults from the shared Vitest config.
- The shared `scripts/run-vitest.mjs` launcher now also adds `--no-maglev` for Vitest child Node processes by default to reduce V8 compile churn during big local runs. Set `OPENCLAW_VITEST_ENABLE_MAGLEV=1` if you need to compare against stock V8 behavior.
- Fast-local iteration note:
- `pnpm test:changed` routes through scoped lanes when the changed paths map cleanly to a smaller suite.

View File

@@ -13,9 +13,10 @@ title: "Tests"
- `pnpm test:coverage`: Runs the unit suite with V8 coverage (via `vitest.unit.config.ts`). Global thresholds are 70% lines/branches/functions/statements. Coverage excludes integration-heavy entrypoints (CLI wiring, gateway/telegram bridges, webchat static server) to keep the target focused on unit-testable logic.
- `pnpm test:coverage:changed`: Runs unit coverage only for files changed since `origin/main`.
- `pnpm test:changed`: expands changed git paths into scoped Vitest lanes when the diff only touches routable source/test files. Config/setup changes still fall back to the native root projects run so wiring edits rerun broadly when needed.
- `pnpm test`: routes explicit file/directory targets through scoped Vitest lanes, but still falls back to the native root projects run when you do a full untargeted sweep.
- `pnpm test`: routes explicit file/directory targets through scoped Vitest lanes. Untargeted runs now execute five sequential shard configs (`vitest.full-core-unit.config.ts`, `vitest.full-core-runtime.config.ts`, `vitest.full-agentic.config.ts`, `vitest.full-auto-reply.config.ts`, `vitest.full-extensions.config.ts`) instead of one giant root-project process.
- Selected `plugin-sdk` and `commands` test files now route through dedicated light lanes that keep only `test/setup.ts`, leaving runtime-heavy cases on their existing lanes.
- Selected `plugin-sdk` and `commands` helper source files also map `pnpm test:changed` to explicit sibling tests in those light lanes, so small helper edits avoid rerunning the heavy runtime-backed suites.
- `auto-reply` now also splits into three dedicated configs (`core`, `top-level`, `reply`) so the reply harness does not dominate the lighter top-level status/token/helper tests.
- Base Vitest config now defaults to `pool: "threads"` and `isolate: false`, with the shared non-isolated runner enabled across the repo configs.
- `pnpm test:channels` runs `vitest.channels.config.ts`.
- `pnpm test:extensions` runs `vitest.extensions.config.ts`.