--- summary: "How to run tests locally (vitest) and when to use force/coverage modes" read_when: - Running or fixing tests title: "Tests" --- - Full testing kit (suites, live, Docker): [Testing](/help/testing) - Update and plugin package validation: [Testing updates and plugins](/help/testing-updates-plugins) ## Agent default Agent sessions run one/few focused tests and cheap static checks locally only for trusted source and when the existing dependency install is ready. Never execute untrusted repository tooling locally. Larger suites, changed gates with typecheck/lint fan-out, builds, Docker, package lanes, E2E, live proof, and cross-platform validation run remotely through Crabbox. Trusted maintainer heavy proof defaults to Blacksmith Testbox. The configured Testbox workflow hydrates credentials, so untrusted contributor or fork code must use secretless fork CI or sanitized direct AWS Crabbox instead. Do not pre-warm for anticipated work. Acquire the backend lazily when the first heavy command is ready, reuse the returned `tbx_...` id for later heavy commands, sync the current checkout on every run, and stop it before handoff. After the first successful reuse, the wrapper records the lease's base, dependency, and Testbox workflow fingerprint under `.crabbox/testbox-leases/`. Source-only edits keep reusing the warmed box. A changed merge base, lockfile, package-manager input, wrapper, or Testbox workflow fails closed and requires a fresh lease. Every run still syncs the current checkout. `OPENCLAW_TESTBOX_ALLOW_STALE=1` is only for intentional diagnostics, not release proof. Local test commands below are for human workflows and bounded agent proof. Remote-provider unavailability must be reported; it is not permission to silently run a broad local gate. For untrusted heavy proof, lazily warm with `--provider aws`. Every run must set `CRABBOX_ENV_ALLOW=CI`, pass `--provider aws --no-hydrate`, and use a fresh temporary remote `HOME` before installing dependencies or running tests. Use a newly warmed lease dedicated to that untrusted source; never reuse a trusted or previously hydrated lease. Launch an installed trusted Crabbox binary from a clean trusted `main` checkout and fetch only the remote PR with `--fresh-pr`; never execute the untrusted checkout's wrapper or config locally. Unset `CRABBOX_AWS_INSTANCE_PROFILE` and fail closed unless resolved `aws.instanceProfile` is empty. Before any install/test, use trusted absolute-path tools to require an IMDSv2 token, prove the IAM credentials endpoint returns 404, and verify remote `git rev-parse HEAD` equals the full reviewed PR head SHA. Bind the lease to that SHA and stop/rewarm when the head changes. Upload trusted `scripts/crabbox-untrusted-bootstrap.sh` from clean `main` alongside `--fresh-pr`; it installs pinned Node/pnpm, verifies the SHA and package-manager pin, isolates `HOME`, installs dependencies, then executes the requested test. If the broker cannot prove no role or no remote PR exists, use secretless fork CI. Do not use `hydrate-github`, `--no-sync`, or a credential-hydrated Testbox workflow. Unset all `CRABBOX_TAILSCALE*` overrides, force `--network public --tailscale=false`, clear exit-node/LAN flags, and require `crabbox inspect` to report public networking with no Tailscale state before uploading any script. ## Routine local order 1. `pnpm test:changed` for changed-scope Vitest proof. 2. `pnpm test ` for one file, directory, or explicit target. 3. `pnpm test` only when you intentionally need the full local Vitest suite. In a Codex worktree or linked/sparse checkout, agents avoid direct local `pnpm test*` / `pnpm check*` / `pnpm crabbox:run`: - Bounded focused proof with ready dependencies: `node scripts/run-vitest.mjs `. - Classify-first changed check: `node scripts/check-changed.mjs`; docs-only, no-change, and small metadata plans stay local when dependencies are ready, while heavy or dependency-missing plans delegate to Testbox. - Explicit kept-lease broad proof: `node scripts/crabbox-wrapper.mjs run --provider blacksmith-testbox ... -- env OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 OPENCLAW_CHANGED_LANES_RAW_SYNC=1 corepack pnpm check:changed` so pnpm runs inside Testbox. - The wrapper's final `exitCode` and timing JSON are the command result. A delegated Blacksmith GitHub Actions run may show `cancelled` after a successful SSH command because the Testbox is stopped from outside the keepalive action; check the wrapper summary and command output before treating that as a failure. - `OPENCLAW_HEAVY_CHECK_LOCK_SCOPE=worktree `: keeps heavy-check serialization inside the current worktree instead of the Git common dir for commands such as `pnpm check:changed` and targeted `pnpm test ...`. Use it only on high-capacity local hosts when you intentionally run independent checks across linked worktrees. ## Core commands Test wrapper runs end with a short `[test] passed|failed|skipped ... in ...` summary; Vitest's own duration line stays the per-shard detail. | Command | What it does | | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pnpm test` | Explicit file/directory targets route through scoped Vitest lanes. Untargeted runs are full-suite proof: fixed shard groups expand to leaf configs for local parallel execution, with the expected shard fanout printed before starting. The extension group always expands to per-extension shard configs instead of one giant root-project process. | | `pnpm test:changed` | Cheap smart changed-test run: precise targets from direct test edits, sibling `*.test.ts` files, explicit source mappings, and the local import graph. Broad/config/package changes are skipped unless they map to precise tests. | | `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed` | Explicit broad changed-test run; use when a test harness/config/package edit should fall back to Vitest's broader changed-test behavior. | | `pnpm test:force` | Frees the configured OpenClaw gateway port (default `18789`), then runs the full suite with an isolated gateway port so server tests do not collide with a running instance. | | `pnpm test:coverage` | Emits an informational V8 coverage report for the default unit lane (`vitest.unit.config.ts`); no coverage thresholds are enforced. | | `pnpm test:coverage:changed` | Unit coverage only for files changed since `origin/main`. | | `pnpm changed:lanes` | Shows the architectural lanes triggered by the diff against `origin/main`. | | `pnpm check:changed` | Classifies the changed lanes before choosing execution. Docs-only, no-change, and small metadata plans stay local when dependencies are ready; plans with typecheck/lint fan-out, other heavy lanes, or missing local dependencies delegate to Crabbox/Testbox outside CI. Does not run Vitest; use `pnpm test:changed` or `pnpm test ` for test proof. | ## Shared test state and process helpers - `src/test-utils/openclaw-test-state.ts`: use from Vitest when a test needs an isolated `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, config fixture, workspace, agent dir, or auth-profile store. - `pnpm test:env-mutations:report`: non-blocking report of tests/harnesses that mutate `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, `OPENCLAW_WORKSPACE_DIR`, or related env keys directly. Use it to find migration candidates for the shared test-state helper. - `test/helpers/openclaw-test-instance.ts`: process-level E2E tests needing a running Gateway, CLI env, log capture, and cleanup in one place. - Docker/Bash E2E lanes that source `scripts/lib/docker-e2e-image.sh` can pass `docker_e2e_test_state_shell_b64