diff --git a/.agents/skills/openclaw-testing/SKILL.md b/.agents/skills/openclaw-testing/SKILL.md index be0ef3ba2d2..bf6bb6c18e2 100644 --- a/.agents/skills/openclaw-testing/SKILL.md +++ b/.agents/skills/openclaw-testing/SKILL.md @@ -41,9 +41,9 @@ Prove the touched surface first. Do not reflexively run the whole suite. ```bash pnpm changed:lanes --json -pnpm check:changed -pnpm test:changed -pnpm test:changed:focused +pnpm check:changed # changed typecheck/lint/guards; no Vitest +pnpm test:changed # cheap smart changed Vitest targets +OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed pnpm test -- --reporter=verbose OPENCLAW_VITEST_MAX_WORKERS=1 pnpm test ``` @@ -51,6 +51,22 @@ OPENCLAW_VITEST_MAX_WORKERS=1 pnpm test Use targeted file paths whenever possible. Avoid raw `vitest`; use the repo `pnpm test` wrapper so project routing, workers, and setup stay correct. +## Command Semantics + +- `pnpm check` and `pnpm check:changed` do not run Vitest tests. They are for + typecheck, lint, and guard proof. +- `pnpm test` and `pnpm test:changed` run Vitest tests. +- `pnpm test:changed` is intentionally cheap by default: direct test edits, + sibling tests, explicit source mappings, and import-graph dependents. +- `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed` is the explicit broad + fallback for harness/config/package edits that genuinely need it. +- Do not run extension sweeps just because core changed. If a core edit is for a + specific plugin bug, run that plugin's tests explicitly. If a public SDK or + contract change needs consumer proof, choose the smallest representative + plugin/contract tests first, then broaden only when the risk justifies it. +- The test wrapper prints a short `[test] passed|failed|skipped ... in ...` + line. Vitest's own duration is still the per-shard detail. + ## CI Debugging Start with current run state, not logs for everything: diff --git a/docs/ci.md b/docs/ci.md index 3118edb835d..1387fb4d3e1 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -94,7 +94,7 @@ CI routing-only edits, selected cheap core-test fixture edits, and narrow plugin Windows Node checks are scoped to Windows-specific process/path wrappers, npm/pnpm/UI runner helpers, package manager config, and the CI workflow surfaces that execute that lane; unrelated source, plugin, install-smoke, and test-only changes stay on the Linux Node lanes so they do not reserve a 16-vCPU Windows worker for coverage that is already exercised by the normal test shards. The separate `install-smoke` workflow reuses the same scope script through its own `preflight` job. It splits smoke coverage into `run_fast_install_smoke` and `run_full_install_smoke`. Pull requests run the fast path for Docker/package surfaces, bundled plugin package/manifest changes, and core plugin/channel/gateway/Plugin SDK surfaces that the Docker smoke jobs exercise. Source-only bundled plugin changes, test-only edits, and docs-only edits do not reserve Docker workers. The fast path builds the root Dockerfile image once, checks the CLI, runs the agents delete shared-workspace CLI smoke, runs the container gateway-network e2e, verifies a bundled extension build arg, and runs the bounded bundled-plugin Docker profile under a 240-second aggregate command timeout with each scenario's Docker run capped separately. The full path keeps QR package install and installer Docker/update coverage for nightly scheduled runs, manual dispatches, workflow-call release checks, and pull requests that truly touch installer/package/Docker surfaces. `main` pushes, including merge commits, do not force the full path; when changed-scope logic would request full coverage on a push, the workflow keeps the fast Docker smoke and leaves the full install smoke to nightly or release validation. The slow Bun global install image-provider smoke is separately gated by `run_bun_global_install_smoke`; it runs on the nightly schedule and from the release checks workflow, and manual `install-smoke` dispatches can opt into it, but pull requests and `main` pushes do not run it. QR and installer Docker tests keep their own install-focused Dockerfiles. Local `test:docker:all` prebuilds one shared live-test image, packs OpenClaw once as an npm tarball, and builds two shared `scripts/e2e/Dockerfile` images: a bare Node/Git runner for installer/update/plugin-dependency lanes and a functional image that installs the same tarball into `/app` for normal functionality lanes. Docker lane definitions live in `scripts/lib/docker-e2e-scenarios.mjs`, planner logic lives in `scripts/lib/docker-e2e-plan.mjs`, and the runner only executes the selected plan. The scheduler selects the image per lane with `OPENCLAW_DOCKER_E2E_BARE_IMAGE` and `OPENCLAW_DOCKER_E2E_FUNCTIONAL_IMAGE`, then runs lanes with `OPENCLAW_SKIP_DOCKER_BUILD=1`; tune the default main-pool slot count of 10 with `OPENCLAW_DOCKER_ALL_PARALLELISM` and the provider-sensitive tail-pool slot count of 10 with `OPENCLAW_DOCKER_ALL_TAIL_PARALLELISM`. Heavy lane caps default to `OPENCLAW_DOCKER_ALL_LIVE_LIMIT=9`, `OPENCLAW_DOCKER_ALL_NPM_LIMIT=10`, and `OPENCLAW_DOCKER_ALL_SERVICE_LIMIT=7` so npm install and multi-service lanes do not overcommit Docker while lighter lanes still fill available slots. Lane starts are staggered by 2 seconds by default to avoid local Docker daemon create storms; override with `OPENCLAW_DOCKER_ALL_START_STAGGER_MS=0` or another millisecond value. The local aggregate preflights Docker, removes stale OpenClaw E2E containers, emits active-lane status, persists lane timings for longest-first ordering, and supports `OPENCLAW_DOCKER_ALL_DRY_RUN=1` for scheduler inspection. It stops scheduling new pooled lanes after the first failure by default, and each lane has a 120-minute fallback timeout overrideable with `OPENCLAW_DOCKER_ALL_LANE_TIMEOUT_MS`; selected live/tail lanes use tighter per-lane caps. `OPENCLAW_DOCKER_ALL_LANES=` runs exact scheduler lanes, including release-only lanes such as `install-e2e` and split bundled update lanes such as `bundled-channel-update-acpx`, while skipping the cleanup smoke so agents can reproduce one failed lane. The reusable live/E2E workflow asks `scripts/test-docker-all.mjs --plan-json` which package, image kind, live image, lane, and credential coverage is required, then `scripts/docker-e2e.mjs` converts that plan into GitHub outputs and summaries. It packs OpenClaw through `scripts/package-openclaw-for-docker.mjs`, validates the tarball inventory, builds and pushes one SHA-tagged bare GHCR Docker E2E image when the plan needs install/update/plugin-dependency lanes, and builds one SHA-tagged functional GHCR Docker E2E image when the plan needs package-installed functionality lanes; if either SHA-tagged image already exists, the workflow skips rebuilding that image but still creates the fresh tarball artifact required by targeted reruns. The release-path Docker suite runs as at most three chunked jobs with `OPENCLAW_SKIP_DOCKER_BUILD=1` so each chunk pulls only the image kind it needs and executes multiple lanes through the same weighted scheduler (`OPENCLAW_DOCKER_ALL_PROFILE=release-path`, `OPENCLAW_DOCKER_ALL_CHUNK=core|package-update|plugins-integrations`). Each chunk uploads `.artifacts/docker-tests/` with lane logs, timings, `summary.json`, `failures.json`, phase timings, scheduler plan JSON, and per-lane rerun commands. The workflow `docker_lanes` input runs selected lanes against the prepared images instead of the three chunk jobs, which keeps failed-lane debugging bounded to one targeted Docker job and prepares a fresh npm tarball for the selected ref; if a selected lane is a live Docker lane, the targeted job builds the live-test image locally for that rerun. Use `pnpm test:docker:rerun ` to download Docker artifacts from a GitHub run and print combined/per-lane targeted rerun commands; use `pnpm test:docker:timings ` for slow-lane and phase critical-path summaries. When Open WebUI is requested with the release-path suite, it runs inside the plugins/integrations chunk instead of reserving a fourth Docker worker; Open WebUI keeps a standalone job only for openwebui-only dispatches. The scheduled live/E2E workflow runs the full release-path Docker suite daily. The bundled update matrix is split by update target so repeated npm update and doctor repair passes can shard with other bundled checks. -Local changed-lane logic lives in `scripts/changed-lanes.mjs` and is executed by `scripts/check-changed.mjs`. That local gate is stricter about architecture boundaries than the broad CI platform scope: core production changes run core prod typecheck plus core tests, core test-only changes run only core test typecheck/tests, extension production changes run extension prod typecheck plus extension tests, and extension test-only changes run only extension test typecheck/tests. Public Plugin SDK or plugin-contract changes expand to extension validation because extensions depend on those core contracts. Release metadata-only version bumps run targeted version/config/root-dependency checks. Unknown root/config changes fail safe to all lanes. +Local changed-lane logic lives in `scripts/changed-lanes.mjs` and is executed by `scripts/check-changed.mjs`. That local check gate is stricter about architecture boundaries than the broad CI platform scope: core production changes run core prod and core test typecheck plus core lint/guards, core test-only changes run only core test typecheck plus core lint, extension production changes run extension prod and extension test typecheck plus extension lint, and extension test-only changes run extension test typecheck plus extension lint. Public Plugin SDK or plugin-contract changes expand to extension typecheck because extensions depend on those core contracts, but Vitest extension sweeps are explicit test work. Release metadata-only version bumps run targeted version/config/root-dependency checks. Unknown root/config changes fail safe to all check lanes. On pushes, the `checks` matrix adds the push-only `compat-node22` lane. On pull requests, that lane is skipped and the matrix stays focused on the normal test/channel lanes. @@ -120,7 +120,7 @@ The CI concurrency key is versioned (`CI-v7-*`) so a GitHub-side zombie in an ol ```bash pnpm changed:lanes # inspect the local changed-lane classifier for origin/main...HEAD -pnpm check:changed # smart local gate: changed typecheck/lint/tests by boundary lane +pnpm check:changed # smart local check gate: changed typecheck/lint/guards by boundary lane pnpm check # fast local gate: production tsgo + sharded lint + parallel fast guards pnpm check:test-types pnpm check:timed # same gate with per-stage timings @@ -128,6 +128,7 @@ pnpm build:strict-smoke pnpm check:architecture pnpm test:gateway:watch-regression pnpm test # vitest tests +pnpm test:changed # cheap smart changed Vitest targets pnpm test:channels pnpm test:contracts:channels pnpm check:docs # docs format + lint + broken links diff --git a/docs/help/testing.md b/docs/help/testing.md index 05374017e9c..8b5cdb338b1 100644 --- a/docs/help/testing.md +++ b/docs/help/testing.md @@ -411,9 +411,9 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost): - Untargeted `pnpm test` runs twelve smaller shard configs (`core-unit-fast`, `core-unit-src`, `core-unit-security`, `core-unit-ui`, `core-unit-support`, `core-support-boundary`, `core-contracts`, `core-bundled`, `core-runtime`, `agentic`, `auto-reply`, `extensions`) instead of one giant native root-project process. This cuts peak RSS on loaded machines and avoids auto-reply/extension work starving unrelated suites. - `pnpm test --watch` still uses the native root `vitest.config.ts` project graph, because a multi-shard watch loop is not practical. - `pnpm test`, `pnpm test:watch`, and `pnpm test:perf:imports` route explicit file/directory targets through scoped lanes first, so `pnpm test extensions/discord/src/monitor/message-handler.preflight.test.ts` avoids paying the full root project startup tax. - - `pnpm test:changed` expands changed git paths into the same scoped lanes when the diff only touches routable source/test files; config/setup edits still fall back to the broad root-project rerun. - - `pnpm check:changed` is the normal smart local gate for narrow work. It classifies the diff into core, core tests, extensions, extension tests, apps, docs, release metadata, live Docker tooling, and tooling, then runs the matching typecheck/lint/test lanes. Public Plugin SDK and plugin-contract changes include one extension validation pass because extensions depend on those core contracts. Release metadata-only version bumps run targeted version/config/root-dependency checks instead of the full suite, with a guard that rejects package changes outside the top-level version field. - - Live Docker ACP harness edits run a focused local gate: shell syntax for the live Docker auth scripts, live Docker scheduler dry-run, ACP bind unit tests, and the ACPX extension tests. `package.json` changes are included only when the diff is limited to `scripts["test:docker:live-*"]`; dependency, export, version, and other package-surface edits still use the broader guards. + - `pnpm test:changed` expands changed git paths into cheap scoped lanes by default: direct test edits, sibling `*.test.ts` files, explicit source mappings, and local import-graph dependents. Config/setup/package edits do not broad-run tests unless you explicitly use `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed`. + - `pnpm check:changed` is the normal smart local check gate for narrow work. It classifies the diff into core, core tests, extensions, extension tests, apps, docs, release metadata, live Docker tooling, and tooling, then runs the matching typecheck, lint, and guard commands. It does not run Vitest tests; call `pnpm test:changed` or explicit `pnpm test ` for test proof. Release metadata-only version bumps run targeted version/config/root-dependency checks, with a guard that rejects package changes outside the top-level version field. + - Live Docker ACP harness edits run focused checks: shell syntax for the live Docker auth scripts and a live Docker scheduler dry-run. `package.json` changes are included only when the diff is limited to `scripts["test:docker:live-*"]`; dependency, export, version, and other package-surface edits still use the broader guards. - Import-light unit tests from agents, commands, plugins, auto-reply helpers, `plugin-sdk`, and similar pure utility areas route through the `unit-fast` lane, which skips `test/setup-openclaw-runtime.ts`; stateful/runtime-heavy files stay on the existing lanes. - Selected `plugin-sdk` and `commands` helper source files also map changed-mode runs to explicit sibling tests in those light lanes, so helper edits avoid rerunning the full heavy suite for that directory. - `auto-reply` has dedicated buckets for top-level core helpers, top-level `reply.*` integration tests, and the `src/auto-reply/reply/**` subtree. CI further splits the reply subtree into agent-runner, dispatch, and commands/state-routing shards so one import-heavy bucket does not own the full Node tail. @@ -458,10 +458,11 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost): - The pre-commit hook is formatting-only. It restages formatted files and does not run lint, typecheck, or tests. - Run `pnpm check:changed` explicitly before handoff or push when you - need the smart local gate. Public Plugin SDK and plugin-contract - changes include one extension validation pass. - - `pnpm test:changed` routes through scoped lanes when the changed paths - map cleanly to a smaller suite. + need the smart local check gate. + - `pnpm test:changed` routes through cheap scoped lanes by default. Use + `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed` only when the agent + decides a harness, config, package, or contract edit really needs broader + Vitest coverage. - `pnpm test:max` and `pnpm test:changed:max` keep the same routing behavior, just with a higher worker cap. - Local worker auto-scaling is intentionally conservative and backs off diff --git a/docs/reference/test.md b/docs/reference/test.md index f8203bb2f9a..c375b83338c 100644 --- a/docs/reference/test.md +++ b/docs/reference/test.md @@ -10,11 +10,12 @@ title: "Tests" - `pnpm test:force`: Kills any lingering gateway process holding the default control port, then runs the full Vitest suite with an isolated gateway port so server tests don’t collide with a running instance. Use this when a prior gateway run left port 18789 occupied. - `pnpm test:coverage`: Runs the unit suite with V8 coverage (via `vitest.unit.config.ts`). This is a loaded-file unit coverage gate, not whole-repo all-file coverage. Thresholds are 70% lines/functions/statements and 55% branches. Because `coverage.all` is false, the gate measures files loaded by the unit coverage suite instead of treating every split-lane source file as uncovered. - `pnpm test:coverage:changed`: Runs unit coverage only for files changed since `origin/main`. -- `pnpm test:changed`: expands changed git paths into scoped Vitest lanes when the diff only touches routable source/test files. Config/setup changes still fall back to the native root projects run so wiring edits rerun broadly when needed. -- `pnpm test:changed:focused`: inner-loop changed test run. It only runs precise targets from direct test edits, sibling `*.test.ts` files, explicit source mappings, and the local import graph. Broad/config/package changes are skipped instead of expanding to the full changed-test fallback. +- `pnpm test:changed`: cheap smart changed test run. It runs precise targets from direct test edits, sibling `*.test.ts` files, explicit source mappings, and the local import graph. Broad/config/package changes are skipped unless they map to precise tests. +- `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed`: explicit broad changed test run. Use it when a test harness/config/package edit should fall back to Vitest's broader changed-test behavior. - `pnpm changed:lanes`: shows the architectural lanes triggered by the diff against `origin/main`. -- `pnpm check:changed`: runs the smart changed gate for the diff against `origin/main`. It runs core work with core test lanes, extension work with extension test lanes, test-only work with test typecheck/tests only, expands public Plugin SDK or plugin-contract changes to one extension validation pass, and keeps release metadata-only version bumps on targeted version/config/root-dependency checks. +- `pnpm check:changed`: runs the smart changed check gate for the diff against `origin/main`. It runs typecheck, lint, and guard commands for the affected architectural lanes, but does not run Vitest tests. Use `pnpm test:changed` or explicit `pnpm test ` for test proof. - `pnpm test`: routes explicit file/directory targets through scoped Vitest lanes. Untargeted runs use fixed shard groups and expand to leaf configs for local parallel execution; the extension group always expands to the per-extension shard configs instead of one giant root-project process. +- Test wrapper runs end with a short `[test] passed|failed|skipped ... in ...` summary. Vitest's own duration line stays the per-shard detail. - Full, extension, and include-pattern shard runs update local timing data in `.artifacts/vitest-shard-timings.json`; later whole-config runs use those timings to balance slow and fast shards. Include-pattern CI shards append the shard name to the timing key, which keeps filtered shard timings visible without replacing whole-config timing data. Set `OPENCLAW_TEST_PROJECTS_TIMINGS=0` to ignore the local timing artifact. - Selected `plugin-sdk` and `commands` test files now route through dedicated light lanes that keep only `test/setup.ts`, leaving runtime-heavy cases on their existing lanes. - Source files with sibling tests map to that sibling before falling back to wider directory globs. Helper edits under `test/helpers/channels` and `test/helpers/plugins` use a local import graph to run importing tests instead of broad-running every shard when the dependency path is precise. diff --git a/package.json b/package.json index 607c298b14d..bc630dc9596 100644 --- a/package.json +++ b/package.json @@ -1480,7 +1480,6 @@ "test:build:singleton": "node scripts/test-built-plugin-singleton.mjs", "test:bundled": "node scripts/run-vitest.mjs run --config test/vitest/vitest.bundled.config.ts", "test:changed": "node scripts/test-projects.mjs --changed origin/main", - "test:changed:focused": "OPENCLAW_TEST_CHANGED_FOCUSED=1 node scripts/test-projects.mjs --changed origin/main", "test:changed:max": "OPENCLAW_VITEST_MAX_WORKERS=8 node scripts/test-projects.mjs --changed origin/main", "test:channels": "node scripts/run-vitest.mjs run --config test/vitest/vitest.channels.config.ts", "test:contracts": "pnpm test:contracts:channels && pnpm test:contracts:plugins", diff --git a/scripts/changed-lanes.mjs b/scripts/changed-lanes.mjs index fdf25890412..7790cb4ee06 100644 --- a/scripts/changed-lanes.mjs +++ b/scripts/changed-lanes.mjs @@ -67,7 +67,7 @@ export function createEmptyChangedLanes() { /** * @param {string[]} changedPaths - * @param {{ packageJsonChangeKind?: "liveDockerTooling" | null }} [options] + * @param {{ packageJsonChangeKind?: "liveDockerTooling" | "tooling" | null }} [options] * @returns {ChangedLaneResult} */ export function detectChangedLanes(changedPaths, options = {}) { @@ -80,6 +80,8 @@ export function detectChangedLanes(changedPaths, options = {}) { let hasNonDocs = false; const packageJsonIsLiveDockerTooling = paths.includes("package.json") && options.packageJsonChangeKind === "liveDockerTooling"; + const packageJsonIsTooling = + paths.includes("package.json") && options.packageJsonChangeKind === "tooling"; if (paths.length === 0) { reasons.push("no changed paths"); @@ -88,6 +90,7 @@ export function detectChangedLanes(changedPaths, options = {}) { if ( !packageJsonIsLiveDockerTooling && + !packageJsonIsTooling && paths.some((changedPath) => RELEASE_METADATA_PATHS.has(changedPath)) && paths.every( (changedPath) => RELEASE_METADATA_PATHS.has(changedPath) || DOCS_PATH_RE.test(changedPath), @@ -115,6 +118,12 @@ export function detectChangedLanes(changedPaths, options = {}) { continue; } + if (changedPath === "package.json" && packageJsonIsTooling) { + lanes.tooling = true; + reasons.push(`${changedPath}: package scripts`); + continue; + } + if (LIVE_DOCKER_TOOLING_PATH_RE.test(changedPath)) { lanes.liveDockerTooling = true; reasons.push(`${changedPath}: live Docker tooling surface`); @@ -195,39 +204,42 @@ export function detectChangedLanes(changedPaths, options = {}) { } /** - * @param {{ base: string; head?: string; includeWorktree?: boolean }} params + * @param {{ base: string; head?: string; includeWorktree?: boolean; cwd?: string }} params * @returns {string[]} */ export function listChangedPathsFromGit(params) { const base = params.base; const head = params.head ?? "HEAD"; + const cwd = params.cwd ?? process.cwd(); if (!base) { return []; } - const rangePaths = runGitNameOnlyDiff([`${base}...${head}`]); + const rangePaths = runGitNameOnlyDiff([`${base}...${head}`], cwd); if (params.includeWorktree === false) { return rangePaths; } return [ ...new Set([ ...rangePaths, - ...runGitNameOnlyDiff(["--cached", "--diff-filter=ACMR"]), - ...runGitNameOnlyDiff(["--diff-filter=ACMR"]), - ...runGitLsFiles(["--others", "--exclude-standard"]), + ...runGitNameOnlyDiff(["--cached", "--diff-filter=ACMR"], cwd), + ...runGitNameOnlyDiff(["--diff-filter=ACMR"], cwd), + ...runGitLsFiles(["--others", "--exclude-standard"], cwd), ]), ].toSorted((left, right) => left.localeCompare(right)); } -function runGitNameOnlyDiff(extraArgs) { +function runGitNameOnlyDiff(extraArgs, cwd = process.cwd()) { const output = execFileSync("git", ["diff", "--name-only", ...extraArgs], { + cwd, stdio: ["ignore", "pipe", "pipe"], encoding: "utf8", }); return output.split("\n").map(normalizeChangedPath).filter(Boolean); } -function runGitLsFiles(extraArgs) { +function runGitLsFiles(extraArgs, cwd = process.cwd()) { const output = execFileSync("git", ["ls-files", ...extraArgs], { + cwd, stdio: ["ignore", "pipe", "pipe"], encoding: "utf8", }); @@ -245,7 +257,10 @@ export function listStagedChangedPaths() { export function classifyPackageJsonChangeFromGit(params) { try { const { before, after } = readPackageJsonBeforeAfter(params); - return isLiveDockerPackageScriptOnlyChange(before, after) ? "liveDockerTooling" : null; + if (isLiveDockerPackageScriptOnlyChange(before, after)) { + return "liveDockerTooling"; + } + return isPackageScriptOnlyChange(before, after) ? "tooling" : null; } catch { return null; } @@ -265,6 +280,20 @@ export function isLiveDockerPackageScriptOnlyChange(before, after) { ); } +export function isPackageScriptOnlyChange(before, after) { + const beforePackage = JSON.parse(before); + const afterPackage = JSON.parse(after); + const beforeScripts = extractPackageScripts(beforePackage); + const afterScripts = extractPackageScripts(afterPackage); + const beforeStripped = stripPackageScripts(beforePackage); + const afterStripped = stripPackageScripts(afterPackage); + + return ( + stableJson(beforeStripped) === stableJson(afterStripped) && + stableJson(beforeScripts) !== stableJson(afterScripts) + ); +} + function readPackageJsonBeforeAfter(params) { const before = readGitText(params.staged ? "HEAD" : params.base, "package.json"); if (params.staged) { @@ -317,6 +346,17 @@ function stripLiveDockerPackageScripts(packageJson) { return clone; } +function extractPackageScripts(packageJson) { + const scripts = packageJson?.scripts; + return scripts && typeof scripts === "object" && !Array.isArray(scripts) ? scripts : {}; +} + +function stripPackageScripts(packageJson) { + const clone = JSON.parse(JSON.stringify(packageJson)); + delete clone.scripts; + return clone; +} + function stableJson(value) { if (Array.isArray(value)) { return `[${value.map(stableJson).join(",")}]`; diff --git a/scripts/check-changed.mjs b/scripts/check-changed.mjs index f8a5df47902..75e5df7c9ec 100644 --- a/scripts/check-changed.mjs +++ b/scripts/check-changed.mjs @@ -14,12 +14,7 @@ import { } from "./lib/local-heavy-check-runtime.mjs"; import { runManagedCommand } from "./lib/managed-child-process.mjs"; import { createSparseTsgoSkipEnv } from "./lib/tsgo-sparse-guard.mjs"; -import { isCiLikeEnv } from "./lib/vitest-local-scheduling.mjs"; -import { resolveChangedTestTargetPlan } from "./test-projects.test-support.mjs"; -export const CHANGED_CHECK_VITEST_NO_OUTPUT_TIMEOUT_MS = "600000"; -const VITEST_NO_OUTPUT_TIMEOUT_ENV_KEY = "OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS"; -const VITEST_NO_OUTPUT_RETRY_ENV_KEY = "OPENCLAW_VITEST_NO_OUTPUT_RETRY"; const LIVE_DOCKER_AUTH_SHELL_TARGETS = [ "scripts/lib/live-docker-auth.sh", "scripts/test-live-acp-bind-docker.sh", @@ -39,35 +34,6 @@ export function createChangedCheckChildEnv(baseEnv = process.env) { }; } -export function createChangedCheckVitestEnv(baseEnv = process.env) { - const resolvedBaseEnv = createChangedCheckChildEnv(baseEnv); - const env = { - ...resolvedBaseEnv, - [VITEST_NO_OUTPUT_TIMEOUT_ENV_KEY]: - resolvedBaseEnv[VITEST_NO_OUTPUT_TIMEOUT_ENV_KEY]?.trim() || - CHANGED_CHECK_VITEST_NO_OUTPUT_TIMEOUT_MS, - [VITEST_NO_OUTPUT_RETRY_ENV_KEY]: - resolvedBaseEnv[VITEST_NO_OUTPUT_RETRY_ENV_KEY]?.trim() || "0", - }; - - const hasWorkerOverride = Boolean( - (resolvedBaseEnv.OPENCLAW_VITEST_MAX_WORKERS ?? resolvedBaseEnv.OPENCLAW_TEST_WORKERS)?.trim(), - ); - const hasParallelOverride = Boolean(resolvedBaseEnv.OPENCLAW_TEST_PROJECTS_PARALLEL?.trim()); - const serialOverride = resolvedBaseEnv.OPENCLAW_TEST_PROJECTS_SERIAL?.trim(); - if ( - !isCiLikeEnv(resolvedBaseEnv) && - !hasWorkerOverride && - !hasParallelOverride && - serialOverride !== "0" - ) { - env.OPENCLAW_TEST_PROJECTS_SERIAL = serialOverride || "1"; - env.OPENCLAW_VITEST_MAX_WORKERS = "1"; - } - - return env; -} - export function createChangedCheckPlan(result, options = {}) { const commands = []; const baseEnv = createChangedCheckChildEnv(options.env ?? process.env); @@ -93,10 +59,6 @@ export function createChangedCheckPlan(result, options = {}) { if (result.docsOnly) { return { commands, - testTargets: [], - runChangedTestsBroad: false, - runFullTests: false, - runExtensionTests: false, summary: "docs-only", }; } @@ -118,10 +80,6 @@ export function createChangedCheckPlan(result, options = {}) { add("root dependency ownership", ["deps:root-ownership:check"]); return { commands, - testTargets: [], - runChangedTestsBroad: false, - runFullTests: false, - runExtensionTests: false, summary: "release metadata", }; } @@ -132,10 +90,6 @@ export function createChangedCheckPlan(result, options = {}) { add("runtime import cycles", ["check:import-cycles"]); return { commands, - testTargets: [], - runChangedTestsBroad: false, - runFullTests: true, - runExtensionTests: false, summary: "all", }; } @@ -189,26 +143,10 @@ export function createChangedCheckPlan(result, options = {}) { OPENCLAW_DOCKER_ALL_DRY_RUN: "1", OPENCLAW_DOCKER_ALL_LIVE_MODE: "only", }); - add( - "ACP bind unit tests", - ["test", "src/gateway/live-agent-probes.test.ts", "src/agents/acp-spawn.test.ts"], - createChangedCheckVitestEnv(baseEnv), - ); - add("ACPX extension tests", ["test:extension", "acpx"], createChangedCheckVitestEnv(baseEnv)); } - const testPlan = resolveChangedTestTargetPlan(result.paths); - const runExtensionTests = result.extensionImpactFromCore; - const testTargets = runExtensionTests - ? testPlan.targets.filter((target) => target !== "extensions") - : testPlan.targets; - const runChangedTestsBroad = testPlan.mode === "broad"; return { commands, - testTargets, - runChangedTestsBroad, - runFullTests: false, - runExtensionTests, summary: Object.entries(lanes) .filter(([, enabled]) => enabled) .map(([lane]) => lane) @@ -244,61 +182,6 @@ export async function runChangedCheck(result, options = {}) { } } - if (plan.runFullTests) { - const status = await runPnpm( - { name: "tests all", args: ["test"], env: createChangedCheckVitestEnv(childEnv) }, - timings, - ); - if (status !== 0) { - printSummary(timings, options); - return status; - } - } else if (plan.runChangedTestsBroad) { - const testArgs = options.explicitPaths - ? ["test"] - : ["test", "--changed", options.base ?? "origin/main"]; - const status = await runPnpm( - { - name: options.explicitPaths ? "tests all" : "tests changed broad", - args: testArgs, - env: createChangedCheckVitestEnv(childEnv), - }, - timings, - ); - if (status !== 0) { - printSummary(timings, options); - return status; - } - } else if (plan.testTargets.length > 0) { - const status = await runPnpm( - { - name: "tests changed", - args: ["test", ...plan.testTargets], - env: createChangedCheckVitestEnv(childEnv), - }, - timings, - ); - if (status !== 0) { - printSummary(timings, options); - return status; - } - } - - if (plan.runExtensionTests) { - const status = await runPnpm( - { - name: "tests extensions", - args: ["test:extensions"], - env: createChangedCheckVitestEnv(childEnv), - }, - timings, - ); - if (status !== 0) { - printSummary(timings, options); - return status; - } - } - printSummary(timings, options); return 0; } finally { @@ -314,17 +197,11 @@ function printPlan(result, plan, options) { const prefix = options.dryRun ? "[check:changed:dry-run]" : "[check:changed]"; console.error(`${prefix} lanes=${plan.summary || "none"}`); if (result.extensionImpactFromCore) { - console.error(`${prefix} core contract changed; extension tests included`); - } - if (plan.runChangedTestsBroad) { - console.error(`${prefix} broad changed tests included`); + console.error(`${prefix} extension-impacting surface; extension typecheck included`); } for (const reason of result.reasons) { console.error(`${prefix} ${reason}`); } - if (plan.testTargets.length > 0) { - console.error(`${prefix} test targets=${plan.testTargets.length}`); - } } async function runPnpm(command, timings) { diff --git a/scripts/test-projects.mjs b/scripts/test-projects.mjs index 832db912b0a..571df0a8743 100644 --- a/scripts/test-projects.mjs +++ b/scripts/test-projects.mjs @@ -1,5 +1,6 @@ import fs from "node:fs"; import { performance } from "node:perf_hooks"; +import { formatMs } from "./lib/check-timing-summary.mjs"; import { acquireLocalHeavyCheckLockSync } from "./lib/local-heavy-check-runtime.mjs"; import { isCiLikeEnv, @@ -271,6 +272,7 @@ async function runVitestSpecsParallel(specs, concurrency) { } async function main() { + const suiteStartedAt = performance.now(); const args = process.argv.slice(2); const baseEnv = resolveLocalVitestEnv(process.env); const { targetArgs } = parseTestProjectsArgs(args, process.cwd()); @@ -309,6 +311,7 @@ async function main() { if (runSpecs.length === 0) { console.error("[test] no changed test targets; skipping Vitest."); + printTestSummary("skipped", 0, performance.now() - suiteStartedAt); return; } @@ -360,8 +363,11 @@ async function main() { concurrency, ); writeShardTimings(timings, process.cwd(), baseEnv); - console.error( - `[test] completed ${parallelSpecs.length} Vitest shards; Vitest summaries above are per-shard, not aggregate totals.`, + printTestSummary( + parallelExitCode === 0 ? "passed" : "failed", + parallelSpecs.length, + performance.now() - suiteStartedAt, + "Vitest summaries above are per-shard, not aggregate totals.", ); releaseLockOnce(); if (parallelExitCode !== 0) { @@ -378,18 +384,24 @@ async function main() { if (!result) { return; } + if (result.timing) { + timings.push(result.timing); + } if (result.code !== 0) { exitCode = exitCode || result.code; if (spec.continueOnFailure !== true) { + printTestSummary("failed", timings.length, performance.now() - suiteStartedAt); releaseLockOnce(); process.exit(result.code); } } - if (result.timing) { - timings.push(result.timing); - } } writeShardTimings(timings, process.cwd(), baseEnv); + printTestSummary( + exitCode === 0 ? "passed" : "failed", + timings.length, + performance.now() - suiteStartedAt, + ); releaseLockOnce(); if (exitCode !== 0) { @@ -397,6 +409,13 @@ async function main() { } } +function printTestSummary(status, shardCount, durationMs, detail) { + const suffix = detail ? `; ${detail}` : ""; + console.error( + `[test] ${status} ${shardCount} Vitest shard${shardCount === 1 ? "" : "s"} in ${formatMs(durationMs)}${suffix}`, + ); +} + main().catch((error) => { releaseLockOnce(); console.error(error); diff --git a/scripts/test-projects.test-support.d.mts b/scripts/test-projects.test-support.d.mts index 65e7f861d67..008b5a5c04d 100644 --- a/scripts/test-projects.test-support.d.mts +++ b/scripts/test-projects.test-support.d.mts @@ -17,7 +17,7 @@ export type VitestRunSpec = { export type ChangedTestTargetOptions = { cwd?: string; env?: Record; - focused?: boolean; + broad?: boolean; }; export const DEFAULT_TEST_PROJECTS_VITEST_NO_OUTPUT_TIMEOUT_MS: string; diff --git a/scripts/test-projects.test-support.mjs b/scripts/test-projects.test-support.mjs index 098afc803d1..c2a48ea397e 100644 --- a/scripts/test-projects.test-support.mjs +++ b/scripts/test-projects.test-support.mjs @@ -1,4 +1,3 @@ -import { execFileSync } from "node:child_process"; import fs from "node:fs"; import os from "node:os"; import path from "node:path"; @@ -40,7 +39,10 @@ import { isBoundaryTestFile, isBundledPluginDependentUnitTestFile, } from "../test/vitest/vitest.unit-paths.mjs"; -import { detectChangedLanes } from "./changed-lanes.mjs"; +import { + detectChangedLanes, + listChangedPathsFromGit as listChangedPathsFromGitSource, +} from "./changed-lanes.mjs"; import { isCiLikeEnv, resolveLocalFullSuiteProfile } from "./lib/vitest-local-scheduling.mjs"; import { resolveVitestCliEntry, resolveVitestNodeArgs } from "./run-vitest.mjs"; @@ -207,7 +209,7 @@ const VITEST_CONFIG_BY_KIND = { utils: UTILS_VITEST_CONFIG, wizard: WIZARD_VITEST_CONFIG, }; -const BROAD_CHANGED_RERUN_PATTERNS = [ +const BROAD_CHANGED_FALLBACK_PATTERNS = [ /^package\.json$/u, /^pnpm-lock\.yaml$/u, /^test\/setup(?:\.shared|\.extensions|-openclaw-runtime)?\.ts$/u, @@ -305,7 +307,7 @@ const SOURCE_ROOTS_FOR_IMPORT_GRAPH = ["src", "extensions", "packages", "ui/src" const IMPORTABLE_FILE_EXTENSIONS = [".ts", ".tsx", ".mts", ".cts"]; const IMPORT_SPECIFIER_PATTERN = /\b(?:import|export)\s+(?:type\s+)?(?:[^'"]*?\s+from\s+)?["']([^"']+)["']|\bimport\s*\(\s*["']([^"']+)["']\s*\)/gu; -const FOCUSED_CHANGED_ENV_KEY = "OPENCLAW_TEST_CHANGED_FOCUSED"; +const BROAD_CHANGED_ENV_KEY = "OPENCLAW_TEST_CHANGED_BROAD"; const VITEST_NO_OUTPUT_TIMEOUT_ENV_KEY = "OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS"; const VITEST_NO_OUTPUT_RETRY_ENV_KEY = "OPENCLAW_VITEST_NO_OUTPUT_RETRY"; export const DEFAULT_TEST_PROJECTS_VITEST_NO_OUTPUT_TIMEOUT_MS = "180000"; @@ -594,36 +596,7 @@ function resolveChannelContractTargetKind(relative) { } function listChangedPathsFromGit(baseRef, cwd) { - return [ - ...new Set([ - ...runGitNameOnlyDiff(cwd, [`${baseRef}...HEAD`]), - ...runGitNameOnlyDiff(cwd, ["--cached", "--diff-filter=ACMR"]), - ...runGitNameOnlyDiff(cwd, ["--diff-filter=ACMR"]), - ...runGitLsFiles(cwd, ["--others", "--exclude-standard"]), - ]), - ].toSorted((left, right) => left.localeCompare(right)); -} - -function runGitNameOnlyDiff(cwd, extraArgs) { - return execFileSync("git", ["diff", "--name-only", ...extraArgs], { - cwd, - encoding: "utf8", - stdio: ["ignore", "pipe", "pipe"], - }) - .split("\n") - .map((line) => normalizePathPattern(line.trim())) - .filter((line) => line.length > 0); -} - -function runGitLsFiles(cwd, extraArgs) { - return execFileSync("git", ["ls-files", ...extraArgs], { - cwd, - encoding: "utf8", - stdio: ["ignore", "pipe", "pipe"], - }) - .split("\n") - .map((line) => normalizePathPattern(line.trim())) - .filter((line) => line.length > 0); + return listChangedPathsFromGitSource({ base: baseRef, cwd }); } function extractChangedBaseRef(args) { @@ -665,7 +638,7 @@ function shouldKeepBroadChangedRun(changedPaths) { return changedPaths.some((changedPath) => PRECISE_SOURCE_TEST_TARGETS.has(changedPath) ? false - : BROAD_CHANGED_RERUN_PATTERNS.some((pattern) => pattern.test(changedPath)), + : BROAD_CHANGED_FALLBACK_PATTERNS.some((pattern) => pattern.test(changedPath)), ); } @@ -685,8 +658,8 @@ function resolveToolingTestTargets(changedPath) { return TOOLING_SOURCE_TEST_TARGETS.get(changedPath) ?? TOOLING_TEST_TARGETS.get(changedPath); } -function shouldUseFocusedChangedTargets(env = process.env) { - const value = env[FOCUSED_CHANGED_ENV_KEY]?.trim().toLowerCase(); +function shouldUseBroadChangedTargets(env = process.env) { + const value = env[BROAD_CHANGED_ENV_KEY]?.trim().toLowerCase(); return ["1", "true", "yes", "on"].includes(value ?? ""); } @@ -741,7 +714,8 @@ export function resolveChangedTestTargetPlan(changedPaths, options = {}) { return { mode: "targets", targets: toolingTargets }; } const changedLanes = detectChangedLanes(changedPaths); - const focused = options.focused ?? shouldUseFocusedChangedTargets(options.env ?? {}); + const env = options.env ?? {}; + const useBroadFallback = options.broad ?? shouldUseBroadChangedTargets(env); const targets = []; for (const changedPath of changedPaths) { const preciseTargets = resolvePreciseChangedTestTargets(changedPath, options); @@ -749,20 +723,21 @@ export function resolveChangedTestTargetPlan(changedPaths, options = {}) { targets.push(...preciseTargets); continue; } - if (focused) { + const needsBroadFallback = shouldKeepBroadChangedRun([changedPath]) || changedLanes.lanes.all; + if (needsBroadFallback) { + if (useBroadFallback) { + return { mode: "broad", targets: [] }; + } continue; } - if (shouldKeepBroadChangedRun([changedPath]) || changedLanes.lanes.all) { - return { mode: "broad", targets: [] }; - } if (isRoutableChangedTarget(changedPath)) { targets.push(changedPath); } } - if (!focused && changedLanes.lanes.all) { + if (useBroadFallback && changedLanes.lanes.all) { return { mode: "broad", targets: [] }; } - if (!focused && changedLanes.extensionImpactFromCore) { + if (useBroadFallback && changedLanes.extensionImpactFromCore) { targets.push("extensions"); } return { mode: "targets", targets: [...new Set(targets)] }; diff --git a/test/scripts/changed-lanes.test.ts b/test/scripts/changed-lanes.test.ts index f29d3ae64bc..ab014980a8d 100644 --- a/test/scripts/changed-lanes.test.ts +++ b/test/scripts/changed-lanes.test.ts @@ -5,12 +5,11 @@ import { afterEach, describe, expect, it } from "vitest"; import { detectChangedLanes, isLiveDockerPackageScriptOnlyChange, + isPackageScriptOnlyChange, } from "../../scripts/changed-lanes.mjs"; import { - CHANGED_CHECK_VITEST_NO_OUTPUT_TIMEOUT_MS, createChangedCheckChildEnv, createChangedCheckPlan, - createChangedCheckVitestEnv, } from "../../scripts/check-changed.mjs"; import { cleanupTempDirs, makeTempRepoRoot } from "../helpers/temp-repo.js"; @@ -216,8 +215,8 @@ describe("scripts/changed-lanes", () => { extensionTests: true, all: false, }); - expect(plan.runExtensionTests).toBe(true); - expect(plan.testTargets).toEqual(["src/plugin-sdk/core.test.ts"]); + expect(plan.commands.map((command) => command.args[0])).toContain("tsgo:extensions"); + expect(plan.commands.map((command) => command.args[0])).toContain("tsgo:extensions:test"); }); it("fails safe for root config changes", () => { @@ -225,8 +224,8 @@ describe("scripts/changed-lanes", () => { const plan = createChangedCheckPlan(result); expect(result.lanes.all).toBe(true); - expect(plan.runFullTests).toBe(true); expect(plan.commands.map((command) => command.args[0])).toContain("tsgo:all"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); it("routes gitignore changes to tooling instead of all lanes", () => { @@ -237,10 +236,9 @@ describe("scripts/changed-lanes", () => { tooling: true, all: false, }); - expect(plan.runFullTests).toBe(false); - expect(plan.runChangedTestsBroad).toBe(false); expect(plan.commands.map((command) => command.args[0])).toContain("lint:scripts"); expect(plan.commands.map((command) => command.args[0])).not.toContain("tsgo:all"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); it("routes live Docker ACP tooling changes through a focused gate", () => { @@ -258,8 +256,6 @@ describe("scripts/changed-lanes", () => { all: false, tooling: false, }); - expect(plan.runFullTests).toBe(false); - expect(plan.runChangedTestsBroad).toBe(false); expect(plan.commands.map((command) => command.name)).toEqual([ "conflict markers", "typecheck core tests", @@ -267,8 +263,6 @@ describe("scripts/changed-lanes", () => { "lint scripts", "live Docker shell syntax", "live Docker scheduler dry run", - "ACP bind unit tests", - "ACPX extension tests", ]); expect( plan.commands.find((command) => command.name === "live Docker shell syntax"), @@ -330,7 +324,6 @@ describe("scripts/changed-lanes", () => { releaseMetadata: false, all: false, }); - expect(plan.runFullTests).toBe(false); expect(plan.commands.map((command) => command.name)).toContain("live Docker scheduler dry run"); }); @@ -400,6 +393,77 @@ describe("scripts/changed-lanes", () => { }); }); + it("classifies normal package script changes from the git diff", () => { + const dir = makeTempRepoRoot(tempDirs, "openclaw-package-scripts-"); + git(dir, ["init", "-q", "--initial-branch=main"]); + writeFileSync( + path.join(dir, "package.json"), + `${JSON.stringify( + { + name: "fixture", + scripts: { + test: "node scripts/test-projects.mjs", + }, + dependencies: { + leftpad: "1.0.0", + }, + }, + null, + 2, + )}\n`, + "utf8", + ); + git(dir, ["add", "package.json"]); + git(dir, [ + "-c", + "user.email=test@example.com", + "-c", + "user.name=Test User", + "commit", + "-q", + "-m", + "initial", + ]); + + writeFileSync( + path.join(dir, "package.json"), + `${JSON.stringify( + { + name: "fixture", + scripts: { + test: "node scripts/test-projects.mjs", + "test:profile": "node scripts/profile-tests.mjs", + }, + dependencies: { + leftpad: "1.0.0", + }, + }, + null, + 2, + )}\n`, + "utf8", + ); + + const output = execFileSync( + process.execPath, + [path.join(repoRoot, "scripts", "changed-lanes.mjs"), "--json", "--base", "HEAD"], + { + cwd: dir, + encoding: "utf8", + env: createNestedGitEnv(), + }, + ); + + expect(JSON.parse(output)).toMatchObject({ + paths: ["package.json"], + lanes: { + tooling: true, + all: false, + liveDockerTooling: false, + }, + }); + }); + it("keeps non-script package changes off the live Docker focused gate", () => { const before = `${JSON.stringify( { name: "fixture", scripts: {}, dependencies: { leftpad: "1.0.0" } }, @@ -422,6 +486,41 @@ describe("scripts/changed-lanes", () => { expect(isLiveDockerPackageScriptOnlyChange(before, after)).toBe(false); }); + it("routes package script-only changes through the tooling gate", () => { + const before = `${JSON.stringify( + { name: "fixture", scripts: { test: "node test.js" }, dependencies: { leftpad: "1.0.0" } }, + null, + 2, + )}\n`; + const after = `${JSON.stringify( + { + name: "fixture", + scripts: { + test: "node test.js", + "test:profile": "node scripts/profile-tests.mjs", + }, + dependencies: { leftpad: "1.0.0" }, + }, + null, + 2, + )}\n`; + + expect(isPackageScriptOnlyChange(before, after)).toBe(true); + + const result = detectChangedLanes(["package.json"], { + packageJsonChangeKind: "tooling", + }); + const plan = createChangedCheckPlan(result); + + expect(result.lanes).toMatchObject({ + tooling: true, + all: false, + liveDockerTooling: false, + }); + expect(plan.commands.map((command) => command.args[0])).toContain("lint:scripts"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("tsgo:all"); + }); + it("keeps release metadata commits off the full changed gate", () => { const result = detectChangedLanes([ "CHANGELOG.md", @@ -443,7 +542,6 @@ describe("scripts/changed-lanes", () => { core: false, apps: false, }); - expect(plan.runFullTests).toBe(false); expect(plan.commands.map((command) => command.args[0])).toEqual([ "check:no-conflict-markers", "release-metadata:check", @@ -519,26 +617,24 @@ describe("scripts/changed-lanes", () => { tooling: true, all: false, }); - expect(plan.testTargets).toEqual(["test/git-hooks-pre-commit.test.ts"]); - expect(plan.runFullTests).toBe(false); + expect(plan.commands.map((command) => command.args[0])).toContain("lint:scripts"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); - it("keeps shared Vitest wiring changes on the broad changed test path", () => { + it("keeps shared Vitest wiring changes out of check test execution", () => { const result = detectChangedLanes(["test/vitest/vitest.shared.config.ts"]); const plan = createChangedCheckPlan(result); - expect(plan.testTargets).toEqual([]); - expect(plan.runChangedTestsBroad).toBe(true); - expect(plan.runFullTests).toBe(false); + expect(plan.commands.map((command) => command.args[0])).toContain("lint:scripts"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); - it("keeps setup changes on the broad changed test path", () => { + it("keeps setup changes out of check test execution", () => { const result = detectChangedLanes(["test/setup.ts"]); const plan = createChangedCheckPlan(result); - expect(plan.testTargets).toEqual([]); - expect(plan.runChangedTestsBroad).toBe(true); - expect(plan.runFullTests).toBe(false); + expect(plan.commands.map((command) => command.args[0])).toContain("lint:scripts"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); it("does not route generated A2UI artifacts as direct Vitest targets", () => { @@ -548,17 +644,16 @@ describe("scripts/changed-lanes", () => { ]); const plan = createChangedCheckPlan(result); - expect(plan.testTargets).toEqual(["test/scripts/bundle-a2ui.test.ts"]); - expect(plan.runChangedTestsBroad).toBe(false); + expect(plan.commands.map((command) => command.args[0])).toContain("tsgo:core"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); it("routes changed extension Vitest configs to only their owning shard", () => { const result = detectChangedLanes(["test/vitest/vitest.extension-discord.config.ts"]); const plan = createChangedCheckPlan(result); - expect(plan.testTargets).toEqual(["test/vitest/vitest.extension-discord.config.ts"]); - expect(plan.runChangedTestsBroad).toBe(false); - expect(plan.runFullTests).toBe(false); + expect(plan.commands.map((command) => command.args[0])).toContain("lint:scripts"); + expect(plan.commands.map((command) => command.args[0])).not.toContain("test"); }); it("keeps an empty changed path list as a no-op", () => { @@ -580,8 +675,6 @@ describe("scripts/changed-lanes", () => { expect(plan.commands).toEqual([ { name: "conflict markers", args: ["check:no-conflict-markers"] }, ]); - expect(plan.runChangedTestsBroad).toBe(false); - expect(plan.runFullTests).toBe(false); }); it("keeps docs-only changes cheap", () => { @@ -592,40 +685,5 @@ describe("scripts/changed-lanes", () => { expect(plan.commands).toEqual([ { name: "conflict markers", args: ["check:no-conflict-markers"] }, ]); - expect(plan.runChangedTestsBroad).toBe(false); - expect(plan.runFullTests).toBe(false); - }); - - it("sets a ten-minute Vitest watchdog for changed checks", () => { - expect(CHANGED_CHECK_VITEST_NO_OUTPUT_TIMEOUT_MS).toBe("600000"); - expect(createChangedCheckVitestEnv({ PATH: "/usr/bin" })).toMatchObject({ - PATH: "/usr/bin", - OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS: CHANGED_CHECK_VITEST_NO_OUTPUT_TIMEOUT_MS, - OPENCLAW_VITEST_NO_OUTPUT_RETRY: "0", - OPENCLAW_TEST_PROJECTS_SERIAL: "1", - OPENCLAW_VITEST_MAX_WORKERS: "1", - }); - - expect( - createChangedCheckVitestEnv({ - OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS: "45000", - OPENCLAW_VITEST_NO_OUTPUT_RETRY: "1", - }), - ).toMatchObject({ - OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS: "45000", - OPENCLAW_VITEST_NO_OUTPUT_RETRY: "1", - }); - }); - - it("does not force serial changed-check tests in CI or when workers are explicit", () => { - expect(createChangedCheckVitestEnv({ CI: "true" })).not.toHaveProperty( - "OPENCLAW_VITEST_MAX_WORKERS", - ); - expect(createChangedCheckVitestEnv({ OPENCLAW_VITEST_MAX_WORKERS: "4" })).toMatchObject({ - OPENCLAW_VITEST_MAX_WORKERS: "4", - }); - expect( - createChangedCheckVitestEnv({ OPENCLAW_TEST_PROJECTS_PARALLEL: "4" }), - ).not.toHaveProperty("OPENCLAW_TEST_PROJECTS_SERIAL"); }); }); diff --git a/test/scripts/test-projects.test.ts b/test/scripts/test-projects.test.ts index 85bd5d7d08d..78450d3ab67 100644 --- a/test/scripts/test-projects.test.ts +++ b/test/scripts/test-projects.test.ts @@ -25,12 +25,23 @@ describe("scripts/test-projects changed-target routing", () => { ).toEqual(["src/shared/string-normalization.test.ts", "src/utils/provider-utils.test.ts"]); }); - it("keeps the broad changed run for Vitest wiring edits", () => { + it("keeps changed mode focused by default for Vitest wiring edits", () => { expect( resolveChangedTargetArgs(["--changed", "origin/main"], process.cwd(), () => [ "test/vitest/vitest.shared.config.ts", "src/utils/provider-utils.ts", ]), + ).toEqual(["src/utils/provider-utils.test.ts"]); + }); + + it("keeps the broad changed run available for Vitest wiring edits", () => { + expect( + resolveChangedTargetArgs( + ["--changed", "origin/main"], + process.cwd(), + () => ["test/vitest/vitest.shared.config.ts", "src/utils/provider-utils.ts"], + { env: { OPENCLAW_TEST_CHANGED_BROAD: "1" } }, + ), ).toBeNull(); }); @@ -129,11 +140,22 @@ describe("scripts/test-projects changed-target routing", () => { ]); }); - it("keeps the broad changed run for shared test helpers", () => { + it("keeps shared test helpers cheap by default when no precise target exists", () => { expect( resolveChangedTargetArgs(["--changed", "origin/main"], process.cwd(), () => [ "test/helpers/channels/plugin.ts", ]), + ).toEqual([]); + }); + + it("keeps the broad changed run available for shared test helpers", () => { + expect( + resolveChangedTargetArgs( + ["--changed", "origin/main"], + process.cwd(), + () => ["test/helpers/channels/plugin.ts"], + { env: { OPENCLAW_TEST_CHANGED_BROAD: "1" } }, + ), ).toBeNull(); }); @@ -174,11 +196,22 @@ describe("scripts/test-projects changed-target routing", () => { ]); }); - it("keeps the broad changed run for unknown root surfaces", () => { + it("keeps unknown root surfaces cheap by default", () => { expect( resolveChangedTargetArgs(["--changed", "origin/main"], process.cwd(), () => [ "unknown/file.txt", ]), + ).toEqual([]); + }); + + it("keeps the broad changed run available for unknown root surfaces", () => { + expect( + resolveChangedTargetArgs( + ["--changed", "origin/main"], + process.cwd(), + () => ["unknown/file.txt"], + { env: { OPENCLAW_TEST_CHANGED_BROAD: "1" } }, + ), ).toBeNull(); }); @@ -204,11 +237,29 @@ describe("scripts/test-projects changed-target routing", () => { ).toEqual([]); }); - it("adds extension tests for public plugin SDK changes", () => { + it("keeps public plugin SDK changes focused by default", () => { const plans = buildVitestRunPlans(["--changed", "origin/main"], process.cwd(), () => [ "src/plugin-sdk/provider-entry.ts", ]); + expect(plans).toEqual([ + { + config: "test/vitest/vitest.unit-fast.config.ts", + forwardedArgs: [], + includePatterns: ["src/plugin-sdk/provider-entry.test.ts"], + watchMode: false, + }, + ]); + }); + + it("adds extension tests for public plugin SDK changes in broad changed mode", () => { + const plans = buildVitestRunPlans( + ["--changed", "origin/main"], + process.cwd(), + () => ["src/plugin-sdk/provider-entry.ts"], + { env: { OPENCLAW_TEST_CHANGED_BROAD: "1" } }, + ); + expect(plans).toEqual([ { config: "test/vitest/vitest.unit-fast.config.ts", @@ -485,11 +536,29 @@ describe("scripts/test-projects changed-target routing", () => { ]); }); - it("routes plugin-sdk source files with sibling tests narrowly plus extension tests", () => { + it("routes plugin-sdk source files with sibling tests narrowly by default", () => { const plans = buildVitestRunPlans(["--changed", "origin/main"], process.cwd(), () => [ "src/plugin-sdk/facade-runtime.ts", ]); + expect(plans).toEqual([ + { + config: "test/vitest/vitest.bundled.config.ts", + forwardedArgs: [], + includePatterns: ["src/plugin-sdk/facade-runtime.test.ts"], + watchMode: false, + }, + ]); + }); + + it("routes plugin-sdk source files with sibling tests plus extensions in broad changed mode", () => { + const plans = buildVitestRunPlans( + ["--changed", "origin/main"], + process.cwd(), + () => ["src/plugin-sdk/facade-runtime.ts"], + { env: { OPENCLAW_TEST_CHANGED_BROAD: "1" } }, + ); + expect(plans).toEqual([ { config: "test/vitest/vitest.bundled.config.ts", @@ -521,22 +590,27 @@ describe("scripts/test-projects changed-target routing", () => { ]); }); - it("keeps focused changed mode to precise targets only", () => { - expect( - resolveChangedTestTargetPlan(["package.json", "src/commands/channels.add.ts"], { - focused: true, - }), - ).toEqual({ + it("keeps changed mode to precise targets by default", () => { + expect(resolveChangedTestTargetPlan(["package.json", "src/commands/channels.add.ts"])).toEqual({ mode: "targets", targets: ["src/commands/channels.add.test.ts"], }); }); - it("uses import-graph targets in focused changed mode", () => { + it("keeps broad changed fallback available through explicit env", () => { expect( - resolveChangedTestTargetPlan(["test/helpers/plugins/plugin-registration.ts"], { - focused: true, - }).targets, + resolveChangedTestTargetPlan(["package.json", "src/commands/channels.add.ts"], { + env: { OPENCLAW_TEST_CHANGED_BROAD: "1" }, + }), + ).toEqual({ + mode: "broad", + targets: [], + }); + }); + + it("uses import-graph targets in default changed mode", () => { + expect( + resolveChangedTestTargetPlan(["test/helpers/plugins/plugin-registration.ts"]).targets, ).toContain("extensions/openrouter/index.test.ts"); });