mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 11:40:42 +00:00
ci: speed up release metadata pre-commit checks
This commit is contained in:
@@ -47,7 +47,7 @@ Jobs are ordered so cheap checks fail before expensive ones run:
|
||||
Scope logic lives in `scripts/ci-changed-scope.mjs` and is covered by unit tests in `src/scripts/ci-changed-scope.test.ts`.
|
||||
The separate `install-smoke` workflow reuses the same scope script through its own `preflight` job. It computes `run_install_smoke` from the narrower changed-smoke signal, so Docker/install smoke only runs for install, packaging, and container-relevant changes.
|
||||
|
||||
Local changed-lane logic lives in `scripts/changed-lanes.mjs` and is executed by `scripts/check-changed.mjs`. That local gate is stricter about architecture boundaries than the broad CI platform scope: core production changes run core prod typecheck plus core tests, core test-only changes run only core test typecheck/tests, extension production changes run extension prod typecheck plus extension tests, and extension test-only changes run only extension test typecheck/tests. Public Plugin SDK or plugin-contract changes expand to extension validation because extensions depend on those core contracts. Unknown root/config changes fail safe to all lanes.
|
||||
Local changed-lane logic lives in `scripts/changed-lanes.mjs` and is executed by `scripts/check-changed.mjs`. That local gate is stricter about architecture boundaries than the broad CI platform scope: core production changes run core prod typecheck plus core tests, core test-only changes run only core test typecheck/tests, extension production changes run extension prod typecheck plus extension tests, and extension test-only changes run only extension test typecheck/tests. Public Plugin SDK or plugin-contract changes expand to extension validation because extensions depend on those core contracts. Release metadata-only version bumps run targeted version/config/root-dependency checks. Unknown root/config changes fail safe to all lanes.
|
||||
|
||||
On pushes, the `checks` matrix adds the push-only `compat-node22` lane. On pull requests, that lane is skipped and the matrix stays focused on the normal test/channel lanes.
|
||||
|
||||
|
||||
@@ -288,7 +288,7 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost):
|
||||
- `pnpm test --watch` still uses the native root `vitest.config.ts` project graph, because a multi-shard watch loop is not practical.
|
||||
- `pnpm test`, `pnpm test:watch`, and `pnpm test:perf:imports` route explicit file/directory targets through scoped lanes first, so `pnpm test extensions/discord/src/monitor/message-handler.preflight.test.ts` avoids paying the full root project startup tax.
|
||||
- `pnpm test:changed` expands changed git paths into the same scoped lanes when the diff only touches routable source/test files; config/setup edits still fall back to the broad root-project rerun.
|
||||
- `pnpm check:changed` is the normal smart local gate for narrow work. It classifies the diff into core, core tests, extensions, extension tests, apps, docs, and tooling, then runs the matching typecheck/lint/test lanes. Public Plugin SDK and plugin-contract changes include extension validation because extensions depend on those core contracts.
|
||||
- `pnpm check:changed` is the normal smart local gate for narrow work. It classifies the diff into core, core tests, extensions, extension tests, apps, docs, release metadata, and tooling, then runs the matching typecheck/lint/test lanes. Public Plugin SDK and plugin-contract changes include extension validation because extensions depend on those core contracts. Release metadata-only version bumps run targeted version/config/root-dependency checks instead of the full suite, with a guard that rejects package changes outside the top-level version field.
|
||||
- Import-light unit tests from agents, commands, plugins, auto-reply helpers, `plugin-sdk`, and similar pure utility areas route through the `unit-fast` lane, which skips `test/setup-openclaw-runtime.ts`; stateful/runtime-heavy files stay on the existing lanes.
|
||||
- Selected `plugin-sdk` and `commands` helper source files also map changed-mode runs to explicit sibling tests in those light lanes, so helper edits avoid rerunning the full heavy suite for that directory.
|
||||
- `auto-reply` now has three dedicated buckets: top-level core helpers, top-level `reply.*` integration tests, and the `src/auto-reply/reply/**` subtree. This keeps the heaviest reply harness work off the cheap status/chunk/token tests.
|
||||
@@ -311,7 +311,7 @@ Think of the suites as “increasing realism” (and increasing flakiness/cost):
|
||||
- The shared `scripts/run-vitest.mjs` launcher now also adds `--no-maglev` for Vitest child Node processes by default to reduce V8 compile churn during big local runs. Set `OPENCLAW_VITEST_ENABLE_MAGLEV=1` if you need to compare against stock V8 behavior.
|
||||
- Fast-local iteration note:
|
||||
- `pnpm changed:lanes` shows which architectural lanes a diff triggers.
|
||||
- The pre-commit hook runs `pnpm check:changed --staged` after staged formatting/linting, so core-only commits do not pay extension test cost unless they touch public extension-facing contracts.
|
||||
- The pre-commit hook runs `pnpm check:changed --staged` after staged formatting/linting, so core-only commits do not pay extension test cost unless they touch public extension-facing contracts. Release metadata-only commits stay on the targeted version/config/root-dependency lane.
|
||||
- `pnpm test:changed` routes through scoped lanes when the changed paths map cleanly to a smaller suite.
|
||||
- `pnpm test:max` and `pnpm test:changed:max` keep the same routing behavior, just with a higher worker cap.
|
||||
- Local worker auto-scaling is intentionally conservative now and also backs off when the host load average is already high, so multiple concurrent Vitest runs do less damage by default.
|
||||
|
||||
@@ -14,7 +14,7 @@ title: "Tests"
|
||||
- `pnpm test:coverage:changed`: Runs unit coverage only for files changed since `origin/main`.
|
||||
- `pnpm test:changed`: expands changed git paths into scoped Vitest lanes when the diff only touches routable source/test files. Config/setup changes still fall back to the native root projects run so wiring edits rerun broadly when needed.
|
||||
- `pnpm changed:lanes`: shows the architectural lanes triggered by the diff against `origin/main`.
|
||||
- `pnpm check:changed`: runs the smart changed gate for the diff against `origin/main`. It runs core work with core test lanes, extension work with extension test lanes, test-only work with test typecheck/tests only, and expands public Plugin SDK or plugin-contract changes to extension validation.
|
||||
- `pnpm check:changed`: runs the smart changed gate for the diff against `origin/main`. It runs core work with core test lanes, extension work with extension test lanes, test-only work with test typecheck/tests only, expands public Plugin SDK or plugin-contract changes to extension validation, and keeps release metadata-only version bumps on targeted version/config/root-dependency checks.
|
||||
- `pnpm test`: routes explicit file/directory targets through scoped Vitest lanes. Untargeted runs use fixed shard groups and expand to leaf configs for local parallel execution; the extension group always expands to the per-extension shard configs instead of one giant root-project process.
|
||||
- Full and extension shard runs update local timing data in `.artifacts/vitest-shard-timings.json`; later runs use those timings to balance slow and fast shards. Set `OPENCLAW_TEST_PROJECTS_TIMINGS=0` to ignore the local timing artifact.
|
||||
- Selected `plugin-sdk` and `commands` test files now route through dedicated light lanes that keep only `test/setup.ts`, leaving runtime-heavy cases on their existing lanes.
|
||||
|
||||
@@ -1387,6 +1387,7 @@
|
||||
"qa:lab:up": "node --import tsx scripts/qa-lab-up.ts",
|
||||
"qa:lab:up:fast": "node --import tsx scripts/qa-lab-up.ts --use-prebuilt-image --bind-ui-dist --skip-ui-build",
|
||||
"qa:lab:watch": "vite build --watch --config extensions/qa-lab/web/vite.config.ts",
|
||||
"release-metadata:check": "node scripts/check-release-metadata-only.mjs",
|
||||
"release:check": "pnpm deps:root-ownership:check && pnpm check:base-config-schema && pnpm check:bundled-channel-config-metadata && pnpm config:docs:check && pnpm plugin-sdk:check-exports && pnpm plugin-sdk:api:check && node --import tsx scripts/release-check.ts",
|
||||
"release:openclaw:npm:check": "node --import tsx scripts/openclaw-npm-release-check.ts",
|
||||
"release:openclaw:npm:verify-published": "node --import tsx scripts/openclaw-npm-postpublish-verify.ts",
|
||||
|
||||
@@ -14,8 +14,21 @@ const TEST_PATH_RE =
|
||||
/(?:^|\/)(?:test|__tests__)\/|(?:\.|\/)(?:test|spec|e2e|browser\.test)\.[cm]?[jt]sx?$/u;
|
||||
const PUBLIC_EXTENSION_CONTRACT_RE =
|
||||
/^(?:src\/plugin-sdk\/|src\/plugins\/contracts\/|src\/channels\/plugins\/|scripts\/lib\/plugin-sdk-entrypoints\.json$|scripts\/sync-plugin-sdk-exports\.mjs$|scripts\/generate-plugin-sdk-api-baseline\.ts$)/u;
|
||||
export const RELEASE_METADATA_PATHS = new Set([
|
||||
"CHANGELOG.md",
|
||||
"apps/android/app/build.gradle.kts",
|
||||
"apps/ios/CHANGELOG.md",
|
||||
"apps/ios/Config/Version.xcconfig",
|
||||
"apps/ios/fastlane/metadata/en-US/release_notes.txt",
|
||||
"apps/ios/version.json",
|
||||
"apps/macos/Sources/OpenClaw/Resources/Info.plist",
|
||||
"docs/.generated/config-baseline.sha256",
|
||||
"docs/install/updating.md",
|
||||
"package.json",
|
||||
"src/config/schema.base.generated.ts",
|
||||
]);
|
||||
|
||||
/** @typedef {"core" | "coreTests" | "extensions" | "extensionTests" | "apps" | "docs" | "tooling" | "all"} ChangedLane */
|
||||
/** @typedef {"core" | "coreTests" | "extensions" | "extensionTests" | "apps" | "docs" | "tooling" | "releaseMetadata" | "all"} ChangedLane */
|
||||
|
||||
/**
|
||||
* @typedef {{
|
||||
@@ -43,6 +56,7 @@ export function createEmptyChangedLanes() {
|
||||
apps: false,
|
||||
docs: false,
|
||||
tooling: false,
|
||||
releaseMetadata: false,
|
||||
all: false,
|
||||
};
|
||||
}
|
||||
@@ -65,6 +79,20 @@ export function detectChangedLanes(changedPaths) {
|
||||
return { paths, lanes, extensionImpactFromCore: false, docsOnly: false, reasons };
|
||||
}
|
||||
|
||||
if (
|
||||
paths.some((changedPath) => RELEASE_METADATA_PATHS.has(changedPath)) &&
|
||||
paths.every(
|
||||
(changedPath) => RELEASE_METADATA_PATHS.has(changedPath) || DOCS_PATH_RE.test(changedPath),
|
||||
)
|
||||
) {
|
||||
lanes.releaseMetadata = true;
|
||||
lanes.docs = paths.some((changedPath) => DOCS_PATH_RE.test(changedPath));
|
||||
for (const changedPath of paths) {
|
||||
reasons.push(`${changedPath}: release metadata`);
|
||||
}
|
||||
return { paths, lanes, extensionImpactFromCore: false, docsOnly: false, reasons };
|
||||
}
|
||||
|
||||
for (const changedPath of paths) {
|
||||
if (DOCS_PATH_RE.test(changedPath)) {
|
||||
lanes.docs = true;
|
||||
|
||||
@@ -10,7 +10,7 @@ import { booleanFlag, parseFlagArgs, stringFlag } from "./lib/arg-utils.mjs";
|
||||
import { printTimingSummary } from "./lib/check-timing-summary.mjs";
|
||||
import { resolveChangedTestTargetPlan } from "./test-projects.test-support.mjs";
|
||||
|
||||
export function createChangedCheckPlan(result) {
|
||||
export function createChangedCheckPlan(result, options = {}) {
|
||||
const commands = [];
|
||||
const add = (name, args) => {
|
||||
if (!commands.some((command) => command.name === name && sameArgs(command.args, args))) {
|
||||
@@ -34,6 +34,28 @@ export function createChangedCheckPlan(result) {
|
||||
const lanes = result.lanes;
|
||||
const runAll = lanes.all;
|
||||
|
||||
if (lanes.releaseMetadata) {
|
||||
add("release metadata guard", [
|
||||
"release-metadata:check",
|
||||
"--",
|
||||
...(options.staged
|
||||
? ["--staged"]
|
||||
: ["--base", options.base ?? "origin/main", "--head", options.head ?? "HEAD"]),
|
||||
]);
|
||||
add("iOS version sync", ["ios:version:check"]);
|
||||
add("config schema baseline", ["config:schema:check"]);
|
||||
add("config docs baseline", ["config:docs:check"]);
|
||||
add("root dependency ownership", ["deps:root-ownership:check"]);
|
||||
return {
|
||||
commands,
|
||||
testTargets: [],
|
||||
runChangedTestsBroad: false,
|
||||
runFullTests: false,
|
||||
runExtensionTests: false,
|
||||
summary: "release metadata",
|
||||
};
|
||||
}
|
||||
|
||||
if (runAll) {
|
||||
add("typecheck all", ["tsgo:all"]);
|
||||
add("lint", ["lint"]);
|
||||
@@ -99,7 +121,7 @@ export function createChangedCheckPlan(result) {
|
||||
}
|
||||
|
||||
export async function runChangedCheck(result, options = {}) {
|
||||
const plan = createChangedCheckPlan(result);
|
||||
const plan = createChangedCheckPlan(result, options);
|
||||
printPlan(result, plan, options);
|
||||
|
||||
if (options.dryRun) {
|
||||
|
||||
151
scripts/check-release-metadata-only.mjs
Normal file
151
scripts/check-release-metadata-only.mjs
Normal file
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env node
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { RELEASE_METADATA_PATHS } from "./changed-lanes.mjs";
|
||||
|
||||
const VERSION_ONLY_TEXT_PATHS = new Set([
|
||||
"apps/android/app/build.gradle.kts",
|
||||
"apps/ios/Config/Version.xcconfig",
|
||||
"apps/ios/version.json",
|
||||
"apps/macos/Sources/OpenClaw/Resources/Info.plist",
|
||||
"src/config/schema.base.generated.ts",
|
||||
]);
|
||||
|
||||
function normalizePath(input) {
|
||||
return String(input ?? "")
|
||||
.trim()
|
||||
.replaceAll("\\", "/")
|
||||
.replace(/^\.\/+/u, "");
|
||||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
const args = { staged: false, base: "origin/main", head: "HEAD", paths: [] };
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const arg = argv[index];
|
||||
if (arg === "--staged") {
|
||||
args.staged = true;
|
||||
} else if (arg === "--base") {
|
||||
args.base = argv[++index] ?? "";
|
||||
} else if (arg === "--head") {
|
||||
args.head = argv[++index] ?? "";
|
||||
} else {
|
||||
args.paths.push(normalizePath(arg));
|
||||
}
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
function git(args) {
|
||||
return execFileSync("git", args, {
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
encoding: "utf8",
|
||||
});
|
||||
}
|
||||
|
||||
function listChangedPaths(args) {
|
||||
if (args.paths.length > 0) {
|
||||
return [...new Set(args.paths.filter(Boolean))].toSorted((left, right) =>
|
||||
left.localeCompare(right),
|
||||
);
|
||||
}
|
||||
const diffArgs = args.staged
|
||||
? ["diff", "--cached", "--name-only", "--diff-filter=ACMR"]
|
||||
: ["diff", "--name-only", "--diff-filter=ACMR", `${args.base}...${args.head}`];
|
||||
return git(diffArgs)
|
||||
.split("\n")
|
||||
.map(normalizePath)
|
||||
.filter(Boolean)
|
||||
.toSorted((left, right) => left.localeCompare(right));
|
||||
}
|
||||
|
||||
function readBlob(ref, filePath) {
|
||||
if (ref === "WORKTREE") {
|
||||
return readFileSync(filePath, "utf8");
|
||||
}
|
||||
return git(["show", `${ref}:${filePath}`]);
|
||||
}
|
||||
|
||||
function refsFor(args) {
|
||||
return args.staged ? { before: "HEAD", after: "" } : { before: args.base, after: args.head };
|
||||
}
|
||||
|
||||
function readBeforeAfter(args, filePath) {
|
||||
const refs = refsFor(args);
|
||||
const before = readBlob(refs.before, filePath);
|
||||
let after = readBlob(refs.after, filePath);
|
||||
if (!args.staged && existsSync(filePath)) {
|
||||
const worktree = readBlob("WORKTREE", filePath);
|
||||
if (worktree !== after) {
|
||||
after = worktree;
|
||||
}
|
||||
}
|
||||
return {
|
||||
before,
|
||||
after,
|
||||
};
|
||||
}
|
||||
|
||||
function stripPackageVersion(raw) {
|
||||
const parsed = JSON.parse(raw);
|
||||
delete parsed.version;
|
||||
return stableJson(parsed);
|
||||
}
|
||||
|
||||
function stableJson(value) {
|
||||
if (Array.isArray(value)) {
|
||||
return `[${value.map(stableJson).join(",")}]`;
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
return `{${Object.keys(value)
|
||||
.toSorted((left, right) => left.localeCompare(right))
|
||||
.map((key) => `${JSON.stringify(key)}:${stableJson(value[key])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function normalizeVersionText(raw) {
|
||||
return raw
|
||||
.replace(/\b20\d{2}\.\d{1,2}\.\d{1,2}(?:-beta\.\d+|-\d+)?\b/gu, "<OPENCLAW_VERSION>")
|
||||
.replace(/\b20\d{6}(?:\d{2})?\b/gu, "<OPENCLAW_BUILD>");
|
||||
}
|
||||
|
||||
function fail(message) {
|
||||
console.error(`[release-metadata] ${message}`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
const paths = listChangedPaths(args);
|
||||
|
||||
for (const filePath of paths) {
|
||||
if (!RELEASE_METADATA_PATHS.has(filePath)) {
|
||||
fail(`${filePath}: not a release metadata path; run the normal changed gate`);
|
||||
}
|
||||
}
|
||||
|
||||
if (paths.includes("package.json")) {
|
||||
const { before, after } = readBeforeAfter(args, "package.json");
|
||||
if (stripPackageVersion(before) !== stripPackageVersion(after)) {
|
||||
fail("package.json changed outside the top-level version field");
|
||||
}
|
||||
}
|
||||
|
||||
for (const filePath of paths) {
|
||||
if (!VERSION_ONLY_TEXT_PATHS.has(filePath)) {
|
||||
continue;
|
||||
}
|
||||
const { before, after } = readBeforeAfter(args, filePath);
|
||||
if (normalizeVersionText(before) !== normalizeVersionText(after)) {
|
||||
fail(`${filePath}: changed outside recognized version/build literals`);
|
||||
}
|
||||
}
|
||||
|
||||
if (process.exitCode) {
|
||||
process.exit(process.exitCode);
|
||||
}
|
||||
console.error(`[release-metadata] ok (${paths.length} files)`);
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -152,7 +152,7 @@ describe("scripts/changed-lanes", () => {
|
||||
});
|
||||
|
||||
it("fails safe for root config changes", () => {
|
||||
const result = detectChangedLanes(["package.json"]);
|
||||
const result = detectChangedLanes(["pnpm-lock.yaml"]);
|
||||
const plan = createChangedCheckPlan(result);
|
||||
|
||||
expect(result.lanes.all).toBe(true);
|
||||
@@ -160,6 +160,93 @@ describe("scripts/changed-lanes", () => {
|
||||
expect(plan.commands.map((command) => command.args[0])).toContain("tsgo:all");
|
||||
});
|
||||
|
||||
it("keeps release metadata commits off the full changed gate", () => {
|
||||
const result = detectChangedLanes([
|
||||
"CHANGELOG.md",
|
||||
"apps/android/app/build.gradle.kts",
|
||||
"apps/ios/CHANGELOG.md",
|
||||
"apps/ios/Config/Version.xcconfig",
|
||||
"apps/ios/fastlane/metadata/en-US/release_notes.txt",
|
||||
"apps/ios/version.json",
|
||||
"apps/macos/Sources/OpenClaw/Resources/Info.plist",
|
||||
"docs/.generated/config-baseline.sha256",
|
||||
"package.json",
|
||||
"src/config/schema.base.generated.ts",
|
||||
]);
|
||||
const plan = createChangedCheckPlan(result, { staged: true });
|
||||
|
||||
expect(result.lanes).toMatchObject({
|
||||
releaseMetadata: true,
|
||||
all: false,
|
||||
core: false,
|
||||
apps: false,
|
||||
});
|
||||
expect(plan.runFullTests).toBe(false);
|
||||
expect(plan.commands.map((command) => command.args[0])).toEqual([
|
||||
"check:no-conflict-markers",
|
||||
"release-metadata:check",
|
||||
"ios:version:check",
|
||||
"config:schema:check",
|
||||
"config:docs:check",
|
||||
"deps:root-ownership:check",
|
||||
]);
|
||||
});
|
||||
|
||||
it("guards release metadata package changes to the top-level version field", () => {
|
||||
const dir = makeTempRepoRoot(tempDirs, "openclaw-release-metadata-");
|
||||
git(dir, ["init", "-q", "--initial-branch=main"]);
|
||||
writeFileSync(
|
||||
path.join(dir, "package.json"),
|
||||
`${JSON.stringify({ name: "fixture", version: "2026.4.20", dependencies: { leftpad: "1.0.0" } }, null, 2)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
git(dir, ["add", "package.json"]);
|
||||
git(dir, [
|
||||
"-c",
|
||||
"user.email=test@example.com",
|
||||
"-c",
|
||||
"user.name=Test User",
|
||||
"commit",
|
||||
"-q",
|
||||
"-m",
|
||||
"initial",
|
||||
]);
|
||||
|
||||
writeFileSync(
|
||||
path.join(dir, "package.json"),
|
||||
`${JSON.stringify({ name: "fixture", version: "2026.4.21", dependencies: { leftpad: "1.0.0" } }, null, 2)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
git(dir, ["add", "package.json"]);
|
||||
expect(() =>
|
||||
execFileSync(
|
||||
process.execPath,
|
||||
[path.join(repoRoot, "scripts", "check-release-metadata-only.mjs"), "--staged"],
|
||||
{
|
||||
cwd: dir,
|
||||
stdio: "pipe",
|
||||
},
|
||||
),
|
||||
).not.toThrow();
|
||||
|
||||
writeFileSync(
|
||||
path.join(dir, "package.json"),
|
||||
`${JSON.stringify({ name: "fixture", version: "2026.4.21", dependencies: { leftpad: "1.0.1" } }, null, 2)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
git(dir, ["add", "package.json"]);
|
||||
expect(() =>
|
||||
execFileSync(
|
||||
process.execPath,
|
||||
[path.join(repoRoot, "scripts", "check-release-metadata-only.mjs"), "--staged"],
|
||||
{
|
||||
cwd: dir,
|
||||
stdio: "pipe",
|
||||
},
|
||||
),
|
||||
).toThrow();
|
||||
});
|
||||
|
||||
it("routes root test/support changes to the tooling test lane instead of all lanes", () => {
|
||||
const result = detectChangedLanes(["test/git-hooks-pre-commit.test.ts"]);
|
||||
const plan = createChangedCheckPlan(result);
|
||||
@@ -222,6 +309,7 @@ describe("scripts/changed-lanes", () => {
|
||||
apps: false,
|
||||
docs: false,
|
||||
tooling: false,
|
||||
releaseMetadata: false,
|
||||
all: false,
|
||||
});
|
||||
expect(plan.commands).toEqual([
|
||||
|
||||
Reference in New Issue
Block a user