Rename Codex runtime prompt snapshot directory (#76274)

* test: rename Codex runtime prompt snapshots

* test: refresh prompt snapshots with node24

* test: format prompt snapshot markdown in generator
This commit is contained in:
pashpashpash
2026-05-02 15:34:59 -07:00
committed by GitHub
parent dc005e1bcc
commit bcb2476ae7
12 changed files with 43 additions and 55 deletions

View File

@@ -54,7 +54,7 @@ Scope logic lives in `scripts/ci-changed-scope.mjs` and is covered by unit tests
- **CI routing-only edits, selected cheap core-test fixture edits, and narrow plugin contract helper/test-routing edits** use a fast Node-only manifest path: `preflight`, security, and a single `checks-fast-core` task. That path skips build artifacts, Node 22 compatibility, channel contracts, full core shards, bundled-plugin shards, and additional guard matrices when the change is limited to the routing or helper surfaces the fast task exercises directly. - **CI routing-only edits, selected cheap core-test fixture edits, and narrow plugin contract helper/test-routing edits** use a fast Node-only manifest path: `preflight`, security, and a single `checks-fast-core` task. That path skips build artifacts, Node 22 compatibility, channel contracts, full core shards, bundled-plugin shards, and additional guard matrices when the change is limited to the routing or helper surfaces the fast task exercises directly.
- **Windows Node checks** are scoped to Windows-specific process/path wrappers, npm/pnpm/UI runner helpers, package manager config, and the CI workflow surfaces that execute that lane; unrelated source, plugin, install-smoke, and test-only changes stay on the Linux Node lanes. - **Windows Node checks** are scoped to Windows-specific process/path wrappers, npm/pnpm/UI runner helpers, package manager config, and the CI workflow surfaces that execute that lane; unrelated source, plugin, install-smoke, and test-only changes stay on the Linux Node lanes.
The slowest Node test families are split or balanced so each job stays small without over-reserving runners: channel contracts run as three weighted shards, small core unit lanes are paired, auto-reply runs as four balanced workers (with the reply subtree split into agent-runner, dispatch, and commands/state-routing shards), and agentic gateway/plugin configs are spread across the existing source-only agentic Node jobs instead of waiting on built artifacts. Broad browser, QA, media, and miscellaneous plugin tests use their dedicated Vitest configs instead of the shared plugin catch-all. Include-pattern shards record timing entries using the CI shard name, so `.artifacts/vitest-shard-timings.json` can distinguish a whole config from a filtered shard. `check-additional` keeps package-boundary compile/canary work together and separates runtime topology architecture from gateway watch coverage; the boundary guard shard runs its small independent guards concurrently inside one job, including `pnpm prompt:snapshots:check` so Codex happy-path prompt drift is pinned to the PR that caused it. Gateway watch, channel tests, and the core support-boundary shard run concurrently inside `build-artifacts` after `dist/` and `dist-runtime/` are already built. The slowest Node test families are split or balanced so each job stays small without over-reserving runners: channel contracts run as three weighted shards, small core unit lanes are paired, auto-reply runs as four balanced workers (with the reply subtree split into agent-runner, dispatch, and commands/state-routing shards), and agentic gateway/plugin configs are spread across the existing source-only agentic Node jobs instead of waiting on built artifacts. Broad browser, QA, media, and miscellaneous plugin tests use their dedicated Vitest configs instead of the shared plugin catch-all. Include-pattern shards record timing entries using the CI shard name, so `.artifacts/vitest-shard-timings.json` can distinguish a whole config from a filtered shard. `check-additional` keeps package-boundary compile/canary work together and separates runtime topology architecture from gateway watch coverage; the boundary guard shard runs its small independent guards concurrently inside one job, including `pnpm prompt:snapshots:check` so Codex runtime happy-path prompt drift is pinned to the PR that caused it. Gateway watch, channel tests, and the core support-boundary shard run concurrently inside `build-artifacts` after `dist/` and `dist-runtime/` are already built.
Android CI runs both `testPlayDebugUnitTest` and `testThirdPartyDebugUnitTest` and then builds the Play debug APK. The third-party flavor has no separate source set or manifest; its unit-test lane still compiles the flavor with the SMS/call-log BuildConfig flags, while avoiding a duplicate debug APK packaging job on every Android-relevant push. Android CI runs both `testPlayDebugUnitTest` and `testThirdPartyDebugUnitTest` and then builds the Play debug APK. The third-party flavor has no separate source set or manifest; its unit-test lane still compiles the flavor with the SMS/call-log BuildConfig flags, while avoiding a duplicate debug APK packaging job on every Android-relevant push.

View File

@@ -111,8 +111,8 @@ in both the global system prompt and channel context.
## Prompt snapshots ## Prompt snapshots
OpenClaw keeps committed happy-path prompt snapshots for the Codex/message-tool OpenClaw keeps committed prompt snapshots for the Codex runtime happy path under
runtime under `test/fixtures/agents/prompt-snapshots/happy-path/`. They render `test/fixtures/agents/prompt-snapshots/codex-runtime-happy-path/`. They render
selected app-server thread/turn params plus a reconstructed model-bound prompt selected app-server thread/turn params plus a reconstructed model-bound prompt
layer stack for Telegram direct, Discord group, and heartbeat turns. That stack layer stack for Telegram direct, Discord group, and heartbeat turns. That stack
includes a pinned Codex `gpt-5.5` model prompt fixture generated from Codex's includes a pinned Codex `gpt-5.5` model prompt fixture generated from Codex's

View File

@@ -5,8 +5,8 @@ import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url"; import { fileURLToPath, pathToFileURL } from "node:url";
import { promisify } from "node:util"; import { promisify } from "node:util";
import { import {
CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR,
createHappyPathPromptSnapshotFiles, createHappyPathPromptSnapshotFiles,
HAPPY_PATH_PROMPT_SNAPSHOT_DIR,
} from "../test/helpers/agents/happy-path-prompt-snapshots.js"; } from "../test/helpers/agents/happy-path-prompt-snapshots.js";
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
@@ -40,7 +40,7 @@ async function writeSnapshotFiles(root: string, files: PromptSnapshotFile[]) {
async function formatSnapshotFiles(root: string, files: PromptSnapshotFile[]) { async function formatSnapshotFiles(root: string, files: PromptSnapshotFile[]) {
const filePaths = files const filePaths = files
.filter((file) => file.path.endsWith(".json")) .filter((file) => file.path.endsWith(".md") || file.path.endsWith(".json"))
.map((file) => path.resolve(root, file.path)); .map((file) => path.resolve(root, file.path));
if (filePaths.length === 0) { if (filePaths.length === 0) {
return; return;
@@ -62,7 +62,9 @@ async function readSnapshotFiles(root: string, files: PromptSnapshotFile[]) {
async function listCommittedSnapshotArtifactPaths(root: string): Promise<string[]> { async function listCommittedSnapshotArtifactPaths(root: string): Promise<string[]> {
let committedEntries: string[]; let committedEntries: string[];
try { try {
committedEntries = await fs.readdir(path.resolve(root, HAPPY_PATH_PROMPT_SNAPSHOT_DIR)); committedEntries = await fs.readdir(
path.resolve(root, CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR),
);
} catch (error) { } catch (error) {
if (!hasErrorCode(error, "ENOENT")) { if (!hasErrorCode(error, "ENOENT")) {
throw error; throw error;
@@ -71,7 +73,7 @@ async function listCommittedSnapshotArtifactPaths(root: string): Promise<string[
} }
return committedEntries return committedEntries
.filter((entry) => entry.endsWith(".md") || entry.endsWith(".json")) .filter((entry) => entry.endsWith(".md") || entry.endsWith(".json"))
.map((entry) => path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, entry)); .map((entry) => path.join(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR, entry));
} }
export async function deleteStalePromptSnapshotFiles( export async function deleteStalePromptSnapshotFiles(
@@ -100,7 +102,9 @@ export async function createFormattedPromptSnapshotFiles(): Promise<PromptSnapsh
async function writeSnapshots() { async function writeSnapshots() {
const files = await createFormattedPromptSnapshotFiles(); const files = await createFormattedPromptSnapshotFiles();
await fs.mkdir(path.resolve(repoRoot, HAPPY_PATH_PROMPT_SNAPSHOT_DIR), { recursive: true }); await fs.mkdir(path.resolve(repoRoot, CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR), {
recursive: true,
});
const deleted = await deleteStalePromptSnapshotFiles(repoRoot, files); const deleted = await deleteStalePromptSnapshotFiles(repoRoot, files);
await writeSnapshotFiles(repoRoot, files); await writeSnapshotFiles(repoRoot, files);
const deletedSummary = deleted.length > 0 ? ` Deleted ${deleted.length} stale file(s).` : ""; const deletedSummary = deleted.length > 0 ? ` Deleted ${deleted.length} stale file(s).` : "";

View File

@@ -568,9 +568,7 @@ Full JSON: `codex-dynamic-tools.discord-group.json`
"type": "string" "type": "string"
}, },
"action": { "action": {
"enum": [ "enum": ["send"],
"send"
],
"type": "string" "type": "string"
}, },
"activityName": { "activityName": {
@@ -938,9 +936,7 @@ Full JSON: `codex-dynamic-tools.discord-group.json`
"type": "string" "type": "string"
} }
}, },
"required": [ "required": ["action"],
"action"
],
"type": "object" "type": "object"
}, },
"name": "message" "name": "message"

View File

@@ -545,9 +545,7 @@ Full JSON: `codex-dynamic-tools.telegram-direct.json`
"type": "string" "type": "string"
}, },
"action": { "action": {
"enum": [ "enum": ["send"],
"send"
],
"type": "string" "type": "string"
}, },
"activityName": { "activityName": {
@@ -915,9 +913,7 @@ Full JSON: `codex-dynamic-tools.telegram-direct.json`
"type": "string" "type": "string"
} }
}, },
"required": [ "required": ["action"],
"action"
],
"type": "object" "type": "object"
}, },
"name": "message" "name": "message"

View File

@@ -547,9 +547,7 @@ Full JSON: `codex-dynamic-tools.heartbeat-turn.json`
"type": "string" "type": "string"
}, },
"action": { "action": {
"enum": [ "enum": ["send"],
"send"
],
"type": "string" "type": "string"
}, },
"activityName": { "activityName": {
@@ -917,9 +915,7 @@ Full JSON: `codex-dynamic-tools.heartbeat-turn.json`
"type": "string" "type": "string"
} }
}, },
"required": [ "required": ["action"],
"action"
],
"type": "object" "type": "object"
}, },
"name": "message" "name": "message"
@@ -939,21 +935,11 @@ Full JSON: `codex-dynamic-tools.heartbeat-turn.json`
"type": "boolean" "type": "boolean"
}, },
"outcome": { "outcome": {
"enum": [ "enum": ["no_change", "progress", "done", "blocked", "needs_attention"],
"no_change",
"progress",
"done",
"blocked",
"needs_attention"
],
"type": "string" "type": "string"
}, },
"priority": { "priority": {
"enum": [ "enum": ["low", "normal", "high"],
"low",
"normal",
"high"
],
"type": "string" "type": "string"
}, },
"reason": { "reason": {
@@ -963,11 +949,7 @@ Full JSON: `codex-dynamic-tools.heartbeat-turn.json`
"type": "string" "type": "string"
} }
}, },
"required": [ "required": ["outcome", "notify", "summary"],
"outcome",
"notify",
"summary"
],
"type": "object" "type": "object"
}, },
"name": "heartbeat_respond" "name": "heartbeat_respond"

View File

@@ -23,7 +23,8 @@ import { normalizeAgentRuntimeTools } from "../../../src/plugin-sdk/agent-harnes
import { createOpenClawCodingTools } from "../../../src/plugin-sdk/agent-harness.js"; import { createOpenClawCodingTools } from "../../../src/plugin-sdk/agent-harness.js";
import { loadBundledPluginTestApiSync } from "../../../src/test-utils/bundled-plugin-public-surface.js"; import { loadBundledPluginTestApiSync } from "../../../src/test-utils/bundled-plugin-public-surface.js";
export const HAPPY_PATH_PROMPT_SNAPSHOT_DIR = "test/fixtures/agents/prompt-snapshots/happy-path"; export const CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR =
"test/fixtures/agents/prompt-snapshots/codex-runtime-happy-path";
export const CODEX_MODEL_PROMPT_FIXTURE_DIR = export const CODEX_MODEL_PROMPT_FIXTURE_DIR =
"test/fixtures/agents/prompt-snapshots/codex-model-catalog"; "test/fixtures/agents/prompt-snapshots/codex-model-catalog";
@@ -697,8 +698,14 @@ function renderReadme(scenarios: PromptScenario[]): string {
"", "",
"Codex model prompt fixtures:", "Codex model prompt fixtures:",
"", "",
`- ${path.relative(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, CODEX_MODEL_PROMPT_FIXTURE_PATH)}`, `- ${path.relative(
`- ${path.relative(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, CODEX_MODEL_PROMPT_SOURCE_PATH)}`, CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR,
CODEX_MODEL_PROMPT_FIXTURE_PATH,
)}`,
`- ${path.relative(
CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR,
CODEX_MODEL_PROMPT_SOURCE_PATH,
)}`,
"", "",
].join("\n"); ].join("\n");
} }
@@ -707,15 +714,15 @@ export function createHappyPathPromptSnapshotFiles(): PromptSnapshotFile[] {
const scenarios = createScenarios(); const scenarios = createScenarios();
return [ return [
{ {
path: path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, "README.md"), path: path.join(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR, "README.md"),
content: renderReadme(scenarios), content: renderReadme(scenarios),
}, },
...scenarios.map((scenario) => ({ ...scenarios.map((scenario) => ({
path: path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, `${scenario.id}.md`), path: path.join(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR, `${scenario.id}.md`),
content: renderScenarioSnapshot(scenario), content: renderScenarioSnapshot(scenario),
})), })),
...scenarios.map((scenario) => ({ ...scenarios.map((scenario) => ({
path: path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, scenario.toolSnapshotFile), path: path.join(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR, scenario.toolSnapshotFile),
content: stableJson(scenario.dynamicTools), content: stableJson(scenario.dynamicTools),
})), })),
].map((file) => ({ ].map((file) => ({

View File

@@ -14,7 +14,7 @@ import {
} from "../../scripts/sync-codex-model-prompt-fixture.js"; } from "../../scripts/sync-codex-model-prompt-fixture.js";
import { import {
CODEX_MODEL_PROMPT_FIXTURE_DIR, CODEX_MODEL_PROMPT_FIXTURE_DIR,
HAPPY_PATH_PROMPT_SNAPSHOT_DIR, CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR,
} from "../helpers/agents/happy-path-prompt-snapshots.js"; } from "../helpers/agents/happy-path-prompt-snapshots.js";
describe("happy path prompt snapshots", () => { describe("happy path prompt snapshots", () => {
@@ -25,22 +25,25 @@ describe("happy path prompt snapshots", () => {
expect(fs.readFileSync(file.path, "utf8"), file.path).toBe(file.content); expect(fs.readFileSync(file.path, "utf8"), file.path).toBe(file.content);
} }
const committed = fs const committed = fs
.readdirSync(HAPPY_PATH_PROMPT_SNAPSHOT_DIR) .readdirSync(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR)
.filter((entry) => entry.endsWith(".md") || entry.endsWith(".json")) .filter((entry) => entry.endsWith(".md") || entry.endsWith(".json"))
.map((entry) => path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, entry)); .map((entry) => path.join(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR, entry));
expect(committed.toSorted()).toEqual([...expectedPaths].toSorted()); expect(committed.toSorted()).toEqual([...expectedPaths].toSorted());
}); });
it("deletes stale generated snapshot artifacts", async () => { it("deletes stale generated snapshot artifacts", async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-prompt-snapshot-stale-")); const root = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-prompt-snapshot-stale-"));
try { try {
const snapshotDir = path.join(root, HAPPY_PATH_PROMPT_SNAPSHOT_DIR); const snapshotDir = path.join(root, CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR);
fs.mkdirSync(snapshotDir, { recursive: true }); fs.mkdirSync(snapshotDir, { recursive: true });
const stalePath = path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, "stale-snapshot.md"); const stalePath = path.join(
CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR,
"stale-snapshot.md",
);
fs.writeFileSync(path.join(root, stalePath), "stale\n"); fs.writeFileSync(path.join(root, stalePath), "stale\n");
const deleted = await deleteStalePromptSnapshotFiles(root, [ const deleted = await deleteStalePromptSnapshotFiles(root, [
{ path: path.join(HAPPY_PATH_PROMPT_SNAPSHOT_DIR, "current.md") }, { path: path.join(CODEX_RUNTIME_HAPPY_PATH_PROMPT_SNAPSHOT_DIR, "current.md") },
]); ]);
expect(deleted).toEqual([stalePath]); expect(deleted).toEqual([stalePath]);