mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-18 10:14:45 +00:00
docs(skills): add release ci workflow skill
This commit is contained in:
90
.agents/skills/openclaw-release-ci/SKILL.md
Normal file
90
.agents/skills/openclaw-release-ci/SKILL.md
Normal file
@@ -0,0 +1,90 @@
|
||||
---
|
||||
name: openclaw-release-ci
|
||||
description: "Run, watch, debug, and summarize OpenClaw full release CI, release checks, live provider gates, install/update proofs, and release-secret preflights."
|
||||
---
|
||||
|
||||
# OpenClaw Release CI
|
||||
|
||||
Use this with `$openclaw-release-maintainer` and `$openclaw-testing` when a release candidate needs full validation, install/update proof, live provider checks, or CI recovery.
|
||||
|
||||
## Guardrails
|
||||
|
||||
- No version bump, tag, npm publish, GitHub release, or release promotion without explicit operator approval.
|
||||
- Validate provider secrets before dispatching expensive full release matrices.
|
||||
- Do not set GitHub secrets from unvalidated 1Password candidates. If a candidate returns 401/403, leave the existing secret alone and report the exact missing provider.
|
||||
- Use `$one-password` for secret reads/writes: one persistent tmux session, targeted items only, no secret output.
|
||||
- Watch one parent run plus compact child summaries. Avoid broad `gh run view` polling loops; REST quota is easy to burn.
|
||||
- Fetch logs only for failed or currently-blocking jobs. If quota is low, stop polling and wait for reset.
|
||||
- Treat live-provider flakes separately from code failures: prove key validity, provider HTTP status, retry evidence, and exact failing lane before editing code.
|
||||
|
||||
## Preflight
|
||||
|
||||
Before full release validation:
|
||||
|
||||
```bash
|
||||
node .agents/skills/openclaw-release-ci/scripts/verify-provider-secrets.mjs --required openai,anthropic,fireworks
|
||||
gh api rate_limit --jq '.resources.core'
|
||||
git status --short --branch
|
||||
git rev-parse HEAD
|
||||
```
|
||||
|
||||
If env lacks keys, use `$one-password` to inject or set them, then rerun the script. The script prints only provider status and HTTP class, never tokens.
|
||||
|
||||
## Dispatch
|
||||
|
||||
Prefer the trusted workflow on `main`, target the exact release SHA:
|
||||
|
||||
```bash
|
||||
gh workflow run full-release-validation.yml \
|
||||
--repo openclaw/openclaw \
|
||||
--ref main \
|
||||
-f ref=<release-sha> \
|
||||
-f provider=openai \
|
||||
-f mode=both \
|
||||
-f release_profile=full \
|
||||
-f rerun_group=all
|
||||
```
|
||||
|
||||
Use `release_profile=stable` unless the operator explicitly asks for the broad advisory provider/media matrix. Use narrow `rerun_group` after focused fixes.
|
||||
|
||||
## Watch
|
||||
|
||||
Use the summary helper instead of repeated raw polling:
|
||||
|
||||
```bash
|
||||
node .agents/skills/openclaw-release-ci/scripts/release-ci-summary.mjs <full-release-run-id>
|
||||
```
|
||||
|
||||
Then watch only when useful:
|
||||
|
||||
```bash
|
||||
gh run watch <full-release-run-id> --repo openclaw/openclaw --exit-status
|
||||
```
|
||||
|
||||
Stop watchers before ending the turn or switching strategy.
|
||||
|
||||
## Failure Triage
|
||||
|
||||
1. Confirm parent SHA and child run IDs.
|
||||
2. List failed jobs only:
|
||||
```bash
|
||||
gh run view <child-run-id> --repo openclaw/openclaw --json jobs \
|
||||
--jq '.jobs[] | select(.conclusion=="failure" or .conclusion=="timed_out" or .conclusion=="cancelled") | [.databaseId,.name,.conclusion,.url] | @tsv'
|
||||
```
|
||||
3. Fetch one failed job log. If rate-limited, note reset time and avoid more REST calls.
|
||||
4. For secret-looking failures, validate the provider endpoint from the same secret source before editing code.
|
||||
5. For live-cache failures, inspect whether it is missing/invalid key, empty text, provider refusal, timeout, or baseline miss. Do not weaken release gates without clear provider evidence.
|
||||
6. Fix narrowly, run local/changed proof, commit, push, rerun the smallest matching group.
|
||||
|
||||
## Evidence
|
||||
|
||||
Record:
|
||||
|
||||
- release SHA
|
||||
- full parent run URL
|
||||
- child run IDs and conclusions: CI, Release Checks, Plugin Prerelease, NPM Telegram
|
||||
- targeted local proof commands
|
||||
- provider-secret preflight result
|
||||
- known gaps or unrelated failures
|
||||
|
||||
For lessons and recovery patterns, read `references/release-ci-notes.md`.
|
||||
4
.agents/skills/openclaw-release-ci/agents/openai.yaml
Normal file
4
.agents/skills/openclaw-release-ci/agents/openai.yaml
Normal file
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "OpenClaw Release CI"
|
||||
short_description: "Verify and debug OpenClaw release validation runs"
|
||||
default_prompt: "Use $openclaw-release-ci to preflight provider secrets, watch full release validation, summarize child runs, and triage only failing release lanes."
|
||||
@@ -0,0 +1,41 @@
|
||||
# Release CI Notes
|
||||
|
||||
## What Went Wrong
|
||||
|
||||
- Full validation was started before all provider keys were proven valid.
|
||||
- GitHub secret presence was confused with key validity.
|
||||
- Repeated `gh run view` and log fetches exhausted REST quota.
|
||||
- Parent run state was less useful than child run evidence.
|
||||
- Live-cache failures needed structured classification: invalid key, empty provider output, timeout, or real cache regression.
|
||||
- Background watchers accumulated and made interruption recovery harder.
|
||||
|
||||
## Better Defaults
|
||||
|
||||
- Run provider-secret preflight first. Require real `/models` or equivalent endpoint checks for release-blocking providers.
|
||||
- Keep one watcher open. Use child summaries every few minutes, not every few seconds.
|
||||
- Fetch failed-job logs only after a job reaches a terminal failing state.
|
||||
- Prefer narrow `rerun_group` recovery after a focused fix.
|
||||
- Leave bad secrets unset. A 401 candidate from 1Password should not overwrite GitHub.
|
||||
- Make the final release evidence note durable: parent URL, child run URLs, SHA, command proof, and gaps.
|
||||
|
||||
## Secret Handling Pattern
|
||||
|
||||
- Use `$one-password`; never run broad env dumps.
|
||||
- Search exact item titles or known ids.
|
||||
- Validate candidates without printing values.
|
||||
- Set GitHub secrets only after endpoint validation succeeds.
|
||||
- After setting, verify metadata with `gh secret list`, not value output.
|
||||
|
||||
## Live Cache Pattern
|
||||
|
||||
- Empty text with token usage is a provider/output issue until proven otherwise.
|
||||
- Retry lane-level mismatches once with a fresh session id.
|
||||
- Keep cache baselines strict, but log enough structured usage to distinguish cache miss from response mismatch.
|
||||
- If a provider key validates locally but fails in Actions, inspect whether the workflow reads the expected secret name.
|
||||
|
||||
## Quota-Safe GitHub Pattern
|
||||
|
||||
- Check `gh api rate_limit --jq '.resources.core'` before log-heavy work.
|
||||
- Use one child-run listing call, then inspect failed jobs only.
|
||||
- If remaining quota is low, pause until reset; do not keep polling.
|
||||
- Prefer GraphQL only for metadata when REST is exhausted; logs still need REST.
|
||||
79
.agents/skills/openclaw-release-ci/scripts/release-ci-summary.mjs
Executable file
79
.agents/skills/openclaw-release-ci/scripts/release-ci-summary.mjs
Executable file
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env node
|
||||
import { execFileSync } from "node:child_process";
|
||||
import process from "node:process";
|
||||
|
||||
const runId = process.argv[2];
|
||||
const repo = process.env.OPENCLAW_RELEASE_REPO || "openclaw/openclaw";
|
||||
|
||||
if (!runId) {
|
||||
console.error("usage: release-ci-summary.mjs <full-release-run-id>");
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
function gh(args) {
|
||||
return execFileSync("gh", args, {
|
||||
encoding: "utf8",
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
}
|
||||
|
||||
function jsonGh(args) {
|
||||
return JSON.parse(gh(args));
|
||||
}
|
||||
|
||||
function rate() {
|
||||
try {
|
||||
return jsonGh(["api", "rate_limit"]).resources.core;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
const core = rate();
|
||||
if (core) {
|
||||
const reset = new Date(core.reset * 1000).toISOString();
|
||||
console.log(`rate: remaining=${core.remaining}/${core.limit} reset=${reset}`);
|
||||
if (core.remaining < 20) {
|
||||
console.error("rate too low for CI summary; wait for reset before polling");
|
||||
process.exit(3);
|
||||
}
|
||||
}
|
||||
|
||||
const parent = jsonGh([
|
||||
"run",
|
||||
"view",
|
||||
runId,
|
||||
"--repo",
|
||||
repo,
|
||||
"--json",
|
||||
"status,conclusion,createdAt,headSha,url,jobs",
|
||||
]);
|
||||
|
||||
console.log(`parent: ${runId} ${parent.status}/${parent.conclusion || "none"}`);
|
||||
console.log(`sha: ${parent.headSha}`);
|
||||
console.log(`url: ${parent.url}`);
|
||||
|
||||
for (const job of parent.jobs ?? []) {
|
||||
const marker = job.conclusion || job.status;
|
||||
console.log(`parent-job: ${marker} ${job.name}`);
|
||||
}
|
||||
|
||||
const since = parent.createdAt;
|
||||
const runList = gh([
|
||||
"api",
|
||||
`repos/${repo}/actions/runs?per_page=100`,
|
||||
"--jq",
|
||||
`.workflow_runs[] | select(.created_at >= "${since}") | select(.name=="CI" or .name=="OpenClaw Release Checks" or .name=="Plugin Prerelease" or .name=="NPM Telegram Beta E2E" or .name=="Full Release Validation") | [.id,.name,.status,.conclusion,.head_sha,.html_url] | @tsv`,
|
||||
]).trim();
|
||||
|
||||
if (!runList) {
|
||||
console.log("children: none found yet");
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
console.log("children:");
|
||||
for (const line of runList.split("\n")) {
|
||||
const [id, name, status, conclusion, sha, url] = line.split("\t");
|
||||
console.log(`child: ${id} ${name} ${status}/${conclusion || "none"} sha=${sha}`);
|
||||
console.log(`child-url: ${url}`);
|
||||
}
|
||||
113
.agents/skills/openclaw-release-ci/scripts/verify-provider-secrets.mjs
Executable file
113
.agents/skills/openclaw-release-ci/scripts/verify-provider-secrets.mjs
Executable file
@@ -0,0 +1,113 @@
|
||||
#!/usr/bin/env node
|
||||
import process from "node:process";
|
||||
|
||||
const args = new Map();
|
||||
for (let index = 2; index < process.argv.length; index += 1) {
|
||||
const arg = process.argv[index];
|
||||
if (!arg.startsWith("--")) continue;
|
||||
const [key, inlineValue] = arg.slice(2).split("=", 2);
|
||||
const value = inlineValue ?? process.argv[index + 1];
|
||||
if (inlineValue === undefined) index += 1;
|
||||
args.set(key, value);
|
||||
}
|
||||
|
||||
const requiredInput = String(args.get("required") ?? "openai,anthropic").trim();
|
||||
const required = new Set(
|
||||
(requiredInput.toLowerCase() === "none" ? "" : requiredInput)
|
||||
.split(",")
|
||||
.map((entry) => entry.trim().toLowerCase())
|
||||
.filter(Boolean),
|
||||
);
|
||||
|
||||
const timeoutMs = Number(args.get("timeout-ms") ?? 10_000);
|
||||
|
||||
function envFirst(names) {
|
||||
for (const name of names) {
|
||||
const value = process.env[name]?.trim();
|
||||
if (value) return { name, value };
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
async function checkProvider(id, config) {
|
||||
const secret = envFirst(config.env);
|
||||
if (!secret) {
|
||||
return { id, ok: false, status: "missing", env: config.env.join("|") };
|
||||
}
|
||||
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
const headers = config.headers(secret.value);
|
||||
const response = await fetch(config.url, {
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
});
|
||||
return {
|
||||
id,
|
||||
ok: response.ok,
|
||||
status: response.ok ? "ok" : `http_${response.status}`,
|
||||
env: secret.name,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
id,
|
||||
ok: false,
|
||||
status: error?.name === "AbortError" ? "timeout" : "error",
|
||||
env: secret.name,
|
||||
};
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
const providers = {
|
||||
openai: {
|
||||
env: ["OPENAI_API_KEY"],
|
||||
url: "https://api.openai.com/v1/models",
|
||||
headers: (token) => ({ authorization: `Bearer ${token}` }),
|
||||
},
|
||||
anthropic: {
|
||||
env: ["ANTHROPIC_API_KEY", "ANTHROPIC_API_TOKEN"],
|
||||
url: "https://api.anthropic.com/v1/models",
|
||||
headers: (token) => ({
|
||||
"anthropic-version": "2023-06-01",
|
||||
"x-api-key": token,
|
||||
}),
|
||||
},
|
||||
fireworks: {
|
||||
env: ["FIREWORKS_API_KEY"],
|
||||
url: "https://api.fireworks.ai/inference/v1/models",
|
||||
headers: (token) => ({ authorization: `Bearer ${token}` }),
|
||||
},
|
||||
openrouter: {
|
||||
env: ["OPENROUTER_API_KEY"],
|
||||
url: "https://openrouter.ai/api/v1/models",
|
||||
headers: (token) => ({ authorization: `Bearer ${token}` }),
|
||||
},
|
||||
};
|
||||
|
||||
const unknown = [...required].filter((id) => !providers[id]);
|
||||
if (unknown.length > 0) {
|
||||
console.error(`unknown providers: ${unknown.join(",")}`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const results = [];
|
||||
for (const id of Object.keys(providers)) {
|
||||
if (required.has(id) || envFirst(providers[id].env)) {
|
||||
results.push(await checkProvider(id, providers[id]));
|
||||
}
|
||||
}
|
||||
|
||||
let failed = false;
|
||||
for (const result of results) {
|
||||
const requiredLabel = required.has(result.id) ? "required" : "optional";
|
||||
console.log(`${result.id}: ${result.status} env=${result.env} ${requiredLabel}`);
|
||||
if (required.has(result.id) && !result.ok) failed = true;
|
||||
}
|
||||
|
||||
if (failed) {
|
||||
console.error("release provider secret preflight failed");
|
||||
process.exit(1);
|
||||
}
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -139,6 +139,8 @@ mantis/
|
||||
!.agents/skills/openclaw-refactor-docs/**
|
||||
!.agents/skills/openclaw-qa-testing/
|
||||
!.agents/skills/openclaw-qa-testing/**
|
||||
!.agents/skills/openclaw-release-ci/
|
||||
!.agents/skills/openclaw-release-ci/**
|
||||
!.agents/skills/openclaw-release-maintainer/
|
||||
!.agents/skills/openclaw-release-maintainer/**
|
||||
!.agents/skills/openclaw-secret-scanning-maintainer/
|
||||
|
||||
Reference in New Issue
Block a user