From a3d0b4307b15a4037ce434eb09e0674c4b65035e Mon Sep 17 00:00:00 2001 From: Vincent Koc Date: Wed, 22 Apr 2026 23:13:29 -0700 Subject: [PATCH] docs(release): use QA Lab all lanes gate --- .../openclaw-release-maintainer/SKILL.md | 73 ++++++++++++------- 1 file changed, 46 insertions(+), 27 deletions(-) diff --git a/.agents/skills/openclaw-release-maintainer/SKILL.md b/.agents/skills/openclaw-release-maintainer/SKILL.md index 0f9cae23a5b..0b8b6e402fc 100644 --- a/.agents/skills/openclaw-release-maintainer/SKILL.md +++ b/.agents/skills/openclaw-release-maintainer/SKILL.md @@ -201,9 +201,18 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts - Source Peter's profile before live release validation so OpenAI and Anthropic credentials are available without printing secrets: `set -a; source "$HOME/.profile"; set +a`. -- Release QA and Parallels validation for this train must use both +- Parallels validation and any local live model QA for this train must use both `OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. If either is missing after sourcing - `.profile`, stop before starting the long lanes and report the missing key. + `.profile`, stop before starting those local long lanes and report the + missing key. +- Live credentialed channel QA is the GitHub Actions workflow + `QA-Lab - All Lanes` (`.github/workflows/qa-live-telegram-convex.yml`), not a + local substitute. Dispatch it from Actions against the release tag and wait + for it to pass before npm preflight/publish readiness. Use a SHA only when it + satisfies the workflow's secret-bearing trust gate: main ancestor or open PR + head. It runs the QA Lab mock parity gate plus live Matrix and live Telegram + lanes using the `qa-live-shared` environment; Telegram uses Convex CI + credential leases. - Default release checks: - `pnpm check` - `pnpm check:test-types` @@ -221,9 +230,12 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts - all Parallels install/update tests: `pnpm test:parallels:npm-update -- --json` plus any needed individual rerun lanes from `openclaw-parallels-smoke` - - all QA release validation: - OpenAI live suite with `openai/gpt-5.4` in fast mode, Anthropic live suite - with `anthropic/claude-opus-4-6`, and the repo-backed character evals + - all QA release validation: dispatch GitHub Actions > `QA-Lab - All Lanes` + against the release tag and require success. This is the release gate for + live credentialed Matrix/Telegram channel coverage. Use a SHA only when it + satisfies the workflow trust gate. Run local OpenAI/Anthropic suites or + repo-backed character evals only when the operator asks for extra model + coverage or a failure needs local debugging. - Post-published beta verification roster: - `node --import tsx scripts/openclaw-npm-postpublish-verify.ts ` - install/update smoke against the published beta channel @@ -231,13 +243,17 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts - Parallels published beta install/update coverage with both OpenAI and Anthropic provider keys available - targeted QA reruns only for areas touched by fixes after the full pre-npm - roster, unless the operator requests the full QA roster again + roster, unless the operator requests the full QA roster again. If the fix + touches live channel QA, credential plumbing, Matrix, Telegram, or the QA + harness, rerun Actions > `QA-Lab - All Lanes`. - Check all release-related build surfaces touched by the release, not only the npm package. - For beta-style full e2e batteries, hard-cap top-level long lanes instead of letting them run indefinitely. Use host `timeout --foreground`/`gtimeout --foreground` caps such as: - `45m` for `OPENCLAW_INSTALL_SMOKE_SKIP_NONROOT=1 pnpm test:install:smoke` - `90m` for `pnpm test:docker:all` - `60m` each for standalone Docker live lanes - - `180m` for the full QA live OpenAI + Anthropic roster + - `180m` for local full QA live OpenAI + Anthropic rosters when explicitly + requested; the default release channel QA gate is Actions > + `QA-Lab - All Lanes` - Parallels caps from the `openclaw-parallels-smoke` skill If a lane hits its cap, stop and inspect/fix the affected lane before continuing; do not continue to wait on the same process. - Actual npm install/update phases are capped at 5 minutes. If `npm install -g`, installer package install, or `openclaw update` takes longer than 300s in release e2e, stop treating the run as healthy progress and debug the installer/updater or harness. @@ -383,73 +399,76 @@ node --import tsx scripts/openclaw-npm-postpublish-verify.ts 6. Create `release/YYYY.M.D` from that post-changelog `main` commit. 7. Make every repo version location match the beta tag before creating it. 8. Commit release preparation changes on the release branch and push the branch. -9. Run the full pre-npm beta test roster from the release branch before any npm - preflight or publish. +9. Run the local build, Docker, and Parallels parts of the full pre-npm beta + test roster from the release branch before any npm preflight or publish. 10. For beta releases, skip mac app build/sign/notarize unless beta scope or a release blocker specifically requires it. For stable releases, include the mac app, signing, notarization, and appcast path. 11. Confirm the target npm version is not already published. 12. Create and push the git tag from the release branch. 13. Create or refresh the matching GitHub release. -14. Start `.github/workflows/openclaw-npm-release.yml` from the release branch +14. Dispatch Actions > `QA-Lab - All Lanes` against the release tag and wait + for the mock parity, live Matrix, and live Telegram credentialed-channel + lanes to pass. +15. Start `.github/workflows/openclaw-npm-release.yml` from the release branch with `preflight_only=true` and choose the intended `npm_dist_tag` (`beta` default; `latest` only for an intentional direct stable publish). Wait for it to pass. Save that run id because the real publish requires it to reuse the prepared npm tarball. -15. For stable releases, start `.github/workflows/macos-release.yml` in +16. For stable releases, start `.github/workflows/macos-release.yml` in `openclaw/openclaw` and wait for the public validation-only run to pass. -16. For stable releases, start +17. For stable releases, start `openclaw/releases-private/.github/workflows/openclaw-macos-validate.yml` with the same tag and wait for the private mac validation lane to pass. -17. For stable releases, start +18. For stable releases, start `openclaw/releases-private/.github/workflows/openclaw-macos-publish.yml` with `preflight_only=true` and wait for it to pass. Save that run id because the real publish requires it to reuse the notarized mac artifacts. -18. If any preflight or validation run fails, fix the issue on a new commit, +19. If any preflight or validation run fails, fix the issue on a new commit, delete the tag and matching GitHub release, recreate them from the fixed commit, and rerun all relevant preflights from scratch before continuing. Never reuse old preflight results after the commit changes. For pushed or published beta tags, do not delete/recreate; increment to the next beta tag. -19. Start `.github/workflows/openclaw-npm-release.yml` from the same branch with +20. Start `.github/workflows/openclaw-npm-release.yml` from the same branch with the same tag for the real publish, choose `npm_dist_tag` (`beta` default, `latest` only when you intentionally want direct stable publish), keep it the same as the preflight run, and pass the successful npm `preflight_run_id`. -20. Wait for `npm-release` approval from `@openclaw/openclaw-release-managers`. -21. Run postpublish verification: +21. Wait for `npm-release` approval from `@openclaw/openclaw-release-managers`. +22. Run postpublish verification: `node --import tsx scripts/openclaw-npm-postpublish-verify.ts `. -22. Run the post-published beta verification roster. If any lane fails after +23. Run the post-published beta verification roster. If any lane fails after the beta tag/package is pushed or published, fix, commit/push/pull, increment to the next beta tag, and restart at the full pre-npm beta test roster for the new beta. If a pre-npm lane fails before any tag/package leaves the machine, fix and rerun the same intended beta attempt. Repeat up to the operator's authorized beta-attempt limit, normally 4. -23. Announce the beta/stable release on Discord best-effort using Peter's bot +24. Announce the beta/stable release on Discord best-effort using Peter's bot token from `.profile`. -24. If the operator requested beta only, stop after beta verification and the +25. If the operator requested beta only, stop after beta verification and the announcement. -25. If the stable release was published to `beta`, start the private +26. If the stable release was published to `beta`, start the private `openclaw/releases-private/.github/workflows/openclaw-npm-dist-tags.yml` workflow after beta validation passes to promote that stable version from `beta` to `latest`, then verify `latest` now points at that version. -26. If the stable release was published directly to `latest` and `beta` should +27. If the stable release was published directly to `latest` and `beta` should follow it, start that same private dist-tag workflow to point `beta` at the stable version, then verify both `latest` and `beta` point at that version. -27. For stable releases, start +28. For stable releases, start `openclaw/releases-private/.github/workflows/openclaw-macos-publish.yml` for the real publish with the successful private mac `preflight_run_id` and wait for success. -28. Verify the successful real private mac run uploaded the `.zip`, `.dmg`, +29. Verify the successful real private mac run uploaded the `.zip`, `.dmg`, and `.dSYM.zip` artifacts to the existing GitHub release in `openclaw/openclaw`. -29. For stable releases, download `macos-appcast-` from the successful +30. For stable releases, download `macos-appcast-` from the successful private mac run, update `appcast.xml` on `main`, and verify the feed. Merge or cherry-pick release branch changes back to `main` after stable succeeds. -30. For beta releases, publish the mac assets only when intentionally requested; +31. For beta releases, publish the mac assets only when intentionally requested; expect no shared production `appcast.xml` artifact and do not update the shared production feed unless a separate beta feed exists. -31. After publish, verify npm and the attached release artifacts. +32. After publish, verify npm and the attached release artifacts. ## GHSA advisory work