mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 09:50:42 +00:00
docs: explain release validation entrypoints
This commit is contained in:
Peter Steinberger
108
docs/ci.md
108
docs/ci.md
@@ -26,6 +26,114 @@ workflow checkout. Profiles cover smoke, package, product, full, and custom
|
||||
Docker lane selections. The optional Telegram lane is published-npm only and
|
||||
reuses the `NPM Telegram Beta E2E` workflow.
|
||||
|
||||
## Package Acceptance
|
||||
|
||||
Use `Package Acceptance` when the question is "does this installable OpenClaw
|
||||
package work as a product?" It is different from normal CI: normal CI validates
|
||||
the source tree, while package acceptance validates a single tarball through the
|
||||
same Docker E2E harness users exercise after install or update.
|
||||
|
||||
The workflow has four jobs:
|
||||
|
||||
1. `resolve_package` checks out `workflow_ref`, resolves one package candidate,
|
||||
writes `.artifacts/docker-e2e-package/openclaw-current.tgz`, writes
|
||||
`.artifacts/docker-e2e-package/package-candidate.json`, uploads both as the
|
||||
`package-under-test` artifact, and prints the source, workflow ref, package
|
||||
ref, version, SHA-256, and profile in the GitHub step summary.
|
||||
2. `docker_acceptance` calls
|
||||
`openclaw-live-and-e2e-checks-reusable.yml` with `ref=workflow_ref` and
|
||||
`package_artifact_name=package-under-test`. The reusable workflow downloads
|
||||
that artifact, validates the tarball inventory, prepares package-digest
|
||||
Docker images when needed, and runs the selected Docker lanes against that
|
||||
package instead of packing the workflow checkout.
|
||||
3. `npm_telegram` optionally calls `NPM Telegram Beta E2E`. It runs only when
|
||||
`telegram_mode` is not `none`, and only for `source=npm`, because that lane
|
||||
installs a published package spec.
|
||||
4. `summary` fails the workflow if package resolution, Docker acceptance, or
|
||||
the optional Telegram lane failed.
|
||||
|
||||
Candidate sources:
|
||||
|
||||
- `source=npm`: accepts only `openclaw@beta`, `openclaw@latest`, or an exact
|
||||
OpenClaw release version such as `openclaw@2026.4.27-beta.2`. Use this for
|
||||
published beta/stable acceptance.
|
||||
- `source=ref`: packs a trusted `package_ref` branch, tag, or full commit SHA.
|
||||
The resolver fetches OpenClaw branches/tags, verifies the selected commit is
|
||||
reachable from repository branch history or a release tag, installs deps in a
|
||||
detached worktree, and packs it with `scripts/package-openclaw-for-docker.mjs`.
|
||||
- `source=url`: downloads an HTTPS `.tgz`; `package_sha256` is required.
|
||||
- `source=artifact`: downloads one `.tgz` from `artifact_run_id` and
|
||||
`artifact_name`; `package_sha256` is optional but should be supplied for
|
||||
externally shared artifacts.
|
||||
|
||||
Keep `workflow_ref` and `package_ref` separate. `workflow_ref` is the trusted
|
||||
workflow/harness code that runs the test. `package_ref` is the source commit
|
||||
that gets packed when `source=ref`. This lets the current test harness validate
|
||||
older trusted source commits without running old workflow logic.
|
||||
|
||||
Profiles map to Docker coverage:
|
||||
|
||||
- `smoke`: `npm-onboard-channel-agent`, `gateway-network`, `config-reload`
|
||||
- `package`: `install-e2e`, `npm-onboard-channel-agent`, `doctor-switch`,
|
||||
`update-channel-switch`, `bundled-channel-deps`, `plugins`, `plugin-update`
|
||||
- `product`: `package` plus `mcp-channels`, `cron-mcp-cleanup`,
|
||||
`openai-web-search-minimal`, `openwebui`
|
||||
- `full`: full Docker release-path chunks with OpenWebUI
|
||||
- `custom`: exact `docker_lanes`; required when `suite_profile=custom`
|
||||
|
||||
Release checks call Package Acceptance with `source=ref`,
|
||||
`package_ref=<release-ref>`, `workflow_ref=<release workflow ref>`, and
|
||||
`suite_profile=package`. That profile is the GitHub-native replacement for most
|
||||
Parallels package/update validation. Cross-OS release checks still cover
|
||||
OS-specific onboarding, installer, and platform behavior; package/update
|
||||
product validation should start with Package Acceptance.
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# Validate the current beta package with product-level coverage.
|
||||
gh workflow run package-acceptance.yml \
|
||||
--ref main \
|
||||
-f workflow_ref=main \
|
||||
-f source=npm \
|
||||
-f package_spec=openclaw@beta \
|
||||
-f suite_profile=product
|
||||
|
||||
# Pack and validate a release branch with the current harness.
|
||||
gh workflow run package-acceptance.yml \
|
||||
--ref main \
|
||||
-f workflow_ref=main \
|
||||
-f source=ref \
|
||||
-f package_ref=release/YYYY.M.D \
|
||||
-f suite_profile=package
|
||||
|
||||
# Validate a tarball URL. SHA-256 is mandatory for source=url.
|
||||
gh workflow run package-acceptance.yml \
|
||||
--ref main \
|
||||
-f workflow_ref=main \
|
||||
-f source=url \
|
||||
-f package_url=https://example.com/openclaw-current.tgz \
|
||||
-f package_sha256=<64-char-sha256> \
|
||||
-f suite_profile=smoke
|
||||
|
||||
# Reuse a tarball uploaded by another Actions run.
|
||||
gh workflow run package-acceptance.yml \
|
||||
--ref main \
|
||||
-f workflow_ref=main \
|
||||
-f source=artifact \
|
||||
-f artifact_run_id=<run-id> \
|
||||
-f artifact_name=package-under-test \
|
||||
-f suite_profile=custom \
|
||||
-f docker_lanes='install-e2e plugin-update'
|
||||
```
|
||||
|
||||
When debugging a failed package acceptance run, start at the `resolve_package`
|
||||
summary to confirm the package source, version, and SHA-256. Then inspect the
|
||||
`docker_acceptance` child run and its Docker artifacts:
|
||||
`.artifacts/docker-tests/**/summary.json`, `failures.json`, lane logs, phase
|
||||
timings, and rerun commands. Prefer rerunning the failed package profile or
|
||||
exact Docker lanes instead of rerunning full release validation.
|
||||
|
||||
QA Lab has dedicated CI lanes outside the main smart-scoped workflow. The
|
||||
`Parity gate` workflow runs on matching PR changes and manual dispatch; it
|
||||
builds the private QA runtime and compares the mock GPT-5.5 and Opus 4.6
|
||||
|
||||
@@ -66,9 +66,9 @@ the maintainer-only release runbook.
|
||||
6. Run `OpenClaw NPM Release` with `preflight_only=true`. Before a tag exists,
|
||||
a full 40-character release-branch SHA is allowed for validation-only
|
||||
preflight. Save the successful `preflight_run_id`.
|
||||
7. Run `Full Release Validation` for the release branch, tag, or full commit
|
||||
SHA. This is the umbrella run for the four big release test boxes: Vitest,
|
||||
Docker, QA Lab, and Package.
|
||||
7. Kick off all pre-release tests with `Full Release Validation` for the
|
||||
release branch, tag, or full commit SHA. This is the one manual entrypoint
|
||||
for the four big release test boxes: Vitest, Docker, QA Lab, and Package.
|
||||
8. If validation fails, fix on the release branch and rerun the smallest failed
|
||||
file, lane, workflow job, package profile, provider, or model allowlist that
|
||||
proves the fix. Rerun the full umbrella only when the changed surface makes
|
||||
@@ -96,15 +96,14 @@ the maintainer-only release runbook.
|
||||
- Run `pnpm build && pnpm ui:build` before `pnpm release:check` so the expected
|
||||
`dist/*` release artifacts and Control UI bundle exist for the pack
|
||||
validation step
|
||||
- Run the manual `Full Release Validation` workflow before release approval
|
||||
when you need the whole release validation suite from one entrypoint. It
|
||||
accepts a branch, tag, or full commit SHA, dispatches manual `CI`, and
|
||||
dispatches `OpenClaw Release Checks` for install smoke, package acceptance,
|
||||
Docker release-path suites, live/E2E, OpenWebUI, QA Lab parity, Matrix, and
|
||||
Telegram lanes.
|
||||
Provide `npm_telegram_package_spec` only after a package has been published
|
||||
and the post-publish Telegram E2E should run too.
|
||||
Example: `gh workflow run full-release-validation.yml --ref main -f ref=release/YYYY.M.D`
|
||||
- Run the manual `Full Release Validation` workflow before release approval to
|
||||
kick off all pre-release test boxes from one entrypoint. It accepts a branch,
|
||||
tag, or full commit SHA, dispatches manual `CI`, and dispatches
|
||||
`OpenClaw Release Checks` for install smoke, package acceptance, Docker
|
||||
release-path suites, live/E2E, OpenWebUI, QA Lab parity, Matrix, and Telegram
|
||||
lanes. Provide `npm_telegram_package_spec` only after a package has been
|
||||
published and the post-publish Telegram E2E should run too. Example:
|
||||
`gh workflow run full-release-validation.yml --ref main -f ref=release/YYYY.M.D`
|
||||
- Run the manual `Package Acceptance` workflow when you want side-channel proof
|
||||
for a package candidate while release work continues. Use `source=npm` for
|
||||
`openclaw@beta`, `openclaw@latest`, or an exact release version; `source=ref`
|
||||
@@ -221,8 +220,9 @@ Validation` or from the `main`/release workflow ref so workflow logic and
|
||||
|
||||
## Release test boxes
|
||||
|
||||
`Full Release Validation` is the manual umbrella that operators use when they
|
||||
want all release validation from one entrypoint:
|
||||
`Full Release Validation` is how operators kick off all pre-release tests from
|
||||
one entrypoint. Run it from the trusted `main` workflow ref and pass the release
|
||||
branch, tag, or full commit SHA as `ref`:
|
||||
|
||||
```bash
|
||||
gh workflow run full-release-validation.yml \
|
||||
@@ -236,9 +236,48 @@ gh workflow run full-release-validation.yml \
|
||||
The workflow resolves the target ref, dispatches manual `CI` with
|
||||
`target_ref=<release-ref>`, dispatches `OpenClaw Release Checks`, and
|
||||
optionally dispatches post-publish Telegram E2E when
|
||||
`npm_telegram_package_spec` is set. A full run is only acceptable when both
|
||||
child workflows succeed or an intentionally skipped optional child is recorded
|
||||
in the summary.
|
||||
`npm_telegram_package_spec` is set. `OpenClaw Release Checks` then fans out
|
||||
install smoke, cross-OS release checks, live/E2E Docker release-path coverage,
|
||||
Package Acceptance, QA Lab parity, live Matrix, and live Telegram. A full run is
|
||||
only acceptable when the `Full Release Validation` summary shows `normal_ci` and
|
||||
`release_checks` as successful, and any optional `npm_telegram` child is either
|
||||
successful or intentionally skipped.
|
||||
|
||||
Use these variants depending on release stage:
|
||||
|
||||
```bash
|
||||
# Validate an unpublished release candidate branch.
|
||||
gh workflow run full-release-validation.yml \
|
||||
--ref main \
|
||||
-f ref=release/YYYY.M.D \
|
||||
-f workflow_ref=main \
|
||||
-f provider=openai \
|
||||
-f mode=both
|
||||
|
||||
# Validate an exact pushed commit.
|
||||
gh workflow run full-release-validation.yml \
|
||||
--ref main \
|
||||
-f ref=<40-char-sha> \
|
||||
-f workflow_ref=main \
|
||||
-f provider=openai \
|
||||
-f mode=both
|
||||
|
||||
# After publishing a beta, add published-package Telegram E2E.
|
||||
gh workflow run full-release-validation.yml \
|
||||
--ref main \
|
||||
-f ref=release/YYYY.M.D \
|
||||
-f workflow_ref=main \
|
||||
-f provider=openai \
|
||||
-f mode=both \
|
||||
-f npm_telegram_package_spec=openclaw@YYYY.M.D-beta.N \
|
||||
-f npm_telegram_provider_mode=mock-openai
|
||||
```
|
||||
|
||||
Do not use the full umbrella as the first rerun after a focused fix. If one box
|
||||
fails, use the failed child workflow, job, Docker lane, package profile, model
|
||||
provider, or QA lane for the next proof. Run the full umbrella again only when
|
||||
the fix changed shared release orchestration or made earlier all-box evidence
|
||||
stale.
|
||||
|
||||
### Vitest
|
||||
|
||||
|
||||
Reference in New Issue
Block a user