diff --git a/docs/concepts/qa-e2e-automation.md b/docs/concepts/qa-e2e-automation.md index becd26cd1d6..f36ae7c8ee6 100644 --- a/docs/concepts/qa-e2e-automation.md +++ b/docs/concepts/qa-e2e-automation.md @@ -52,6 +52,18 @@ pnpm qa:lab:watch rebuilds that bundle on change, and the browser auto-reloads when the QA Lab asset hash changes. +For a disposable Linux VM lane without bringing Docker into the QA path, run: + +```bash +pnpm openclaw qa suite --runner multipass --scenario channel-chat-baseline +``` + +This boots a fresh Multipass guest, installs dependencies, builds OpenClaw +inside the guest, runs `qa suite` on the mock-openai lane, then copies the +normal QA report and summary back into `.artifacts/qa-e2e/...` on the host. +It reuses the same scenario-selection behavior as `qa suite` on the host and +only changes where that suite runs. + ## Repo-backed seeds Seed assets live in `qa/`: diff --git a/docs/help/testing.md b/docs/help/testing.md index 511c1a88fdd..f09606f509a 100644 --- a/docs/help/testing.md +++ b/docs/help/testing.md @@ -28,6 +28,7 @@ Most days: - Direct file targeting now routes extension/channel paths too: `pnpm test extensions/discord/src/monitor/message-handler.preflight.test.ts` - Prefer targeted runs first when you are iterating on a single failure. - Docker-backed QA site: `pnpm qa:lab:up` +- Linux VM-backed QA lane: `pnpm openclaw qa suite --runner multipass --scenario channel-chat-baseline` When you touch tests or want extra confidence: @@ -41,6 +42,21 @@ When debugging real providers/models (requires real creds): Tip: when you only need one failing case, prefer narrowing live tests via the allowlist env vars described below. +## QA-specific runners + +These commands sit beside the main test suites when you need QA-lab realism: + +- `pnpm openclaw qa suite` + - Runs repo-backed QA scenarios directly on the host. +- `pnpm openclaw qa suite --runner multipass` + - Runs the same QA suite inside a disposable Multipass Linux VM. + - Keeps the same scenario-selection behavior as `qa suite` on the host. + - Reuses the same provider/model selection flags as `qa suite`; the runner only changes where the suite executes. + - Writes the normal QA report + summary plus Multipass logs under + `.artifacts/qa-e2e/...`. +- `pnpm qa:lab:up` + - Starts the Docker-backed QA site for operator-style QA work. + ## Test suites (what runs where) Think of the suites as “increasing realism” (and increasing flakiness/cost):