docs(qa): reorg, audit against code, and refresh stale content

Reorg
- Rename the architecture page title to "QA overview" (slug stays
  /concepts/qa-e2e-automation so inbound links keep working).
- Move "Adding a channel to QA" + scenario-helper-name reference from
  testing.md into qa-e2e-automation.md under "Transport adapters". Architecture
  belongs with the architecture page.
- Drop the duplicate live-transport coverage table from testing.md; canonical
  copy stays in qa-e2e-automation.md under a new "Live transport coverage"
  heading so qa-matrix.md can deep-link to it.
- Slim testing.md QA-specific runners section to ops only, with cross-links.

Audit (against extensions/qa-lab/src/cli.ts, qa-channel/src/config-schema.ts,
and live-transport runtimes)
- qa-e2e-automation.md gains a "Command surface" table covering all 14
  openclaw qa <subcommand> forms; previously only ~7 of 14 were named.
- Document missing OPENCLAW_QA_TELEGRAM_CAPTURE_CONTENT and
  OPENCLAW_QA_DISCORD_CAPTURE_CONTENT env vars (Matrix already had it).
- Cross-link qa coverage from the Reporting section.
- qa-channel.md completes the config-key list (enabled, name, accounts,
  defaultAccount were missing from the schema doc) and pollTimeoutMs range.
- Drop stale "Follow-up work" framing in qa-channel.md (provider/model matrix,
  scenario discovery, orchestration) — all three already shipped.
- Replace "vertical slice" language with current behavior; fix misplaced
  debugger-UI paragraph.

Discoverability
- Add a Note callout to testing.md pointing at the three QA pages
  (QA overview, Matrix QA, QA channel) so maintainers landing on testing.md
  see the QA stack in the prologue.

Glossary entries for the renamed/new doc titles.
This commit is contained in:
Gustavo Madeira Santana
2026-04-27 13:39:46 -04:00
parent abca187df5
commit dd1a94f089
4 changed files with 165 additions and 187 deletions

View File

@@ -7,27 +7,16 @@ read_when:
- You are iterating on end-to-end QA automation
---
`qa-channel` is a bundled synthetic message transport for automated OpenClaw QA.
`qa-channel` is a bundled synthetic message transport for automated OpenClaw QA. It is not a production channel — it exists to exercise the same channel plugin boundary used by real transports while keeping state deterministic and fully inspectable.
It is not a production channel. It exists to exercise the same channel plugin
boundary used by real transports while keeping state deterministic and fully
inspectable.
## What it does today
## What it does
- Slack-class target grammar:
- `dm:<user>`
- `channel:<room>`
- `thread:<room>/<thread>`
- HTTP-backed synthetic bus for:
- inbound message injection
- outbound transcript capture
- thread creation
- reactions
- edits
- deletes
- search and read actions
- Bundled host-side self-check runner that writes a Markdown report
- HTTP-backed synthetic bus for inbound message injection, outbound transcript capture, thread creation, reactions, edits, deletes, and search/read actions.
- Host-side self-check runner that writes a Markdown report to `.artifacts/qa-e2e/`.
## Config
@@ -45,68 +34,53 @@ inspectable.
}
```
Supported account keys:
Account keys:
- `baseUrl`
- `botUserId`
- `botDisplayName`
- `pollTimeoutMs`
- `allowFrom`
- `defaultTo`
- `actions.messages`
- `actions.reactions`
- `actions.search`
- `actions.threads`
- `enabled` — master toggle for this account.
- `name` — optional display label.
- `baseUrl` — synthetic bus URL.
- `botUserId` — Matrix-style bot user id used in target grammar.
- `botDisplayName` — display name for outbound messages.
- `pollTimeoutMs` — long-poll wait window. Integer between 100 and 30000.
- `allowFrom` — sender allowlist (user ids or `"*"`).
- `defaultTo` — fallback target when none is supplied.
- `actions.messages` / `actions.reactions` / `actions.search` / `actions.threads` — per-action tool gating.
## Runner
Multi-account keys at the top level:
Current vertical slice:
- `accounts` — record of named per-account overrides keyed by account id.
- `defaultAccount` — preferred account id when multiple are configured.
## Runners
Host-side self-check (writes a Markdown report under `.artifacts/qa-e2e/`):
```bash
pnpm qa:e2e
```
This now routes through the bundled `qa-lab` extension. It starts the in-repo
QA bus, boots the bundled `qa-channel` runtime slice, runs a deterministic
self-check, and writes a Markdown report under `.artifacts/qa-e2e/`.
This routes through `qa-lab`, starts the in-repo QA bus, boots the bundled `qa-channel` runtime slice, and runs a deterministic self-check.
Private debugger UI:
```bash
pnpm qa:lab:up
```
That one command builds the QA site, starts the Docker-backed gateway + QA Lab
stack, and prints the QA Lab URL. From that site you can pick scenarios, choose
the model lane, launch individual runs, and watch results live.
Full repo-backed QA suite:
Full repo-backed scenario suite:
```bash
pnpm openclaw qa suite
```
That launches the private QA debugger at a local URL, separate from the
shipped Control UI bundle.
Runs scenarios in parallel against the QA gateway lane. See [QA overview](/concepts/qa-e2e-automation) for scenarios, profiles, and provider modes.
## Scope
Docker-backed QA site (gateway + QA Lab debugger UI in one stack):
Current scope is intentionally narrow:
```bash
pnpm qa:lab:up
```
- bus + plugin transport
- threaded routing grammar
- channel-owned message actions
- Markdown reporting
- Docker-backed QA site with run controls
Follow-up work will add:
- provider/model matrix execution
- richer scenario discovery
- OpenClaw-native orchestration later
Builds the QA site, starts the Docker-backed gateway + QA Lab stack, and prints the QA Lab URL. From there you can pick scenarios, choose the model lane, launch individual runs, and watch results live. The QA Lab debugger is separate from the shipped Control UI bundle.
## Related
- [QA overview](/concepts/qa-e2e-automation) — overall stack, transport adapters, scenario authoring
- [Matrix QA](/concepts/qa-matrix) — example live-transport runner that drives a real channel
- [Pairing](/channels/pairing)
- [Groups](/channels/groups)
- [Channels overview](/channels)