mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-01 04:00:22 +00:00
qa-lab: standardize live transport scenario coverage
This commit is contained in:
@@ -64,6 +64,36 @@ the real Matrix plugin inside a QA gateway child. The live transport lane keeps
|
||||
the child config scoped to the transport under test, so Matrix runs without
|
||||
`qa-channel` in the child config.
|
||||
|
||||
Live transport lanes now share one smaller contract instead of each inventing
|
||||
their own scenario list shape:
|
||||
|
||||
- Baseline contract:
|
||||
- canary
|
||||
- mention gating
|
||||
- sender allowlist block
|
||||
- top-level reply shape
|
||||
- restart resume
|
||||
- Capability add-ons:
|
||||
- thread follow-up
|
||||
- thread isolation
|
||||
- reaction observation
|
||||
- help command
|
||||
|
||||
Current coverage:
|
||||
|
||||
- Matrix:
|
||||
- baseline contract
|
||||
- thread follow-up
|
||||
- thread isolation
|
||||
- reaction observation
|
||||
- Telegram:
|
||||
- canary
|
||||
- help command
|
||||
|
||||
This keeps `qa-channel` as the broad product-behavior suite while Matrix,
|
||||
Telegram, and future live transports share one explicit transport-contract
|
||||
checklist.
|
||||
|
||||
For a disposable Linux VM lane without bringing Docker into the QA path, run:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -70,6 +70,34 @@ These commands sit beside the main test suites when you need QA-lab realism:
|
||||
- Provisions three temporary Matrix users (`driver`, `sut`, `observer`) plus one private room, then starts a QA gateway child with the real Matrix plugin as the SUT transport.
|
||||
- Uses the pinned stable Tuwunel image `ghcr.io/matrix-construct/tuwunel:v1.5.1` by default. Override with `OPENCLAW_QA_MATRIX_TUWUNEL_IMAGE` when you need to test a different image.
|
||||
- Writes a Matrix QA report, summary, and observed-events artifact under `.artifacts/qa-e2e/...`.
|
||||
- `pnpm openclaw qa telegram`
|
||||
- Runs the Telegram live QA lane against a real private group using the driver and SUT bot tokens from env.
|
||||
- Writes a Telegram QA report, summary, and observed-messages artifact under `.artifacts/qa-e2e/...`.
|
||||
|
||||
Live transport lanes share one standard contract so new transports do not drift:
|
||||
|
||||
- Baseline:
|
||||
- canary
|
||||
- mention gating
|
||||
- sender allowlist block
|
||||
- top-level reply shape
|
||||
- restart resume
|
||||
- Capability add-ons:
|
||||
- thread follow-up
|
||||
- thread isolation
|
||||
- reaction observation
|
||||
- help command
|
||||
|
||||
Current lane coverage:
|
||||
|
||||
- Matrix:
|
||||
- baseline contract
|
||||
- thread follow-up
|
||||
- thread isolation
|
||||
- reaction observation
|
||||
- Telegram:
|
||||
- canary
|
||||
- help command
|
||||
|
||||
## Test suites (what runs where)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user