mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 14:20:44 +00:00
QA Lab: add provider registry
Move mock and live provider behavior behind provider-owned definitions so suite, manual, Matrix, and transport lanes share defaults, auth staging, model config, and standalone server startup. Add AIMock as a first-class local provider mode while keeping mock-openai as the scenario-aware deterministic lane.
This commit is contained in:
@@ -151,6 +151,22 @@ The baseline list should stay broad enough to cover:
|
||||
- repo-reading and docs-reading
|
||||
- one small build task such as Lobster Invaders
|
||||
|
||||
## Provider mock lanes
|
||||
|
||||
`qa suite` has two local provider mock lanes:
|
||||
|
||||
- `mock-openai` is the scenario-aware OpenClaw mock. It remains the default
|
||||
deterministic mock lane for repo-backed QA and parity gates.
|
||||
- `aimock` starts an AIMock-backed provider server for experimental protocol,
|
||||
fixture, record/replay, and chaos coverage. It is additive and does not
|
||||
replace the `mock-openai` scenario dispatcher.
|
||||
|
||||
Provider-lane implementation lives under `extensions/qa-lab/src/providers/`.
|
||||
Each provider owns its defaults, local server startup, gateway model config,
|
||||
auth-profile staging needs, and live/mock capability flags. Shared suite and
|
||||
gateway code should route through the provider registry instead of branching on
|
||||
provider names.
|
||||
|
||||
## Transport adapters
|
||||
|
||||
`qa-lab` owns a generic transport seam for markdown QA scenarios.
|
||||
|
||||
@@ -52,6 +52,10 @@ These commands sit beside the main test suites when you need QA-lab realism:
|
||||
gateway workers, up to 64 workers or the selected scenario count. Use
|
||||
`--concurrency <count>` to tune the worker count, or `--concurrency 1` for
|
||||
the older serial lane.
|
||||
- Supports provider modes `live-frontier`, `mock-openai`, and `aimock`.
|
||||
`aimock` starts a local AIMock-backed provider server for experimental
|
||||
fixture and protocol-mock coverage without replacing the scenario-aware
|
||||
`mock-openai` lane.
|
||||
- `pnpm openclaw qa suite --runner multipass`
|
||||
- Runs the same QA suite inside a disposable Multipass Linux VM.
|
||||
- Keeps the same scenario-selection behavior as `qa suite` on the host.
|
||||
@@ -65,6 +69,9 @@ These commands sit beside the main test suites when you need QA-lab realism:
|
||||
`.artifacts/qa-e2e/...`.
|
||||
- `pnpm qa:lab:up`
|
||||
- Starts the Docker-backed QA site for operator-style QA work.
|
||||
- `pnpm openclaw qa aimock`
|
||||
- Starts only the local AIMock provider server for direct protocol smoke
|
||||
testing.
|
||||
- `pnpm openclaw qa matrix`
|
||||
- Runs the Matrix live QA lane against a disposable Docker-backed Tuwunel homeserver.
|
||||
- This QA host is repo/dev-only today. Packaged OpenClaw installs do not ship
|
||||
|
||||
Reference in New Issue
Block a user