mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 15:50:46 +00:00
QA: organize scenarios by theme
This commit is contained in:
@@ -4,7 +4,8 @@ Seed QA assets for the private `qa-lab` extension.
|
||||
|
||||
Files:
|
||||
|
||||
- `scenarios.md` - canonical QA scenario pack, kickoff mission, and operator identity.
|
||||
- `scenarios/index.md` - canonical QA scenario pack, kickoff mission, and operator identity.
|
||||
- `scenarios/<theme>/*.md` - one runnable scenario per markdown file.
|
||||
- `frontier-harness-plan.md` - big-model bakeoff and tuning loop for harness work.
|
||||
- `convex-credential-broker/` - standalone Convex v1 lease broker for pooled live credentials.
|
||||
|
||||
|
||||
@@ -3,6 +3,6 @@
|
||||
Canonical scenario source now lives in:
|
||||
|
||||
- `qa/scenarios/index.md`
|
||||
- `qa/scenarios/*.md`
|
||||
- `qa/scenarios/<theme>/*.md`
|
||||
|
||||
Each QA scenario has its own markdown file.
|
||||
|
||||
@@ -23,7 +23,7 @@ execution:
|
||||
prompt: |-
|
||||
Subagent fanout synthesis check: delegate exactly two bounded subagents sequentially.
|
||||
Subagent 1: verify that `HEARTBEAT.md` exists and report `ok` if it does.
|
||||
Subagent 2: verify that `repo/qa/scenarios/subagent-fanout-synthesis.md` exists and report `ok` if it does.
|
||||
Subagent 2: verify that `repo/qa/scenarios/agents/subagent-fanout-synthesis.md` exists and report `ok` if it does.
|
||||
Wait for both subagents to finish.
|
||||
Then reply with exactly these two lines and nothing else:
|
||||
subagent-1: ok
|
||||
@@ -4,12 +4,28 @@ Single source of truth for repo-backed QA suite bootstrap data.
|
||||
`qa-lab` should treat this directory as a generic markdown scenario pack:
|
||||
|
||||
- `index.md` defines pack-level bootstrap data
|
||||
- each `*.md` scenario defines one runnable test via `qa-scenario` + `qa-flow`
|
||||
- scenario markdown may also define required plugins and gateway config patching
|
||||
- each nested `*.md` scenario defines one runnable test via `qa-scenario` + `qa-flow`
|
||||
- scenario markdown may also define category metadata, required plugins, lane filters,
|
||||
and gateway config patching
|
||||
|
||||
- kickoff mission
|
||||
- QA operator identity
|
||||
- scenario files under `./`
|
||||
- scenario files under one-level theme directories
|
||||
|
||||
Theme directories:
|
||||
|
||||
- `agents/` - agent behavior, instructions, and subagent flows
|
||||
- `channels/` - DM, shared channel, thread, and message-action behavior
|
||||
- `character/` - persona and style eval scenarios
|
||||
- `config/` - config patch, apply, and restart behavior
|
||||
- `media/` - image understanding and generation
|
||||
- `memory/` - recall, ranking, active memory, and thread isolation
|
||||
- `models/` - provider capabilities and model switching
|
||||
- `plugins/` - plugin, skill, and MCP tool integration
|
||||
- `runtime/` - turn recovery, compaction, approval, and inventory behavior
|
||||
- `scheduling/` - cron and recurring work
|
||||
- `ui/` - Control UI plus qa-channel flows
|
||||
- `workspace/` - repo-reading and workspace artifact tasks
|
||||
|
||||
```yaml qa-pack
|
||||
version: 1
|
||||
|
||||
Reference in New Issue
Block a user