mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 11:10:45 +00:00
feat(qa): add Mantis Discord status reaction scenario (#76747)
* feat(qa): add Mantis Discord status reaction scenario * fix(qa): retry Discord rate limits in Mantis runs * refactor(qa): reuse Discord API retry helper * fix(qa): import Discord API through package surface * fix(ci): generate Discord boundary declarations * fix(ci): keep xai boundary overrides stable
This commit is contained in:
committed by
GitHub
parent
1e8de7661e
commit
77a50db9ea
@@ -84,14 +84,16 @@ pnpm openclaw qa mantis run \
|
||||
```
|
||||
|
||||
The GitHub smoke workflow is `Mantis Discord Smoke`. The before and after GitHub
|
||||
workflow should accept equivalent inputs:
|
||||
workflow for the first real scenario is `Mantis Discord Status Reactions`. It
|
||||
accepts:
|
||||
|
||||
- `transport`: `discord` for the first version.
|
||||
- `scenario`: one or more scenario ids.
|
||||
- `baseline_ref`: default `origin/main` or the linked issue's reported bad tag.
|
||||
- `candidate_ref`: the PR head SHA.
|
||||
- `machine_provider`: `aws` by default, with later `hetzner` fallback.
|
||||
- `post_to_pr`: whether ClawSweeper should comment with the result.
|
||||
- `baseline_ref`: the ref expected to reproduce queued-only behavior.
|
||||
- `candidate_ref`: the ref expected to show `queued -> thinking -> done`.
|
||||
|
||||
It checks out the workflow harness ref, builds separate baseline and candidate
|
||||
worktrees, runs `discord-status-reactions-tool-only` against each worktree, and
|
||||
uploads `baseline/`, `candidate/`, `comparison.json`, and `mantis-report.md` as
|
||||
Actions artifacts.
|
||||
|
||||
ClawSweeper command examples:
|
||||
|
||||
@@ -179,6 +181,25 @@ lifecycle transition in tool-only mode. Candidate evidence should show lifecycle
|
||||
status reactions running when `messages.statusReactions.enabled` is explicitly
|
||||
true.
|
||||
|
||||
The executable first slice is the opt-in Discord live QA scenario:
|
||||
|
||||
```bash
|
||||
pnpm openclaw qa discord \
|
||||
--scenario discord-status-reactions-tool-only \
|
||||
--provider-mode live-frontier \
|
||||
--model openai/gpt-5.4 \
|
||||
--alt-model openai/gpt-5.4 \
|
||||
--fast \
|
||||
--output-dir .artifacts/qa-e2e/mantis/discord-status-reactions-candidate
|
||||
```
|
||||
|
||||
It configures the SUT with always-on guild handling, `visibleReplies:
|
||||
"message_tool"`, `ackReaction: "👀"`, and explicit status reactions. The oracle
|
||||
polls the real Discord triggering message and expects the observed sequence
|
||||
`👀 -> 🤔 -> 👍`. Artifacts include `discord-qa-reaction-timelines.json`,
|
||||
`discord-status-reactions-tool-only-timeline.html`, and
|
||||
`discord-status-reactions-tool-only-timeline.png`.
|
||||
|
||||
## Existing QA Pieces
|
||||
|
||||
Mantis should build on the existing private QA stack instead of starting from
|
||||
|
||||
@@ -225,7 +225,7 @@ Output artifacts:
|
||||
pnpm openclaw qa discord
|
||||
```
|
||||
|
||||
Targets one real private Discord guild channel with two bots: a driver bot controlled by the harness and a SUT bot started by the child OpenClaw gateway through the bundled Discord plugin. Verifies channel mention handling and that the SUT bot has registered the native `/help` command with Discord.
|
||||
Targets one real private Discord guild channel with two bots: a driver bot controlled by the harness and a SUT bot started by the child OpenClaw gateway through the bundled Discord plugin. Verifies channel mention handling, that the SUT bot has registered the native `/help` command with Discord, and opt-in Mantis evidence scenarios.
|
||||
|
||||
Required env when `--credential-source env`:
|
||||
|
||||
@@ -244,12 +244,25 @@ Scenarios (`extensions/qa-lab/src/live-transports/discord/discord-live.runtime.t
|
||||
- `discord-canary`
|
||||
- `discord-mention-gating`
|
||||
- `discord-native-help-command-registration`
|
||||
- `discord-status-reactions-tool-only` — opt-in Mantis scenario. Runs by itself because it switches the SUT to always-on, tool-only guild replies with `messages.statusReactions.enabled=true`, then captures a REST reaction timeline plus an HTML/PNG visual artifact.
|
||||
|
||||
Run the Mantis status-reaction scenario explicitly:
|
||||
|
||||
```bash
|
||||
pnpm openclaw qa discord \
|
||||
--scenario discord-status-reactions-tool-only \
|
||||
--provider-mode live-frontier \
|
||||
--model openai/gpt-5.4 \
|
||||
--alt-model openai/gpt-5.4 \
|
||||
--fast
|
||||
```
|
||||
|
||||
Output artifacts:
|
||||
|
||||
- `discord-qa-report.md`
|
||||
- `discord-qa-summary.json`
|
||||
- `discord-qa-observed-messages.json` — bodies redacted unless `OPENCLAW_QA_DISCORD_CAPTURE_CONTENT=1`.
|
||||
- `discord-qa-reaction-timelines.json` and `discord-status-reactions-tool-only-timeline.png` when the status-reaction scenario runs.
|
||||
|
||||
### Convex credential pool
|
||||
|
||||
|
||||
Reference in New Issue
Block a user