mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 05:20:43 +00:00
docs: typography hygiene + 2 in-body H1 removals across 5 pages
Replaced 112 typography characters (curly quotes, apostrophes, em/en dashes, non-breaking hyphens) with ASCII equivalents per docs/CLAUDE.md heading and content hygiene rules. - docs/help/gpt55-codex-agentic-parity.md: 22 chars; removed the duplicate '# GPT-5.5 / Codex Agentic Parity in OpenClaw' H1 (Mintlify renders the title from frontmatter; the in-body H1 with the slash produced a brittle anchor). - docs/platforms/mac/menu-bar.md: 21 chars; removed the duplicate '# Menu Bar Status Logic' H1. - docs/tools/acp-agents.md: 23 chars - docs/concepts/qa-matrix.md: 23 chars - docs/concepts/qa-e2e-automation.md: 23 chars
This commit is contained in:
@@ -240,7 +240,7 @@ can write back through the mounted workspace.
|
||||
|
||||
## Telegram, Discord, and Slack QA reference
|
||||
|
||||
Matrix has a [dedicated page](/concepts/qa-matrix) because of its scenario count and Docker-backed homeserver provisioning. Telegram, Discord, and Slack are smaller — a handful of scenarios each, no profile system, against pre-existing real channels — so their reference lives here.
|
||||
Matrix has a [dedicated page](/concepts/qa-matrix) because of its scenario count and Docker-backed homeserver provisioning. Telegram, Discord, and Slack are smaller - a handful of scenarios each, no profile system, against pre-existing real channels - so their reference lives here.
|
||||
|
||||
### Shared CLI flags
|
||||
|
||||
@@ -248,7 +248,7 @@ These lanes register through `extensions/qa-lab/src/live-transports/shared/live-
|
||||
|
||||
| Flag | Default | Description |
|
||||
| ------------------------------------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
||||
| `--scenario <id>` | — | Run only this scenario. Repeatable. |
|
||||
| `--scenario <id>` | - | Run only this scenario. Repeatable. |
|
||||
| `--output-dir <path>` | `<repo>/.artifacts/qa-e2e/{telegram,discord,slack}-<timestamp>` | Where reports/summary/observed messages and the output log are written. Relative paths resolve against `--repo-root`. |
|
||||
| `--repo-root <path>` | `process.cwd()` | Repository root when invoking from a neutral cwd. |
|
||||
| `--sut-account <id>` | `sut` | Temporary account id inside the QA gateway config. |
|
||||
@@ -270,7 +270,7 @@ Targets one real private Telegram group with two distinct bots (driver + SUT). T
|
||||
|
||||
Required env when `--credential-source env`:
|
||||
|
||||
- `OPENCLAW_QA_TELEGRAM_GROUP_ID` — numeric chat id (string).
|
||||
- `OPENCLAW_QA_TELEGRAM_GROUP_ID` - numeric chat id (string).
|
||||
- `OPENCLAW_QA_TELEGRAM_DRIVER_BOT_TOKEN`
|
||||
- `OPENCLAW_QA_TELEGRAM_SUT_BOT_TOKEN`
|
||||
|
||||
@@ -294,8 +294,8 @@ Scenarios (`extensions/qa-lab/src/live-transports/telegram/telegram-live.runtime
|
||||
Output artifacts:
|
||||
|
||||
- `telegram-qa-report.md`
|
||||
- `telegram-qa-summary.json` — includes per-reply RTT (driver send → observed SUT reply) starting with the canary.
|
||||
- `telegram-qa-observed-messages.json` — bodies redacted unless `OPENCLAW_QA_TELEGRAM_CAPTURE_CONTENT=1`.
|
||||
- `telegram-qa-summary.json` - includes per-reply RTT (driver send → observed SUT reply) starting with the canary.
|
||||
- `telegram-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_TELEGRAM_CAPTURE_CONTENT=1`.
|
||||
|
||||
### Discord QA
|
||||
|
||||
@@ -311,7 +311,7 @@ Required env when `--credential-source env`:
|
||||
- `OPENCLAW_QA_DISCORD_CHANNEL_ID`
|
||||
- `OPENCLAW_QA_DISCORD_DRIVER_BOT_TOKEN`
|
||||
- `OPENCLAW_QA_DISCORD_SUT_BOT_TOKEN`
|
||||
- `OPENCLAW_QA_DISCORD_SUT_APPLICATION_ID` — must match the SUT bot user id returned by Discord (the lane fails fast otherwise).
|
||||
- `OPENCLAW_QA_DISCORD_SUT_APPLICATION_ID` - must match the SUT bot user id returned by Discord (the lane fails fast otherwise).
|
||||
|
||||
Optional:
|
||||
|
||||
@@ -322,7 +322,7 @@ Scenarios (`extensions/qa-lab/src/live-transports/discord/discord-live.runtime.t
|
||||
- `discord-canary`
|
||||
- `discord-mention-gating`
|
||||
- `discord-native-help-command-registration`
|
||||
- `discord-status-reactions-tool-only` — opt-in Mantis scenario. Runs by itself because it switches the SUT to always-on, tool-only guild replies with `messages.statusReactions.enabled=true`, then captures a REST reaction timeline plus HTML/PNG visual artifacts. Mantis before/after reports also preserve scenario-provided MP4 artifacts as `baseline.mp4` and `candidate.mp4`.
|
||||
- `discord-status-reactions-tool-only` - opt-in Mantis scenario. Runs by itself because it switches the SUT to always-on, tool-only guild replies with `messages.statusReactions.enabled=true`, then captures a REST reaction timeline plus HTML/PNG visual artifacts. Mantis before/after reports also preserve scenario-provided MP4 artifacts as `baseline.mp4` and `candidate.mp4`.
|
||||
|
||||
Run the Mantis status-reaction scenario explicitly:
|
||||
|
||||
@@ -339,7 +339,7 @@ Output artifacts:
|
||||
|
||||
- `discord-qa-report.md`
|
||||
- `discord-qa-summary.json`
|
||||
- `discord-qa-observed-messages.json` — bodies redacted unless `OPENCLAW_QA_DISCORD_CAPTURE_CONTENT=1`.
|
||||
- `discord-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_DISCORD_CAPTURE_CONTENT=1`.
|
||||
- `discord-qa-reaction-timelines.json` and `discord-status-reactions-tool-only-timeline.png` when the status-reaction scenario runs.
|
||||
|
||||
### Slack QA
|
||||
@@ -375,16 +375,16 @@ Output artifacts:
|
||||
|
||||
- `slack-qa-report.md`
|
||||
- `slack-qa-summary.json`
|
||||
- `slack-qa-observed-messages.json` — bodies redacted unless `OPENCLAW_QA_SLACK_CAPTURE_CONTENT=1`.
|
||||
- `slack-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_SLACK_CAPTURE_CONTENT=1`.
|
||||
|
||||
#### Setting up the Slack workspace
|
||||
|
||||
The lane needs two distinct Slack apps in one workspace, plus a channel both bots are members of:
|
||||
|
||||
- `channelId` — the `Cxxxxxxxxxx` id of a channel both bots have been invited to. Use a dedicated channel; the lane posts on every run.
|
||||
- `driverBotToken` — bot token (`xoxb-...`) of the **Driver** app.
|
||||
- `sutBotToken` — bot token (`xoxb-...`) of the **SUT** app, which must be a separate Slack app from the driver so its bot user id is distinct.
|
||||
- `sutAppToken` — app-level token (`xapp-...`) of the SUT app with `connections:write`, used by Socket Mode so the SUT app can receive events.
|
||||
- `channelId` - the `Cxxxxxxxxxx` id of a channel both bots have been invited to. Use a dedicated channel; the lane posts on every run.
|
||||
- `driverBotToken` - bot token (`xoxb-...`) of the **Driver** app.
|
||||
- `sutBotToken` - bot token (`xoxb-...`) of the **SUT** app, which must be a separate Slack app from the driver so its bot user id is distinct.
|
||||
- `sutAppToken` - app-level token (`xapp-...`) of the SUT app with `connections:write`, used by Socket Mode so the SUT app can receive events.
|
||||
|
||||
Prefer a Slack workspace dedicated to QA over reusing a production workspace.
|
||||
|
||||
@@ -417,7 +417,7 @@ Go to [api.slack.com/apps](https://api.slack.com/apps) → _Create New App_ →
|
||||
}
|
||||
```
|
||||
|
||||
Copy the _Bot User OAuth Token_ (`xoxb-...`) — that becomes `driverBotToken`. The driver only needs to post messages and identify itself; no events, no Socket Mode.
|
||||
Copy the _Bot User OAuth Token_ (`xoxb-...`) - that becomes `driverBotToken`. The driver only needs to post messages and identify itself; no events, no Socket Mode.
|
||||
|
||||
**2. Create the SUT app**
|
||||
|
||||
@@ -504,7 +504,7 @@ In the QA workspace, create a channel (e.g. `#openclaw-qa`) and invite both bots
|
||||
/invite @OpenClaw QA SUT
|
||||
```
|
||||
|
||||
Copy the `Cxxxxxxxxxx` id from _channel info → About → Channel ID_ — that becomes `channelId`. A public channel works; if you use a private channel both apps already have `groups:history` so the harness's history reads will still succeed.
|
||||
Copy the `Cxxxxxxxxxx` id from _channel info → About → Channel ID_ - that becomes `channelId`. A public channel works; if you use a private channel both apps already have `groups:history` so the harness's history reads will still succeed.
|
||||
|
||||
**4. Register the credentials**
|
||||
|
||||
@@ -545,7 +545,7 @@ pnpm openclaw qa slack \
|
||||
--output-dir .artifacts/qa-e2e/slack-local
|
||||
```
|
||||
|
||||
A green run completes in well under 30 seconds and `slack-qa-report.md` shows both `slack-canary` and `slack-mention-gating` at status `pass`. If the lane hangs for ~90 seconds and exits with `Convex credential pool exhausted for kind "slack"`, either the pool is empty or every row is leased — `qa credentials list --kind slack --status all --json` will tell you which.
|
||||
A green run completes in well under 30 seconds and `slack-qa-report.md` shows both `slack-canary` and `slack-mention-gating` at status `pass`. If the lane hangs for ~90 seconds and exits with `Convex credential pool exhausted for kind "slack"`, either the pool is empty or every row is leased - `qa credentials list --kind slack --status all --json` will tell you which.
|
||||
|
||||
### Convex credential pool
|
||||
|
||||
@@ -553,9 +553,9 @@ Telegram, Discord, and Slack lanes can lease credentials from a shared Convex po
|
||||
|
||||
Payload shapes the broker validates on `admin/add`:
|
||||
|
||||
- Telegram (`kind: "telegram"`): `{ groupId: string, driverToken: string, sutToken: string }` — `groupId` must be a numeric chat-id string.
|
||||
- Telegram (`kind: "telegram"`): `{ groupId: string, driverToken: string, sutToken: string }` - `groupId` must be a numeric chat-id string.
|
||||
- Discord (`kind: "discord"`): `{ guildId: string, channelId: string, driverBotToken: string, sutBotToken: string, sutApplicationId: string }`.
|
||||
- Slack (`kind: "slack"`): `{ channelId: string, driverBotToken: string, sutBotToken: string, sutAppToken: string }` — `channelId` must match `^[A-Z][A-Z0-9]+$` (a Slack id like `Cxxxxxxxxxx`). See [Setting up the Slack workspace](#setting-up-the-slack-workspace) for app and scope provisioning.
|
||||
- Slack (`kind: "slack"`): `{ channelId: string, driverBotToken: string, sutBotToken: string, sutAppToken: string }` - `channelId` must match `^[A-Z][A-Z0-9]+$` (a Slack id like `Cxxxxxxxxxx`). See [Setting up the Slack workspace](#setting-up-the-slack-workspace) for app and scope provisioning.
|
||||
|
||||
Operational env vars and the Convex broker endpoint contract live in [Testing → Shared Telegram credentials via Convex](/help/testing#shared-telegram-credentials-via-convex-v1) (the section name predates Discord support; the broker semantics are identical for both kinds).
|
||||
|
||||
@@ -690,7 +690,7 @@ Preferred generic helpers for new scenarios:
|
||||
- `formatTransportTranscript`
|
||||
- `resetTransport`
|
||||
|
||||
Compatibility aliases remain available for existing scenarios — `waitForQaChannelReady`, `waitForOutboundMessage`, `waitForNoOutbound`, `formatConversationTranscript`, `resetBus` — but new scenario authoring should use the generic names. The aliases exist to avoid a flag-day migration, not as the model going forward.
|
||||
Compatibility aliases remain available for existing scenarios - `waitForQaChannelReady`, `waitForOutboundMessage`, `waitForNoOutbound`, `formatConversationTranscript`, `resetBus` - but new scenario authoring should use the generic names. The aliases exist to avoid a flag-day migration, not as the model going forward.
|
||||
|
||||
## Reporting
|
||||
|
||||
@@ -702,7 +702,7 @@ The report should answer:
|
||||
- What stayed blocked
|
||||
- What follow-up scenarios are worth adding
|
||||
|
||||
For the inventory of available scenarios — useful when sizing follow-up work or wiring a new transport — run `pnpm openclaw qa coverage` (add `--json` for machine-readable output).
|
||||
For the inventory of available scenarios - useful when sizing follow-up work or wiring a new transport - run `pnpm openclaw qa coverage` (add `--json` for machine-readable output).
|
||||
|
||||
For character and style checks, run the same scenario across multiple live model
|
||||
refs and write a judged Markdown report:
|
||||
|
||||
@@ -9,7 +9,7 @@ title: "Matrix QA"
|
||||
|
||||
The Matrix QA lane runs the bundled `@openclaw/matrix` plugin against a disposable Tuwunel homeserver in Docker, with temporary driver, SUT, and observer accounts plus seeded rooms. It is the live transport-real coverage for Matrix.
|
||||
|
||||
This is maintainer-only tooling. Packaged OpenClaw releases intentionally omit `qa-lab`, so `openclaw qa` is only available from a source checkout. Source checkouts load the bundled runner directly — no plugin install step is needed.
|
||||
This is maintainer-only tooling. Packaged OpenClaw releases intentionally omit `qa-lab`, so `openclaw qa` is only available from a source checkout. Source checkouts load the bundled runner directly - no plugin install step is needed.
|
||||
|
||||
For broader QA framework context, see [QA overview](/concepts/qa-e2e-automation).
|
||||
|
||||
@@ -24,7 +24,7 @@ Plain `pnpm openclaw qa matrix` runs `--profile all` and does not stop on first
|
||||
## What the lane does
|
||||
|
||||
1. Provisions a disposable Tuwunel homeserver in Docker (default image `ghcr.io/matrix-construct/tuwunel:v1.5.1`, server name `matrix-qa.test`, port `28008`).
|
||||
2. Registers three temporary users — `driver` (sends inbound traffic), `sut` (the OpenClaw Matrix account under test), `observer` (third-party traffic capture).
|
||||
2. Registers three temporary users - `driver` (sends inbound traffic), `sut` (the OpenClaw Matrix account under test), `observer` (third-party traffic capture).
|
||||
3. Seeds rooms required by the selected scenarios (main, threading, media, restart, secondary, allowlist, E2EE, verification DM, etc.).
|
||||
4. Starts a child OpenClaw gateway with the real Matrix plugin scoped to the SUT account; `qa-channel` is not loaded in the child.
|
||||
5. Runs scenarios in sequence, observing events through the driver/observer Matrix clients.
|
||||
@@ -42,7 +42,7 @@ pnpm openclaw qa matrix [options]
|
||||
| --------------------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `--profile <profile>` | `all` | Scenario profile. See [Profiles](#profiles). |
|
||||
| `--fail-fast` | off | Stop after the first failed check or scenario. |
|
||||
| `--scenario <id>` | — | Run only this scenario. Repeatable. See [Scenarios](#scenarios). |
|
||||
| `--scenario <id>` | - | Run only this scenario. Repeatable. See [Scenarios](#scenarios). |
|
||||
| `--output-dir <path>` | `<repo>/.artifacts/qa-e2e/matrix-<timestamp>` | Where reports, summary, observed events, and the output log are written. Relative paths resolve against `--repo-root`. |
|
||||
| `--repo-root <path>` | `process.cwd()` | Repository root when invoking from a neutral working directory. |
|
||||
| `--sut-account <id>` | `sut` | Matrix account id inside the QA gateway config. |
|
||||
@@ -70,7 +70,7 @@ The selected profile decides which scenarios run.
|
||||
| `fast` | Release-gate subset that exercises the live transport contract: canary, mention gating, allowlist block, reply shape, restart resume, thread follow-up, thread isolation, reaction observation, and exec approval metadata delivery. |
|
||||
| `transport` | Transport-level threading, DM, room, autojoin, mention/allowlist, approval, and reaction scenarios. |
|
||||
| `media` | Image, audio, video, PDF, EPUB attachment coverage. |
|
||||
| `e2ee-smoke` | Minimum E2EE coverage — basic encrypted reply, thread follow-up, bootstrap success. |
|
||||
| `e2ee-smoke` | Minimum E2EE coverage - basic encrypted reply, thread follow-up, bootstrap success. |
|
||||
| `e2ee-deep` | Exhaustive E2EE state-loss, backup, key, and recovery scenarios. |
|
||||
| `e2ee-cli` | `openclaw matrix encryption setup` and `verify *` CLI scenarios driven through the QA harness. |
|
||||
|
||||
@@ -80,17 +80,17 @@ The exact mapping lives in `extensions/qa-matrix/src/runners/contract/scenario-c
|
||||
|
||||
The full scenario id list is the `MatrixQaScenarioId` union in `extensions/qa-matrix/src/runners/contract/scenario-catalog.ts:15`. Categories include:
|
||||
|
||||
- threading — `matrix-thread-*`, `matrix-subagent-thread-spawn`
|
||||
- top-level / DM / room — `matrix-top-level-reply-shape`, `matrix-room-*`, `matrix-dm-*`
|
||||
- streaming and tool progress — `matrix-room-partial-streaming-preview`, `matrix-room-quiet-streaming-preview`, `matrix-room-tool-progress-*`, `matrix-room-block-streaming`
|
||||
- media — `matrix-media-type-coverage`, `matrix-room-image-understanding-attachment`, `matrix-attachment-only-ignored`, `matrix-unsupported-media-safe`
|
||||
- routing — `matrix-room-autojoin-invite`, `matrix-secondary-room-*`
|
||||
- reactions — `matrix-reaction-*`
|
||||
- approvals — `matrix-approval-*` (exec/plugin metadata, chunked fallback, deny reactions, threads, and `target: "both"` routing)
|
||||
- restart and replay — `matrix-restart-*`, `matrix-stale-sync-replay-dedupe`, `matrix-room-membership-loss`, `matrix-homeserver-restart-resume`, `matrix-initial-catchup-then-incremental`
|
||||
- mention gating, bot-to-bot, and allowlists — `matrix-mention-*`, `matrix-allowbots-*`, `matrix-allowlist-*`, `matrix-multi-actor-ordering`, `matrix-inbound-edit-*`, `matrix-mxid-prefixed-command-block`, `matrix-observer-allowlist-override`
|
||||
- E2EE — `matrix-e2ee-*` (basic reply, thread follow-up, bootstrap, recovery key lifecycle, state-loss variants, server backup behavior, device hygiene, SAS / QR / DM verification, restart, artifact redaction)
|
||||
- E2EE CLI — `matrix-e2ee-cli-*` (encryption setup, idempotent setup, bootstrap failure, recovery-key lifecycle, multi-account, gateway-reply round-trip, self-verification)
|
||||
- threading - `matrix-thread-*`, `matrix-subagent-thread-spawn`
|
||||
- top-level / DM / room - `matrix-top-level-reply-shape`, `matrix-room-*`, `matrix-dm-*`
|
||||
- streaming and tool progress - `matrix-room-partial-streaming-preview`, `matrix-room-quiet-streaming-preview`, `matrix-room-tool-progress-*`, `matrix-room-block-streaming`
|
||||
- media - `matrix-media-type-coverage`, `matrix-room-image-understanding-attachment`, `matrix-attachment-only-ignored`, `matrix-unsupported-media-safe`
|
||||
- routing - `matrix-room-autojoin-invite`, `matrix-secondary-room-*`
|
||||
- reactions - `matrix-reaction-*`
|
||||
- approvals - `matrix-approval-*` (exec/plugin metadata, chunked fallback, deny reactions, threads, and `target: "both"` routing)
|
||||
- restart and replay - `matrix-restart-*`, `matrix-stale-sync-replay-dedupe`, `matrix-room-membership-loss`, `matrix-homeserver-restart-resume`, `matrix-initial-catchup-then-incremental`
|
||||
- mention gating, bot-to-bot, and allowlists - `matrix-mention-*`, `matrix-allowbots-*`, `matrix-allowlist-*`, `matrix-multi-actor-ordering`, `matrix-inbound-edit-*`, `matrix-mxid-prefixed-command-block`, `matrix-observer-allowlist-override`
|
||||
- E2EE - `matrix-e2ee-*` (basic reply, thread follow-up, bootstrap, recovery key lifecycle, state-loss variants, server backup behavior, device hygiene, SAS / QR / DM verification, restart, artifact redaction)
|
||||
- E2EE CLI - `matrix-e2ee-cli-*` (encryption setup, idempotent setup, bootstrap failure, recovery-key lifecycle, multi-account, gateway-reply round-trip, self-verification)
|
||||
|
||||
Pass `--scenario <id>` (repeatable) to run a hand-picked set; combine with `--profile all` to ignore profile gating.
|
||||
|
||||
@@ -112,10 +112,10 @@ Pass `--scenario <id>` (repeatable) to run a hand-picked set; combine with `--pr
|
||||
|
||||
Written to `--output-dir`:
|
||||
|
||||
- `matrix-qa-report.md` — Markdown protocol report (what passed, failed, was skipped, and why).
|
||||
- `matrix-qa-summary.json` — Structured summary suitable for CI parsing and dashboards.
|
||||
- `matrix-qa-observed-events.json` — Observed Matrix events from the driver and observer clients. Bodies are redacted unless `OPENCLAW_QA_MATRIX_CAPTURE_CONTENT=1`; approval metadata is summarized with selected safe fields and truncated command preview.
|
||||
- `matrix-qa-output.log` — Combined stdout/stderr from the run. If `OPENCLAW_RUN_NODE_OUTPUT_LOG` is set, the outer launcher's log is reused instead.
|
||||
- `matrix-qa-report.md` - Markdown protocol report (what passed, failed, was skipped, and why).
|
||||
- `matrix-qa-summary.json` - Structured summary suitable for CI parsing and dashboards.
|
||||
- `matrix-qa-observed-events.json` - Observed Matrix events from the driver and observer clients. Bodies are redacted unless `OPENCLAW_QA_MATRIX_CAPTURE_CONTENT=1`; approval metadata is summarized with selected safe fields and truncated command preview.
|
||||
- `matrix-qa-output.log` - Combined stdout/stderr from the run. If `OPENCLAW_RUN_NODE_OUTPUT_LOG` is set, the outer launcher's log is reused instead.
|
||||
|
||||
The default output dir is `<repo>/.artifacts/qa-e2e/matrix-<timestamp>` so successive runs do not overwrite each other.
|
||||
|
||||
@@ -133,7 +133,7 @@ Matrix is one of three live transport lanes (Matrix, Telegram, Discord) that sha
|
||||
|
||||
## Related
|
||||
|
||||
- [QA overview](/concepts/qa-e2e-automation) — overall QA stack and live transport contract
|
||||
- [QA Channel](/channels/qa-channel) — synthetic channel adapter for repo-backed scenarios
|
||||
- [Testing](/help/testing) — running tests and adding QA coverage
|
||||
- [Matrix](/channels/matrix) — the channel plugin under test
|
||||
- [QA overview](/concepts/qa-e2e-automation) - overall QA stack and live transport contract
|
||||
- [QA Channel](/channels/qa-channel) - synthetic channel adapter for repo-backed scenarios
|
||||
- [Testing](/help/testing) - running tests and adding QA coverage
|
||||
- [Matrix](/channels/matrix) - the channel plugin under test
|
||||
|
||||
@@ -7,8 +7,6 @@ read_when:
|
||||
- Reviewing the strict-agentic, tool-schema, elevation, and replay fixes
|
||||
---
|
||||
|
||||
# GPT-5.5 / Codex Agentic Parity in OpenClaw
|
||||
|
||||
OpenClaw already worked well with tool-using frontier models, but GPT-5.5 and Codex-style models were still underperforming in a few practical ways:
|
||||
|
||||
- they could stop after planning instead of doing the work
|
||||
@@ -25,11 +23,11 @@ This parity program fixes those gaps in four reviewable slices.
|
||||
|
||||
This slice adds an opt-in `strict-agentic` execution contract for embedded Pi GPT-5 runs.
|
||||
|
||||
When enabled, OpenClaw stops accepting plan-only turns as “good enough” completion. If the model only says what it intends to do and does not actually use tools or make progress, OpenClaw retries with an act-now steer and then fails closed with an explicit blocked state instead of silently ending the task.
|
||||
When enabled, OpenClaw stops accepting plan-only turns as "good enough" completion. If the model only says what it intends to do and does not actually use tools or make progress, OpenClaw retries with an act-now steer and then fails closed with an explicit blocked state instead of silently ending the task.
|
||||
|
||||
This improves the GPT-5.5 experience most on:
|
||||
|
||||
- short “ok do it” follow-ups
|
||||
- short "ok do it" follow-ups
|
||||
- code tasks where the first step is obvious
|
||||
- flows where `update_plan` should be progress tracking rather than filler text
|
||||
|
||||
@@ -86,21 +84,21 @@ The goal is not to make GPT-5.5 imitate Opus. The goal is to give GPT-5.5 a runt
|
||||
|
||||
That changes the user experience from:
|
||||
|
||||
- “the model had a good plan but stopped”
|
||||
- "the model had a good plan but stopped"
|
||||
|
||||
to:
|
||||
|
||||
- “the model either acted, or OpenClaw surfaced the exact reason it could not”
|
||||
- "the model either acted, or OpenClaw surfaced the exact reason it could not"
|
||||
|
||||
## Before vs after for GPT-5.5 users
|
||||
|
||||
| Before this program | After PR A-D |
|
||||
| ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| GPT-5.5 could stop after a reasonable plan without taking the next tool step | PR A turns “plan only” into “act now or surface a blocked state” |
|
||||
| GPT-5.5 could stop after a reasonable plan without taking the next tool step | PR A turns "plan only" into "act now or surface a blocked state" |
|
||||
| Strict tool schemas could reject parameter-free or OpenAI/Codex-shaped tools in confusing ways | PR C makes provider-owned tool registration and invocation more predictable |
|
||||
| `/elevated full` guidance could be vague or wrong in blocked runtimes | PR B gives GPT-5.5 and the user truthful runtime and permission hints |
|
||||
| Replay or compaction failures could feel like the task silently disappeared | PR C surfaces paused, blocked, abandoned, and replay-invalid outcomes explicitly |
|
||||
| “GPT-5.5 feels worse than Opus” was mostly anecdotal | PR D turns that into the same scenario pack, the same metrics, and a hard pass/fail gate |
|
||||
| "GPT-5.5 feels worse than Opus" was mostly anecdotal | PR D turns that into the same scenario pack, the same metrics, and a hard pass/fail gate |
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -142,7 +140,7 @@ The first-wave parity pack currently covers five scenarios:
|
||||
|
||||
### `approval-turn-tool-followthrough`
|
||||
|
||||
Checks that the model does not stop at “I’ll do that” after a short approval. It should take the first concrete action in the same turn.
|
||||
Checks that the model does not stop at "I'll do that" after a short approval. It should take the first concrete action in the same turn.
|
||||
|
||||
### `model-switch-tool-continuity`
|
||||
|
||||
@@ -210,8 +208,8 @@ Use the verdict in `qa-agentic-parity-summary.json` as the final machine-readabl
|
||||
|
||||
- `pass` means GPT-5.5 covered the same scenarios as Opus 4.6 and did not regress on the agreed aggregate metrics.
|
||||
- `fail` means at least one hard gate tripped: weaker completion, worse unintended stops, weaker valid tool use, any fake-success case, or mismatched scenario coverage.
|
||||
- “shared/base CI issue” is not itself a parity result. If CI noise outside PR D blocks a run, the verdict should wait for a clean merged-runtime execution instead of being inferred from branch-era logs.
|
||||
- Auth, proxy, DNS, and `/elevated full` truthfulness still come from PR B’s deterministic suites, so the final release claim needs both: a passing PR D parity verdict and green PR B truthfulness coverage.
|
||||
- "shared/base CI issue" is not itself a parity result. If CI noise outside PR D blocks a run, the verdict should wait for a clean merged-runtime execution instead of being inferred from branch-era logs.
|
||||
- Auth, proxy, DNS, and `/elevated full` truthfulness still come from PR B's deterministic suites, so the final release claim needs both: a passing PR D parity verdict and green PR B truthfulness coverage.
|
||||
|
||||
## Who should enable `strict-agentic`
|
||||
|
||||
@@ -219,7 +217,7 @@ Use `strict-agentic` when:
|
||||
|
||||
- the agent is expected to act immediately when a next step is obvious
|
||||
- GPT-5.5 or Codex-family models are the primary runtime
|
||||
- you prefer explicit blocked states over “helpful” recap-only replies
|
||||
- you prefer explicit blocked states over "helpful" recap-only replies
|
||||
|
||||
Keep the default contract when:
|
||||
|
||||
|
||||
@@ -5,22 +5,20 @@ read_when:
|
||||
title: "Menu bar"
|
||||
---
|
||||
|
||||
# Menu Bar Status Logic
|
||||
|
||||
## What is shown
|
||||
|
||||
- We surface the current agent work state in the menu bar icon and in the first status row of the menu.
|
||||
- Health status is hidden while work is active; it returns when all sessions are idle.
|
||||
- A root “Context” submenu contains recent sessions instead of expanding them directly in the root menu.
|
||||
- The “Nodes” block in the root menu lists **devices** only (paired nodes via `node.list`), not client/presence entries.
|
||||
- A root “Usage” section appears below Context when provider usage snapshots are available, followed by usage-cost details when available.
|
||||
- A root "Context" submenu contains recent sessions instead of expanding them directly in the root menu.
|
||||
- The "Nodes" block in the root menu lists **devices** only (paired nodes via `node.list`), not client/presence entries.
|
||||
- A root "Usage" section appears below Context when provider usage snapshots are available, followed by usage-cost details when available.
|
||||
|
||||
## State model
|
||||
|
||||
- Sessions: events arrive with `runId` (per-run) plus `sessionKey` in the payload. The “main” session is the key `main`; if absent, we fall back to the most recently updated session.
|
||||
- Priority: main always wins. If main is active, its state is shown immediately. If main is idle, the most recently active non‑main session is shown. We do not flip‑flop mid‑activity; we only switch when the current session goes idle or main becomes active.
|
||||
- Sessions: events arrive with `runId` (per-run) plus `sessionKey` in the payload. The "main" session is the key `main`; if absent, we fall back to the most recently updated session.
|
||||
- Priority: main always wins. If main is active, its state is shown immediately. If main is idle, the most recently active non-main session is shown. We do not flip-flop mid-activity; we only switch when the current session goes idle or main becomes active.
|
||||
- Activity kinds:
|
||||
- `job`: high‑level command execution (`state: started|streaming|done|error`).
|
||||
- `job`: high-level command execution (`state: started|streaming|done|error`).
|
||||
- `tool`: `phase: start|result` with `toolName` and `meta/args`.
|
||||
|
||||
## IconState enum (Swift)
|
||||
@@ -42,13 +40,13 @@ title: "Menu bar"
|
||||
### Visual mapping
|
||||
|
||||
- `idle`: normal critter.
|
||||
- `workingMain`: badge with glyph, full tint, leg “working” animation.
|
||||
- `workingMain`: badge with glyph, full tint, leg "working" animation.
|
||||
- `workingOther`: badge with glyph, muted tint, no scurry.
|
||||
- `overridden`: uses the chosen glyph/tint regardless of activity.
|
||||
|
||||
## Context submenu
|
||||
|
||||
- The root menu shows one “Context” row with a session count/status and opens a submenu.
|
||||
- The root menu shows one "Context" row with a session count/status and opens a submenu.
|
||||
- The Context submenu header shows the active session count for the last 24 hours.
|
||||
- Each session row keeps its token bar, age, preview, thinking/verbose, reset, compact, and delete actions.
|
||||
- Loading, disconnected, and session-load error messages appear inside the Context submenu.
|
||||
@@ -62,7 +60,7 @@ title: "Menu bar"
|
||||
|
||||
## Event ingestion
|
||||
|
||||
- Source: control‑channel `agent` events (`ControlChannel.handleAgentEvent`).
|
||||
- Source: control-channel `agent` events (`ControlChannel.handleAgentEvent`).
|
||||
- Parsed fields:
|
||||
- `stream: "job"` with `data.state` for start/stop.
|
||||
- `stream: "tool"` with `data.phase`, `name`, optional `meta`/`args`.
|
||||
@@ -74,7 +72,7 @@ title: "Menu bar"
|
||||
|
||||
## Debug override
|
||||
|
||||
- Settings ▸ Debug ▸ “Icon override” picker:
|
||||
- Settings ▸ Debug ▸ "Icon override" picker:
|
||||
- `System (auto)` (default)
|
||||
- `Working: main` (per tool kind)
|
||||
- `Working: other` (per tool kind)
|
||||
@@ -84,7 +82,7 @@ title: "Menu bar"
|
||||
## Testing checklist
|
||||
|
||||
- Trigger main session job: verify icon switches immediately and status row shows main label.
|
||||
- Trigger non‑main session job while main idle: icon/status shows non‑main; stays stable until it finishes.
|
||||
- Trigger non-main session job while main idle: icon/status shows non-main; stays stable until it finishes.
|
||||
- Start main while other active: icon flips to main instantly.
|
||||
- Rapid tool bursts: ensure badge does not flicker (TTL grace on tool results).
|
||||
- Health row reappears once all sessions idle.
|
||||
|
||||
@@ -77,7 +77,7 @@ an unavailable backend.
|
||||
- The target id is allowed by `acp.allowedAgents` when that allowlist is set.
|
||||
- The harness command can start on the Gateway host.
|
||||
- Provider auth is present for that harness (`claude`, `codex`, `gemini`, `opencode`, `droid`, etc.).
|
||||
- The selected model exists for that harness — model ids are not portable across harnesses.
|
||||
- The selected model exists for that harness - model ids are not portable across harnesses.
|
||||
- The requested `cwd` exists and is accessible, or omit `cwd` and let the backend use its default.
|
||||
- Permission mode matches the work. Non-interactive sessions cannot click native permission prompts, so write/exec-heavy coding runs usually need an ACPX permission profile that can proceed headlessly.
|
||||
|
||||
@@ -86,7 +86,7 @@ an unavailable backend.
|
||||
|
||||
OpenClaw plugin tools and built-in OpenClaw tools are **not** exposed to
|
||||
ACP harnesses by default. Enable the explicit MCP bridges in
|
||||
[ACP agents — setup](/tools/acp-agents-setup) only when the harness
|
||||
[ACP agents - setup](/tools/acp-agents-setup) only when the harness
|
||||
should call those tools directly.
|
||||
|
||||
## Supported harness targets
|
||||
@@ -182,10 +182,10 @@ Quick `/acp` flow from chat:
|
||||
|
||||
</Accordion>
|
||||
<Accordion title="Model / provider / runtime selection cheat sheet">
|
||||
- `openai-codex/*` — PI Codex OAuth/subscription route.
|
||||
- `openai/*` plus `agentRuntime.id: "codex"` — native Codex app-server embedded runtime.
|
||||
- `/codex ...` — native Codex conversation control.
|
||||
- `/acp ...` or `runtime: "acp"` — explicit ACP/acpx control.
|
||||
- `openai-codex/*` - PI Codex OAuth/subscription route.
|
||||
- `openai/*` plus `agentRuntime.id: "codex"` - native Codex app-server embedded runtime.
|
||||
- `/codex ...` - native Codex conversation control.
|
||||
- `/acp ...` or `runtime: "acp"` - explicit ACP/acpx control.
|
||||
|
||||
</Accordion>
|
||||
<Accordion title="ACP-routing natural-language triggers">
|
||||
@@ -244,7 +244,7 @@ For Claude Code through ACP, the stack is:
|
||||
ACP Claude is a **harness session** with ACP controls, session resume,
|
||||
background-task tracking, and optional conversation/thread binding.
|
||||
|
||||
CLI backends are separate text-only local fallback runtimes — see
|
||||
CLI backends are separate text-only local fallback runtimes - see
|
||||
[CLI Backends](/gateway/cli-backends).
|
||||
|
||||
For operators, the practical rule is:
|
||||
@@ -256,15 +256,15 @@ For operators, the practical rule is:
|
||||
|
||||
### Mental model
|
||||
|
||||
- **Chat surface** — where people keep talking (Discord channel, Telegram topic, iMessage chat).
|
||||
- **ACP session** — the durable Codex/Claude/Gemini runtime state OpenClaw routes to.
|
||||
- **Child thread/topic** — an optional extra messaging surface created only by `--thread ...`.
|
||||
- **Runtime workspace** — the filesystem location (`cwd`, repo checkout, backend workspace) where the harness runs. Independent of the chat surface.
|
||||
- **Chat surface** - where people keep talking (Discord channel, Telegram topic, iMessage chat).
|
||||
- **ACP session** - the durable Codex/Claude/Gemini runtime state OpenClaw routes to.
|
||||
- **Child thread/topic** - an optional extra messaging surface created only by `--thread ...`.
|
||||
- **Runtime workspace** - the filesystem location (`cwd`, repo checkout, backend workspace) where the harness runs. Independent of the chat surface.
|
||||
|
||||
### Current-conversation binds
|
||||
|
||||
`/acp spawn <harness> --bind here` pins the current conversation to the
|
||||
spawned ACP session — no child thread, same chat surface. OpenClaw keeps
|
||||
spawned ACP session - no child thread, same chat surface. OpenClaw keeps
|
||||
owning transport, auth, safety, and delivery. Follow-up messages in that
|
||||
conversation route to the same session; `/new` and `/reset` reset the
|
||||
session in place; `/acp close` removes the binding.
|
||||
@@ -284,9 +284,9 @@ Examples:
|
||||
<Accordion title="Binding rules and exclusivity">
|
||||
- `--bind here` and `--thread ...` are mutually exclusive.
|
||||
- `--bind here` only works on channels that advertise current-conversation binding; OpenClaw returns a clear unsupported message otherwise. Bindings persist across gateway restarts.
|
||||
- On Discord, `spawnSessions` gates child thread creation for `--thread auto|here` — not `--bind here`.
|
||||
- On Discord, `spawnSessions` gates child thread creation for `--thread auto|here` - not `--bind here`.
|
||||
- If you spawn to a different ACP agent without `--cwd`, OpenClaw inherits the **target agent's** workspace by default. Missing inherited paths (`ENOENT`/`ENOTDIR`) fall back to the backend default; other access errors (e.g. `EACCES`) surface as spawn errors.
|
||||
- Gateway management commands stay local in bound conversations — `/acp ...` commands are handled by OpenClaw even when normal follow-up text routes to the bound ACP session; `/status` and `/unfocus` also stay local whenever command handling is enabled for that surface.
|
||||
- Gateway management commands stay local in bound conversations - `/acp ...` commands are handled by OpenClaw even when normal follow-up text routes to the bound ACP session; `/status` and `/unfocus` also stay local whenever command handling is enabled for that surface.
|
||||
|
||||
</Accordion>
|
||||
<Accordion title="Thread-bound sessions">
|
||||
@@ -676,7 +676,7 @@ background work. The delivery path depends on that shape.
|
||||
|
||||
```json
|
||||
{
|
||||
"task": "Continue where we left off — fix the remaining test failures",
|
||||
"task": "Continue where we left off - fix the remaining test failures",
|
||||
"runtime": "acp",
|
||||
"agentId": "codex",
|
||||
"resumeSessionId": "<previous-session-id>"
|
||||
@@ -685,7 +685,7 @@ background work. The delivery path depends on that shape.
|
||||
|
||||
Common use cases:
|
||||
|
||||
- Hand off a Codex session from your laptop to your phone — tell your agent to pick up where you left off.
|
||||
- Hand off a Codex session from your laptop to your phone - tell your agent to pick up where you left off.
|
||||
- Continue a coding session you started interactively in the CLI, now headlessly through your agent.
|
||||
- Pick up work that was interrupted by a gateway restart or idle timeout.
|
||||
|
||||
@@ -696,7 +696,7 @@ background work. The delivery path depends on that shape.
|
||||
- `resumeSessionId` is a host-local ACP/harness resume id, not an OpenClaw channel session key; OpenClaw still checks ACP spawn policy and target agent policy before dispatch, while the ACP backend or harness owns authorization for loading that upstream id.
|
||||
- `resumeSessionId` restores the upstream ACP conversation history; `thread` and `mode` still apply normally to the new OpenClaw session you are creating, so `mode: "session"` still requires `thread: true`.
|
||||
- The target agent must support `session/load` (Codex and Claude Code do).
|
||||
- If the session id is not found, the spawn fails with a clear error — no silent fallback to a new session.
|
||||
- If the session id is not found, the spawn fails with a clear error - no silent fallback to a new session.
|
||||
|
||||
</Accordion>
|
||||
<Accordion title="Post-deploy smoke test">
|
||||
@@ -709,7 +709,7 @@ background work. The delivery path depends on that shape.
|
||||
4. Verify `accepted=yes`, a real `childSessionKey`, and no validator error.
|
||||
5. Clean up the temporary bridge session.
|
||||
|
||||
Keep the gate on `mode: "run"` and skip `streamTo: "parent"` —
|
||||
Keep the gate on `mode: "run"` and skip `streamTo: "parent"` -
|
||||
thread-bound `mode: "session"` and stream-relay paths are separate
|
||||
richer integration passes.
|
||||
|
||||
@@ -793,18 +793,18 @@ operations:
|
||||
| ---------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `/acp model <id>` | runtime config key `model` | For Codex ACP, OpenClaw normalizes `openai-codex/<model>` to the adapter model id and maps slash reasoning suffixes such as `openai-codex/gpt-5.4/high` to `reasoning_effort`. |
|
||||
| `/acp set thinking <level>` | runtime config key `thinking` | For Codex ACP, OpenClaw sends the corresponding `reasoning_effort` where the adapter supports one. |
|
||||
| `/acp permissions <profile>` | runtime config key `approval_policy` | — |
|
||||
| `/acp timeout <seconds>` | runtime config key `timeout` | — |
|
||||
| `/acp permissions <profile>` | runtime config key `approval_policy` | - |
|
||||
| `/acp timeout <seconds>` | runtime config key `timeout` | - |
|
||||
| `/acp cwd <path>` | runtime cwd override | Direct update. |
|
||||
| `/acp set <key> <value>` | generic | `key=cwd` uses the cwd override path. |
|
||||
| `/acp reset-options` | clears all runtime overrides | — |
|
||||
| `/acp reset-options` | clears all runtime overrides | - |
|
||||
|
||||
## acpx harness, plugin setup, and permissions
|
||||
|
||||
For acpx harness configuration (Claude Code / Codex / Gemini CLI
|
||||
aliases), the plugin-tools and OpenClaw-tools MCP bridges, and ACP
|
||||
permission modes, see
|
||||
[ACP agents — setup](/tools/acp-agents-setup).
|
||||
[ACP agents - setup](/tools/acp-agents-setup).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -835,7 +835,7 @@ permission modes, see
|
||||
|
||||
## Related
|
||||
|
||||
- [ACP agents — setup](/tools/acp-agents-setup)
|
||||
- [ACP agents - setup](/tools/acp-agents-setup)
|
||||
- [Agent send](/tools/agent-send)
|
||||
- [CLI Backends](/gateway/cli-backends)
|
||||
- [Codex harness](/plugins/codex-harness)
|
||||
|
||||
Reference in New Issue
Block a user