61 Commits

Author SHA1 Message Date
Vincent Koc
2b8d91d9ee docs: typography hygiene + 2 in-body H1 removals across 5 pages
Replaced 112 typography characters (curly quotes, apostrophes, em/en
dashes, non-breaking hyphens) with ASCII equivalents per
docs/CLAUDE.md heading and content hygiene rules.

- docs/help/gpt55-codex-agentic-parity.md: 22 chars; removed the
  duplicate '# GPT-5.5 / Codex Agentic Parity in OpenClaw' H1 (Mintlify
  renders the title from frontmatter; the in-body H1 with the slash
  produced a brittle anchor).
- docs/platforms/mac/menu-bar.md: 21 chars; removed the duplicate
  '# Menu Bar Status Logic' H1.
- docs/tools/acp-agents.md: 23 chars
- docs/concepts/qa-matrix.md: 23 chars
- docs/concepts/qa-e2e-automation.md: 23 chars
2026-05-05 19:34:52 -07:00
Peter Steinberger
057d3a43c0 feat(mantis): capture logged-in discord web evidence 2026-05-06 02:43:49 +01:00
Peter Steinberger
ad2d13cc67 fix(discord): preserve thread reply file attachments 2026-05-06 01:16:57 +01:00
Kevin Lin
dd643b52df test: expand slack live qa coverage (#77713) 2026-05-05 16:11:07 -07:00
Peter Steinberger
430814ebc1 docs: add Mantis Slack desktop runbook 2026-05-05 23:48:49 +01:00
Peter Steinberger
26bc40c1a4 perf: add Mantis Slack hydrate timings 2026-05-05 21:07:07 +01:00
Peter Steinberger
e8a9c766c2 perf: speed up Mantis Slack desktop smoke 2026-05-05 19:57:26 +01:00
Peter Steinberger
35266879de feat: add Mantis visual task video QA 2026-05-05 05:35:12 +01:00
Vincent Koc
e03fe1e289 fix(telegram): reuse preview for long text finals (#77658)
* fix(telegram): reuse preview for long text finals

* test(qa): cover long telegram finals

* fix(qa): satisfy extension lint

* fix(qa): keep telegram long final fixture to two chunks

* test(telegram): cover three chunk finals

* fix(telegram): force long final preview boundary
2026-05-04 21:19:44 -07:00
Vincent Koc
b062bb670d docs(channels): inline Slack manifest into Quick Setup with Recommended/Minimal variants
The Quick Setup steps in docs/channels/slack.md previously sent users to
the `#manifest-and-scope-checklist` anchor lower on the page to copy the
manifest, breaking the copy-paste flow. Pull the manifest inline as a
Mintlify <CodeGroup> for both Socket Mode and HTTP Request URLs tabs and
add a Minimal variant for workspaces that restrict scopes (drops
files:*, reactions:*, pins:*, mpim:*, emoji:read, usergroups:read while
keeping DMs, channel/group history, mentions, App Home, and slash
commands). Recommended matches extensions/slack/src/setup-shared.ts.
Existing Manifest and scope checklist section stays as the canonical
per-scope reference.

Cross-link from docs/concepts/qa-e2e-automation.md so QA maintainers see
the production manifest reference, while keeping the QA Driver/SUT pair
of manifests inline (the lane intentionally needs two distinct apps so
its shape is different from a single-app production install).
2026-05-04 18:16:15 -07:00
Sarah Fortune
d6e991db49 Add instructions for how to setup slack for QA tests (#77606) 2026-05-04 17:38:16 -07:00
Peter Steinberger
f632f5e60b feat(qa): add mantis Slack desktop smoke 2026-05-04 03:47:27 +01:00
Peter Steinberger
57b2d29761 feat(qa): add Mantis desktop browser smoke 2026-05-04 01:30:20 +01:00
Vincent Koc
31cafbb802 test(qa): add Slack live transport lane 2026-05-03 15:19:55 -07:00
Peter Steinberger
d4af125b52 feat(qa): add Mantis before-after CLI 2026-05-03 21:27:43 +01:00
Peter Steinberger
77a50db9ea feat(qa): add Mantis Discord status reaction scenario (#76747)
* feat(qa): add Mantis Discord status reaction scenario

* fix(qa): retry Discord rate limits in Mantis runs

* refactor(qa): reuse Discord API retry helper

* fix(qa): import Discord API through package surface

* fix(ci): generate Discord boundary declarations

* fix(ci): keep xai boundary overrides stable
2026-05-03 17:00:06 +01:00
Peter Steinberger
0bf06e953f feat: add Mantis Discord smoke runner (#76696)
* docs: add Mantis QA system design

* feat: add Mantis Discord smoke runner

* fix: harden Mantis Discord smoke

* fix: redact Mantis Discord artifacts

* fix: satisfy Mantis redaction lint

* fix: redact Mantis mismatch failures

* test: avoid promise assertions in Mantis tests
2026-05-03 15:25:56 +01:00
Vincent Koc
b9eb31b54c ci: fold parity into QA release validation (#74622)
Summary:
- The PR deletes the standalone `Parity gate` workflow, renames QA parity wording from gate to lane, routes docs toward QA/release validation, and adjusts the Docker E2E boundary guard for package-backed live lanes.
- Reproducibility: not applicable. as a CI/docs refactor. The high-confidence review path is static comparison of the repaired PR diff against current workflow/docs code plus exact-head check status.

ClawSweeper fixups:
- Included follow-up commit: ci: fold parity into QA release validation
- Ran the ClawSweeper repair loop before final review.

Validation:
- ClawSweeper review passed for head 3482654058.
- Required merge gates passed before the squash merge.

Prepared head SHA: 3482654058
Review: https://github.com/openclaw/openclaw/pull/74622#issuecomment-4359168336

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
2026-05-02 19:57:15 +00:00
Gustavo Madeira Santana
d59f001507 test(qa-matrix): cover allowBots modes 2026-04-28 00:47:40 -04:00
Gustavo Madeira Santana
c5678194d4 docs(qa): document Telegram and Discord QA lanes against code
Both lanes had only one paragraph each in qa-e2e-automation.md. Adds a
"Telegram and Discord QA reference" section verified against
extensions/qa-lab/src/live-transports/{telegram,discord}/* with:

- shared CLI flags table (--scenario, --output-dir, --repo-root, --sut-account,
  --provider-mode, --model, --alt-model, --fast, --credential-source,
  --credential-role) — none of these were enumerated for either lane.
- Telegram QA: 8 scenario ids
  (telegram-canary/-mention-gating/-mentioned-message-reply/-help-command/
  -commands-command/-tools-compact-command/-whoami-command/-context-command),
  output artifact paths (telegram-qa-report.md, -summary.json,
  -observed-messages.json), and the redaction toggle.
- Discord QA: 3 scenario ids
  (discord-canary/-mention-gating/-native-help-command-registration), output
  artifact paths, and the SUT-application-id-must-match-bot-user-id check.
- Convex credential pool: documents Discord support (only Telegram was
  mentioned before) and the per-kind payload shapes for the
  admin/add validator. Cross-links to testing.md for the broker endpoint
  contract.

Slims the duplicate Operator-flow paragraphs for Telegram and Discord into a
single one-block pointer that links to the new reference section.
2026-04-27 13:48:03 -04:00
Gustavo Madeira Santana
dd1a94f089 docs(qa): reorg, audit against code, and refresh stale content
Reorg
- Rename the architecture page title to "QA overview" (slug stays
  /concepts/qa-e2e-automation so inbound links keep working).
- Move "Adding a channel to QA" + scenario-helper-name reference from
  testing.md into qa-e2e-automation.md under "Transport adapters". Architecture
  belongs with the architecture page.
- Drop the duplicate live-transport coverage table from testing.md; canonical
  copy stays in qa-e2e-automation.md under a new "Live transport coverage"
  heading so qa-matrix.md can deep-link to it.
- Slim testing.md QA-specific runners section to ops only, with cross-links.

Audit (against extensions/qa-lab/src/cli.ts, qa-channel/src/config-schema.ts,
and live-transport runtimes)
- qa-e2e-automation.md gains a "Command surface" table covering all 14
  openclaw qa <subcommand> forms; previously only ~7 of 14 were named.
- Document missing OPENCLAW_QA_TELEGRAM_CAPTURE_CONTENT and
  OPENCLAW_QA_DISCORD_CAPTURE_CONTENT env vars (Matrix already had it).
- Cross-link qa coverage from the Reporting section.
- qa-channel.md completes the config-key list (enabled, name, accounts,
  defaultAccount were missing from the schema doc) and pollTimeoutMs range.
- Drop stale "Follow-up work" framing in qa-channel.md (provider/model matrix,
  scenario discovery, orchestration) — all three already shipped.
- Replace "vertical slice" language with current behavior; fix misplaced
  debugger-UI paragraph.

Discoverability
- Add a Note callout to testing.md pointing at the three QA pages
  (QA overview, Matrix QA, QA channel) so maintainers landing on testing.md
  see the QA stack in the prologue.

Glossary entries for the renamed/new doc titles.
2026-04-27 13:40:11 -04:00
Peter Steinberger
6987132aed ci: add Matrix QA profiles 2026-04-27 05:43:14 +01:00
Peter Steinberger
7ca2f9fed5 test(docker): align package harness image 2026-04-27 01:22:58 +01:00
Peter Steinberger
42db865673 test(docker): run observability on shared image 2026-04-27 00:49:36 +01:00
Vincent Koc
5d7c6e6bda test(docker): add observability smoke
Add Docker aggregate observability coverage for QA-lab OTEL and Prometheus diagnostics.
2026-04-26 16:43:56 -07:00
Vincent Koc
82ddcf24f5 feat(diagnostics): add harness lifecycle telemetry 2026-04-25 23:34:34 -07:00
Vincent Koc
a0ca546997 test(qa): add local otel smoke harness 2026-04-25 19:30:46 -07:00
Peter Steinberger
768bbc7cc0 docs: update OpenAI GPT-5.5 API guidance 2026-04-25 18:14:10 +01:00
Peter Steinberger
88c91675e2 test: stabilize qa suite concurrency 2026-04-24 20:39:33 +01:00
Patrick Erichsen
3a18801343 Add Discord live QA lane (#70792)
* Add Discord live QA lane

* Add Discord native command QA coverage
2026-04-23 19:48:37 -07:00
Peter Steinberger
903308dbf2 fix: stabilize qa lab mock suite 2026-04-24 02:46:33 +01:00
Peter Steinberger
14c4143723 docs: clarify OpenAI GPT-5.5 auth routes 2026-04-23 23:49:17 +01:00
Vincent Koc
2777b089b5 docs: normalize frontmatter titles to sentence case 2026-04-23 13:15:17 -07:00
Vincent Koc
4a2cd533ac docs: remove duplicate H1 where frontmatter title already sets it 2026-04-23 13:11:14 -07:00
Peter Steinberger
89051c6bf6 docs(openai): document GPT-5.5 defaults 2026-04-23 20:19:15 +01:00
Ayaan Zaidi
c22a21759b test(qa): report telegram reply rtt 2026-04-23 14:33:17 +05:30
Josh Avant
d5b326523f qa-lab: make live lanes CI-ready for v1 E2E automation (#69122)
* qa-lab: harden CI defaults and failure semantics for live lanes

* qa-lab: add unit tests for suite progress logging defaults

* qa-lab: cover malformed multipass summary edge cases

* qa-lab: share suite summary failure counting helper

* qa-lab: test allow-failures parse wiring and sanitize progress ids

* fix: note qa CI live-lane defaults in changelog (#69122) (thanks @joshavant)
2026-04-19 21:13:27 -05:00
Gustavo Madeira Santana
82fe6f50ef QA: organize scenarios by theme 2026-04-17 11:03:47 -04:00
Gustavo Madeira Santana
bb7e9823a8 QA Lab: add provider registry
Move mock and live provider behavior behind provider-owned definitions so suite, manual, Matrix, and transport lanes share defaults, auth staging, model config, and standalone server startup.

Add AIMock as a first-class local provider mode while keeping mock-openai as the scenario-aware deterministic lane.
2026-04-16 21:24:59 -04:00
Gustavo Madeira Santana
56a9fd4b34 QA Matrix: capture full runner output 2026-04-16 16:18:54 -04:00
Peter Steinberger
20266c14cb feat(qa-lab): add control ui qa-channel roundtrip scenario 2026-04-12 19:41:06 -07:00
Peter Steinberger
ca2d297c50 docs(qa): clarify markdown scenario contract 2026-04-12 11:59:50 -07:00
Marcus Castro
000fc7f233 refactor(qa): add shared QA channel contract and harden worker startup (#64562)
* refactor(qa): add shared transport contract and suite migration

* refactor(qa): harden worker gateway startup

* fix(qa): scope waits and sanitize shutdown artifacts

* fix(qa): confine artifacts and redact preserved logs

* fix(qa): block symlink escapes in artifact paths

* fix(gateway): clear shutdown race timers

* fix(qa): harden shutdown cleanup paths

* fix(qa): sanitize gateway logs in thrown errors

* fix(qa): harden suite startup and artifact paths

* fix(qa): stage bundled plugins from mutated config

* fix(qa): broaden gateway log bearer redaction

* fix(qa-channel): restore runtime export

* fix(qa): stop failed gateway startups as a process tree

* fix(qa-channel): load runtime hook from api surface
2026-04-12 15:02:57 -03:00
Gustavo Madeira Santana
25445a9f2e qa-lab: add Matrix live transport QA lane (#64489)
Merged via squash.

Prepared head SHA: ae9bb37751
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
2026-04-10 19:35:08 -04:00
Peter Steinberger
3027efaf21 test: raise QA suite default concurrency 2026-04-10 13:45:57 +01:00
Peter Steinberger
4c14f55c62 test: parallelize QA suite scenarios 2026-04-10 13:45:57 +01:00
Shakker
655cfb477a docs: clarify multipass live auth support 2026-04-09 23:53:13 +01:00
Shakker
a04c331cc1 docs: document qa multipass runner 2026-04-09 23:53:13 +01:00
Peter Steinberger
be46d0ddc6 test: update character eval public panel 2026-04-09 01:25:59 +01:00
Peter Steinberger
39cc6b7dc7 fix: stabilize character eval and Qwen model routing 2026-04-09 01:04:09 +01:00