Commit Graph

13 Commits

Author SHA1 Message Date
Peter Steinberger
9ef37d1907 test: tighten assertions and harness coverage 2026-05-08 05:28:12 +01:00
Peter Steinberger
f632f5e60b feat(qa): add mantis Slack desktop smoke 2026-05-04 03:47:27 +01:00
Peter Steinberger
4856cbb017 feat(qa): publish Mantis desktop screenshots 2026-05-04 01:52:14 +01:00
Peter Steinberger
57b2d29761 feat(qa): add Mantis desktop browser smoke 2026-05-04 01:30:20 +01:00
Peter Steinberger
d4af125b52 feat(qa): add Mantis before-after CLI 2026-05-03 21:27:43 +01:00
Peter Steinberger
0bf06e953f feat: add Mantis Discord smoke runner (#76696)
* docs: add Mantis QA system design

* feat: add Mantis Discord smoke runner

* fix: harden Mantis Discord smoke

* fix: redact Mantis Discord artifacts

* fix: satisfy Mantis redaction lint

* fix: redact Mantis mismatch failures

* test: avoid promise assertions in Mantis tests
2026-05-03 15:25:56 +01:00
Josh Avant
d5b326523f qa-lab: make live lanes CI-ready for v1 E2E automation (#69122)
* qa-lab: harden CI defaults and failure semantics for live lanes

* qa-lab: add unit tests for suite progress logging defaults

* qa-lab: cover malformed multipass summary edge cases

* qa-lab: share suite summary failure counting helper

* qa-lab: test allow-failures parse wiring and sanitize progress ids

* fix: note qa CI live-lane defaults in changelog (#69122) (thanks @joshavant)
2026-04-19 21:13:27 -05:00
Gustavo Madeira Santana
3a1e469732 QA: track scenario coverage intent 2026-04-17 14:05:49 -04:00
Gustavo Madeira Santana
bb7e9823a8 QA Lab: add provider registry
Move mock and live provider behavior behind provider-owned definitions so suite, manual, Matrix, and transport lanes share defaults, auth staging, model config, and standalone server startup.

Add AIMock as a first-class local provider mode while keeping mock-openai as the scenario-aware deterministic lane.
2026-04-16 21:24:59 -04:00
Gustavo Madeira Santana
85eac42d34 QA: remove runner install fallback catalog
Drop the generated qa-runner catalog and the missing/install placeholder
path for repo-private QA runners. The host should discover bundled QA
commands from manifest plus runtime surface only.

Also trim stale qa-matrix install docs and package metadata so the
source-only QA policy stays consistent.
2026-04-14 17:37:18 -04:00
Gustavo Madeira Santana
82a2db71e8 refactor(qa): split Matrix QA into optional plugin (#66723)
Merged via squash.

Prepared head SHA: 27241bd089
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
2026-04-14 16:28:57 -04:00
Josh Avant
3d07dfbb65 feat(qa-lab): add Convex credential broker and admin CLI (#65596)
* QA Lab: add Convex credential source for Telegram lane

* QA Lab: scaffold Convex credential broker

* QA Lab: add Convex credential admin CLI

* QA Lab: harden Convex credential security paths

* QA Broker: validate Telegram payloads on admin add

* fix: note QA Convex credential broker in changelog (#65596) (thanks @joshavant)
2026-04-12 22:03:42 -05:00
Gustavo Madeira Santana
25445a9f2e qa-lab: add Matrix live transport QA lane (#64489)
Merged via squash.

Prepared head SHA: ae9bb37751
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
2026-04-10 19:35:08 -04:00