Commit Graph

26 Commits

Author SHA1 Message Date
Dallin Romney
c1b9e36221 test(qa): migrate channel thread and DM isolation scenarios (#101112)
* test(qa): migrate channel thread isolation scenarios

* test(qa): remove stale Matrix scenario fixture

* test(qa): use canonical coverage taxonomy ids

* test(qa): preserve Matrix flow runtime identities

* test(qa): expose Slack thread polling helper

* test(qa): select Matrix for native thread smoke

* test(qa): keep Matrix subagent spawn opt-in

* test(qa): target selected live account in scenario patches
2026-07-06 17:52:38 -07:00
Dallin Romney
66b4dcf184 chore(docker): execute Compose and package artifact proofs (#101045)
* test(docker): prove Compose and package artifacts

* test(docker): use bounded container cleanup

* test(docker): bound package proof launch

* test(qa): keep Docker artifact proofs release-only

* test(docker): simplify artifact proof names

* test(docker): satisfy identity writer lint
2026-07-06 13:05:34 -07:00
Vincent Koc
e75223286d fix(qa): require live channel driver for access-gate scenarios 2026-07-06 21:34:27 +02:00
Shakker
65e12328aa feat: refactor the Control UI architecture
Refactor the Control UI around route-owned page lifecycle and state while preserving existing behavior and design.

Prepared head SHA: bd51b6fa76
Co-authored-by: Shakker <165377636+shakkernerd@users.noreply.github.com>
Reviewed-by: @shakkernerd
2026-07-04 23:19:38 +01:00
Dallin Romney
677c8ff846 improve: enforce canonical QA scenario ownership (#99628)
* improve: enforce canonical QA scenario ownership

* docs: refresh QA documentation map

* docs: condense QA ownership guidance

* docs: remove QA migration guidance

* refactor: simplify QA scenario ownership checks

* docs: align QA coverage authoring guidance
2026-07-03 15:51:18 -07:00
Dallin Romney
15bfadf2bd fix: count maturity coverage ids (#96543) 2026-06-24 14:56:51 -07:00
Vincent Koc
c8ca44739a fix(qa): exclude archived categories from all profile 2026-06-23 12:08:33 +08:00
Dallin Romney
4460fa78c3 feat(qa): add "all" taxonomy profile (#95947)
* qa: add all maturity profile

* test: update qa coverage profile expectations
2026-06-22 20:49:08 -07:00
Dallin Romney
27711b500c ci: add maturity scorecard renderer (#94272) (#95901)
* ci: add maturity scorecard renderer

* ci: render qa scorecard evidence

* ci: type maturity docs renderer

* ci: tighten maturity artifact inputs

* ci: move maturity renderer under qa scripts

* ci: share maturity score schema

* ci: centralize maturity taxonomy validation

* ci: move maturity scores under qa

* ci: remove docs maturity score source

* docs: simplify maturity scorecard output

* docs: commit generated maturity scorecard

* docs: group maturity pages

* docs: simplify maturity scorecard dates

* docs: promote maturity nav tab

* docs: clean up maturity pages

* docs: remove maturity outline page

* docs: filter maturity taxonomy doc links

* docs: simplify maturity taxonomy tables

* docs: keep artifact taxonomy links

* docs: simplify lts scorecard display

* docs: clarify maturity score definitions

* docs: derive maturity coverage from evidence

* docs: hide maturity scorecard until evidence

* docs: remove placeholder maturity pages

* docs: keep maturity scores out of pr

* ci: open maturity scorecard docs pr
2026-06-22 18:55:06 -07:00
Dallin Romney
63b13ea837 feat(qa): crabline channel driver (#91502)
* feat(qa): add crabline channel driver seam

* feat: run crabline channel driver smoke

* chore: keep crabline qa dependency dev-only

* refactor(qa): keep crabline driver details opaque

* chore(qa): pin crabline to merged driver API

* feat(qa): drive channel driver from profiles

* fix(qa): declare crabline runtime peer

* feat(qa): resolve crabline channel from scenarios

* feat(qa): treat unsupported profile channels as coverage gaps

* Revert "feat(qa): treat unsupported profile channels as coverage gaps"

This reverts commit 65a9701655.

* fix(qa): adapt crabline driver to chat sdk cli

* refactor(qa): pass channel driver metadata directly

* chore(qa): update crabline provider pin

* chore(qa): default channel scenarios to driver

* chore: repair qa dependency lockfile

* chore: allow native qa dependency builds

* fix(qa): satisfy crabline driver lint

* fix(qa): satisfy crabline ci gates

* Use crabline transport for smoke QA profile

* fix(qa): keep crabline driver opt-in

* fix(qa): reuse crabline telegram driver token

* fix(qa): route smoke profile through crabline

* fix(qa): run full smoke profile lane

* fix(qa): remove smoke scenario workflow filter

* fix: stabilize crabline smoke qa profile

* fix: pin crabline qa dependency

* test: keep crabline smoke credential-free

* fix: skip visible reasoning lane for crabline smoke

* fix: unblock crabline qa ci

* Update crabline dependency

* Pin crabline to merged main

* Use Crabline fake provider servers
2026-06-22 15:24:59 -07:00
Dallin Romney
e12cf72b17 Standardize QA coverage IDs on dotted names (#94702)
* fix: standardize qa coverage ids

* test: avoid qa coverage id assertion spread
2026-06-18 17:25:26 -07:00
Dallin Romney
e17d111990 fix: require all taxonomy coverage ids (#94296) 2026-06-17 16:38:14 -07:00
Colin Johnson
591313e80a qa-lab: support script-backed evidence scenarios (#94276)
* qa: add script scenario execution kind

* fix(qa-lab): carry suite profile into script producer evidence and simplify artifact path resolution

* fix(qa-lab): keep out-of-repo producer artifacts absolute to avoid ../ traversal refs

---------

Co-authored-by: Dallin Romney <dallinromney@gmail.com>
2026-06-17 15:09:25 -07:00
Dallin Romney
450060d7a2 test(qa): expand smoke-ci and release categories and coverage (#93175)
* test(qa): add smoke ci primary coverage evidence

* test(qa): remove overstated primary coverage claims

* test(qa): make release profile include smoke ci

* test(qa): trim taxonomy formatting churn

* test(qa): avoid hardcoded profile names in coverage test

* test(qa): make release profile cover taxonomy

* test(qa): type profile fixture all category flag

* test(qa): include channel delivery in smoke ci profile
2026-06-15 18:05:52 -07:00
Dallin Romney
1f8c4d3958 simplify QA evidence profile and mappings/coverage shape (#93153)
* test(qa): simplify evidence coverage shape

* test(qa): collapse evidence scorecard metadata

* test(qa): document evidence schema version
2026-06-14 22:26:58 -07:00
Dallin Romney
fef8394079 Convert QA scenarios to YAML files (#92915)
* refactor: load QA scenarios from YAML

* docs: update personal QA scenario docs

* test: keep QA scenarios YAML-only
2026-06-14 17:31:18 -07:00
Dallin Romney
a3e9dfee0e Simplify QA scorecard mapping shape (#92558)
* test(qa): simplify scorecard mapping shape

* test(qa): use typed scorecard evidence refs

* test(qa): map scorecard categories by coverage id

* feat: align qa coverage with taxonomy features

* refactor: keep qa coverage ids canonical

* refactor: minimize qa coverage id churn

* test: align qa coverage id assertions

* test: update qa evidence coverage expectations

* refactor qa taxonomy coverage ids

* style qa taxonomy coverage ids

* test qa coverage lint fix

* test qa coverage type fix
2026-06-14 00:16:33 -07:00
Dallin Romney
561b293c7a Run Vitest and Playwright scenarios from qa suite (#92606)
* test(qa): run vitest and playwright scenarios from qa suite

* fix(qa): harden scenario suite dispatch

* refactor(qa): share scenario path utilities

* refactor(qa): share test file scenario runner

* refactor(qa): route test file scenarios through suite runtime

* refactor(qa): use explicit suite runtime result kind

* test(qa): write suite evidence artifact

* refactor(qa): clarify suite execution dispatch

* fix(qa): keep test-file scenarios out of flow-only runners

* refactor(qa): export mixed scenario suite runner
2026-06-13 01:06:10 -07:00
Dallin Romney
d8b3e523ff Add QA scorecard taxonomy validation (#91500)
Merged via squash.

Prepared head SHA: a9aec907d4
Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com>
Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com>
Reviewed-by: @RomneyDa
2026-06-12 17:07:51 -07:00
Peter Steinberger
58912f8fd8 docs: document channel extension sources 2026-06-04 21:59:00 -04:00
Vincent Koc
7f05be041e fix(diagnostics): harden observability exports and smokes (#85371)
* test(diagnostics): widen observability smokes

* fix(diagnostics): sanitize observability exports

* docs(diagnostics): format otel export docs
2026-05-23 15:27:43 +08:00
Vincent Koc
dcd98bf1ef test(qa-lab): report scenario pack coverage 2026-05-22 22:35:31 +08:00
Vincent Koc
fda0baf98d test(qa-lab): report live transport coverage lanes 2026-05-22 19:31:32 +08:00
Shakker
e6755fdef9 test: tighten qa and memory array assertions 2026-05-09 05:26:01 +01:00
Peter Steinberger
a3e48fd259 test: clarify qa coverage inventory assertions 2026-05-08 08:39:52 +01:00
Gustavo Madeira Santana
3a1e469732 QA: track scenario coverage intent 2026-04-17 14:05:49 -04:00