Peter Steinberger
b835337cd6
test: filter live qa scenario lanes
2026-04-21 12:43:30 +01:00
Peter Steinberger
05ba1335d9
fix: tolerate qa cli json startup logs
2026-04-21 07:37:48 +01:00
Peter Steinberger
6ce17db11a
fix: gate max thinking by model support
2026-04-21 07:02:43 +01:00
Peter Steinberger
f5be489266
test: add gpt-5.4 thinking visibility QA
2026-04-21 06:13:39 +01:00
Peter Steinberger
f1a544ef6d
perf: avoid sort-for-single selection
2026-04-20 23:20:31 +01:00
Peter Steinberger
dc4e90bbd2
fix(qa-lab): restore transport helper contracts
2026-04-20 21:03:34 +01:00
Peter Steinberger
fbba29319f
refactor: share qa credential helpers
2026-04-20 21:00:42 +01:00
Peter Steinberger
0882b85d5a
refactor: share qa runtime helpers
2026-04-20 20:58:28 +01:00
Peter Steinberger
6fbfb8b7a3
test: share character eval fixtures
2026-04-20 20:56:30 +01:00
Peter Steinberger
f587887122
test: share qa temp dir harness
2026-04-20 20:54:45 +01:00
Peter Steinberger
b97f993b0c
test: share qa docker healthy deps
2026-04-20 18:07:04 +01:00
Peter Steinberger
98a5f737d7
perf(test): streamline qa gateway child tests
2026-04-20 16:14:15 +01:00
Peter Steinberger
1148f245c8
build(deps): declare extension runtime dependencies
2026-04-20 16:07:14 +01:00
Peter Steinberger
09171eee8d
perf(test): shorten qa channel readiness polling
2026-04-20 16:00:59 +01:00
Peter Steinberger
e50b8e3e99
perf(test): mock qa lab capture store
2026-04-20 15:55:08 +01:00
Peter Steinberger
372ca5e81e
perf(test): skip manual lane settle delay in unit tests
2026-04-20 15:51:28 +01:00
Peter Steinberger
44030ac4fd
perf(test): cache qa scenario catalog
2026-04-20 15:17:07 +01:00
Peter Steinberger
beff874340
perf(test): trim active memory and qa lab hotspots
2026-04-20 15:04:38 +01:00
Peter Steinberger
869950564f
build: update dependencies
2026-04-20 13:18:32 +01:00
Peter Steinberger
8116e638f3
chore: release 2026.4.20
2026-04-20 13:16:40 +01:00
Gustavo Madeira Santana
ea37a833dc
test(matrix): add stale sync replay dedupe scenario
2026-04-20 05:51:04 -04:00
Josh Avant
d5b326523f
qa-lab: make live lanes CI-ready for v1 E2E automation ( #69122 )
...
* qa-lab: harden CI defaults and failure semantics for live lanes
* qa-lab: add unit tests for suite progress logging defaults
* qa-lab: cover malformed multipass summary edge cases
* qa-lab: share suite summary failure counting helper
* qa-lab: test allow-failures parse wiring and sanitize progress ids
* fix: note qa CI live-lane defaults in changelog (#69122 ) (thanks @joshavant)
2026-04-19 21:13:27 -05:00
Peter Steinberger
ac8f0c9c0d
chore: prepare 2026.4.19-beta.1 release
2026-04-19 02:09:43 +01:00
Peter Steinberger
4fa961d4f1
refactor(lint): enable map spread rule
2026-04-18 20:37:12 +01:00
Peter Steinberger
1f1ff0567a
refactor(lint): reduce map spread patterns
2026-04-18 19:27:43 +01:00
Peter Steinberger
5cc4426f88
test: align qa multipass pnpm expectation
2026-04-18 18:39:03 +01:00
Peter Steinberger
753183e081
build(deps): update workspace dependencies
2026-04-18 18:04:56 +01:00
Peter Steinberger
66385670e4
refactor: reduce unnecessary dynamic imports
2026-04-18 16:15:33 +01:00
Peter Steinberger
3f2e73b723
chore(release): bump version to 2026.4.18
2026-04-18 15:46:33 +01:00
Gustavo Madeira Santana
a50ec27d3b
Tests: speed up QA lab startup
2026-04-17 22:19:17 -04:00
Vincent Koc
5edf876a5e
test(auth): add codex oauth red-blue coverage
2026-04-17 14:14:03 -07:00
Gustavo Madeira Santana
3a1e469732
QA: track scenario coverage intent
2026-04-17 14:05:49 -04:00
Gustavo Madeira Santana
82fe6f50ef
QA: organize scenarios by theme
2026-04-17 11:03:47 -04:00
Peter Steinberger
24431e5114
build: declare qa-lab aimock runtime dependency
2026-04-17 02:57:18 +01:00
Gustavo Madeira Santana
bb7e9823a8
QA Lab: add provider registry
...
Move mock and live provider behavior behind provider-owned definitions so suite, manual, Matrix, and transport lanes share defaults, auth staging, model config, and standalone server startup.
Add AIMock as a first-class local provider mode while keeping mock-openai as the scenario-aware deterministic lane.
2026-04-16 21:24:59 -04:00
Peter Steinberger
0dc4c4076c
chore: bump version to 2026.4.16
2026-04-17 00:45:04 +01:00
Gustavo Madeira Santana
4db162db7f
QA: split lab runtime and extend Matrix coverage ( #67430 )
...
Merged via squash.
Prepared head SHA: 790418b93b5599352+gumadeiras@users.noreply.github.com >
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com >
Reviewed-by: @gumadeiras
2026-04-16 03:08:39 -04:00
Gustavo Madeira Santana
8ecb6bbb12
QA: accept nodejs as Node runtime
2026-04-16 02:39:52 -04:00
Gustavo Madeira Santana
d5933af80b
QA: fix private runtime source loading ( #67428 )
...
Merged via squash.
Prepared head SHA: b8bf2b6be65599352+gumadeiras@users.noreply.github.com >
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com >
Reviewed-by: @gumadeiras
2026-04-15 21:59:16 -04:00
Peter Steinberger
b3fa5880dd
build(extensions): bump bundled plugin versions to 2026.4.15-beta.1
2026-04-15 15:06:13 +01:00
Gustavo Madeira Santana
963ad1df06
QA: extend Matrix live contract coverage
2026-04-15 07:36:35 -04:00
Mason Huang
edfa074e0f
Tests: align pnpm test expectations with main ( #67001 )
...
Merged via squash.
Prepared head SHA: 29c80680538814856+hxy91819@users.noreply.github.com >
Co-authored-by: hxy91819 <8814856+hxy91819@users.noreply.github.com >
Reviewed-by: @hxy91819
2026-04-15 18:31:23 +08:00
Gustavo Madeira Santana
fb92ca1a4d
QA: genericize mock streaming fixtures
2026-04-14 23:44:41 -04:00
Gustavo Madeira Santana
85eac42d34
QA: remove runner install fallback catalog
...
Drop the generated qa-runner catalog and the missing/install placeholder
path for repo-private QA runners. The host should discover bundled QA
commands from manifest plus runtime surface only.
Also trim stale qa-matrix install docs and package metadata so the
source-only QA policy stays consistent.
2026-04-14 17:37:18 -04:00
Gustavo Madeira Santana
653100488d
QA: fix matrix runner staging and host registration
2026-04-14 17:18:25 -04:00
Gustavo Madeira Santana
82a2db71e8
refactor(qa): split Matrix QA into optional plugin ( #66723 )
...
Merged via squash.
Prepared head SHA: 27241bd0895599352+gumadeiras@users.noreply.github.com >
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com >
Reviewed-by: @gumadeiras
2026-04-14 16:28:57 -04:00
Peter Steinberger
d2240a9476
test: harden qa-lab concurrent web scenarios
2026-04-14 13:42:02 +01:00
Vincent Koc
33a698fe10
fix(qa-lab): correct scenario catalog type
2026-04-14 09:39:20 +01:00
Vincent Koc
900681751d
test(qa-lab): seed broken-turn recovery scenarios ( #66416 )
2026-04-14 09:03:49 +01:00
Vincent Koc
e63cbe831b
test(qa-lab): cover GPT-style broken turns
2026-04-14 01:39:49 +01:00
