Vincent Koc
d1b268f7f7
fix(qa): normalize completed wait envelopes
2026-06-23 05:13:35 +02:00
Vincent Koc
2205f50016
test(qa): satisfy history reply lint
2026-06-23 04:01:11 +02:00
Vincent Koc
d716dfd532
test(qa): wait for live history replies in flow scenarios
2026-06-23 04:01:11 +02:00
Vincent Koc
5822e8074d
test(qa): accept completed agent wait status
2026-06-23 04:01:11 +02:00
Dallin Romney
27711b500c
ci: add maturity scorecard renderer ( #94272 ) ( #95901 )
...
* ci: add maturity scorecard renderer
* ci: render qa scorecard evidence
* ci: type maturity docs renderer
* ci: tighten maturity artifact inputs
* ci: move maturity renderer under qa scripts
* ci: share maturity score schema
* ci: centralize maturity taxonomy validation
* ci: move maturity scores under qa
* ci: remove docs maturity score source
* docs: simplify maturity scorecard output
* docs: commit generated maturity scorecard
* docs: group maturity pages
* docs: simplify maturity scorecard dates
* docs: promote maturity nav tab
* docs: clean up maturity pages
* docs: remove maturity outline page
* docs: filter maturity taxonomy doc links
* docs: simplify maturity taxonomy tables
* docs: keep artifact taxonomy links
* docs: simplify lts scorecard display
* docs: clarify maturity score definitions
* docs: derive maturity coverage from evidence
* docs: hide maturity scorecard until evidence
* docs: remove placeholder maturity pages
* docs: keep maturity scores out of pr
* ci: open maturity scorecard docs pr
2026-06-22 18:55:06 -07:00
Vincent Koc
def4b51485
fix(qa): gate smoke profile scenarios by channel driver
2026-06-23 09:34:52 +08:00
Vincent Koc
43f2b61f3b
test(qa): keep image generation fixture on mock lane
2026-06-23 02:35:02 +02:00
Vincent Koc
086c629556
test(qa): scope provider-sensitive flow fixtures
2026-06-23 02:17:20 +02:00
Vincent Koc
befe04f465
test(qa): accept Sonnet max thinking support
2026-06-23 01:57:43 +02:00
Vincent Koc
264b37e9d2
test(qa): avoid redacted config cleanup patch
2026-06-23 01:39:39 +02:00
Dallin Romney
de9c94cbbb
feat(qa): forward shared suite flags to multipass runner ( #91506 )
2026-06-22 15:48:05 -07:00
Dallin Romney
63b13ea837
feat(qa): crabline channel driver ( #91502 )
...
* feat(qa): add crabline channel driver seam
* feat: run crabline channel driver smoke
* chore: keep crabline qa dependency dev-only
* refactor(qa): keep crabline driver details opaque
* chore(qa): pin crabline to merged driver API
* feat(qa): drive channel driver from profiles
* fix(qa): declare crabline runtime peer
* feat(qa): resolve crabline channel from scenarios
* feat(qa): treat unsupported profile channels as coverage gaps
* Revert "feat(qa): treat unsupported profile channels as coverage gaps"
This reverts commit 65a9701655 .
* fix(qa): adapt crabline driver to chat sdk cli
* refactor(qa): pass channel driver metadata directly
* chore(qa): update crabline provider pin
* chore(qa): default channel scenarios to driver
* chore: repair qa dependency lockfile
* chore: allow native qa dependency builds
* fix(qa): satisfy crabline driver lint
* fix(qa): satisfy crabline ci gates
* Use crabline transport for smoke QA profile
* fix(qa): keep crabline driver opt-in
* fix(qa): reuse crabline telegram driver token
* fix(qa): route smoke profile through crabline
* fix(qa): run full smoke profile lane
* fix(qa): remove smoke scenario workflow filter
* fix: stabilize crabline smoke qa profile
* fix: pin crabline qa dependency
* test: keep crabline smoke credential-free
* fix: skip visible reasoning lane for crabline smoke
* fix: unblock crabline qa ci
* Update crabline dependency
* Pin crabline to merged main
* Use Crabline fake provider servers
2026-06-22 15:24:59 -07:00
Dallin Romney
ed2dfee7d7
feat(qa): expose active memory toggles to scenarios ( #95858 )
2026-06-22 14:26:37 -07:00
Vincent Koc
96c6f8022c
fix(auto-reply): suppress quoted silent replies
2026-06-23 02:24:29 +08:00
Vincent Koc
a4eb49a176
refactor(qa): share gateway message text extraction
2026-06-22 20:25:26 +08:00
Vincent Koc
db21588636
refactor(qa): share suite summary file loading
2026-06-22 20:24:20 +08:00
Vincent Koc
d3f7f7d1fc
chore(deadcode): remove unused test-only helpers
2026-06-22 15:48:43 +08:00
Vincent Koc
7cc21ef59d
fix(qa): stabilize smoke-ci scenarios
2026-06-22 15:41:53 +08:00
Vincent Koc
abdd81db11
test(qa): include long-context soak scenario
2026-06-22 14:28:04 +08:00
Vincent Koc
4506d8bad6
fix(qa): surface tool-search bridge results
2026-06-22 08:01:44 +02:00
Vincent Koc
fd13192adc
fix(qa-lab): version UI assets from repo root
2026-06-21 23:43:48 +02:00
Vincent Koc
1c63da09d8
fix(qa-lab): cap agent wait timeout payloads
2026-06-21 23:34:38 +02:00
Marcus Castro
b50a5aebba
fix(whatsapp): preserve native quote replies ( #95483 )
...
* fix(whatsapp): preserve native quote replies
* feat(plugin-sdk): add quote-reply live transport standard
* test(qa-lab): add WhatsApp quote-reply live scenarios
2026-06-21 11:26:58 -03:00
Vincent Koc
8a7906c716
fix(qa): reject fractional live token usage
2026-06-21 14:49:29 +02:00
Vincent Koc
c85113e30e
fix(qa): escape tool coverage markdown cells
2026-06-21 14:46:38 +02:00
Vincent Koc
d9dfcd6c8a
fix(qa): reject impossible confidence counts
2026-06-21 14:38:42 +02:00
Vincent Koc
5e86c7eef4
fix(qa): reject non-object mock Anthropic JSON
2026-06-21 13:13:37 +02:00
Vincent Koc
6f3af56952
fix(qa): reject non-object mock OpenAI JSON
2026-06-21 13:10:50 +02:00
Vincent Koc
f69ba12a37
fix(qa): reject malformed mock OpenAI JSON
2026-06-21 12:58:20 +02:00
Vincent Koc
b5fc9514c0
fix(qa): reject coerced evidence artifact indexes
2026-06-21 12:50:37 +02:00
Vincent Koc
600bace853
fix(qa): validate QA bus poll numbers
2026-06-21 12:41:22 +02:00
Vincent Koc
b8a5dac1a2
fix(qa-lab): resolve Windows PowerShell path
2026-06-21 12:40:57 +02:00
Vincent Koc
66e5cfdd86
fix(qa): reject QA Lab host port collisions
2026-06-21 12:23:52 +02:00
Vincent Koc
a70b34a3cb
fix(qa-lab): resolve Windows node lookup tool
2026-06-21 12:14:39 +02:00
Vincent Koc
735d70d9db
test(extensions): pin Windows taskkill test roots
2026-06-21 11:37:59 +02:00
Vincent Koc
61d1fd1f72
fix(qa-lab): resolve Windows taskkill path
2026-06-21 11:26:30 +02:00
Dallin Romney
5dd30c3995
test: fold HTTP API script proof into QA Lab ( #94700 )
...
* test: fold HTTP API script proof into QA Lab
* test: remove folded HTTP API script tests
* test: relax QA native scenario catalog inventory
* test: trim folded QA Lab script cruft
* test: align folded QA coverage ids
* test: keep native QA evidence out of parity tiers
* test: update mirrored QA routing expectation
* test: preserve chat tools profile build guard
* test: avoid overclaiming gateway tool API coverage
* test: pin folded QA coverage ids
2026-06-20 23:10:35 -07:00
Vincent Koc
5b22409389
fix(qa-lab): taskkill gateway children on windows
2026-06-21 08:07:00 +02:00
Vincent Koc
b970d57175
fix(qa-lab): taskkill timed-out cli trees on windows
2026-06-21 08:01:55 +02:00
Vincent Koc
c43822077a
fix(qa-lab): force taskkill scenario trees on windows
2026-06-21 07:56:03 +02:00
Vincent Koc
ed500dda25
fix(qa): use corepack for lab docker build fallback
2026-06-20 17:45:09 +02:00
Vincent Koc
7fc5a72433
fix(qa): cap chunked credential lease payloads
2026-06-20 17:34:38 +02:00
Vincent Koc
c5623e72f3
fix(qa): quote generated compose paths
2026-06-20 17:08:40 +02:00
Vincent Koc
947c21ee5a
refactor(qa): reuse qa shell quote helper
2026-06-20 17:05:10 +02:00
Vincent Koc
99f58ae6d6
fix(qa): quote qa docker stop command
2026-06-20 16:59:14 +02:00
Vincent Koc
a6e41a0cc1
fix(qa-lab): kill script timeout trees on windows
2026-06-20 12:58:52 +02:00
Vincent Koc
1ede829fbf
fix(qa-lab): leave vitest timeout cleanup to wrapper
2026-06-20 12:53:39 +02:00
Vincent Koc
b93b07ee1b
test(qa-lab): use temp harness in scenario runner tests
2026-06-20 12:53:39 +02:00
Vincent Koc
405e5072fd
fix(qa-lab): bound test file scenario commands
2026-06-20 12:53:39 +02:00
Vincent Koc
88cf142c98
fix(qa-lab): preserve model catalog abort grace
2026-06-20 11:18:43 +02:00