Vincent Koc
6922ddd936
fix(qa): isolate runner discovery facades ( #117203 )
2026-08-01 13:40:11 +08:00
Harjoth Khara
102c2e1a64
fix(slack): recover MPIM thread context from outbound replies ( #116660 )
...
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com >
2026-07-31 23:12:56 +08:00
Peter Steinberger
82231b425b
fix(qa-channel): isolate account evidence and exclude deleted messages ( #116744 )
...
* fix(qa): scope transport verdicts to visible account replies
* refactor(qa): remove obsolete outbound wait aliases
---------
Co-authored-by: Peter Steinberger <steipete@macos.shared >
2026-07-31 04:19:41 -07:00
Patrick Erichsen
61cf4b9246
test(buzz): add live QA Lab channel lane ( #116298 )
...
* feat(buzz): add live QA runner
* ci(qa): add gated Buzz live lane
* docs(qa): document Buzz live testing
* fix(buzz): keep QA relay type private
* test(buzz): type QA suite mock
* fix(buzz): preserve racing QA reply relations
* fix(qa): harden Buzz credential redaction
* fix(buzz): ignore retained QA relay events
* fix(buzz): clean up observer setup failures
* test(buzz): return complete racing messages
* fix(buzz): default QA credentials to file
* fix(qa): validate pooled Buzz secrets
* fix(qa): redact structured Buzz auth tags
* fix(buzz): preserve QA credential env override
* fix(qa): redact inspected Buzz auth tags
* test(buzz): limit live QA to supported scenarios
* ci(qa): run Buzz lane in release checks
* fix(qa): require secure remote Buzz relays
* fix(qa): satisfy Buzz QA validation gates
* fix(qa): inject credential leasing into transport adapters
2026-07-30 17:44:27 -07:00
joshavant
2e882fcf0d
docs(qa): record Tuwunel manifest provenance
2026-07-30 14:57:44 -05:00
joshavant
dd4728305f
test(qa-lab): use upload-safe Tuwunel fixture
2026-07-30 14:57:44 -05:00
joshavant
fe654eaacd
test(qa-lab): refresh Matrix Tuwunel fixture
2026-07-30 14:57:44 -05:00
Dallin Romney
714a46d7f2
fix(qa): align live scenario selection ( #115767 )
2026-07-30 16:59:01 +08:00
Dallin Romney
f027d7bcf6
refactor(qa): make Matrix sharding execution-owned ( #115589 )
...
* refactor(qa): make sharding execution-owned
* test(qa): simplify transport selection coverage
2026-07-29 15:23:37 +08:00
Dallin Romney
781744e125
fix(qa): run profile scenarios on every eligible channel ( #115540 )
...
* fix(qa): resolve channels for portable live scenarios
* fix(qa): expand profile scenarios across eligible channels
2026-07-29 13:14:37 +08:00
Peter Steinberger
dd5b9c8c04
fix(slack): prevent duplicate MPIM mention replies ( #115528 )
...
* fix(slack): make MPIM app mentions type-safe
Resolve typeless app mentions through explicit metadata, the scoped event cache, then conversations.info so modern C-prefixed MPIMs cannot be misclassified by event ordering. Add a real Slack QA scenario that requires one reply and cleans up its temporary MPIM.
Co-authored-by: moeealii <75953662+moeealii@users.noreply.github.com >
* fix(slack): type unresolved mention metadata
* fix(slack): satisfy MPIM type contracts
---------
Co-authored-by: moeealii <75953662+moeealii@users.noreply.github.com >
2026-07-29 00:23:06 -04:00
Dallin Romney
9feb6ad161
refactor(qa): make scenario selection taxonomy-owned ( #114055 )
...
* refactor(qa): remove scenario execution profiles
* fix(qa): preserve smoke CI membership
* test(qa): drop redundant smoke pack order check
* refactor(qa): remove Matrix runner profiles
* refactor(qa): simplify Matrix CI orchestration
* refactor(qa): derive execution sets from taxonomy
* fix(qa): keep Matrix shard helper private
* refactor(qa): derive profile scenarios from taxonomy
* refactor(qa): reconcile taxonomy selection
* refactor(qa): use canonical profile planning
* refactor(qa): select canonical coverage owners
* fix(qa): keep profile selector internal
* refactor(qa): remove representative coverage owners
2026-07-28 22:23:41 +08:00
Dallin Romney
9310586b09
fix(qa-lab): unify runner control plane ( #112838 )
2026-07-27 14:05:14 +08:00
Dallin Romney
132d91e427
refactor(qa): make taxonomy coverage ids unique ( #110280 )
2026-07-22 16:28:44 +09:00
Dallin Romney
3946bcc20c
fix(qa): run isolated Matrix evidence partitions in parallel ( #112465 )
...
* fix(qa): parallelize isolated Matrix evidence partitions
* refactor(qa): plan adapter instance concurrency
* fix(qa): let Docker assign Matrix harness ports
* refactor(qa): use instance isolation for Matrix concurrency
* test(ci): expect maturity concurrency six
* refactor(qa): remove Matrix harness manifest
2026-07-22 14:45:52 +09:00
Dallin Romney
9a2f25c82b
refactor(qa): make channel drivers run-level ( #108602 )
2026-07-16 00:59:03 -07:00
Dallin Romney
2d9b78fff7
refactor(qa): canonicalize live scenario coverage ( #108464 )
2026-07-15 16:50:41 -07:00
Dallin Romney
ca45a784f3
docs: align QA transport and release workflow contracts ( #108463 )
...
* docs: align QA transport workflow contracts
* docs: clarify Mantis env credentials
2026-07-15 14:51:37 -07:00
Dallin Romney
af71400c8e
refactor(qa): route live channel selectors through suite host ( #108429 )
...
* refactor(qa): route live channel selectors through suite host
* fix(qa): repair live selector CI
* fix(qa): remove retired runner exports
* fix(qa): remove obsolete live evidence
2026-07-15 13:35:34 -07:00
Dallin Romney
77d07dc3e9
refactor(qa): migrate Telegram scenarios into QA Lab ( #108430 )
...
* refactor(qa): migrate Telegram scenarios into QA Lab
* refactor(qa): remove retired Telegram runner exports
2026-07-15 12:55:00 -07:00
Dallin Romney
6c4e76ea8b
ci(qa): reuse Matrix live workflow for releases ( #103604 )
...
* ci(qa): reuse Matrix live workflow for releases
* docs(testing): restore Tailscale isolation flag
* fix(ci): make Matrix artifact names ref-safe
* test(ci): type Matrix workflow profiles
* style(testing): format Tailscale isolation guidance
2026-07-15 01:48:59 -07:00
Dallin Romney
ad34552473
refactor(qa): migrate Matrix scenarios into QA Lab ( #103589 )
...
* refactor(qa): migrate Matrix scenarios into QA Lab
* fix(qa): build Matrix boundary declarations
* fix(qa): preserve Matrix preview boundaries
* fix(qa): preserve Matrix hot reload semantics
* fix(qa): harden Matrix destructive scenario failures
* fix(qa): harden Matrix scenario isolation
* fix(qa): close Matrix negative scenario blind spots
* fix(qa): isolate Matrix substrate state
* fix(qa): harden Matrix transport substrate
* fix(qa): preserve Matrix profile and event parity
* fix(qa): preserve explicit scenario models
* fix(qa): align Matrix scenario coverage taxonomy
* fix(qa): format Matrix allowlist cleanup
* fix(qa): satisfy migrated Matrix CI contracts
* fix(qa): reconcile Matrix migration with current main
* fix(qa): break scenario flow import cycle
* fix(qa): reconcile Matrix max-lines ownership
* fix(qa): address Matrix review boundaries
* fix(qa): remove stale Matrix lint suppression
* fix(qa): adopt split Matrix E2EE flows
* fix(qa): export Matrix scenario record guard
* fix(qa): align Matrix migration with privatized helpers
* refactor(qa): finish Matrix QA Lab ownership
* fix(qa): preserve Matrix suite defaults
* fix(qa): reconcile Matrix cleanup with current main
* test(qa): follow canonical Matrix profile size
* fix(qa): guard stale Matrix QA package output
* docs(qa): redirect retired Matrix QA pages
* refactor(qa): finish Matrix runner rename
* test(qa): assert Matrix defaults through profile resolver
* docs: refresh QA cleanup map
* fix(qa): privatize Matrix storage discovery
2026-07-15 01:22:20 -07:00
Peter Steinberger
fe0a6db2b4
fix(slack): warn on disabled channel drops ( #105790 )
2026-07-12 17:50:14 -07:00
efrazer-oai
d09d200293
fix(slack): show commentary independently of tool progress ( #103995 )
...
* fix(slack): separate commentary progress from tools
* fix(slack): preserve legacy preamble defaults
* test(slack): add live commentary progress proof
* fix(slack): support scoped progress drafts
* test(slack): harden commentary progress proof
* test(slack): type commentary matrix cases
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com >
2026-07-12 17:52:53 +01:00
Peter Steinberger
934a974c29
feat(slack): support native data visualizations ( #104539 )
2026-07-11 09:18:39 -07:00
Peter Steinberger
fa77fe10d5
chore: migrate active GPT-5.5 references to GPT-5.6 ( #104452 )
...
* chore(models): migrate active GPT-5.5 references
* test(workboard): expect GPT-5.6 Sol default
* chore: keep release notes in PR body
* test(models): align picker fixtures with Sol default
* test: update PDF default model expectation
* test(qa): migrate thinking smoke to Luna
* test(gateway): align mock catalog with Sol default
* ci: retrigger exact-head PR checks
* test(gateway): document default catalog invariant
2026-07-11 06:30:57 -07:00
Dallin Romney
c1b9e36221
test(qa): migrate channel thread and DM isolation scenarios ( #101112 )
...
* test(qa): migrate channel thread isolation scenarios
* test(qa): remove stale Matrix scenario fixture
* test(qa): use canonical coverage taxonomy ids
* test(qa): preserve Matrix flow runtime identities
* test(qa): expose Slack thread polling helper
* test(qa): select Matrix for native thread smoke
* test(qa): keep Matrix subagent spawn opt-in
* test(qa): target selected live account in scenario patches
2026-07-06 17:52:38 -07:00
Dallin Romney
b29d472e41
refactor(qa): add canonical live channel adapters ( #99707 )
2026-07-06 09:24:34 -07:00
Gorkem Erdogan
e7c0673fc5
fix(slack): react action rejects emoji glyphs; member-info userId affordance unclear ( #100375 )
...
* fix(slack): normalize react emoji glyphs and clarify member-info userId param
Slack's reactions.add/remove only accept shortcode names, never a raw
Unicode glyph, but the react action's emoji param had no description
steering models away from passing one, so calls like
emoji="✅ " failed with invalid_name. Consolidates the glyph-to-shortcode
map that already existed privately in the ack-reaction dispatch path into
the shared normalizeSlackEmojiName export in actions.ts, the layer that
owns the actual Slack API calls, so the message-tool react action gets the
same normalization.
Also tightens the generic userId param description so models stop trying
target on member-info, which has no target mode and requires userId
directly.
* fix(slack): preserve emoji reaction semantics
* fix(slack): default reactions to inbound message
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com >
2026-07-06 00:15:32 +01:00
Kevin Lin
5fdaa04eca
feat(qa-lab): add Codex Slack approval scenarios ( #91519 )
...
* feat(qa-lab): add codex slack approval scenarios
* fix(qa-lab): harden Codex Slack approval proof
* fix(qa-lab): validate Codex approval model
* test(qa-lab): verify approved Codex operation
* fix(qa-lab): normalize Slack QA account routing
* fix(qa-lab): read Codex tool result transcripts
* fix(qa-lab): harden Codex approval cleanup
* fix(qa-lab): support Codex Mantis checkpoints
* fix(qa-lab): repair Mantis pnpm bootstrap
* fix(qa-lab): normalize Codex tool result text
* fix(qa-lab): scope Mantis pnpm fallback
* fix(qa-lab): bootstrap pinned pnpm without npm
* fix(qa-lab): terminate pnpm metadata record
* fix(qa-lab): bootstrap official node for mantis
* fix(qa-lab): make mantis pnpm shim executable
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com >
2026-07-05 22:47:18 +01:00
ooiuuii
1e7694a8a5
fix(whatsapp): serialize overlapping Web sends ( #99050 )
...
* fix(whatsapp): serialize web sendMessage calls
* fix(whatsapp): make send serialization socket-owned
* test(qa): narrow live gateway connection fields
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com >
2026-07-05 02:32:36 -04:00
Peter Steinberger
f7d7148cf0
docs: rewrite published docs grounded in current source ( #100142 )
...
Source-grounded rewrite of 529 published docs pages with per-unit information-loss verification: 1,713 factual corrections cited to src/**, generated surfaces regenerated, frontmatter titles preserved for i18n, release notes pages untouched. All docs gates green.
Closes #100141
2026-07-05 00:32:47 -04:00
Marcus Castro
fa3c9de459
test(qa-lab): harden whatsapp qa scenarios ( #95622 )
...
* fix(whatsapp): preserve group participant identity in QA driver
* fix(whatsapp): infer same-chat action targets
* test(qa-lab): record whatsapp scenario posture
* test(qa-lab): harden whatsapp live scenario coverage
* docs(qa): describe whatsapp qa lane coverage
2026-07-01 00:01:09 -03:00
Dallin Romney
5dae3d49e6
Update QA Lab Crabline integration ( #97941 )
...
* Update QA Lab Crabline integration
* Refresh docs map
2026-06-29 16:07:38 -07:00
Vincent Koc
ae06d846fa
docs(qa): clarify Matrix smoke provider mode
2026-06-24 19:02:57 +08:00
Dallin Romney
9666db607e
test(qa): clean up smoke taxonomy profile ( #96320 )
2026-06-24 00:43:00 -07:00
Dallin Romney
4a503ed45e
docs: add maturity docs routes ( #91483 )
2026-06-23 23:59:47 -07:00
Dallin Romney
4460fa78c3
feat(qa): add "all" taxonomy profile ( #95947 )
...
* qa: add all maturity profile
* test: update qa coverage profile expectations
2026-06-22 20:49:08 -07:00
Dallin Romney
63b13ea837
feat(qa): crabline channel driver ( #91502 )
...
* feat(qa): add crabline channel driver seam
* feat: run crabline channel driver smoke
* chore: keep crabline qa dependency dev-only
* refactor(qa): keep crabline driver details opaque
* chore(qa): pin crabline to merged driver API
* feat(qa): drive channel driver from profiles
* fix(qa): declare crabline runtime peer
* feat(qa): resolve crabline channel from scenarios
* feat(qa): treat unsupported profile channels as coverage gaps
* Revert "feat(qa): treat unsupported profile channels as coverage gaps"
This reverts commit 65a97016558705514a1c1ec74870de9b0a6e7886.
* fix(qa): adapt crabline driver to chat sdk cli
* refactor(qa): pass channel driver metadata directly
* chore(qa): update crabline provider pin
* chore(qa): default channel scenarios to driver
* chore: repair qa dependency lockfile
* chore: allow native qa dependency builds
* fix(qa): satisfy crabline driver lint
* fix(qa): satisfy crabline ci gates
* Use crabline transport for smoke QA profile
* fix(qa): keep crabline driver opt-in
* fix(qa): reuse crabline telegram driver token
* fix(qa): route smoke profile through crabline
* fix(qa): run full smoke profile lane
* fix(qa): remove smoke scenario workflow filter
* fix: stabilize crabline smoke qa profile
* fix: pin crabline qa dependency
* test: keep crabline smoke credential-free
* fix: skip visible reasoning lane for crabline smoke
* fix: unblock crabline qa ci
* Update crabline dependency
* Pin crabline to merged main
* Use Crabline fake provider servers
2026-06-22 15:24:59 -07:00
Marcus Castro
b50a5aebba
fix(whatsapp): preserve native quote replies ( #95483 )
...
* fix(whatsapp): preserve native quote replies
* feat(plugin-sdk): add quote-reply live transport standard
* test(qa-lab): add WhatsApp quote-reply live scenarios
2026-06-21 11:26:58 -03:00
Dallin Romney
e12cf72b17
Standardize QA coverage IDs on dotted names ( #94702 )
...
* fix: standardize qa coverage ids
* test: avoid qa coverage id assertion spread
2026-06-18 17:25:26 -07:00
Dallin Romney
c4ae2be947
fix: taxonomy coverage id cleanup ( #94304 )
...
* fix: split taxonomy coverage id features
* fix: clean taxonomy feature row names
* docs: clarify taxonomy coverage id semantics
* docs: tighten coverage id guidance
* fix: keep taxonomy features product shaped
* fix: narrow sdk artifact coverage bundle
* fix: name taxonomy coverage ids clearly
* fix: polish taxonomy feature descriptions
2026-06-18 15:16:58 -07:00
Colin Johnson
591313e80a
qa-lab: support script-backed evidence scenarios ( #94276 )
...
* qa: add script scenario execution kind
* fix(qa-lab): carry suite profile into script producer evidence and simplify artifact path resolution
* fix(qa-lab): keep out-of-repo producer artifacts absolute to avoid ../ traversal refs
---------
Co-authored-by: Dallin Romney <dallinromney@gmail.com >
2026-06-17 15:09:25 -07:00
Dallin Romney
e32929e12c
Add slim evidence mode for QA profile evidence ( #93179 )
...
* test(qa): compact profile evidence execution metadata
* docs(qa): document compact profile evidence
* test(qa): support compact evidence mode
* test(qa): rename compact evidence mode to slim
* docs(qa): trim slim evidence wording
* fix(qa): avoid commander runtime import
2026-06-15 14:50:40 -07:00
Dallin Romney
3d38c9a633
test(qa): embed profile scorecard evidence ( #93109 )
...
* test(qa): embed profile scorecard evidence
* test(qa): fix profile runner return lint
* test(qa): satisfy suite command lint return
2026-06-14 20:51:38 -07:00
Dallin Romney
e8db9c3bc0
test(qa): add qa run --profile and unified output summary/evidence ( #91587 )
...
* test(qa): add mapped qa run profiles
* test(qa): document mapped profile runner
* test(qa): validate run profiles from mapping
* test(qa): preserve root profile parsing
* test(qa): simplify taxonomy profile dispatch
* test(qa): align tool coverage CLI expectation
* test(qa): fix profile dispatch fixture type
* test(qa): share profile runner option types
* test(qa): split shared cli runner options
* test(qa): unify profile suite artifacts
* fix(qa): filter profile scenarios by provider lane
* test(qa): drop native scenario subreports
* fix(qa): keep native log refs repo-relative
* fix(cli): preserve qa run root profile parsing
* fix(qa): avoid qa profile flag collision
* fix(qa): reject profile flags without qa profile
2026-06-14 18:08:42 -07:00
Dallin Romney
fef8394079
Convert QA scenarios to YAML files ( #92915 )
...
* refactor: load QA scenarios from YAML
* docs: update personal QA scenario docs
* test: keep QA scenarios YAML-only
2026-06-14 17:31:18 -07:00
Dallin Romney
1affe4fcdf
Fold Telegram RTT sampling into live QA evidence ( #92550 )
...
* refactor(qa): fold telegram rtt into live evidence
* test: default package telegram rtt samples
* refactor(qa-lab): fold telegram rtt into live evidence
* fix(qa-lab): keep package telegram rtt optional for focused runs
* fix(qa-lab): avoid stale rtt evidence on failed samples
* fix(qa-lab): pass telegram live env into credential leasing
* fix(qa-lab): update telegram canary remediation artifacts
* docs(qa): remove stale telegram observed artifact guidance
* fix(qa-lab): clarify telegram empty-reply remediation
* fix(qa-lab): honor telegram rtt timeout
* ci(qa): drop stale telegram capture env
* refactor: align telegram evidence coverage fields
* fix: ignore stale telegram observed artifacts
* fix: preserve telegram rtt coverage mapping
* fix: omit unused telegram rtt catch binding
* docs: document telegram rtt check selector
2026-06-14 17:02:33 +08:00
Dallin Romney
561b293c7a
Run Vitest and Playwright scenarios from qa suite ( #92606 )
...
* test(qa): run vitest and playwright scenarios from qa suite
* fix(qa): harden scenario suite dispatch
* refactor(qa): share scenario path utilities
* refactor(qa): share test file scenario runner
* refactor(qa): route test file scenarios through suite runtime
* refactor(qa): use explicit suite runtime result kind
* test(qa): write suite evidence artifact
* refactor(qa): clarify suite execution dispatch
* fix(qa): keep test-file scenarios out of flow-only runners
* refactor(qa): export mixed scenario suite runner
2026-06-13 01:06:10 -07:00
Dallin Romney
4809ac70fa
Add QA evidence artifact output ( #91484 )
...
* feat: add qa evidence summary normalization
* chore: rename qa evidence target environment
* chore: align qa evidence profile terminology
* chore: align qa evidence summary fields
* chore: add qa evidence taxonomy ref
* test: remove stale multipass evidence example
* test(qa): normalize vitest and playwright evidence
* test(qa): slim evidence summary metadata
* test(qa): clarify evidence summary inputs
* test(qa): rename scenario specs in evidence flow
* test(qa): treat evidence profiles as mapping strings
* test(qa): use neutral evidence test identity
* test(qa): nest evidence summary joins
* refactor(qa): normalize live evidence summaries
* fix(qa): accept normalized telegram rtt summaries
* fix(qa): normalize evidence lane summaries
* fix(qa): align evidence summaries with requirements
* refactor(qa): tighten evidence summary builders
* refactor(qa): restore standard evidence ids
* fix(qa): keep legacy summaries out of rtt evidence
* refactor(qa): make package evidence provenance explicit
* test(qa): keep script tests out of qa lab internals
* refactor(qa): rename scenario evidence definitions
* refactor(qa): clean evidence summary wording
* test(qa): fix evidence summary test inputs
* refactor(qa): simplify evidence identity fields
* refactor(qa): tighten evidence summary inputs
* refactor(qa): rename evidence artifact
2026-06-12 16:12:58 -07:00
