refactor(qa): add shared QA channel contract and harden worker startup (#64562)

* refactor(qa): add shared transport contract and suite migration

* refactor(qa): harden worker gateway startup

* fix(qa): scope waits and sanitize shutdown artifacts

* fix(qa): confine artifacts and redact preserved logs

* fix(qa): block symlink escapes in artifact paths

* fix(gateway): clear shutdown race timers

* fix(qa): harden shutdown cleanup paths

* fix(qa): sanitize gateway logs in thrown errors

* fix(qa): harden suite startup and artifact paths

* fix(qa): stage bundled plugins from mutated config

* fix(qa): broaden gateway log bearer redaction

* fix(qa-channel): restore runtime export

* fix(qa): stop failed gateway startups as a process tree

* fix(qa-channel): load runtime hook from api surface
This commit is contained in:
Marcus Castro
2026-04-12 15:02:57 -03:00
committed by GitHub
parent fcae3bf943
commit 000fc7f233
38 changed files with 2177 additions and 473 deletions

View File

@@ -87,6 +87,78 @@ transport coverage matrix.
| Matrix | x | x | x | x | x | x | x | x | |
| Telegram | x | | | | | | | | x |
### Adding a channel to QA
Adding a channel to the markdown QA system requires exactly two things:
1. A transport adapter for the channel.
2. A scenario pack that exercises the channel contract.
Do not add a channel-specific QA runner when the shared `qa-lab` runner can
own the flow.
`qa-lab` owns the shared mechanics:
- suite startup and teardown
- worker concurrency
- artifact writing
- report generation
- scenario execution
- compatibility aliases for older `qa-channel` scenarios
The channel adapter owns the transport contract:
- how the gateway is configured for that transport
- how readiness is checked
- how inbound events are injected
- how outbound messages are observed
- how transcripts and normalized transport state are exposed
- how transport-backed actions are executed
- how transport-specific reset or cleanup is handled
The minimum adoption bar for a new channel is:
1. Implement the transport adapter on the shared `qa-lab` seam.
2. Register the adapter in the transport registry.
3. Keep transport-specific mechanics inside the adapter or the channel harness.
4. Author or adapt markdown scenarios under `qa/scenarios/`.
5. Use the generic scenario helpers for new scenarios.
6. Keep existing compatibility aliases working unless the repo is doing an intentional migration.
The decision rule is strict:
- If behavior can be expressed once in `qa-lab`, put it in `qa-lab`.
- If behavior depends on one channel transport, keep it in that adapter or plugin harness.
- If a scenario needs a new capability that more than one channel can use, add a generic helper instead of a channel-specific branch in `suite.ts`.
- If a behavior is only meaningful for one transport, keep the scenario transport-specific and make that explicit in the scenario contract.
Preferred generic helper names for new scenarios are:
- `waitForTransportReady`
- `waitForChannelReady`
- `injectInboundMessage`
- `injectOutboundMessage`
- `waitForTransportOutboundMessage`
- `waitForChannelOutboundMessage`
- `waitForNoTransportOutbound`
- `getTransportSnapshot`
- `readTransportMessage`
- `readTransportTranscript`
- `formatTransportTranscript`
- `resetTransport`
Compatibility aliases remain available for existing scenarios, including:
- `waitForQaChannelReady`
- `waitForOutboundMessage`
- `waitForNoOutbound`
- `formatConversationTranscript`
- `resetBus`
New channel work should use the generic helper names.
Compatibility aliases exist to avoid a flag day migration, not as the model for
new scenario authoring.
## Test suites (what runs where)
Think of the suites as “increasing realism” (and increasing flakiness/cost):