Dallin Romney 4809ac70fa Add QA evidence artifact output (#91484) ...

* feat: add qa evidence summary normalization

* chore: rename qa evidence target environment

* chore: align qa evidence profile terminology

* chore: align qa evidence summary fields

* chore: add qa evidence taxonomy ref

* test: remove stale multipass evidence example

* test(qa): normalize vitest and playwright evidence

* test(qa): slim evidence summary metadata

* test(qa): clarify evidence summary inputs

* test(qa): rename scenario specs in evidence flow

* test(qa): treat evidence profiles as mapping strings

* test(qa): use neutral evidence test identity

* test(qa): nest evidence summary joins

* refactor(qa): normalize live evidence summaries

* fix(qa): accept normalized telegram rtt summaries

* fix(qa): normalize evidence lane summaries

* fix(qa): align evidence summaries with requirements

* refactor(qa): tighten evidence summary builders

* refactor(qa): restore standard evidence ids

* fix(qa): keep legacy summaries out of rtt evidence

* refactor(qa): make package evidence provenance explicit

* test(qa): keep script tests out of qa lab internals

* refactor(qa): rename scenario evidence definitions

* refactor(qa): clean evidence summary wording

* test(qa): fix evidence summary test inputs

* refactor(qa): simplify evidence identity fields

* refactor(qa): tighten evidence summary inputs

* refactor(qa): rename evidence artifact

2026-06-12 16:12:58 -07:00

..

fixtures

…

arg-utils.test.ts

fix(test): reject missing numeric flag values

2026-06-06 23:01:21 +02:00

audit-seams.test.ts

…

barnacle-auto-response.test.ts

…

bench-cli-startup.test.ts

fix(test): fail startup bench missing RSS samples

2026-06-06 20:39:07 +02:00

bench-gateway-child-test-support.ts

…

bench-gateway-restart.test.ts

fix(test): require restart bench resource evidence

2026-06-06 21:18:58 +02:00

bench-gateway-startup.test.ts

fix(test): reject missing gateway bench flag values

2026-06-06 21:03:45 +02:00

bench-test-changed.test.ts

fix(test): require resource evidence in perf scripts

2026-06-07 01:11:24 +02:00

bounded-response-text.test.ts

fix(e2e): cancel timed out response reads

2026-06-07 12:32:56 +02:00

bounded-response.test.ts

…

browser-cdp-snapshot.test.ts

fix(e2e): bound browser snapshot diagnostics

2026-06-07 12:39:48 +02:00

build-all.test.ts

fix(build): ship export session html assets

2026-06-06 09:14:12 -07:00

build-and-run-mac.test.ts

fix(mac): scope build-and-run cleanup

2026-06-07 06:35:44 +02:00

build-diffs-viewer-runtime.test.ts

…

bundled-plugin-assets.test.ts

…

bundled-plugin-build-entries.test.ts

test(plugins): guard dedicated channel sidecars

2026-06-10 13:05:24 +05:30

bundled-plugin-install-uninstall-probe.test.ts

fix(qqbot): migrate group tool policy config (#91128)

2026-06-07 02:33:06 -07:00

bundled-plugin-source-utils.test.ts

…

changed-lanes.test.ts

…

channel-contract-test-plan.test.ts

…

channel-message-flows.test.ts

fix(dev): require telegram final message id

2026-06-07 07:48:13 +02:00

check-changelog-attributions.test.ts

…

check-channel-agnostic-boundaries.test.ts

…

check-cli-bootstrap-imports.test.ts

…

check-cli-startup-memory.test.ts

fix(test): reject loose startup memory limits

2026-06-07 02:33:15 +02:00

check-composite-action-input-interpolation.test.ts

…

check-deadcode-unused-files.test.ts

fix(webchat): finalize provider failure lifecycle (#91895)

2026-06-10 12:33:21 -04:00

check-dependency-pins.test.ts

…

check-deprecated-api-usage.test.ts

…

check-docs-i18n-glossary.test.ts

fix(docs): reject missing glossary diff refs

2026-06-06 22:23:09 +02:00

check-docs-mdx.test.ts

fix(docs): reject missing mdx report paths

2026-06-06 22:02:20 +02:00

check-dynamic-import-warts.test.ts

…

check-extension-package-tsc-boundary.test.ts

…

check-extension-wildcard-reexports.test.ts

…

check-file-utils.test.ts

…

check-gateway-cpu-scenarios.test.ts

fix(qa): reject empty gauntlet pass evidence

2026-06-07 04:56:32 +02:00

check-gateway-watch-regression.test.ts

refactor: store Zalo hosted media in plugin state

2026-06-06 22:56:48 -07:00

check-memory-fd-repro.test.ts

…

check-no-conflict-markers.test.ts

…

check-no-random-messaging-tmp.test.ts

…

check-no-raw-window-open.test.ts

…

check-openclaw-package-tarball.test.ts

…

check-opengrep-rule-metadata.test.ts

…

check-package-patches.test.ts

…

check-plugin-sdk-wildcard-reexports.test.ts

…

check-release-metadata-only.test.ts

fix(ci): reject missing release metadata refs

2026-06-06 22:15:34 +02:00

check-runtime-sidecar-loaders.test.ts

…

check-workflows.test.ts

…

check.test.ts

…

ci-docker-pull-retry.test.ts

…

ci-node-test-plan.test.ts

perf(ci): isolate Docker tooling tests

2026-06-11 18:13:36 +09:00

ci-run-timings.test.ts

fix(ci): reject ambiguous run timing args

2026-06-06 22:50:49 +02:00

ci-workflow-guards.test.ts

fix: retry workflow sanity checkout fetches

2026-06-09 17:39:01 +01:00

claude-auth-status.test.ts

fix(test): support macOS Bash 3 script suites

2026-06-10 15:37:15 +09:00

clawdock-helpers.test.ts

…

cli-startup-bench-spawner.test.ts

fix(test): require startup bench report overlap

2026-06-06 20:53:08 +02:00

close-duplicate-prs-after-merge.test.ts

…

codesign-mac-app.test.ts

…

codex-app-server-protocol-source.test.ts

fix: preserve non-oneOf schema array order (#91891)

2026-06-10 15:58:30 -07:00

codex-media-path-client.test.ts

fix(test): bound remaining child output collectors

2026-06-07 12:25:30 +02:00

committer.test.ts

…

config-reload-log-scanner.test.ts

…

control-ui-i18n.test.ts

…

crabbox-wrapper.test.ts

fix(test): fail stale Crabbox broker auth

2026-06-06 18:46:28 +02:00

create-dmg.test.ts

…

cron-mcp-cleanup-docker-client.test.ts

fix(e2e): require cron cleanup success status

2026-06-07 07:43:33 +02:00

dependency-changes-report.test.ts

fix(report): reject missing dependency report paths

2026-06-06 22:07:31 +02:00

dependency-guard-script.test.ts

…

dependency-guard-workflow.test.ts

…

dependency-ownership-surface-report.test.ts

fix(report): require ownership markdown path

2026-06-06 22:18:03 +02:00

dependency-vulnerability-gate.test.ts

…

dev-tooling-safety.test.ts

fix(dev): bound anthropic prompt log tails

2026-06-07 10:22:50 +02:00

docker-all-scheduler.test.ts

fix(e2e): preserve docker cleanup failure artifacts

2026-06-06 17:37:12 +02:00

docker-build-helper.test.ts

test(ci): relax docker signal wait

2026-06-11 21:52:19 +09:00

docker-e2e-helper-cli.test.ts

fix(test): require docker timing limit values

2026-06-06 22:48:19 +02:00

docker-e2e-observability.test.ts

fix(test): execute docker observability proof

2026-06-07 12:22:10 +02:00

docker-e2e-plan.test.ts

fix(e2e): run openwebui release chat smoke

2026-06-07 07:18:56 +02:00

docker-stats-resource-ceiling.test.ts

test(release): ignore terminal docker stats samples

2026-06-09 01:02:24 +02:00

docs-sync-publish.test.ts

fix(docs): require sync provenance values

2026-06-06 22:43:04 +02:00

e2e-agent-turn-output.test.ts

fix(test): reject failed agent reply markers

2026-06-07 00:45:00 +02:00

e2e-helper-env-limits.test.ts

fix(test): bound remaining child output collectors

2026-06-07 12:25:30 +02:00

e2e-mock-config-limits.test.ts

…

e2e-run-with-pty.test.ts

fix(test): bound remaining child output collectors

2026-06-07 12:25:30 +02:00

e2e-shell-tempfiles.test.ts

…

e2e-temp-state-dir.test.ts

…

e2e-websocket-open.test.ts

…

embedded-run-abort-leak.test.ts

…

ensure-cli-startup-build.test.ts

…

ensure-extension-memory-build.test.ts

…

ensure-playwright-chromium.test.ts

…

extension-import-boundary-checker.test.ts

fix(tooling): bound extension boundary source reads

2026-06-07 12:36:13 +02:00

extension-source-classifier.test.ts

…

firecrawl-compare.test.ts

…

fixture-config.test.ts

…

fixtures-workspace.test.ts

fix(e2e): bound workspace fixture output

2026-06-07 09:27:18 +02:00

format-generated-module.test.ts

fix(build): respect PATH-less pnpm environments

2026-06-10 18:06:31 +09:00

full-release-validation-at-sha.test.ts

fix(release): require validation dispatch values

2026-06-06 22:39:58 +02:00

gateway-frame-payload.test.ts

…

gateway-network-client.test.ts

fix(e2e): require gateway network health payload

2026-06-07 07:16:19 +02:00

gateway-smoke.test.ts

fix(dev): align gateway smoke auth contract

2026-06-07 12:07:05 +02:00

gateway-ws-client.test.ts

test: harden stalled websocket cleanup

2026-06-11 15:32:58 +01:00

generate-dependency-release-evidence.test.ts

fix(test): reject missing dependency evidence values

2026-06-06 22:12:08 +02:00

generate-npm-shrinkwrap.test.ts

…

gh-read.test.ts

…

github-activity-helper.test.ts

fix(test): support macOS Bash 3 script suites

2026-06-10 15:37:15 +09:00

install-cli.test.ts

…

install-ps1.test.ts

…

install-sh.test.ts

fix(installer): stop after failed Node package installs

2026-06-11 22:58:43 +09:00

ios-node-e2e.test.ts

fix(test): bound child output buffers

2026-06-07 12:19:08 +02:00

ios-pin-version.test.ts

fix(release): use monthly patch versions

2026-06-06 12:26:32 -07:00

ios-pull-gateway-log.test.ts

fix(dev): harden smoke log diagnostics

2026-06-07 08:42:16 +02:00

ios-team-id.test.ts

…

ios-version.test-support.ts

…

ios-version.test.ts

fix(qqbot): migrate group tool policy config (#91128)

2026-06-07 02:33:06 -07:00

kitchen-sink-plugin-assertions.test.ts

feat(plugins): allow installed trusted policy contracts

2026-06-10 16:18:23 +10:00

kitchen-sink-rpc-walk.test.ts

test(e2e): widen kitchen sink RPC coverage

2026-06-10 16:42:47 +09:00

kova-ci-summary.test.ts

fix(ci): require kova resource metrics

2026-06-07 00:10:37 +02:00

kova-report-gate.test.ts

fix(ci): require kova partial evidence

2026-06-07 02:10:23 +02:00

label-open-issues.test.ts

…

limit-edge-case-live-proof.test.ts

…

lint-suppressions.test.ts

…

list-prod-store-packages.test.ts

fix docker store seed target packages (#91547)

2026-06-08 23:38:46 -04:00

live-docker-auth.test.ts

fix(release): cap docker e2e cpus

2026-06-09 01:02:22 +02:00

live-docker-stage.test.ts

…

live-plugin-tool-assertions.test.ts

fix(e2e): require live tool result proof

2026-06-07 09:44:35 +02:00

local-heavy-check-runtime.test.ts

…

managed-child-process.test.ts

…

mantis-build-telegram-desktop-proof-evidence.test.ts

…

mantis-build-telegram-evidence.test.ts

Add QA evidence artifact output (#91484)

2026-06-12 16:12:58 -07:00

mantis-publish-pr-evidence.test.ts

…

mantis-telegram-desktop-proof-workflow.test.ts

…

mcp-channel-limits.test.ts

…

mcp-channels-harness.test.ts

…

mcp-code-mode-gateway-client.test.ts

test(e2e): share mcp code mode validation

2026-06-06 18:16:57 +02:00

mcp-connect-timeout.test.ts

…

mcp-websocket-open.test.ts

…

measure-rpc-rtt.test.ts

fix(qa): validate rpc rtt smoke payloads

2026-06-07 07:09:01 +02:00

merge-head-diff-base.test.ts

fix(ci): reject missing merge diff refs

2026-06-06 22:25:06 +02:00

mock-openai-http.test.ts

…

notarize-mac-artifact.test.ts

…

npm-onboard-channel-agent-assertions.test.ts

fix(test): validate auth profile env refs

2026-06-06 23:09:43 +02:00

npm-runner.test.ts

…

npm-telegram-live.test.ts

Add QA evidence artifact output (#91484)

2026-06-12 16:12:58 -07:00

npm-telegram-rtt-driver.test.ts

Add QA evidence artifact output (#91484)

2026-06-12 16:12:58 -07:00

npm-verify-exec.test.ts

…

onboard-log-contains.test.ts

fix(e2e): bound onboard log polling reads

2026-06-07 07:14:37 +02:00

openai-chat-tools-client.test.ts

fix(test): bound child output buffers

2026-06-07 12:19:08 +02:00

openai-image-auth-docker-client.test.ts

…

openai-web-search-minimal-assertions.test.ts

fix(test): require native web search proof

2026-06-07 00:02:29 +02:00

openai-web-search-minimal-client.test.ts

fix(e2e): require web search success marker

2026-06-07 07:42:24 +02:00

openclaw-cross-os-release-checks.test.ts

fix(ci): disable memory slot in release smoke config

2026-06-10 14:56:21 +09:00

openclaw-cross-os-release-workflow.test.ts

…

openclaw-e2e-instance.test.ts

fix(e2e): bound shared helper log output

2026-06-07 10:19:16 +02:00

openclaw-performance-source-summary.test.ts

test(sqlite): add state perf query plan harness

2026-06-11 14:49:26 -07:00

openclaw-performance-workflow.test.ts

fix(ci): require live kova evidence

2026-06-07 00:48:54 +02:00

openclaw-test-state.test.ts

…

openwebui-probe.test.ts

fix(test): bound remaining child output collectors

2026-06-07 12:25:30 +02:00

oxlint-config.test.ts

…

package-acceptance-workflow.test.ts

fix(release): gate beta publish on plugin verification

2026-06-11 20:42:58 +09:00

package-changelog.test.ts

…

package-mac-app.test.ts

fix(mac): fail release packaging without Swift compat lib

2026-06-06 21:06:32 +02:00

package-mac-dist.test.ts

fix(mac): fail closed on invalid release build floors

2026-06-06 19:55:22 +02:00

package-openclaw-for-docker.test.ts

fix(package): require docker package option values

2026-06-06 22:36:47 +02:00

package-root-args.test.ts

fix(package): require package root values

2026-06-06 22:26:53 +02:00

parallels-npm-update-smoke.test.ts

fix(e2e): bound parallels update logs

2026-06-07 12:00:57 +02:00

parallels-package-log-progress-extract.test.ts

…

parallels-smoke-model.test.ts

fix(release): guard Parallels skip-restore lanes

2026-06-10 08:27:59 +09:00

parallels-update-job-timeout.test.ts

…

plugin-boundary-report.test.ts

…

plugin-contract-test-plan.test.ts

…

plugin-gateway-gauntlet.test.ts

fix(qa): allow gauntlet observations to fail

2026-06-07 07:31:56 +02:00

plugin-lifecycle-measure.test.ts

fix(e2e): fail lifecycle resource spikes

2026-06-07 06:11:15 +02:00

plugin-lifecycle-probe.test.ts

fix(test): require lifecycle uninstall config proof

2026-06-06 23:46:46 +02:00

plugin-npm-package-manifest-args.test.ts

fix(plugin): require package manifest run target

2026-06-06 22:31:42 +02:00

plugin-npm-runtime-build-args.test.ts

fix(plugin): require runtime build package targets

2026-06-06 22:29:12 +02:00

plugin-prerelease-test-plan.test.ts

…

plugin-sdk-surface-report.test.ts

…

plugin-update-unchanged-docker.test.ts

test(update): align corrupt plugin repair guidance

2026-06-11 04:33:25 +09:00

plugins-assertions.test.ts

fix(test): bound child output buffers

2026-06-07 12:19:08 +02:00

pnpm-audit-prod.test.ts

fix(security): require audit severity values

2026-06-06 22:19:18 +02:00

pnpm-runner.test.ts

fix(build): respect PATH-less pnpm environments

2026-06-10 18:06:31 +09:00

postinstall-bundled-plugins.test.ts

…

preinstall-package-manager-warning.test.ts

…

prepare-extension-package-boundary-artifacts.test.ts

…

prepare-git-hooks.test.ts

…

print-cli-backend-live-metadata.test.ts

…

profile-extension-memory.test.ts

fix(test): fail extension memory import failures

2026-06-06 20:35:27 +02:00

prompt-snapshots.test.ts

…

qa-e2e.test.ts

…

qa-lab-up.test.ts

…

qa-otel-smoke.test.ts

fix(test): bound otel collector output

2026-06-07 09:42:15 +02:00

real-behavior-proof-check.test.ts

…

real-behavior-proof-policy.test.ts

…

release-beta-smoke.test.ts

…

release-beta-verifier.test.ts

fix(release): gate beta publish on plugin verification

2026-06-11 20:42:58 +09:00

release-candidate-checklist.test.ts

…

release-check.test.ts

…

release-media-memory-scenario.test.ts

…

release-scenarios-assertions.test.ts

fix(e2e): bound release scenario JSON artifacts

2026-06-07 12:43:33 +02:00

release-user-journey-assertions.test.ts

fix(e2e): bound release user journey JSON artifacts

2026-06-07 12:45:43 +02:00

release-workflow-matrix-plan.test.ts

…

report-cli-helpers.test.ts

fix(test): reject missing report artifact paths

2026-06-06 22:13:37 +02:00

resolve-openclaw-package-candidate-ip-bypass.test.ts

…

resolve-openclaw-package-candidate.test.ts

…

restart-mac.test.ts

fix(mac): scope restart app cleanup

2026-06-07 06:58:46 +02:00

root-dependency-ownership-audit.test.ts

…

root-package-overrides.test.ts

…

rtt-harness.test.ts

Add QA evidence artifact output (#91484)

2026-06-12 16:12:58 -07:00

run-additional-boundary-checks.test.ts

…

run-android-gradle.test.ts

…

run-opengrep.test.ts

…

run-oxlint.test.ts

…

run-tsgo.test.ts

…

run-vitest-profile.test.ts

fix(test): require vitest profile output dir

2026-06-06 22:16:53 +02:00

run-vitest.test.ts

perf(ci): isolate Docker tooling tests

2026-06-11 18:13:36 +09:00

run-with-env.test.ts

fix(scripts): reject loose run env kill delay

2026-06-07 02:35:08 +02:00

runtime-postbuild-stamp.test.ts

…

runtime-postbuild.test.ts

…

sandbox-common-smoke-workflow.test.ts

…

secret-provider-integrations.test.ts

fix(e2e): require secret probe success

2026-06-07 11:22:43 +02:00

secret-scanning-maintainer.test.ts

…

session-log-mentions.test.ts

…

setup-pnpm-store-cache-ensure-node.test.ts

…

source-file-scan-cache.test.ts

fix(tooling): bound source scan file reads

2026-06-07 12:14:09 +02:00

stage-bundled-plugin-runtime.test.ts

…

summarize-cpuprofile.test.ts

fix(perf): reject empty cpu profiles

2026-06-06 21:21:17 +02:00

targeted-docker-lane-groups.test.ts

…

telegram-bot-api.test.ts

…

telegram-user-crabbox-proof.test.ts

fix(e2e): bound telegram desktop proof logs

2026-06-07 10:42:28 +02:00

telegram-user-credential.test.ts

fix(qqbot): migrate group tool policy config (#91128)

2026-06-07 02:33:06 -07:00

test-built-status-message-runtime.test.ts

…

test-device-pair-telegram.test.ts

fix(dev): lazy-load telegram pairing smoke

2026-06-07 08:48:40 +02:00

test-env-mutation-report.test.ts

fix(test): reject missing env mutation roots

2026-06-06 22:05:23 +02:00

test-extension.test.ts

…

test-force.test.ts

…

test-group-report.test.ts

fix(test): bound group report child output

2026-06-07 11:40:47 +02:00

test-helpers.ts

fix(test): support macOS Bash 3 script suites

2026-06-10 15:37:15 +09:00

test-hotspots.test.ts

fix(test): require resource evidence in perf scripts

2026-06-07 01:11:24 +02:00

test-install-sh-docker.test.ts

fix(e2e): preserve dist during bun restore

2026-06-07 07:54:05 +02:00

test-live-cli-backend-docker.test.ts

…

test-live-codex-harness-docker.test.ts

fix(e2e): bound codex live failure logs

2026-06-07 10:44:06 +02:00

test-live-media.test.ts

fix(test): require live media providers

2026-06-07 00:25:05 +02:00

test-live-shard.test.ts

fix(test): require enabled live shard proof

2026-06-07 06:47:02 +02:00

test-live.test.ts

…

test-perf-budget.test.ts

fix(test): require perf budget source

2026-06-07 01:24:10 +02:00

test-projects.test.ts

perf(ci): isolate Docker tooling tests

2026-06-11 18:13:36 +09:00

test-report-utils.test.ts

fix(test): support macOS Bash 3 script suites

2026-06-10 15:37:15 +09:00

tool-search-gateway-e2e.test.ts

fix(e2e): require tool-search session proof

2026-06-07 09:31:21 +02:00

transitive-manifest-risk-report.test.ts

…

ts-guard-utils.test.ts

…

ts-topology.test.ts

…

tsdown-build.test.ts

…

ui.test.ts

…

upgrade-survivor-assertions.test.ts

test(ci): restore upgrade survivor session fixture (#92049)

2026-06-11 08:06:19 +09:00

upgrade-survivor-baselines.test.ts

…

upgrade-survivor-config-recipe.test.ts

…

upgrade-survivor-probe-gateway.test.ts

fix(test): bound remaining child output collectors

2026-06-07 12:25:30 +02:00

verify-docker-attestations.test.ts

fix(security): require docker attestation platform values

2026-06-06 22:33:56 +02:00

verify-plugin-npm-published-runtime.test.ts

…

verify.test.ts

…

vitest-local-scheduling.test.ts

…

vitest-process-group.test.ts

…

vitest-shard-timings.test.ts

…

watch-node.test.ts

…

webchat-auto-tts-live-proof.test.ts

…

website-installer-sync-workflow.test.ts

…

write-cli-startup-metadata.test.ts

…

zai-fallback-repro.test.ts

fix(e2e): require zai fallback evidence

2026-06-07 11:33:36 +02:00