refactor(qa): split Matrix QA into optional plugin (#66723)

Merged via squash.

Prepared head SHA: 27241bd089
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras

CHANGELOG.md

@@ -8,6 +8,7 @@ Docs: https://docs.openclaw.ai
 
 - Docs/showcase: add a scannable hero, complete section jump links, and a responsive video grid for community examples. (#48493) Thanks @jchopard69.
 - Agents/local models: add `agents.defaults.localModelMode: "lean"` to drop heavyweight default tools like `browser`, `cron`, and `message`, reducing prompt size for weaker local-model setups without changing the normal path. Thanks @ImLukeF.
+- QA/Matrix: split Matrix live QA into a source-linked `qa-matrix` runner and keep repo-private `qa-*` surfaces out of packaged and published builds. (#66723) Thanks @gumadeiras.
 
 ### Fixes
 

docs/cli/plugins.md

@@ -197,6 +197,12 @@ Use `--link` to avoid copying a local directory (adds to `plugins.load.paths`):
 openclaw plugins install -l ./my-plugin
 ```
 
+Repo QA example (source-linked dev surface; not shipped in packaged installs):
+
+```bash
+openclaw plugins install -l ./extensions/qa-matrix
+```
+
 `--force` is not supported with `--link` because linked installs reuse the
 source path instead of copying over a managed install target.
 

docs/help/testing.md

@@ -67,9 +67,13 @@ These commands sit beside the main test suites when you need QA-lab realism:
   - Starts the Docker-backed QA site for operator-style QA work.
 - `pnpm openclaw qa matrix`
   - Runs the Matrix live QA lane against a disposable Docker-backed Tuwunel homeserver.
+  - This QA host is repo/dev-only today. Packaged OpenClaw installs do not ship
+    `qa-lab`, so they do not expose `openclaw qa`.
+  - Repo checkouts can link the in-tree plugin directly:
+    `openclaw plugins install -l ./extensions/qa-matrix`.
   - Provisions three temporary Matrix users (`driver`, `sut`, `observer`) plus one private room, then starts a QA gateway child with the real Matrix plugin as the SUT transport.
   - Uses the pinned stable Tuwunel image `ghcr.io/matrix-construct/tuwunel:v1.5.1` by default. Override with `OPENCLAW_QA_MATRIX_TUWUNEL_IMAGE` when you need to test a different image.
-  - Matrix currently supports only `--credential-source env` because the lane provisions disposable users locally.
+  - Matrix does not expose shared credential-source flags because the lane provisions disposable users locally.
   - Writes a Matrix QA report, summary, and observed-events artifact under `.artifacts/qa-e2e/...`.
 - `pnpm openclaw qa telegram`
   - Runs the Telegram live QA lane against a real private group using the driver and SUT bot tokens from env.
@@ -170,11 +174,12 @@ Adding a channel to the markdown QA system requires exactly two things:
 1. A transport adapter for the channel.
 2. A scenario pack that exercises the channel contract.
 
-Do not add a channel-specific QA runner when the shared `qa-lab` runner can
+Do not add a new top-level QA command root when the shared `qa-lab` host can
 own the flow.
 
-`qa-lab` owns the shared mechanics:
+`qa-lab` owns the shared host mechanics:
 
+- the `openclaw qa` command root
 - suite startup and teardown
 - worker concurrency
 - artifact writing
@@ -182,8 +187,9 @@ own the flow.
 - scenario execution
 - compatibility aliases for older `qa-channel` scenarios
 
-The channel adapter owns the transport contract:
+Runner plugins own the transport contract:
 
+- how `openclaw qa <runner>` is mounted beneath the shared `qa` root
 - how the gateway is configured for that transport
 - how readiness is checked
 - how inbound events are injected
@@ -194,17 +200,20 @@ The channel adapter owns the transport contract:
 
 The minimum adoption bar for a new channel is:
 
-1. Implement the transport adapter on the shared `qa-lab` seam.
+1. Keep `qa-lab` as the owner of the shared `qa` root.
-2. Register the adapter in the transport registry.
+2. Implement the transport runner on the shared `qa-lab` host seam.
-3. Keep transport-specific mechanics inside the adapter or the channel harness.
+3. Keep transport-specific mechanics inside the runner plugin or channel harness.
-4. Author or adapt markdown scenarios under `qa/scenarios/`.
+4. Mount the runner as `openclaw qa <runner>` instead of registering a competing root command.
-5. Use the generic scenario helpers for new scenarios.
+   Runner plugins should declare `qaRunners` in `openclaw.plugin.json` and export a matching `qaRunnerCliRegistrations` array from `runtime-api.ts`.
-6. Keep existing compatibility aliases working unless the repo is doing an intentional migration.
+   Keep `runtime-api.ts` light; lazy CLI and runner execution should stay behind separate entrypoints.
+5. Author or adapt markdown scenarios under `qa/scenarios/`.
+6. Use the generic scenario helpers for new scenarios.
+7. Keep existing compatibility aliases working unless the repo is doing an intentional migration.
 
 The decision rule is strict:
 
 - If behavior can be expressed once in `qa-lab`, put it in `qa-lab`.
-- If behavior depends on one channel transport, keep it in that adapter or plugin harness.
+- If behavior depends on one channel transport, keep it in that runner plugin or plugin harness.
 - If a scenario needs a new capability that more than one channel can use, add a generic helper instead of a channel-specific branch in `suite.ts`.
 - If a behavior is only meaningful for one transport, keep the scenario transport-specific and make that explicit in the scenario contract.
 

docs/plugins/manifest.md

@@ -56,6 +56,8 @@ Use it for:
   plugin before runtime loads
 - static capability ownership snapshots used for bundled compat wiring and
   contract coverage
+- cheap QA runner metadata that the shared `openclaw qa` host can inspect
+  before plugin runtime loads
 - channel-specific config metadata that should merge into catalog and validation
   surfaces without loading runtime
 - config UI hints
@@ -158,6 +160,7 @@ Those belong in your plugin code and `package.json`.
 | `providerAuthChoices`               | No       | `object[]`                       | Cheap auth-choice metadata for onboarding pickers, preferred-provider resolution, and simple CLI flag wiring.                                                                                                |
 | `activation`                        | No       | `object`                         | Cheap activation hints for provider, command, channel, route, and capability-triggered loading. Metadata only; plugin runtime still owns actual behavior.                                                    |
 | `setup`                             | No       | `object`                         | Cheap setup/onboarding descriptors that discovery and setup surfaces can inspect without loading plugin runtime.                                                                                             |
+| `qaRunners`                         | No       | `object[]`                       | Cheap QA runner descriptors used by the shared `openclaw qa` host before plugin runtime loads.                                                                                                               |
 | `contracts`                         | No       | `object`                         | Static bundled capability snapshot for speech, realtime transcription, realtime voice, media-understanding, image-generation, music-generation, video-generation, web-fetch, web search, and tool ownership. |
 | `channelConfigs`                    | No       | `Record<string, object>`         | Manifest-owned channel config metadata merged into discovery and validation surfaces before runtime loads.                                                                                                   |
 | `skills`                            | No       | `string[]`                       | Skill directories to load, relative to the plugin root.                                                                                                                                                      |
@@ -219,6 +222,29 @@ uses this metadata for diagnostics without importing plugin runtime code.
 Use `activation` when the plugin can cheaply declare which control-plane events
 should activate it later.
 
+## qaRunners reference
+
+Use `qaRunners` when a plugin contributes one or more transport runners beneath
+the shared `openclaw qa` root. Keep this metadata cheap and static; the plugin
+runtime still owns actual CLI registration through a lightweight
+`runtime-api.ts` surface that exports `qaRunnerCliRegistrations`.
+
+```json
+{
+  "qaRunners": [
+    {
+      "commandName": "matrix",
+      "description": "Run the Docker-backed Matrix live QA lane against a disposable homeserver"
+    }
+  ]
+}
+```
+
+| Field         | Required | Type     | What it means                                                      |
+| ------------- | -------- | -------- | ------------------------------------------------------------------ |
+| `commandName` | Yes      | `string` | Subcommand mounted beneath `openclaw qa`, for example `matrix`.    |
+| `description` | No       | `string` | Fallback help text used when the shared host needs a stub command. |
+
 This block is metadata only. It does not register runtime behavior, and it does
 not replace `register(...)`, `setupEntry`, or other runtime/plugin entrypoints.
 Current consumers use it as a narrowing hint before broader plugin loading, so

extensions/qa-lab/runtime-api.ts

@@ -1 +1,2 @@
 export * from "./src/runtime-api.js";
+export { startQaLiveLaneGateway } from "./src/live-transports/shared/live-gateway.runtime.js";

extensions/qa-lab/src/cli.runtime.test.ts

@@ -8,7 +8,6 @@ const {
   runQaSuiteFromRuntime,
   runQaCharacterEval,
   runQaMultipass,
-  runMatrixQaLive,
   runTelegramQaLive,
   startQaLabServer,
   writeQaDockerHarnessFiles,
@@ -20,7 +19,6 @@ const {
   runQaSuiteFromRuntime: vi.fn(),
   runQaCharacterEval: vi.fn(),
   runQaMultipass: vi.fn(),
-  runMatrixQaLive: vi.fn(),
   runTelegramQaLive: vi.fn(),
   startQaLabServer: vi.fn(),
   writeQaDockerHarnessFiles: vi.fn(),
@@ -52,10 +50,6 @@ vi.mock("./multipass.runtime.js", () => ({
   runQaMultipass,
 }));
 
-vi.mock("./live-transports/matrix/matrix-live.runtime.js", () => ({
-  runMatrixQaLive,
-}));
-
 vi.mock("./live-transports/telegram/telegram-live.runtime.js", () => ({
   runTelegramQaLive,
 }));
@@ -88,7 +82,6 @@ import {
   runQaParityReportCommand,
   runQaSuiteCommand,
 } from "./cli.runtime.js";
-import { runQaMatrixCommand } from "./live-transports/matrix/cli.runtime.js";
 import { runQaTelegramCommand } from "./live-transports/telegram/cli.runtime.js";
 
 describe("qa cli runtime", () => {
@@ -100,7 +93,6 @@ describe("qa cli runtime", () => {
     runQaCharacterEval.mockReset();
     runQaManualLane.mockReset();
     runQaMultipass.mockReset();
-    runMatrixQaLive.mockReset();
     runTelegramQaLive.mockReset();
     startQaLabServer.mockReset();
     writeQaDockerHarnessFiles.mockReset();
@@ -139,13 +131,6 @@ describe("qa cli runtime", () => {
       vmName: "openclaw-qa-test",
       scenarioIds: ["channel-chat-baseline"],
     });
-    runMatrixQaLive.mockResolvedValue({
-      outputDir: "/tmp/matrix",
-      reportPath: "/tmp/matrix/report.md",
-      summaryPath: "/tmp/matrix/summary.json",
-      observedEventsPath: "/tmp/matrix/observed.json",
-      scenarios: [],
-    });
     runTelegramQaLive.mockResolvedValue({
       outputDir: "/tmp/telegram",
       reportPath: "/tmp/telegram/report.md",
@@ -226,30 +211,6 @@ describe("qa cli runtime", () => {
     });
   });
 
-  it("resolves matrix qa repo-root-relative paths before dispatching", async () => {
-    await runQaMatrixCommand({
-      repoRoot: "/tmp/openclaw-repo",
-      outputDir: ".artifacts/qa/matrix",
-      providerMode: "live-frontier",
-      primaryModel: "openai/gpt-5.4",
-      alternateModel: "openai/gpt-5.4",
-      fastMode: true,
-      scenarioIds: ["matrix-thread-follow-up"],
-      sutAccountId: "sut-live",
-    });
-
-    expect(runMatrixQaLive).toHaveBeenCalledWith({
-      repoRoot: path.resolve("/tmp/openclaw-repo"),
-      outputDir: path.resolve("/tmp/openclaw-repo", ".artifacts/qa/matrix"),
-      providerMode: "live-frontier",
-      primaryModel: "openai/gpt-5.4",
-      alternateModel: "openai/gpt-5.4",
-      fastMode: true,
-      scenarioIds: ["matrix-thread-follow-up"],
-      sutAccountId: "sut-live",
-    });
-  });
-
   it("rejects output dirs that escape the repo root", () => {
     expect(() => resolveRepoRelativeOutputDir("/tmp/openclaw-repo", "../outside")).toThrow(
       "--output-dir must stay within the repo root.",
@@ -273,20 +234,6 @@ describe("qa cli runtime", () => {
     );
   });
 
-  it("defaults matrix qa runs onto the live provider lane", async () => {
-    await runQaMatrixCommand({
-      repoRoot: "/tmp/openclaw-repo",
-      scenarioIds: ["matrix-thread-follow-up"],
-    });
-
-    expect(runMatrixQaLive).toHaveBeenCalledWith(
-      expect.objectContaining({
-        repoRoot: path.resolve("/tmp/openclaw-repo"),
-        providerMode: "live-frontier",
-      }),
-    );
-  });
-
   it("normalizes legacy live-openai suite runs onto the frontier provider mode", async () => {
     await runQaSuiteCommand({
       repoRoot: "/tmp/openclaw-repo",

extensions/qa-lab/src/cli.test.ts

@@ -1,22 +1,76 @@
 import { Command } from "commander";
+import type { QaRunnerCliContribution } from "openclaw/plugin-sdk/qa-runner-runtime";
 import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
 
+const TEST_QA_RUNNER = {
+  pluginId: "qa-runner-test",
+  commandName: "runner-test",
+  description: "Run the test live QA lane",
+  npmSpec: "@openclaw/qa-runner-test",
+} as const;
+
+function createAvailableQaRunnerContribution() {
+  return {
+    pluginId: TEST_QA_RUNNER.pluginId,
+    commandName: TEST_QA_RUNNER.commandName,
+    status: "available" as const,
+    registration: {
+      commandName: TEST_QA_RUNNER.commandName,
+      register: vi.fn((qa: Command) => {
+        qa.command(TEST_QA_RUNNER.commandName).action(() => undefined);
+      }),
+    },
+  } satisfies QaRunnerCliContribution;
+}
+
+function createMissingQaRunnerContribution(): QaRunnerCliContribution {
+  return {
+    pluginId: TEST_QA_RUNNER.pluginId,
+    commandName: TEST_QA_RUNNER.commandName,
+    description: TEST_QA_RUNNER.description,
+    status: "missing",
+    npmSpec: TEST_QA_RUNNER.npmSpec,
+  };
+}
+
+function createBlockedQaRunnerContribution(): QaRunnerCliContribution {
+  return {
+    pluginId: TEST_QA_RUNNER.pluginId,
+    commandName: TEST_QA_RUNNER.commandName,
+    description: TEST_QA_RUNNER.description,
+    status: "blocked",
+  };
+}
+
+function createConflictingQaRunnerContribution(commandName: string): QaRunnerCliContribution {
+  return {
+    pluginId: TEST_QA_RUNNER.pluginId,
+    commandName,
+    description: TEST_QA_RUNNER.description,
+    status: "blocked",
+  };
+}
+
 const {
   runQaCredentialsAddCommand,
   runQaCredentialsListCommand,
   runQaCredentialsRemoveCommand,
-  runQaMatrixCommand,
   runQaTelegramCommand,
 } = vi.hoisted(() => ({
   runQaCredentialsAddCommand: vi.fn(),
   runQaCredentialsListCommand: vi.fn(),
   runQaCredentialsRemoveCommand: vi.fn(),
-  runQaMatrixCommand: vi.fn(),
   runQaTelegramCommand: vi.fn(),
 }));
 
-vi.mock("./live-transports/matrix/cli.runtime.js", () => ({
+const { listQaRunnerCliContributions } = vi.hoisted(() => ({
-  runQaMatrixCommand,
+  listQaRunnerCliContributions: vi.fn<() => QaRunnerCliContribution[]>(() => [
+    createAvailableQaRunnerContribution(),
+  ]),
+}));
+
+vi.mock("openclaw/plugin-sdk/qa-runner-runtime", () => ({
+  listQaRunnerCliContributions,
 }));
 
 vi.mock("./live-transports/telegram/cli.runtime.js", () => ({
@@ -36,63 +90,71 @@ describe("qa cli registration", () => {
 
   beforeEach(() => {
     program = new Command();
-    registerQaLabCli(program);
     runQaCredentialsAddCommand.mockReset();
     runQaCredentialsListCommand.mockReset();
     runQaCredentialsRemoveCommand.mockReset();
-    runQaMatrixCommand.mockReset();
     runQaTelegramCommand.mockReset();
+    listQaRunnerCliContributions
+      .mockReset()
+      .mockReturnValue([createAvailableQaRunnerContribution()]);
+    registerQaLabCli(program);
   });
 
   afterEach(() => {
     vi.clearAllMocks();
   });
 
-  it("registers the matrix and telegram live transport subcommands", () => {
+  it("registers discovered and built-in live transport subcommands", () => {
     const qa = program.commands.find((command) => command.name() === "qa");
     expect(qa).toBeDefined();
     expect(qa?.commands.map((command) => command.name())).toEqual(
-      expect.arrayContaining(["matrix", "telegram", "credentials"]),
+      expect.arrayContaining([TEST_QA_RUNNER.commandName, "telegram", "credentials"]),
     );
   });
 
-  it("routes matrix CLI flags into the lane runtime", async () => {
+  it("delegates discovered qa runner registration through the generic host seam", () => {
-    await program.parseAsync([
+    const [{ registration }] = listQaRunnerCliContributions.mock.results[0]?.value;
-      "node",
+    expect(registration.register).toHaveBeenCalledTimes(1);
-      "openclaw",
+  });
-      "qa",
-      "matrix",
-      "--repo-root",
-      "/tmp/openclaw-repo",
-      "--output-dir",
-      ".artifacts/qa/matrix",
-      "--provider-mode",
-      "mock-openai",
-      "--model",
-      "mock-openai/gpt-5.4",
-      "--alt-model",
-      "mock-openai/gpt-5.4-alt",
-      "--scenario",
-      "matrix-thread-follow-up",
-      "--scenario",
-      "matrix-thread-isolation",
-      "--fast",
-      "--sut-account",
-      "sut-live",
-    ]);
 
-    expect(runQaMatrixCommand).toHaveBeenCalledWith({
+  it("keeps Telegram credential flags on the shared host CLI", () => {
-      repoRoot: "/tmp/openclaw-repo",
+    const qa = program.commands.find((command) => command.name() === "qa");
-      outputDir: ".artifacts/qa/matrix",
+    const telegram = qa?.commands.find((command) => command.name() === "telegram");
-      providerMode: "mock-openai",
+    const optionNames = telegram?.options.map((option) => option.long) ?? [];
-      primaryModel: "mock-openai/gpt-5.4",
+
-      alternateModel: "mock-openai/gpt-5.4-alt",
+    expect(optionNames).toEqual(
-      fastMode: true,
+      expect.arrayContaining(["--credential-source", "--credential-role"]),
-      scenarioIds: ["matrix-thread-follow-up", "matrix-thread-isolation"],
+    );
-      sutAccountId: "sut-live",
+  });
-      credentialSource: undefined,
+
-      credentialRole: undefined,
+  it("shows an install hint when a discovered runner plugin is unavailable", async () => {
-    });
+    listQaRunnerCliContributions.mockReset().mockReturnValue([createMissingQaRunnerContribution()]);
+    const missingProgram = new Command();
+    registerQaLabCli(missingProgram);
+
+    await expect(
+      missingProgram.parseAsync(["node", "openclaw", "qa", TEST_QA_RUNNER.commandName]),
+    ).rejects.toThrow(`openclaw plugins install ${TEST_QA_RUNNER.npmSpec}`);
+  });
+
+  it("shows an enable hint when a discovered runner plugin is installed but blocked", async () => {
+    listQaRunnerCliContributions.mockReset().mockReturnValue([createBlockedQaRunnerContribution()]);
+    const blockedProgram = new Command();
+    registerQaLabCli(blockedProgram);
+
+    await expect(
+      blockedProgram.parseAsync(["node", "openclaw", "qa", TEST_QA_RUNNER.commandName]),
+    ).rejects.toThrow(`Enable or allow plugin "${TEST_QA_RUNNER.pluginId}"`);
+  });
+
+  it("rejects discovered runners that collide with built-in qa subcommands", () => {
+    listQaRunnerCliContributions
+      .mockReset()
+      .mockReturnValue([createConflictingQaRunnerContribution("manual")]);
+
+    expect(() => registerQaLabCli(new Command())).toThrow(
+      'QA runner command "manual" conflicts with an existing qa subcommand',
+    );
   });
 
   it("routes telegram CLI defaults into the lane runtime", async () => {

extensions/qa-lab/src/cli.ts

@@ -1,6 +1,6 @@
 import type { Command } from "commander";
 import { collectString } from "./cli-options.js";
-import { LIVE_TRANSPORT_QA_CLI_REGISTRATIONS } from "./live-transports/cli.js";
+import { listLiveTransportQaCliRegistrations } from "./live-transports/cli.js";
 import type { QaProviderModeInput } from "./run-config.js";
 import { hasQaScenarioPack } from "./scenario-catalog.js";
 
@@ -183,6 +183,12 @@ export function isQaLabCliAvailable(): boolean {
   return hasQaScenarioPack();
 }
 
+function assertNoQaSubcommandCollision(qa: Command, commandName: string) {
+  if (qa.commands.some((command) => command.name() === commandName)) {
+    throw new Error(`QA runner command "${commandName}" conflicts with an existing qa subcommand`);
+  }
+}
+
 export function registerQaLabCli(program: Command) {
   const qa = program
     .command("qa")
@@ -284,10 +290,6 @@ export function registerQaLabCli(program: Command) {
       },
     );
 
-  for (const lane of LIVE_TRANSPORT_QA_CLI_REGISTRATIONS) {
-    lane.register(qa);
-  }
-
   qa.command("character-eval")
     .description("Run the character QA scenario across live models and write a judged report")
     .option("--repo-root <path>", "Repository root to target when running from a neutral cwd")
@@ -579,4 +581,9 @@ export function registerQaLabCli(program: Command) {
     .action(async (opts: { host?: string; port?: number }) => {
       await runQaMockOpenAi(opts);
     });
+
+  for (const lane of listLiveTransportQaCliRegistrations()) {
+    assertNoQaSubcommandCollision(qa, lane.commandName);
+    lane.register(qa);
+  }
 }

extensions/qa-lab/src/live-transports/cli.ts

@@ -1,8 +1,78 @@
-import { matrixQaCliRegistration } from "./matrix/cli.js";
+import { listQaRunnerCliContributions } from "openclaw/plugin-sdk/qa-runner-runtime";
 import type { LiveTransportQaCliRegistration } from "./shared/live-transport-cli.js";
 import { telegramQaCliRegistration } from "./telegram/cli.js";
 
+function createMissingQaRunnerCliRegistration(params: {
+  commandName: string;
+  description: string;
+  npmSpec: string;
+}): LiveTransportQaCliRegistration {
+  return {
+    commandName: params.commandName,
+    register(qa) {
+      qa.command(params.commandName)
+        .description(params.description)
+        .action(() => {
+          throw new Error(
+            `QA runner "${params.commandName}" not installed. Install it with "openclaw plugins install ${params.npmSpec}".`,
+          );
+        });
+    },
+  };
+}
+
+function createBlockedQaRunnerCliRegistration(params: {
+  commandName: string;
+  description?: string;
+  pluginId: string;
+}): LiveTransportQaCliRegistration {
+  return {
+    commandName: params.commandName,
+    register(qa) {
+      qa.command(params.commandName)
+        .description(params.description ?? `Run the ${params.commandName} live QA lane`)
+        .action(() => {
+          throw new Error(
+            `QA runner "${params.commandName}" is installed but not active. Enable or allow plugin "${params.pluginId}" in your OpenClaw config, then try again.`,
+          );
+        });
+    },
+  };
+}
+
+function createQaRunnerCliRegistration(
+  runner: ReturnType<typeof listQaRunnerCliContributions>[number],
+): LiveTransportQaCliRegistration {
+  if (runner.status === "available") {
+    return runner.registration;
+  }
+  if (runner.status === "blocked") {
+    return createBlockedQaRunnerCliRegistration({
+      commandName: runner.commandName,
+      description: runner.description,
+      pluginId: runner.pluginId,
+    });
+  }
+  return createMissingQaRunnerCliRegistration({
+    commandName: runner.commandName,
+    description:
+      runner.description ??
+      `Run the ${runner.commandName} live QA lane (install ${runner.npmSpec} first)`,
+    npmSpec: runner.npmSpec,
+  });
+}
+
 export const LIVE_TRANSPORT_QA_CLI_REGISTRATIONS: readonly LiveTransportQaCliRegistration[] = [
   telegramQaCliRegistration,
-  matrixQaCliRegistration,
 ];
+
+export function listLiveTransportQaCliRegistrations(): readonly LiveTransportQaCliRegistration[] {
+  const liveRegistrations = [...LIVE_TRANSPORT_QA_CLI_REGISTRATIONS];
+  const discoveredRunners = listQaRunnerCliContributions();
+
+  for (const runner of discoveredRunners) {
+    liveRegistrations.push(createQaRunnerCliRegistration(runner));
+  }
+
+  return liveRegistrations;
+}

extensions/qa-lab/src/live-transports/shared/live-transport-cli.ts

@@ -33,6 +33,11 @@ export type LiveTransportQaCliRegistration = {
   register(qa: Command): void;
 };
 
+export type LiveTransportQaCredentialCliOptions = {
+  sourceDescription?: string;
+  roleDescription?: string;
+};
+
 export function createLazyCliRuntimeLoader<T>(load: () => Promise<T>) {
   let promise: Promise<T> | null = null;
   return async () => {
@@ -61,13 +66,14 @@ export function mapLiveTransportQaCommanderOptions(
 export function registerLiveTransportQaCli(params: {
   qa: Command;
   commandName: string;
+  credentialOptions?: LiveTransportQaCredentialCliOptions;
   description: string;
   outputDirHelp: string;
   scenarioHelp: string;
   sutAccountHelp: string;
   run: (opts: LiveTransportQaCommandOptions) => Promise<void>;
 }) {
-  params.qa
+  const command = params.qa
     .command(params.commandName)
     .description(params.description)
     .option("--repo-root <path>", "Repository root to target when running from a neutral cwd")
@@ -81,22 +87,27 @@ export function registerLiveTransportQaCli(params: {
     .option("--alt-model <ref>", "Alternate provider/model ref")
     .option("--scenario <id>", params.scenarioHelp, collectString, [])
     .option("--fast", "Enable provider fast mode where supported", false)
-    .option("--sut-account <id>", params.sutAccountHelp, "sut")
+    .option("--sut-account <id>", params.sutAccountHelp, "sut");
-    .option(
+
+  if (params.credentialOptions) {
+    command.option(
       "--credential-source <source>",
-      "Credential source for live lanes: env or convex (default: env)",
+      params.credentialOptions.sourceDescription ??
-    )
+        "Credential source for live lanes: env or convex (default: env)",
-    .option(
+    );
-      "--credential-role <role>",
+    if (params.credentialOptions.roleDescription) {
-      "Credential role for convex auth: maintainer or ci (default: maintainer)",
+      command.option("--credential-role <role>", params.credentialOptions.roleDescription);
-    )
+    }
-    .action(async (opts: LiveTransportQaCommanderOptions) => {
+  }
-      await params.run(mapLiveTransportQaCommanderOptions(opts));
+
-    });
+  command.action(async (opts: LiveTransportQaCommanderOptions) => {
+    await params.run(mapLiveTransportQaCommanderOptions(opts));
+  });
 }
 
 export function createLiveTransportQaCliRegistration(params: {
   commandName: string;
+  credentialOptions?: LiveTransportQaCredentialCliOptions;
   description: string;
   outputDirHelp: string;
   scenarioHelp: string;
@@ -109,6 +120,7 @@ export function createLiveTransportQaCliRegistration(params: {
       registerLiveTransportQaCli({
         qa,
         commandName: params.commandName,
+        credentialOptions: params.credentialOptions,
         description: params.description,
         outputDirHelp: params.outputDirHelp,
         scenarioHelp: params.scenarioHelp,

extensions/qa-lab/src/live-transports/telegram/cli.ts

@@ -20,6 +20,10 @@ async function runQaTelegram(opts: LiveTransportQaCommandOptions) {
 export const telegramQaCliRegistration: LiveTransportQaCliRegistration =
   createLiveTransportQaCliRegistration({
     commandName: "telegram",
+    credentialOptions: {
+      sourceDescription: "Credential source for Telegram QA: env or convex (default: env)",
+      roleDescription: "Credential role for convex auth: maintainer or ci (default: maintainer)",
+    },
     description: "Run the manual Telegram live QA lane against a private bot-to-bot group harness",
     outputDirHelp: "Telegram QA artifact directory",
     scenarioHelp: "Run only the named Telegram QA scenario (repeatable)",

extensions/qa-lab/src/runtime-api.ts

@@ -3,6 +3,7 @@ export type { OpenClawConfig } from "openclaw/plugin-sdk/config-runtime";
 export { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
 export { callGatewayFromCli } from "openclaw/plugin-sdk/browser-node-runtime";
 export type { PluginRuntime } from "openclaw/plugin-sdk/runtime-store";
+export { defaultQaRuntimeModelForMode } from "./model-selection.runtime.js";
 export {
   buildQaTarget,
   createQaBusThread,

extensions/qa-lab/src/self-check.ts

@@ -81,7 +81,7 @@ export async function runQaSelfCheckAgainstState(params: {
     timeline,
     notes: params.notes ?? [
       "Vertical slice: qa-channel + qa-lab bus + private debugger surface.",
-      "Docker orchestration, matrix runs, and auto-fix loops remain follow-up work.",
+      "Docker orchestration, additional QA runners, and auto-fix loops remain follow-up work.",
     ],
   });
 

extensions/qa-matrix/cli.runtime.ts

@@ -0,0 +1 @@
+export { runQaMatrixCommand } from "./src/cli.runtime.js";

extensions/qa-matrix/cli.ts

@@ -0,0 +1 @@
+export { qaRunnerCliRegistrations, registerMatrixQaCli } from "./src/cli.js";

extensions/qa-matrix/index.ts

@@ -0,0 +1,8 @@
+import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
+
+export default definePluginEntry({
+  id: "qa-matrix",
+  name: "QA Matrix",
+  description: "Matrix QA transport runner and substrate",
+  register() {},
+});

extensions/qa-matrix/openclaw.plugin.json

@@ -0,0 +1,16 @@
+{
+  "id": "qa-matrix",
+  "name": "QA Matrix",
+  "description": "Matrix QA transport runner and substrate",
+  "qaRunners": [
+    {
+      "commandName": "matrix",
+      "description": "Run the Docker-backed Matrix live QA lane against a disposable homeserver"
+    }
+  ],
+  "configSchema": {
+    "type": "object",
+    "additionalProperties": false,
+    "properties": {}
+  }
+}

extensions/qa-matrix/package.json

@@ -0,0 +1,34 @@
+{
+  "name": "@openclaw/qa-matrix",
+  "version": "2026.4.12",
+  "description": "OpenClaw Matrix QA runner plugin",
+  "type": "module",
+  "devDependencies": {
+    "@openclaw/plugin-sdk": "workspace:*",
+    "openclaw": "workspace:*"
+  },
+  "peerDependencies": {
+    "openclaw": ">=2026.4.12"
+  },
+  "peerDependenciesMeta": {
+    "openclaw": {
+      "optional": true
+    }
+  },
+  "openclaw": {
+    "extensions": [
+      "./index.ts"
+    ],
+    "install": {
+      "npmSpec": "@openclaw/qa-matrix",
+      "defaultChoice": "npm",
+      "minHostVersion": ">=2026.4.12"
+    },
+    "compat": {
+      "pluginApi": ">=2026.4.12"
+    },
+    "build": {
+      "openclawVersion": "2026.4.12"
+    }
+  }
+}

extensions/qa-matrix/runtime-api.ts

@@ -0,0 +1 @@
+export { qaRunnerCliRegistrations } from "./cli.js";

extensions/qa-matrix/runtime.ts

@@ -0,0 +1 @@
+export { runMatrixQaLive } from "./src/runners/contract/runtime.js";

extensions/qa-matrix/src/cli-options.ts

@@ -0,0 +1,4 @@
+export function collectString(value: string, previous: string[]) {
+  const trimmed = value.trim();
+  return trimmed ? [...previous, trimmed] : previous;
+}

extensions/qa-matrix/src/cli-paths.ts

@@ -0,0 +1,16 @@
+import path from "node:path";
+
+export function resolveRepoRelativeOutputDir(repoRoot: string, outputDir?: string) {
+  if (!outputDir) {
+    return undefined;
+  }
+  if (path.isAbsolute(outputDir)) {
+    throw new Error("--output-dir must be a relative path inside the repo root.");
+  }
+  const resolved = path.resolve(repoRoot, outputDir);
+  const relative = path.relative(repoRoot, resolved);
+  if (relative.startsWith("..") || path.isAbsolute(relative)) {
+    throw new Error("--output-dir must stay within the repo root.");
+  }
+  return resolved;
+}

extensions/qa-matrix/src/cli.runtime.test.ts

@@ -2,7 +2,7 @@ import { describe, expect, it, vi } from "vitest";
 
 const runMatrixQaLive = vi.hoisted(() => vi.fn());
 
-vi.mock("./matrix-live.runtime.js", () => ({
+vi.mock("./runners/contract/runtime.js", () => ({
   runMatrixQaLive,
 }));
 

extensions/qa-matrix/src/cli.runtime.ts

@@ -1,9 +1,9 @@
-import type { LiveTransportQaCommandOptions } from "../shared/live-transport-cli.js";
+import { runMatrixQaLive } from "./runners/contract/runtime.js";
+import type { LiveTransportQaCommandOptions } from "./shared/live-transport-cli.js";
 import {
   printLiveTransportQaArtifacts,
   resolveLiveTransportQaRunOptions,
-} from "../shared/live-transport-cli.runtime.js";
+} from "./shared/live-transport-cli.runtime.js";
-import { runMatrixQaLive } from "./matrix-live.runtime.js";
 
 export async function runQaMatrixCommand(opts: LiveTransportQaCommandOptions) {
   const runOptions = resolveLiveTransportQaRunOptions(opts);

extensions/qa-matrix/src/cli.test.ts

@@ -0,0 +1,29 @@
+import { Command } from "commander";
+import { describe, expect, it } from "vitest";
+import { matrixQaCliRegistration } from "./cli.js";
+
+describe("matrix qa cli registration", () => {
+  it("keeps disposable Matrix lane flags focused", () => {
+    const qa = new Command();
+
+    matrixQaCliRegistration.register(qa);
+
+    const matrix = qa.commands.find((command) => command.name() === "matrix");
+    const optionNames = matrix?.options.map((option) => option.long) ?? [];
+
+    expect(optionNames).toEqual(
+      expect.arrayContaining([
+        "--repo-root",
+        "--output-dir",
+        "--provider-mode",
+        "--model",
+        "--alt-model",
+        "--scenario",
+        "--fast",
+        "--sut-account",
+      ]),
+    );
+    expect(optionNames).not.toContain("--credential-source");
+    expect(optionNames).not.toContain("--credential-role");
+  });
+});

extensions/qa-matrix/src/cli.ts

@@ -4,7 +4,7 @@ import {
   createLiveTransportQaCliRegistration,
   type LiveTransportQaCliRegistration,
   type LiveTransportQaCommandOptions,
-} from "../shared/live-transport-cli.js";
+} from "./shared/live-transport-cli.js";
 
 type MatrixQaCliRuntime = typeof import("./cli.runtime.js");
 
@@ -27,6 +27,8 @@ export const matrixQaCliRegistration: LiveTransportQaCliRegistration =
     run: runQaMatrix,
   });
 
+export const qaRunnerCliRegistrations = [matrixQaCliRegistration] as const;
+
 export function registerMatrixQaCli(qa: Command) {
   matrixQaCliRegistration.register(qa);
 }

extensions/qa-matrix/src/docker-runtime.ts

@@ -0,0 +1,274 @@
+import { createServer } from "node:net";
+import { runExec } from "openclaw/plugin-sdk/process-runtime";
+import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
+
+export type RunCommand = (
+  command: string,
+  args: string[],
+  cwd: string,
+) => Promise<{ stdout: string; stderr: string }>;
+
+export type FetchLike = (input: string) => Promise<{ ok: boolean }>;
+
+export async function fetchHealthUrl(url: string): Promise<{ ok: boolean }> {
+  const { response, release } = await fetchWithSsrFGuard({
+    url,
+    init: {
+      signal: AbortSignal.timeout(2_000),
+    },
+    policy: { allowPrivateNetwork: true },
+    auditContext: "qa-matrix-docker-health-check",
+  });
+  try {
+    return { ok: response.ok };
+  } finally {
+    await release();
+  }
+}
+
+export function describeError(error: unknown) {
+  if (error instanceof Error) {
+    return error.message;
+  }
+  if (typeof error === "string") {
+    return error;
+  }
+  return JSON.stringify(error);
+}
+
+async function isPortFree(port: number) {
+  return await new Promise<boolean>((resolve) => {
+    const server = createServer();
+    server.once("error", () => resolve(false));
+    server.listen(port, "127.0.0.1", () => {
+      server.close(() => resolve(true));
+    });
+  });
+}
+
+async function findFreePort() {
+  return await new Promise<number>((resolve, reject) => {
+    const server = createServer();
+    server.once("error", reject);
+    server.listen(0, () => {
+      const address = server.address();
+      if (!address || typeof address === "string") {
+        server.close();
+        reject(new Error("failed to find free port"));
+        return;
+      }
+      server.close((error) => {
+        if (error) {
+          reject(error);
+          return;
+        }
+        resolve(address.port);
+      });
+    });
+  });
+}
+
+export async function resolveHostPort(preferredPort: number, pinned: boolean) {
+  if (pinned || (await isPortFree(preferredPort))) {
+    return preferredPort;
+  }
+  return await findFreePort();
+}
+
+function trimCommandOutput(output: string) {
+  const trimmed = output.trim();
+  if (!trimmed) {
+    return "";
+  }
+  const lines = trimmed.split("\n");
+  return lines.length <= 120 ? trimmed : lines.slice(-120).join("\n");
+}
+
+export async function execCommand(command: string, args: string[], cwd: string) {
+  try {
+    return await runExec(command, args, { cwd, maxBuffer: 10 * 1024 * 1024 });
+  } catch (error) {
+    const failedProcess = error as Error & { stdout?: string; stderr?: string };
+    const renderedStdout = trimCommandOutput(failedProcess.stdout ?? "");
+    const renderedStderr = trimCommandOutput(failedProcess.stderr ?? "");
+    throw new Error(
+      [
+        `Command failed: ${[command, ...args].join(" ")}`,
+        renderedStderr ? `stderr:\n${renderedStderr}` : "",
+        renderedStdout ? `stdout:\n${renderedStdout}` : "",
+      ]
+        .filter(Boolean)
+        .join("\n\n"),
+      { cause: error },
+    );
+  }
+}
+
+export async function waitForHealth(
+  url: string,
+  deps: {
+    label?: string;
+    composeFile?: string;
+    fetchImpl: FetchLike;
+    sleepImpl: (ms: number) => Promise<unknown>;
+    timeoutMs?: number;
+    pollMs?: number;
+  },
+) {
+  const timeoutMs = deps.timeoutMs ?? 360_000;
+  const pollMs = deps.pollMs ?? 1_000;
+  const startMs = Date.now();
+  const deadline = startMs + timeoutMs;
+  let lastError: unknown = null;
+
+  while (Date.now() < deadline) {
+    try {
+      const response = await deps.fetchImpl(url);
+      if (response.ok) {
+        return;
+      }
+      lastError = new Error(`Health check returned non-OK for ${url}`);
+    } catch (error) {
+      lastError = error;
+    }
+    await deps.sleepImpl(pollMs);
+  }
+
+  const elapsedSec = Math.round((Date.now() - startMs) / 1000);
+  const service = deps.label ?? url;
+  const lines = [
+    `${service} did not become healthy within ${elapsedSec}s (limit ${Math.round(timeoutMs / 1000)}s).`,
+    lastError ? `Last error: ${describeError(lastError)}` : "",
+    `Hint: check container logs with \`docker compose -f ${deps.composeFile ?? "<compose-file>"} logs\` and verify the port is not already in use.`,
+  ];
+  throw new Error(lines.filter(Boolean).join("\n"));
+}
+
+async function isHealthy(url: string, fetchImpl: FetchLike) {
+  try {
+    const response = await fetchImpl(url);
+    return response.ok;
+  } catch {
+    return false;
+  }
+}
+
+function normalizeDockerServiceStatus(row?: { Health?: string; State?: string }) {
+  const health = row?.Health?.trim();
+  if (health) {
+    return health;
+  }
+  const state = row?.State?.trim();
+  if (state) {
+    return state;
+  }
+  return "unknown";
+}
+
+function parseDockerComposePsRows(stdout: string) {
+  const trimmed = stdout.trim();
+  if (!trimmed) {
+    return [] as Array<{ Health?: string; State?: string }>;
+  }
+
+  try {
+    const parsed = JSON.parse(trimmed) as
+      | Array<{ Health?: string; State?: string }>
+      | { Health?: string; State?: string };
+    if (Array.isArray(parsed)) {
+      return parsed;
+    }
+    return [parsed];
+  } catch {
+    return trimmed
+      .split("\n")
+      .map((line) => line.trim())
+      .filter(Boolean)
+      .map((line) => JSON.parse(line) as { Health?: string; State?: string });
+  }
+}
+
+export async function waitForDockerServiceHealth(
+  service: string,
+  composeFile: string,
+  repoRoot: string,
+  runCommand: RunCommand,
+  sleepImpl: (ms: number) => Promise<unknown>,
+  timeoutMs = 360_000,
+  pollMs = 1_000,
+) {
+  const startMs = Date.now();
+  const deadline = startMs + timeoutMs;
+  let lastStatus = "unknown";
+
+  while (Date.now() < deadline) {
+    try {
+      const { stdout } = await runCommand(
+        "docker",
+        ["compose", "-f", composeFile, "ps", "--format", "json", service],
+        repoRoot,
+      );
+      const rows = parseDockerComposePsRows(stdout);
+      const row = rows[0];
+      lastStatus = normalizeDockerServiceStatus(row);
+      if (lastStatus === "healthy" || lastStatus === "running") {
+        return;
+      }
+    } catch (error) {
+      lastStatus = describeError(error);
+    }
+    await sleepImpl(pollMs);
+  }
+
+  const elapsedSec = Math.round((Date.now() - startMs) / 1000);
+  throw new Error(
+    [
+      `${service} did not become healthy within ${elapsedSec}s (limit ${Math.round(timeoutMs / 1000)}s).`,
+      `Last status: ${lastStatus}`,
+      `Hint: check container logs with \`docker compose -f ${composeFile} logs ${service}\`.`,
+    ].join("\n"),
+  );
+}
+
+export async function resolveComposeServiceUrl(
+  service: string,
+  port: number,
+  composeFile: string,
+  repoRoot: string,
+  runCommand: RunCommand,
+  fetchImpl?: FetchLike,
+) {
+  const { stdout: containerStdout } = await runCommand(
+    "docker",
+    ["compose", "-f", composeFile, "ps", "-q", service],
+    repoRoot,
+  );
+  const containerId = containerStdout.trim();
+  if (!containerId) {
+    return null;
+  }
+  const { stdout: ipStdout } = await runCommand(
+    "docker",
+    [
+      "inspect",
+      "--format",
+      "{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}",
+      containerId,
+    ],
+    repoRoot,
+  );
+  const ip = ipStdout.trim();
+  if (!ip) {
+    return null;
+  }
+  const baseUrl = `http://${ip}:${port}/`;
+  if (!fetchImpl) {
+    return baseUrl;
+  }
+  return (await isHealthy(`${baseUrl}healthz`, fetchImpl)) ? baseUrl : null;
+}
+
+export const __testing = {
+  fetchHealthUrl,
+  normalizeDockerServiceStatus,
+};

extensions/qa-matrix/src/report.ts

@@ -0,0 +1,100 @@
+export type QaReportCheck = {
+  name: string;
+  status: "pass" | "fail" | "skip";
+  details?: string;
+};
+
+export type QaReportScenario = {
+  name: string;
+  status: "pass" | "fail" | "skip";
+  details?: string;
+  steps?: QaReportCheck[];
+};
+
+function pushDetailsBlock(lines: string[], label: string, details: string, indent = "") {
+  if (!details.includes("\n")) {
+    lines.push(`${indent}- ${label}: ${details}`);
+    return;
+  }
+  lines.push(`${indent}- ${label}:`);
+  lines.push("", "```text", details, "```");
+}
+
+export function renderQaMarkdownReport(params: {
+  title: string;
+  startedAt: Date;
+  finishedAt: Date;
+  checks?: QaReportCheck[];
+  scenarios?: QaReportScenario[];
+  timeline?: string[];
+  notes?: string[];
+}) {
+  const checks = params.checks ?? [];
+  const scenarios = params.scenarios ?? [];
+  const passCount =
+    checks.filter((check) => check.status === "pass").length +
+    scenarios.filter((scenario) => scenario.status === "pass").length;
+  const failCount =
+    checks.filter((check) => check.status === "fail").length +
+    scenarios.filter((scenario) => scenario.status === "fail").length;
+
+  const lines = [
+    `# ${params.title}`,
+    "",
+    `- Started: ${params.startedAt.toISOString()}`,
+    `- Finished: ${params.finishedAt.toISOString()}`,
+    `- Duration ms: ${params.finishedAt.getTime() - params.startedAt.getTime()}`,
+    `- Passed: ${passCount}`,
+    `- Failed: ${failCount}`,
+    "",
+  ];
+
+  if (checks.length > 0) {
+    lines.push("## Checks", "");
+    for (const check of checks) {
+      lines.push(`- [${check.status === "pass" ? "x" : " "}] ${check.name}`);
+      if (check.details) {
+        pushDetailsBlock(lines, "Details", check.details, "  ");
+      }
+    }
+  }
+
+  if (scenarios.length > 0) {
+    lines.push("", "## Scenarios", "");
+    for (const scenario of scenarios) {
+      lines.push(`### ${scenario.name}`);
+      lines.push("");
+      lines.push(`- Status: ${scenario.status}`);
+      if (scenario.details) {
+        pushDetailsBlock(lines, "Details", scenario.details);
+      }
+      if (scenario.steps?.length) {
+        lines.push("- Steps:");
+        for (const step of scenario.steps) {
+          lines.push(`  - [${step.status === "pass" ? "x" : " "}] ${step.name}`);
+          if (step.details) {
+            pushDetailsBlock(lines, "Details", step.details, "    ");
+          }
+        }
+      }
+      lines.push("");
+    }
+  }
+
+  if (params.timeline && params.timeline.length > 0) {
+    lines.push("## Timeline", "");
+    for (const item of params.timeline) {
+      lines.push(`- ${item}`);
+    }
+  }
+
+  if (params.notes && params.notes.length > 0) {
+    lines.push("", "## Notes", "");
+    for (const note of params.notes) {
+      lines.push(`- ${note}`);
+    }
+  }
+
+  lines.push("");
+  return lines.join("\n");
+}

extensions/qa-matrix/src/run-config.ts

@@ -0,0 +1,9 @@
+export type QaProviderMode = "mock-openai" | "live-frontier";
+export type QaProviderModeInput = QaProviderMode | "live-openai";
+
+export function normalizeQaProviderMode(input: unknown): QaProviderMode {
+  if (input === "mock-openai") {
+    return "mock-openai";
+  }
+  return "live-frontier";
+}

extensions/qa-matrix/src/runners/contract/model-selection.test.ts

@@ -0,0 +1,51 @@
+import { beforeEach, describe, expect, it, vi } from "vitest";
+
+const loadQaLabRuntimeModule = vi.hoisted(() => vi.fn());
+const defaultQaRuntimeModelForMode = vi.hoisted(() => vi.fn());
+
+vi.mock("openclaw/plugin-sdk/qa-lab-runtime", () => ({
+  loadQaLabRuntimeModule,
+}));
+
+describe("matrix qa model selection", () => {
+  beforeEach(() => {
+    defaultQaRuntimeModelForMode.mockReset().mockImplementation((mode, options) =>
+      options?.alternate ? `${mode}:alt` : `${mode}:primary`,
+    );
+    loadQaLabRuntimeModule.mockReset().mockReturnValue({
+      defaultQaRuntimeModelForMode,
+    });
+  });
+
+  it("delegates default model selection through qa-lab runtime defaults", async () => {
+    const { resolveMatrixQaModels } = await import("./model-selection.js");
+
+    expect(resolveMatrixQaModels({ providerMode: "live-openai" })).toEqual({
+      providerMode: "live-frontier",
+      primaryModel: "live-frontier:primary",
+      alternateModel: "live-frontier:alt",
+    });
+    expect(defaultQaRuntimeModelForMode).toHaveBeenNthCalledWith(1, "live-frontier");
+    expect(defaultQaRuntimeModelForMode).toHaveBeenNthCalledWith(2, "live-frontier", {
+      alternate: true,
+    });
+  });
+
+  it("preserves explicit model overrides", async () => {
+    const { resolveMatrixQaModels } = await import("./model-selection.js");
+
+    expect(
+      resolveMatrixQaModels({
+        providerMode: "mock-openai",
+        primaryModel: "custom-primary",
+        alternateModel: "custom-alt",
+      }),
+    ).toEqual({
+      providerMode: "mock-openai",
+      primaryModel: "custom-primary",
+      alternateModel: "custom-alt",
+    });
+    expect(loadQaLabRuntimeModule).not.toHaveBeenCalled();
+    expect(defaultQaRuntimeModelForMode).not.toHaveBeenCalled();
+  });
+});

extensions/qa-matrix/src/runners/contract/model-selection.ts

@@ -0,0 +1,33 @@
+import { loadQaLabRuntimeModule } from "openclaw/plugin-sdk/qa-lab-runtime";
+import { normalizeQaProviderMode, type QaProviderModeInput } from "../../run-config.js";
+
+export type ResolvedMatrixQaModels = {
+  providerMode: ReturnType<typeof normalizeQaProviderMode>;
+  primaryModel: string;
+  alternateModel: string;
+};
+
+export function resolveMatrixQaModels(params: {
+  providerMode?: QaProviderModeInput;
+  primaryModel?: string;
+  alternateModel?: string;
+}): ResolvedMatrixQaModels {
+  const providerMode = normalizeQaProviderMode(params.providerMode ?? "live-frontier");
+  const primaryModel = params.primaryModel?.trim();
+  const alternateModel = params.alternateModel?.trim();
+  if (primaryModel && alternateModel) {
+    return {
+      providerMode,
+      primaryModel,
+      alternateModel,
+    };
+  }
+
+  const qaLabRuntime = loadQaLabRuntimeModule();
+  return {
+    providerMode,
+    primaryModel: primaryModel || qaLabRuntime.defaultQaRuntimeModelForMode(providerMode),
+    alternateModel:
+      alternateModel || qaLabRuntime.defaultQaRuntimeModelForMode(providerMode, { alternate: true }),
+  };
+}

extensions/qa-matrix/src/runners/contract/runtime.test.ts

@@ -1,6 +1,6 @@
 import type { OpenClawConfig } from "openclaw/plugin-sdk/config-runtime";
 import { afterEach, describe, expect, it, vi } from "vitest";
-import { __testing as liveTesting } from "./matrix-live.runtime.js";
+import { __testing as liveTesting } from "./runtime.js";
 
 afterEach(() => {
   vi.useRealTimers();

extensions/qa-matrix/src/runners/contract/runtime.ts

@@ -4,22 +4,20 @@ import path from "node:path";
 import { setTimeout as sleep } from "node:timers/promises";
 import type { OpenClawConfig } from "openclaw/plugin-sdk/config-runtime";
 import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
-import { startQaGatewayChild } from "../../gateway-child.js";
+import { loadQaLabRuntimeModule } from "openclaw/plugin-sdk/qa-lab-runtime";
 import type { QaReportCheck } from "../../report.js";
 import { renderQaMarkdownReport } from "../../report.js";
+import { type QaProviderModeInput } from "../../run-config.js";
 import {
-  defaultQaModelForMode,
+  appendLiveLaneIssue,
-  normalizeQaProviderMode,
+  buildLiveLaneArtifactsError,
-  type QaProviderModeInput,
+} from "../../shared/live-lane-helpers.js";
-} from "../../run-config.js";
-import { startQaLiveLaneGateway } from "../shared/live-gateway.runtime.js";
-import { appendLiveLaneIssue, buildLiveLaneArtifactsError } from "../shared/live-lane-helpers.js";
 import {
   provisionMatrixQaRoom,
   type MatrixQaObservedEvent,
   type MatrixQaProvisionResult,
-} from "./matrix-driver-client.js";
+} from "../../substrate/client.js";
-import { startMatrixQaHarness } from "./matrix-harness.runtime.js";
+import { startMatrixQaHarness } from "../../substrate/harness.runtime.js";
 import {
   MATRIX_QA_SCENARIOS,
   buildMatrixReplyDetails,
@@ -28,7 +26,22 @@ import {
   runMatrixQaScenario,
   type MatrixQaCanaryArtifact,
   type MatrixQaScenarioArtifacts,
-} from "./matrix-live-scenarios.js";
+} from "./scenarios.js";
+import { resolveMatrixQaModels } from "./model-selection.js";
+
+type MatrixQaGatewayChild = {
+  call(
+    method: string,
+    params: Record<string, unknown>,
+    options?: { timeoutMs?: number },
+  ): Promise<unknown>;
+  restart(): Promise<void>;
+};
+
+type MatrixQaLiveLaneGatewayHarness = {
+  gateway: MatrixQaGatewayChild;
+  stop(): Promise<void>;
+};
 
 type MatrixQaScenarioResult = {
   artifacts?: MatrixQaScenarioArtifacts;
@@ -214,7 +227,7 @@ function isMatrixAccountReady(entry?: {
 }
 
 async function waitForMatrixChannelReady(
-  gateway: Awaited<ReturnType<typeof startQaGatewayChild>>,
+  gateway: MatrixQaGatewayChild,
   accountId: string,
   opts?: {
     pollMs?: number;
@@ -255,6 +268,27 @@ async function waitForMatrixChannelReady(
   throw new Error(`matrix account "${accountId}" did not become ready`);
 }
 
+async function startMatrixQaLiveLaneGateway(params: {
+  repoRoot: string;
+  transport: {
+    requiredPluginIds: readonly string[];
+    createGatewayConfig: (params: {
+      baseUrl: string;
+    }) => Pick<OpenClawConfig, "channels" | "messages">;
+  };
+  transportBaseUrl: string;
+  providerMode: "mock-openai" | "live-frontier";
+  primaryModel: string;
+  alternateModel: string;
+  fastMode?: boolean;
+  controlUiEnabled?: boolean;
+  mutateConfig?: (cfg: OpenClawConfig) => OpenClawConfig;
+}): Promise<MatrixQaLiveLaneGatewayHarness> {
+  return (await loadQaLabRuntimeModule().startQaLiveLaneGateway(
+    params,
+  )) as MatrixQaLiveLaneGatewayHarness;
+}
+
 export async function runMatrixQaLive(params: {
   fastMode?: boolean;
   outputDir?: string;
@@ -271,9 +305,11 @@ export async function runMatrixQaLive(params: {
     path.join(repoRoot, ".artifacts", "qa-e2e", `matrix-${Date.now().toString(36)}`);
   await fs.mkdir(outputDir, { recursive: true });
 
-  const providerMode = normalizeQaProviderMode(params.providerMode ?? "live-frontier");
+  const { providerMode, primaryModel, alternateModel } = resolveMatrixQaModels({
-  const primaryModel = params.primaryModel?.trim() || defaultQaModelForMode(providerMode);
+    providerMode: params.providerMode,
-  const alternateModel = params.alternateModel?.trim() || defaultQaModelForMode(providerMode, true);
+    primaryModel: params.primaryModel,
+    alternateModel: params.alternateModel,
+  });
   const sutAccountId = params.sutAccountId?.trim() || "sut";
   const scenarios = findMatrixQaScenarios(params.scenarioIds);
   const observedEvents: MatrixQaObservedEvent[] = [];
@@ -317,12 +353,12 @@ export async function runMatrixQaLive(params: {
   const scenarioResults: MatrixQaScenarioResult[] = [];
   const cleanupErrors: string[] = [];
   let canaryArtifact: MatrixQaCanaryArtifact | undefined;
-  let gatewayHarness: Awaited<ReturnType<typeof startQaLiveLaneGateway>> | null = null;
+  let gatewayHarness: MatrixQaLiveLaneGatewayHarness | null = null;
   let canaryFailed = false;
   const syncState: { driver?: string; observer?: string } = {};
 
   try {
-    gatewayHarness = await startQaLiveLaneGateway({
+    gatewayHarness = await startMatrixQaLiveLaneGateway({
       repoRoot,
       transport: {
         requiredPluginIds: [],
@@ -555,5 +591,6 @@ export const __testing = {
   buildMatrixQaConfig,
   buildObservedEventsArtifact,
   isMatrixAccountReady,
+  resolveMatrixQaModels,
   waitForMatrixChannelReady,
 };

extensions/qa-matrix/src/runners/contract/scenarios.test.ts

@@ -3,19 +3,19 @@ const { createMatrixQaClient } = vi.hoisted(() => ({
   createMatrixQaClient: vi.fn(),
 }));
 
-vi.mock("./matrix-driver-client.js", () => ({
+vi.mock("../../substrate/client.js", () => ({
   createMatrixQaClient,
 }));
 
 import {
   LIVE_TRANSPORT_BASELINE_STANDARD_SCENARIO_IDS,
   findMissingLiveTransportStandardScenarios,
-} from "../shared/live-transport-scenarios.js";
+} from "../../shared/live-transport-scenarios.js";
 import {
   __testing as scenarioTesting,
   MATRIX_QA_SCENARIOS,
   runMatrixQaScenario,
-} from "./matrix-live-scenarios.js";
+} from "./scenarios.js";
 
 describe("matrix live qa scenarios", () => {
   beforeEach(() => {

extensions/qa-matrix/src/runners/contract/scenarios.ts

@@ -3,8 +3,8 @@ import {
   collectLiveTransportStandardScenarioCoverage,
   selectLiveTransportScenarios,
   type LiveTransportScenarioDefinition,
-} from "../shared/live-transport-scenarios.js";
+} from "../../shared/live-transport-scenarios.js";
-import { createMatrixQaClient, type MatrixQaObservedEvent } from "./matrix-driver-client.js";
+import { createMatrixQaClient, type MatrixQaObservedEvent } from "../../substrate/client.js";
 
 export type MatrixQaScenarioId =
   | "matrix-thread-follow-up"

extensions/qa-matrix/src/runtime-api.test.ts

@@ -0,0 +1,9 @@
+import { describe, expect, it } from "vitest";
+
+describe("matrix qa runtime api surface", () => {
+  it("keeps runner discovery lightweight", async () => {
+    const runtimeApi = await import("../runtime-api.js");
+
+    expect(Object.keys(runtimeApi).toSorted()).toEqual(["qaRunnerCliRegistrations"]);
+  });
+});

extensions/qa-matrix/src/shared/live-lane-helpers.ts

@@ -0,0 +1,18 @@
+import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
+
+export function appendLiveLaneIssue(issues: string[], label: string, error: unknown) {
+  issues.push(`${label}: ${formatErrorMessage(error)}`);
+}
+
+export function buildLiveLaneArtifactsError(params: {
+  heading: string;
+  artifacts: Record<string, string>;
+  details?: string[];
+}) {
+  return [
+    params.heading,
+    ...(params.details ?? []),
+    "Artifacts:",
+    ...Object.entries(params.artifacts).map(([label, filePath]) => `- ${label}: ${filePath}`),
+  ].join("\n");
+}

extensions/qa-matrix/src/shared/live-transport-cli.runtime.ts

@@ -0,0 +1,40 @@
+import path from "node:path";
+import { resolveRepoRelativeOutputDir } from "../cli-paths.js";
+import type { QaProviderMode } from "../run-config.js";
+import { normalizeQaProviderMode } from "../run-config.js";
+import type { LiveTransportQaCommandOptions } from "./live-transport-cli.js";
+
+export function resolveLiveTransportQaRunOptions(
+  opts: LiveTransportQaCommandOptions,
+): LiveTransportQaCommandOptions & {
+  repoRoot: string;
+  providerMode: QaProviderMode;
+} {
+  return {
+    repoRoot: path.resolve(opts.repoRoot ?? process.cwd()),
+    outputDir: resolveRepoRelativeOutputDir(
+      path.resolve(opts.repoRoot ?? process.cwd()),
+      opts.outputDir,
+    ),
+    providerMode:
+      opts.providerMode === undefined
+        ? "live-frontier"
+        : normalizeQaProviderMode(opts.providerMode),
+    primaryModel: opts.primaryModel,
+    alternateModel: opts.alternateModel,
+    fastMode: opts.fastMode,
+    scenarioIds: opts.scenarioIds,
+    sutAccountId: opts.sutAccountId,
+    credentialSource: opts.credentialSource?.trim(),
+    credentialRole: opts.credentialRole?.trim(),
+  };
+}
+
+export function printLiveTransportQaArtifacts(
+  laneLabel: string,
+  artifacts: Record<string, string>,
+) {
+  for (const [label, filePath] of Object.entries(artifacts)) {
+    process.stdout.write(`${laneLabel} ${label}: ${filePath}\n`);
+  }
+}

extensions/qa-matrix/src/shared/live-transport-cli.ts

@@ -0,0 +1,132 @@
+import type { Command } from "commander";
+import { collectString } from "../cli-options.js";
+import type { QaProviderModeInput } from "../run-config.js";
+
+export type LiveTransportQaCommandOptions = {
+  repoRoot?: string;
+  outputDir?: string;
+  providerMode?: QaProviderModeInput;
+  primaryModel?: string;
+  alternateModel?: string;
+  fastMode?: boolean;
+  scenarioIds?: string[];
+  sutAccountId?: string;
+  credentialSource?: string;
+  credentialRole?: string;
+};
+
+type LiveTransportQaCommanderOptions = {
+  repoRoot?: string;
+  outputDir?: string;
+  providerMode?: QaProviderModeInput;
+  model?: string;
+  altModel?: string;
+  scenario?: string[];
+  fast?: boolean;
+  sutAccount?: string;
+  credentialSource?: string;
+  credentialRole?: string;
+};
+
+export type LiveTransportQaCliRegistration = {
+  commandName: string;
+  register(qa: Command): void;
+};
+
+export type LiveTransportQaCredentialCliOptions = {
+  sourceDescription?: string;
+  roleDescription?: string;
+};
+
+export function createLazyCliRuntimeLoader<T>(load: () => Promise<T>) {
+  let promise: Promise<T> | null = null;
+  return async () => {
+    promise ??= load();
+    return await promise;
+  };
+}
+
+export function mapLiveTransportQaCommanderOptions(
+  opts: LiveTransportQaCommanderOptions,
+): LiveTransportQaCommandOptions {
+  return {
+    repoRoot: opts.repoRoot,
+    outputDir: opts.outputDir,
+    providerMode: opts.providerMode,
+    primaryModel: opts.model,
+    alternateModel: opts.altModel,
+    fastMode: opts.fast,
+    scenarioIds: opts.scenario,
+    sutAccountId: opts.sutAccount,
+    credentialSource: opts.credentialSource,
+    credentialRole: opts.credentialRole,
+  };
+}
+
+export function registerLiveTransportQaCli(params: {
+  qa: Command;
+  commandName: string;
+  credentialOptions?: LiveTransportQaCredentialCliOptions;
+  description: string;
+  outputDirHelp: string;
+  scenarioHelp: string;
+  sutAccountHelp: string;
+  run: (opts: LiveTransportQaCommandOptions) => Promise<void>;
+}) {
+  const command = params.qa
+    .command(params.commandName)
+    .description(params.description)
+    .option("--repo-root <path>", "Repository root to target when running from a neutral cwd")
+    .option("--output-dir <path>", params.outputDirHelp)
+    .option(
+      "--provider-mode <mode>",
+      "Provider mode: mock-openai or live-frontier (legacy live-openai still works)",
+      "live-frontier",
+    )
+    .option("--model <ref>", "Primary provider/model ref")
+    .option("--alt-model <ref>", "Alternate provider/model ref")
+    .option("--scenario <id>", params.scenarioHelp, collectString, [])
+    .option("--fast", "Enable provider fast mode where supported", false)
+    .option("--sut-account <id>", params.sutAccountHelp, "sut");
+
+  if (params.credentialOptions) {
+    command.option(
+      "--credential-source <source>",
+      params.credentialOptions.sourceDescription ??
+        "Credential source for live lanes: env or convex (default: env)",
+    );
+    if (params.credentialOptions.roleDescription) {
+      command.option("--credential-role <role>", params.credentialOptions.roleDescription);
+    }
+  }
+
+  command.action(async (opts: LiveTransportQaCommanderOptions) => {
+    await params.run(mapLiveTransportQaCommanderOptions(opts));
+  });
+}
+
+export function createLiveTransportQaCliRegistration(params: {
+  commandName: string;
+  credentialOptions?: LiveTransportQaCredentialCliOptions;
+  description: string;
+  outputDirHelp: string;
+  scenarioHelp: string;
+  sutAccountHelp: string;
+  run: (opts: LiveTransportQaCommandOptions) => Promise<void>;
+}): LiveTransportQaCliRegistration {
+  return {
+    commandName: params.commandName,
+    register(qa: Command) {
+      registerLiveTransportQaCli({
+        qa,
+        commandName: params.commandName,
+        credentialOptions: params.credentialOptions,
+        description: params.description,
+        outputDirHelp: params.outputDirHelp,
+        scenarioHelp: params.scenarioHelp,
+        sutAccountHelp: params.sutAccountHelp,
+        run: params.run,
+      });
+    },
+  };
+}

extensions/qa-matrix/src/shared/live-transport-scenarios.ts

@@ -0,0 +1,149 @@
+export type LiveTransportStandardScenarioId =
+  | "canary"
+  | "mention-gating"
+  | "allowlist-block"
+  | "top-level-reply-shape"
+  | "restart-resume"
+  | "thread-follow-up"
+  | "thread-isolation"
+  | "reaction-observation"
+  | "help-command";
+
+export type LiveTransportScenarioDefinition<TId extends string = string> = {
+  id: TId;
+  standardId?: LiveTransportStandardScenarioId;
+  timeoutMs: number;
+  title: string;
+};
+
+export type LiveTransportStandardScenarioDefinition = {
+  description: string;
+  id: LiveTransportStandardScenarioId;
+  title: string;
+};
+
+export const LIVE_TRANSPORT_STANDARD_SCENARIOS: readonly LiveTransportStandardScenarioDefinition[] =
+  [
+    {
+      id: "canary",
+      title: "Transport canary",
+      description: "The lane can trigger one known-good reply on the real transport.",
+    },
+    {
+      id: "mention-gating",
+      title: "Mention gating",
+      description: "Messages without the required mention do not trigger a reply.",
+    },
+    {
+      id: "allowlist-block",
+      title: "Sender allowlist block",
+      description: "Non-allowlisted senders do not trigger a reply.",
+    },
+    {
+      id: "top-level-reply-shape",
+      title: "Top-level reply shape",
+      description: "Top-level replies stay top-level when the lane is configured that way.",
+    },
+    {
+      id: "restart-resume",
+      title: "Restart resume",
+      description: "The lane still responds after a gateway restart.",
+    },
+    {
+      id: "thread-follow-up",
+      title: "Thread follow-up",
+      description: "Threaded prompts receive threaded replies with the expected relation metadata.",
+    },
+    {
+      id: "thread-isolation",
+      title: "Thread isolation",
+      description: "Fresh top-level prompts stay out of prior threads.",
+    },
+    {
+      id: "reaction-observation",
+      title: "Reaction observation",
+      description: "Reaction events are observed and normalized correctly.",
+    },
+    {
+      id: "help-command",
+      title: "Help command",
+      description: "The transport-specific help command path replies successfully.",
+    },
+  ] as const;
+
+export const LIVE_TRANSPORT_BASELINE_STANDARD_SCENARIO_IDS: readonly LiveTransportStandardScenarioId[] =
+  [
+    "canary",
+    "mention-gating",
+    "allowlist-block",
+    "top-level-reply-shape",
+    "restart-resume",
+  ] as const;
+
+const LIVE_TRANSPORT_STANDARD_SCENARIO_ID_SET = new Set(
+  LIVE_TRANSPORT_STANDARD_SCENARIOS.map((scenario) => scenario.id),
+);
+
+function assertKnownStandardScenarioIds(ids: readonly LiveTransportStandardScenarioId[]) {
+  for (const id of ids) {
+    if (!LIVE_TRANSPORT_STANDARD_SCENARIO_ID_SET.has(id)) {
+      throw new Error(`unknown live transport standard scenario id: ${id}`);
+    }
+  }
+}
+
+export function selectLiveTransportScenarios<TDefinition extends { id: string }>(params: {
+  ids?: string[];
+  laneLabel: string;
+  scenarios: readonly TDefinition[];
+}) {
+  if (!params.ids || params.ids.length === 0) {
+    return [...params.scenarios];
+  }
+  const requested = new Set(params.ids);
+  const selected = params.scenarios.filter((scenario) => params.ids?.includes(scenario.id));
+  const missingIds = [...requested].filter(
+    (id) => !selected.some((scenario) => scenario.id === id),
+  );
+  if (missingIds.length > 0) {
+    throw new Error(`unknown ${params.laneLabel} QA scenario id(s): ${missingIds.join(", ")}`);
+  }
+  return selected;
+}
+
+export function collectLiveTransportStandardScenarioCoverage<TId extends string>(params: {
+  alwaysOnStandardScenarioIds?: readonly LiveTransportStandardScenarioId[];
+  scenarios: readonly LiveTransportScenarioDefinition<TId>[];
+}) {
+  const coverage: LiveTransportStandardScenarioId[] = [];
+  const seen = new Set<LiveTransportStandardScenarioId>();
+  const append = (id: LiveTransportStandardScenarioId | undefined) => {
+    if (!id || seen.has(id)) {
+      return;
+    }
+    seen.add(id);
+    coverage.push(id);
+  };
+
+  assertKnownStandardScenarioIds(params.alwaysOnStandardScenarioIds ?? []);
+  for (const id of params.alwaysOnStandardScenarioIds ?? []) {
+    append(id);
+  }
+  for (const scenario of params.scenarios) {
+    if (scenario.standardId) {
+      assertKnownStandardScenarioIds([scenario.standardId]);
+    }
+    append(scenario.standardId);
+  }
+  return coverage;
+}
+
+export function findMissingLiveTransportStandardScenarios(params: {
+  coveredStandardScenarioIds: readonly LiveTransportStandardScenarioId[];
+  expectedStandardScenarioIds: readonly LiveTransportStandardScenarioId[];
+}) {
+  assertKnownStandardScenarioIds(params.coveredStandardScenarioIds);
+  assertKnownStandardScenarioIds(params.expectedStandardScenarioIds);
+  const covered = new Set(params.coveredStandardScenarioIds);
+  return params.expectedStandardScenarioIds.filter((id) => !covered.has(id));
+}

extensions/qa-matrix/src/substrate/client.test.ts

@@ -4,7 +4,7 @@ import {
   createMatrixQaClient,
   provisionMatrixQaRoom,
   type MatrixQaObservedEvent,
-} from "./matrix-driver-client.js";
+} from "./client.js";
 
 function resolveRequestUrl(input: RequestInfo | URL) {
   if (typeof input === "string") {

extensions/qa-matrix/src/substrate/harness.runtime.test.ts

@@ -2,11 +2,7 @@ import { mkdtemp, readFile, rm } from "node:fs/promises";
 import os from "node:os";
 import path from "node:path";
 import { describe, expect, it, vi } from "vitest";
-import {
+import { __testing, startMatrixQaHarness, writeMatrixQaHarnessFiles } from "./harness.runtime.js";
-  __testing,
-  startMatrixQaHarness,
-  writeMatrixQaHarnessFiles,
-} from "./matrix-harness.runtime.js";
 
 describe("matrix harness runtime", () => {
   it("writes a pinned Tuwunel compose file and redacted manifest", async () => {

extensions/qa-matrix/src/substrate/harness.runtime.ts

@@ -11,7 +11,7 @@ import {
   waitForHealth,
   type FetchLike,
   type RunCommand,
-} from "../../docker-runtime.js";
+} from "../docker-runtime.js";
 
 const MATRIX_QA_DEFAULT_IMAGE = "ghcr.io/matrix-construct/tuwunel:v1.5.1";
 const MATRIX_QA_DEFAULT_SERVER_NAME = "matrix-qa.test";

package.json

@@ -30,9 +30,8 @@
     "!dist/**/*.map",
     "!dist/plugin-sdk/.tsbuildinfo",
     "!dist/extensions/qa-channel/**",
-    "dist/extensions/qa-channel/runtime-api.js",
     "!dist/extensions/qa-lab/**",
-    "dist/extensions/qa-lab/runtime-api.js",
+    "!dist/extensions/qa-matrix/**",
     "docs/",
     "!docs/.generated/**",
     "!docs/.i18n/zh-CN.tm.jsonl",
@@ -766,6 +765,14 @@
       "types": "./dist/plugin-sdk/matrix-thread-bindings.d.ts",
       "default": "./dist/plugin-sdk/matrix-thread-bindings.js"
     },
+    "./plugin-sdk/qa-lab-runtime": {
+      "types": "./dist/plugin-sdk/qa-lab-runtime.d.ts",
+      "default": "./dist/plugin-sdk/qa-lab-runtime.js"
+    },
+    "./plugin-sdk/qa-runner-runtime": {
+      "types": "./dist/plugin-sdk/qa-runner-runtime.d.ts",
+      "default": "./dist/plugin-sdk/qa-runner-runtime.js"
+    },
     "./plugin-sdk/mattermost": {
       "types": "./dist/plugin-sdk/mattermost.d.ts",
       "default": "./dist/plugin-sdk/mattermost.js"
@@ -1236,6 +1243,8 @@
     "proxy:install-ca": "node --import tsx scripts/proxy-install-ca.mjs",
     "proxy:run": "node scripts/run-node.mjs proxy run",
     "proxy:start": "node scripts/run-node.mjs proxy start",
+    "qa-runners:check": "node --import tsx scripts/generate-qa-runner-catalog.ts --check",
+    "qa-runners:gen": "node --import tsx scripts/generate-qa-runner-catalog.ts --write",
     "qa:e2e": "node --import tsx scripts/qa-e2e.ts",
     "qa:lab:build": "vite build --config extensions/qa-lab/web/vite.config.ts",
     "qa:lab:ui": "pnpm openclaw qa ui",

pnpm-lock.yaml

@@ -994,6 +994,15 @@ importers:
         specifier: workspace:*
         version: link:../..
 
+  extensions/qa-matrix:
+    devDependencies:
+      '@openclaw/plugin-sdk':
+        specifier: workspace:*
+        version: link:../../packages/plugin-sdk
+      openclaw:
+        specifier: workspace:*
+        version: link:../..
+
   extensions/qianfan:
     devDependencies:
       '@openclaw/plugin-sdk':

scripts/build-all.mjs

@@ -10,11 +10,6 @@ export const BUILD_ALL_STEPS = [
   { label: "canvas:a2ui:bundle", kind: "pnpm", pnpmArgs: ["canvas:a2ui:bundle"] },
   { label: "tsdown", kind: "node", args: ["scripts/tsdown-build.mjs"] },
   { label: "runtime-postbuild", kind: "node", args: ["scripts/runtime-postbuild.mjs"] },
-  {
-    label: "write-npm-update-compat-sidecars",
-    kind: "node",
-    args: ["scripts/write-npm-update-compat-sidecars.mjs"],
-  },
   { label: "build-stamp", kind: "node", args: ["scripts/build-stamp.mjs"] },
   {
     label: "build:plugin-sdk:dts",

scripts/generate-qa-runner-catalog.ts

@@ -0,0 +1,35 @@
+#!/usr/bin/env node
+import path from "node:path";
+import { writeBundledQaRunnerCatalog } from "../src/plugins/qa-runner-catalog.js";
+
+const args = new Set(process.argv.slice(2));
+const checkOnly = args.has("--check");
+const writeMode = args.has("--write");
+
+if (checkOnly === writeMode) {
+  console.error("Use exactly one of --check or --write.");
+  process.exit(1);
+}
+
+const repoRoot = process.cwd();
+const result = await writeBundledQaRunnerCatalog({
+  repoRoot,
+  check: checkOnly,
+});
+
+if (checkOnly) {
+  if (result.changed) {
+    console.error(
+      [
+        "QA runner catalog drift detected.",
+        `Expected current: ${path.relative(repoRoot, result.jsonPath)}`,
+        "If this QA runner metadata change is intentional, run `pnpm qa-runners:gen` and commit the updated baseline file.",
+        "If not intentional, fix the bundled plugin metadata drift first.",
+      ].join("\n"),
+    );
+    process.exit(1);
+  }
+  console.log(`OK ${path.relative(repoRoot, result.jsonPath)}`);
+} else {
+  console.log(`Wrote ${path.relative(repoRoot, result.jsonPath)}`);
+}

scripts/lib/bundled-plugin-build-entries.mjs

@@ -8,7 +8,7 @@ import {
 import { shouldBuildBundledCluster } from "./optional-bundled-clusters.mjs";
 
 const TOP_LEVEL_PUBLIC_SURFACE_EXTENSIONS = new Set([".ts", ".js", ".mts", ".cts", ".mjs", ".cjs"]);
-const NON_PACKAGED_BUNDLED_PLUGIN_DIRS = new Set(["qa-channel", "qa-lab"]);
+const NON_PACKAGED_BUNDLED_PLUGIN_DIRS = new Set(["qa-channel", "qa-lab", "qa-matrix"]);
 const toPosixPath = (value) => value.replaceAll("\\", "/");
 
 function readBundledPluginPackageJson(packageJsonPath) {

scripts/lib/npm-update-compat-sidecars.mjs

@@ -1,16 +0,0 @@
-export const NPM_UPDATE_COMPAT_SIDECARS = [
-  {
-    path: "dist/extensions/qa-channel/runtime-api.js",
-    content:
-      "// Compatibility stub for older OpenClaw updaters. The QA channel implementation is not packaged.\nexport {};\n",
-  },
-  {
-    path: "dist/extensions/qa-lab/runtime-api.js",
-    content:
-      "// Compatibility stub for older OpenClaw updaters. The QA lab implementation is not packaged.\nexport {};\n",
-  },
-];
-
-export const NPM_UPDATE_COMPAT_SIDECAR_PATHS = new Set(
-  NPM_UPDATE_COMPAT_SIDECARS.map((entry) => entry.path),
-);

scripts/lib/plugin-sdk-entrypoints.json

@@ -179,6 +179,8 @@
   "matrix-runtime-surface",
   "matrix-surface",
   "matrix-thread-bindings",
+  "qa-lab-runtime",
+  "qa-runner-runtime",
   "mattermost",
   "mattermost-policy",
   "memory-core",

scripts/lib/qa-runner-catalog.json

@@ -0,0 +1,8 @@
+[
+  {
+    "pluginId": "qa-matrix",
+    "commandName": "matrix",
+    "description": "Run the Docker-backed Matrix live QA lane against a disposable homeserver",
+    "npmSpec": "@openclaw/qa-matrix"
+  }
+]

scripts/openclaw-npm-postpublish-verify.ts

@@ -20,7 +20,6 @@ import {
   collectRootDistBundledRuntimeMirrors,
   collectRuntimeDependencySpecs,
 } from "./lib/bundled-plugin-root-runtime-mirrors.mjs";
-import { NPM_UPDATE_COMPAT_SIDECAR_PATHS } from "./lib/npm-update-compat-sidecars.mjs";
 import { runInstalledWorkspaceBootstrapSmoke } from "./lib/workspace-bootstrap-smoke.mjs";
 import { parseReleaseVersion, resolveNpmCommandInvocation } from "./openclaw-npm-release-check.ts";
 
@@ -44,13 +43,6 @@ type InstalledBundledExtensionManifestRecord = {
 const MAX_BUNDLED_EXTENSION_MANIFEST_BYTES = 1024 * 1024;
 const LEGACY_CONTEXT_ENGINE_UNRESOLVED_RUNTIME_MARKER =
   "Failed to load legacy context engine runtime.";
-const NPM_UPDATE_COMPAT_EXTENSION_DIRS = new Set(
-  [...NPM_UPDATE_COMPAT_SIDECAR_PATHS].map((relativePath) => {
-    const pathParts = relativePath.split("/");
-    pathParts.pop();
-    return pathParts.join("/");
-  }),
-);
 
 export type PublishedInstallScenario = {
   name: string;
@@ -183,20 +175,6 @@ function collectExpectedBundledExtensionPackageIds(
   return ids;
 }
 
-function isNpmUpdateCompatOnlyExtensionDir(params: {
-  extensionId: string;
-  packageRoot: string;
-}): boolean {
-  const relativeExtensionDir = `dist/extensions/${params.extensionId}`;
-  if (!NPM_UPDATE_COMPAT_EXTENSION_DIRS.has(relativeExtensionDir)) {
-    return false;
-  }
-
-  return [...NPM_UPDATE_COMPAT_SIDECAR_PATHS]
-    .filter((relativePath) => relativePath.startsWith(`${relativeExtensionDir}/`))
-    .every((relativePath) => existsSync(join(params.packageRoot, relativePath)));
-}
-
 function readBundledExtensionPackageJsons(packageRoot: string): {
   manifests: InstalledBundledExtensionManifestRecord[];
   errors: string[];
@@ -218,9 +196,6 @@ function readBundledExtensionPackageJsons(packageRoot: string): {
     const extensionDirPath = join(extensionsDir, entry.name);
     const packageJsonPath = join(extensionsDir, entry.name, "package.json");
     if (!existsSync(packageJsonPath)) {
-      if (isNpmUpdateCompatOnlyExtensionDir({ extensionId: entry.name, packageRoot })) {
-        continue;
-      }
       if (expectedPackageIds === null || expectedPackageIds.has(entry.name)) {
         errors.push(`installed bundled extension manifest missing: ${packageJsonPath}.`);
       }

scripts/openclaw-npm-release-check.ts

@@ -9,7 +9,6 @@ import {
   resolveNpmDistTagMirrorAuth as resolveNpmDistTagMirrorAuthBase,
   parseReleaseVersion as parseReleaseVersionBase,
 } from "./lib/npm-publish-plan.mjs";
-import { NPM_UPDATE_COMPAT_SIDECAR_PATHS } from "./lib/npm-update-compat-sidecars.mjs";
 import { WORKSPACE_TEMPLATE_PACK_PATHS } from "./lib/workspace-bootstrap-smoke.mjs";
 
 type PackageJson = {
@@ -465,9 +464,6 @@ function collectPackedTarballErrors(): string[] {
 export function collectForbiddenPackedPathErrors(paths: Iterable<string>): string[] {
   const errors: string[] = [];
   for (const packedPath of paths) {
-    if (NPM_UPDATE_COMPAT_SIDECAR_PATHS.has(packedPath)) {
-      continue;
-    }
     const matchedRule = FORBIDDEN_PACKED_PATH_RULES.find((rule) =>
       packedPath.startsWith(rule.prefix),
     );

scripts/write-npm-update-compat-sidecars.mjs

@@ -1,10 +0,0 @@
-#!/usr/bin/env node
-
-import fs from "node:fs";
-import path from "node:path";
-import { NPM_UPDATE_COMPAT_SIDECARS } from "./lib/npm-update-compat-sidecars.mjs";
-
-for (const entry of NPM_UPDATE_COMPAT_SIDECARS) {
-  fs.mkdirSync(path.dirname(entry.path), { recursive: true });
-  fs.writeFileSync(entry.path, entry.content, "utf8");
-}

src/plugin-sdk/qa-lab-runtime.ts

@@ -0,0 +1,39 @@
+import { loadBundledPluginPublicSurfaceModuleSync } from "./facade-runtime.js";
+
+type QaLabRuntimeSurface = {
+  defaultQaRuntimeModelForMode: (
+    mode: string,
+    options?: {
+      alternate?: boolean;
+      preferredLiveModel?: string;
+    },
+  ) => string;
+  startQaLiveLaneGateway: (...args: unknown[]) => Promise<unknown>;
+};
+
+function isMissingQaLabRuntimeError(error: unknown) {
+  return (
+    error instanceof Error &&
+    (error.message === "Unable to resolve bundled plugin public surface qa-lab/runtime-api.js" ||
+      error.message.startsWith("Unable to open bundled plugin public surface "))
+  );
+}
+
+export function loadQaLabRuntimeModule(): QaLabRuntimeSurface {
+  return loadBundledPluginPublicSurfaceModuleSync<QaLabRuntimeSurface>({
+    dirName: "qa-lab",
+    artifactBasename: "runtime-api.js",
+  });
+}
+
+export function isQaLabRuntimeAvailable(): boolean {
+  try {
+    loadQaLabRuntimeModule();
+    return true;
+  } catch (error) {
+    if (isMissingQaLabRuntimeError(error)) {
+      return false;
+    }
+    throw error;
+  }
+}

src/plugin-sdk/qa-runner-runtime.integration.test.ts

@@ -0,0 +1,143 @@
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { clearPluginDiscoveryCache } from "../plugins/discovery.js";
+import { clearPluginManifestRegistryCache } from "../plugins/manifest-registry.js";
+import { resetFacadeRuntimeStateForTest } from "./facade-runtime.js";
+
+const ORIGINAL_ENV = {
+  OPENCLAW_DISABLE_BUNDLED_PLUGINS: process.env.OPENCLAW_DISABLE_BUNDLED_PLUGINS,
+  OPENCLAW_CONFIG_PATH: process.env.OPENCLAW_CONFIG_PATH,
+  OPENCLAW_DISABLE_PLUGIN_DISCOVERY_CACHE: process.env.OPENCLAW_DISABLE_PLUGIN_DISCOVERY_CACHE,
+  OPENCLAW_DISABLE_PLUGIN_MANIFEST_CACHE: process.env.OPENCLAW_DISABLE_PLUGIN_MANIFEST_CACHE,
+  OPENCLAW_PLUGIN_DISCOVERY_CACHE_MS: process.env.OPENCLAW_PLUGIN_DISCOVERY_CACHE_MS,
+  OPENCLAW_PLUGIN_MANIFEST_CACHE_MS: process.env.OPENCLAW_PLUGIN_MANIFEST_CACHE_MS,
+  OPENCLAW_TEST_FAST: process.env.OPENCLAW_TEST_FAST,
+} as const;
+
+const tempDirs: string[] = [];
+
+function makeTempDir(prefix: string): string {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
+  tempDirs.push(dir);
+  return dir;
+}
+
+function resetQaRunnerRuntimeState() {
+  clearPluginDiscoveryCache();
+  clearPluginManifestRegistryCache();
+  resetFacadeRuntimeStateForTest();
+}
+
+describe("plugin-sdk qa-runner-runtime linked plugin smoke", () => {
+  beforeEach(() => {
+    resetQaRunnerRuntimeState();
+    process.env.OPENCLAW_DISABLE_BUNDLED_PLUGINS = "1";
+    process.env.OPENCLAW_DISABLE_PLUGIN_DISCOVERY_CACHE = "1";
+    process.env.OPENCLAW_DISABLE_PLUGIN_MANIFEST_CACHE = "1";
+    process.env.OPENCLAW_PLUGIN_DISCOVERY_CACHE_MS = "0";
+    process.env.OPENCLAW_PLUGIN_MANIFEST_CACHE_MS = "0";
+    process.env.OPENCLAW_TEST_FAST = "1";
+  });
+
+  afterEach(() => {
+    resetQaRunnerRuntimeState();
+    for (const dir of tempDirs.splice(0)) {
+      fs.rmSync(dir, { recursive: true, force: true });
+    }
+    for (const [key, value] of Object.entries(ORIGINAL_ENV)) {
+      if (value === undefined) {
+        delete process.env[key];
+      } else {
+        process.env[key] = value;
+      }
+    }
+  });
+
+  it("loads an activated qa runner from a linked plugin path", async () => {
+    const stateDir = makeTempDir("openclaw-qa-runner-state-");
+    const pluginDir = path.join(stateDir, "extensions", "qa-linked");
+    const configPath = path.join(stateDir, "openclaw.json");
+
+    fs.writeFileSync(
+      configPath,
+      JSON.stringify({
+        plugins: {},
+      }),
+      "utf8",
+    );
+    process.env.OPENCLAW_CONFIG_PATH = configPath;
+
+    fs.mkdirSync(pluginDir, { recursive: true });
+    fs.writeFileSync(
+      path.join(pluginDir, "openclaw.plugin.json"),
+      JSON.stringify({
+        id: "qa-linked",
+        qaRunners: [
+          {
+            commandName: "linked",
+            description: "Run the linked QA lane",
+          },
+        ],
+        configSchema: {
+          type: "object",
+          additionalProperties: false,
+          properties: {},
+        },
+      }),
+      "utf8",
+    );
+    fs.writeFileSync(
+      path.join(pluginDir, "package.json"),
+      JSON.stringify({
+        name: "@openclaw/qa-linked",
+        type: "module",
+        openclaw: {
+          extensions: ["./index.js"],
+          install: {
+            npmSpec: "@openclaw/qa-linked",
+          },
+        },
+      }),
+      "utf8",
+    );
+    fs.writeFileSync(path.join(pluginDir, "index.js"), 'export default {};\n', "utf8");
+    fs.writeFileSync(
+      path.join(pluginDir, "runtime-api.js"),
+      [
+        "export const qaRunnerCliRegistrations = [",
+        "  {",
+        '    commandName: "linked",',
+        "    register() {}",
+        "  }",
+        "];",
+      ].join("\n"),
+      "utf8",
+    );
+
+    const module = await import("./qa-runner-runtime.js");
+
+    expect(module.listQaRunnerCliContributions()).toEqual(
+      expect.arrayContaining([
+        {
+          pluginId: "qa-linked",
+          commandName: "linked",
+          description: "Run the linked QA lane",
+          status: "available",
+          registration: {
+            commandName: "linked",
+            register: expect.any(Function),
+          },
+        },
+        {
+          pluginId: "qa-matrix",
+          commandName: "matrix",
+          description: "Run the Docker-backed Matrix live QA lane against a disposable homeserver",
+          status: "missing",
+          npmSpec: "@openclaw/qa-matrix",
+        },
+      ]),
+    );
+  });
+});

src/plugin-sdk/qa-runner-runtime.test.ts

@@ -0,0 +1,183 @@
+import type { Command } from "commander";
+import { beforeEach, describe, expect, it, vi } from "vitest";
+
+const loadPluginManifestRegistry = vi.hoisted(() => vi.fn());
+const tryLoadActivatedBundledPluginPublicSurfaceModuleSync = vi.hoisted(() => vi.fn());
+const listBundledQaRunnerCatalog = vi.hoisted(() =>
+  vi.fn<
+    () => Array<{
+      pluginId: string;
+      commandName: string;
+      description?: string;
+      npmSpec: string;
+    }>
+  >(() => []),
+);
+
+vi.mock("../plugins/manifest-registry.js", () => ({
+  loadPluginManifestRegistry,
+}));
+
+vi.mock("../plugins/qa-runner-catalog.js", () => ({
+  listBundledQaRunnerCatalog,
+}));
+
+vi.mock("./facade-runtime.js", () => ({
+  tryLoadActivatedBundledPluginPublicSurfaceModuleSync,
+}));
+
+describe("plugin-sdk qa-runner-runtime", () => {
+  beforeEach(() => {
+    loadPluginManifestRegistry.mockReset().mockReturnValue({
+      plugins: [],
+      diagnostics: [],
+    });
+    listBundledQaRunnerCatalog.mockReset().mockReturnValue([]);
+    tryLoadActivatedBundledPluginPublicSurfaceModuleSync.mockReset();
+  });
+
+  it("stays cold until runner discovery is requested", async () => {
+    await import("./qa-runner-runtime.js");
+
+    expect(loadPluginManifestRegistry).not.toHaveBeenCalled();
+    expect(tryLoadActivatedBundledPluginPublicSurfaceModuleSync).not.toHaveBeenCalled();
+  });
+
+  it("returns activated runner registrations declared in plugin manifests", async () => {
+    const register = vi.fn((qa: Command) => qa);
+    loadPluginManifestRegistry.mockReturnValue({
+      plugins: [
+        {
+          id: "qa-matrix",
+          qaRunners: [
+            {
+              commandName: "matrix",
+              description: "Run the Matrix live QA lane",
+            },
+          ],
+          rootDir: "/tmp/qa-matrix",
+        },
+      ],
+      diagnostics: [],
+    });
+    tryLoadActivatedBundledPluginPublicSurfaceModuleSync.mockReturnValue({
+      qaRunnerCliRegistrations: [{ commandName: "matrix", register }],
+    });
+
+    const module = await import("./qa-runner-runtime.js");
+
+    expect(module.listQaRunnerCliContributions()).toEqual([
+      {
+        pluginId: "qa-matrix",
+        commandName: "matrix",
+        description: "Run the Matrix live QA lane",
+        status: "available",
+        registration: {
+          commandName: "matrix",
+          register,
+        },
+      },
+    ]);
+    expect(tryLoadActivatedBundledPluginPublicSurfaceModuleSync).toHaveBeenCalledWith({
+      dirName: "qa-matrix",
+      artifactBasename: "runtime-api.js",
+    });
+  });
+
+  it("reports declared runners as blocked when the plugin is present but not activated", async () => {
+    loadPluginManifestRegistry.mockReturnValue({
+      plugins: [
+        {
+          id: "qa-matrix",
+          qaRunners: [{ commandName: "matrix" }],
+          rootDir: "/tmp/qa-matrix",
+        },
+      ],
+      diagnostics: [],
+    });
+    tryLoadActivatedBundledPluginPublicSurfaceModuleSync.mockReturnValue(null);
+
+    const module = await import("./qa-runner-runtime.js");
+
+    expect(module.listQaRunnerCliContributions()).toEqual([
+      {
+        pluginId: "qa-matrix",
+        commandName: "matrix",
+        status: "blocked",
+      },
+    ]);
+  });
+
+  it("reports missing optional runners from the generated catalog", async () => {
+    listBundledQaRunnerCatalog.mockReturnValue([
+      {
+        pluginId: "qa-matrix",
+        commandName: "matrix",
+        description: "Run the Matrix live QA lane",
+        npmSpec: "@openclaw/qa-matrix",
+      },
+    ]);
+
+    const module = await import("./qa-runner-runtime.js");
+
+    expect(module.listQaRunnerCliContributions()).toEqual([
+      {
+        pluginId: "qa-matrix",
+        commandName: "matrix",
+        description: "Run the Matrix live QA lane",
+        status: "missing",
+        npmSpec: "@openclaw/qa-matrix",
+      },
+    ]);
+  });
+
+  it("fails fast when two plugins declare the same qa runner command", async () => {
+    loadPluginManifestRegistry.mockReturnValue({
+      plugins: [
+        {
+          id: "alpha",
+          qaRunners: [{ commandName: "matrix" }],
+          rootDir: "/tmp/alpha",
+        },
+        {
+          id: "beta",
+          qaRunners: [{ commandName: "matrix" }],
+          rootDir: "/tmp/beta",
+        },
+      ],
+      diagnostics: [],
+    });
+    tryLoadActivatedBundledPluginPublicSurfaceModuleSync.mockReturnValue(null);
+
+    const module = await import("./qa-runner-runtime.js");
+
+    expect(() => module.listQaRunnerCliContributions()).toThrow(
+      'QA runner command "matrix" declared by both "alpha" and "beta"',
+    );
+  });
+
+  it("fails when runtime registrations include an undeclared command", async () => {
+    loadPluginManifestRegistry.mockReturnValue({
+      plugins: [
+        {
+          id: "qa-matrix",
+          qaRunners: [{ commandName: "matrix" }],
+          rootDir: "/tmp/qa-matrix",
+        },
+      ],
+      diagnostics: [],
+    });
+    tryLoadActivatedBundledPluginPublicSurfaceModuleSync.mockReturnValue({
+      qaRunnerCliRegistrations: [
+        { commandName: "matrix", register: vi.fn() },
+        { commandName: "extra", register: vi.fn() },
+      ],
+    });
+
+    const module = await import("./qa-runner-runtime.js");
+
+    expect(() => module.listQaRunnerCliContributions()).toThrow(
+      'QA runner plugin "qa-matrix" exported "extra" from runtime-api.js but did not declare it in openclaw.plugin.json',
+    );
+  });
+});

src/plugin-sdk/qa-runner-runtime.ts

@@ -0,0 +1,161 @@
+import type { Command } from "commander";
+import type { PluginManifestRecord } from "../plugins/manifest-registry.js";
+import { loadPluginManifestRegistry } from "../plugins/manifest-registry.js";
+import { listBundledQaRunnerCatalog } from "../plugins/qa-runner-catalog.js";
+import { tryLoadActivatedBundledPluginPublicSurfaceModuleSync } from "./facade-runtime.js";
+
+export type QaRunnerCliRegistration = {
+  commandName: string;
+  register(qa: Command): void;
+};
+
+type QaRunnerRuntimeSurface = {
+  qaRunnerCliRegistrations?: readonly QaRunnerCliRegistration[];
+};
+
+export type QaRunnerCliContribution =
+  | {
+      pluginId: string;
+      commandName: string;
+      description?: string;
+      status: "available";
+      registration: QaRunnerCliRegistration;
+    }
+  | {
+      pluginId: string;
+      commandName: string;
+      description?: string;
+      status: "blocked";
+    }
+  | {
+      pluginId: string;
+      commandName: string;
+      description?: string;
+      status: "missing";
+      npmSpec: string;
+    };
+
+function listDeclaredQaRunnerPlugins(): Array<
+  PluginManifestRecord & {
+    qaRunners: NonNullable<PluginManifestRecord["qaRunners"]>;
+  }
+> {
+  return loadPluginManifestRegistry({ cache: true })
+    .plugins.filter(
+      (
+        plugin,
+      ): plugin is PluginManifestRecord & {
+        qaRunners: NonNullable<PluginManifestRecord["qaRunners"]>;
+      } => Array.isArray(plugin.qaRunners) && plugin.qaRunners.length > 0,
+    )
+    .toSorted((left, right) => {
+      const idCompare = left.id.localeCompare(right.id);
+      if (idCompare !== 0) {
+        return idCompare;
+      }
+      return left.rootDir.localeCompare(right.rootDir);
+    });
+}
+
+function indexRuntimeRegistrations(
+  pluginId: string,
+  surface: QaRunnerRuntimeSurface,
+): ReadonlyMap<string, QaRunnerCliRegistration> {
+  const registrations = surface.qaRunnerCliRegistrations ?? [];
+  const registrationByCommandName = new Map<string, QaRunnerCliRegistration>();
+  for (const registration of registrations) {
+    if (!registration?.commandName || typeof registration.register !== "function") {
+      throw new Error(`QA runner plugin "${pluginId}" exported an invalid CLI registration`);
+    }
+    if (registrationByCommandName.has(registration.commandName)) {
+      throw new Error(
+        `QA runner plugin "${pluginId}" exported duplicate CLI registration "${registration.commandName}"`,
+      );
+    }
+    registrationByCommandName.set(registration.commandName, registration);
+  }
+  return registrationByCommandName;
+}
+
+function buildKnownQaRunnerCatalog(): readonly QaRunnerCliContribution[] {
+  const knownRunners = listBundledQaRunnerCatalog();
+  const seenCommandNames = new Map<string, string>();
+  return knownRunners.map((runner) => {
+    const previousOwner = seenCommandNames.get(runner.commandName);
+    if (previousOwner) {
+      throw new Error(
+        `QA runner command "${runner.commandName}" declared by both "${previousOwner}" and "${runner.pluginId}"`,
+      );
+    }
+    seenCommandNames.set(runner.commandName, runner.pluginId);
+    return {
+      pluginId: runner.pluginId,
+      commandName: runner.commandName,
+      ...(runner.description ? { description: runner.description } : {}),
+      status: "missing" as const,
+      npmSpec: runner.npmSpec,
+    };
+  });
+}
+
+export function listQaRunnerCliContributions(): readonly QaRunnerCliContribution[] {
+  const contributions = new Map<string, QaRunnerCliContribution>();
+
+  for (const runner of buildKnownQaRunnerCatalog()) {
+    contributions.set(runner.commandName, runner);
+  }
+
+  for (const plugin of listDeclaredQaRunnerPlugins()) {
+    const runtimeSurface =
+      tryLoadActivatedBundledPluginPublicSurfaceModuleSync<QaRunnerRuntimeSurface>({
+        dirName: plugin.id,
+        artifactBasename: "runtime-api.js",
+      });
+    const runtimeRegistrationByCommandName = runtimeSurface
+      ? indexRuntimeRegistrations(plugin.id, runtimeSurface)
+      : null;
+    const declaredCommandNames = new Set(plugin.qaRunners.map((runner) => runner.commandName));
+
+    for (const runner of plugin.qaRunners) {
+      const previous = contributions.get(runner.commandName);
+      if (previous && previous.pluginId !== plugin.id) {
+        throw new Error(
+          `QA runner command "${runner.commandName}" declared by both "${previous.pluginId}" and "${plugin.id}"`,
+        );
+      }
+
+      const registration = runtimeRegistrationByCommandName?.get(runner.commandName);
+      if (!runtimeSurface) {
+        contributions.set(runner.commandName, {
+          pluginId: plugin.id,
+          commandName: runner.commandName,
+          ...(runner.description ? { description: runner.description } : {}),
+          status: "blocked",
+        });
+        continue;
+      }
+      if (!registration) {
+        throw new Error(
+          `QA runner plugin "${plugin.id}" declared "${runner.commandName}" in openclaw.plugin.json but did not export a matching CLI registration`,
+        );
+      }
+      contributions.set(runner.commandName, {
+        pluginId: plugin.id,
+        commandName: runner.commandName,
+        ...(runner.description ? { description: runner.description } : {}),
+        status: "available",
+        registration,
+      });
+    }
+
+    for (const commandName of runtimeRegistrationByCommandName?.keys() ?? []) {
+      if (!declaredCommandNames.has(commandName)) {
+        throw new Error(
+          `QA runner plugin "${plugin.id}" exported "${commandName}" from runtime-api.js but did not declare it in openclaw.plugin.json`,
+        );
+      }
+    }
+  }
+
+  return [...contributions.values()];
+}

src/plugins/bundled-plugin-metadata.test.ts

@@ -131,11 +131,12 @@ describe("bundled plugin metadata", () => {
     },
   );
 
-  it("excludes private QA sidecars from the packaged runtime sidecar baseline", () => {
+  it("excludes non-packaged QA sidecars from the packaged runtime sidecar baseline", () => {
     expect(BUNDLED_RUNTIME_SIDECAR_PATHS).not.toContain(
       "dist/extensions/qa-channel/runtime-api.js",
     );
     expect(BUNDLED_RUNTIME_SIDECAR_PATHS).not.toContain("dist/extensions/qa-lab/runtime-api.js");
+    expect(BUNDLED_RUNTIME_SIDECAR_PATHS).not.toContain("dist/extensions/qa-matrix/runtime-api.js");
   });
 
   it("captures setup-entry metadata for bundled channel plugins", () => {

src/plugins/install.test.ts

@@ -1427,6 +1427,21 @@ describe("installPluginFromArchive", () => {
     ).toBe(true);
   });
 
+  it("does not flag the real qa-matrix plugin as dangerous install code", async () => {
+    const pluginDir = path.resolve(process.cwd(), "extensions", "qa-matrix");
+
+    const scanResult = await installSecurityScan.scanPackageInstallSource({
+      extensions: ["./index.ts"],
+      logger: { warn: vi.fn() },
+      packageDir: pluginDir,
+      pluginId: "qa-matrix",
+      packageName: "@openclaw/qa-matrix",
+      manifestId: "qa-matrix",
+    });
+
+    expect(scanResult?.blocked).toBeUndefined();
+  });
+
   it("keeps blocked dependency package checks active when forced unsafe install is set", async () => {
     const { pluginDir, extensionsDir } = setupPluginInstallDirs();
 

src/plugins/manifest-registry.test.ts

@@ -499,6 +499,33 @@ describe("loadPluginManifestRegistry", () => {
     });
   });
 
+  it("preserves qa runner descriptors from plugin manifests", () => {
+    const dir = makeTempDir();
+    writeManifest(dir, {
+      id: "qa-matrix",
+      qaRunners: [
+        {
+          commandName: "matrix",
+          description: "Run the Matrix live QA lane",
+        },
+      ],
+      configSchema: { type: "object" },
+    });
+
+    const registry = loadSingleCandidateRegistry({
+      idHint: "qa-matrix",
+      rootDir: dir,
+      origin: "bundled",
+    });
+
+    expect(registry.plugins[0]?.qaRunners).toEqual([
+      {
+        commandName: "matrix",
+        description: "Run the Matrix live QA lane",
+      },
+    ]);
+  });
+
   it("preserves channel config metadata from plugin manifests", () => {
     const dir = makeTempDir();
     writeManifest(dir, {

src/plugins/manifest-registry.ts

@@ -34,6 +34,7 @@ import {
   type PluginManifestChannelConfig,
   type PluginManifestContracts,
   type PluginManifestModelSupport,
+  type PluginManifestQaRunner,
   type PluginManifestSetup,
 } from "./manifest.js";
 import { checkMinHostVersion } from "./min-host-version.js";
@@ -92,6 +93,7 @@ export type PluginManifestRecord = {
   providerAuthChoices?: PluginManifest["providerAuthChoices"];
   activation?: PluginManifestActivation;
   setup?: PluginManifestSetup;
+  qaRunners?: PluginManifestQaRunner[];
   skills: string[];
   settingsFiles?: string[];
   hooks: string[];
@@ -333,6 +335,7 @@ function buildRecord(params: {
     providerAuthChoices: params.manifest.providerAuthChoices,
     activation: params.manifest.activation,
     setup: params.manifest.setup,
+    qaRunners: params.manifest.qaRunners,
     skills: params.manifest.skills ?? [],
     settingsFiles: [],
     hooks: [],

src/plugins/manifest.ts

@@ -80,6 +80,13 @@ export type PluginManifestSetup = {
   requiresRuntime?: boolean;
 };
 
+export type PluginManifestQaRunner = {
+  /** Subcommand mounted beneath `openclaw qa`, for example `matrix`. */
+  commandName: string;
+  /** Optional user-facing help text for fallback host stubs. */
+  description?: string;
+};
+
 export type PluginManifestConfigLiteral = string | number | boolean | null;
 
 export type PluginManifestDangerousConfigFlag = {
@@ -174,6 +181,8 @@ export type PluginManifest = {
   activation?: PluginManifestActivation;
   /** Cheap setup/onboarding metadata exposed before plugin runtime loads. */
   setup?: PluginManifestSetup;
+  /** Cheap QA runner metadata exposed before plugin runtime loads. */
+  qaRunners?: PluginManifestQaRunner[];
   skills?: string[];
   name?: string;
   description?: string;
@@ -484,6 +493,28 @@ function normalizeManifestSetup(value: unknown): PluginManifestSetup | undefined
   return Object.keys(setup).length > 0 ? setup : undefined;
 }
 
+function normalizeManifestQaRunners(value: unknown): PluginManifestQaRunner[] | undefined {
+  if (!Array.isArray(value)) {
+    return undefined;
+  }
+  const normalized: PluginManifestQaRunner[] = [];
+  for (const entry of value) {
+    if (!isRecord(entry)) {
+      continue;
+    }
+    const commandName = normalizeOptionalString(entry.commandName) ?? "";
+    if (!commandName) {
+      continue;
+    }
+    const description = normalizeOptionalString(entry.description) ?? "";
+    normalized.push({
+      commandName,
+      ...(description ? { description } : {}),
+    });
+  }
+  return normalized.length > 0 ? normalized : undefined;
+}
+
 function normalizeProviderAuthChoices(
   value: unknown,
 ): PluginManifestProviderAuthChoice[] | undefined {
@@ -673,6 +704,7 @@ export function loadPluginManifest(
   const providerAuthChoices = normalizeProviderAuthChoices(raw.providerAuthChoices);
   const activation = normalizeManifestActivation(raw.activation);
   const setup = normalizeManifestSetup(raw.setup);
+  const qaRunners = normalizeManifestQaRunners(raw.qaRunners);
   const skills = normalizeTrimmedStringList(raw.skills);
   const contracts = normalizeManifestContracts(raw.contracts);
   const configContracts = normalizeManifestConfigContracts(raw.configContracts);
@@ -706,6 +738,7 @@ export function loadPluginManifest(
       providerAuthChoices,
       activation,
       setup,
+      qaRunners,
       skills,
       name,
       description,

src/plugins/qa-runner-catalog.ts

@@ -0,0 +1,74 @@
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { listBundledPluginMetadata } from "./bundled-plugin-metadata.js";
+
+export type QaRunnerCatalogEntry = {
+  pluginId: string;
+  commandName: string;
+  description?: string;
+  npmSpec: string;
+};
+
+const QA_RUNNER_CATALOG_JSON_PATH = fileURLToPath(
+  new URL("../../scripts/lib/qa-runner-catalog.json", import.meta.url),
+);
+
+export function listBundledQaRunnerCatalog(): readonly QaRunnerCatalogEntry[] {
+  if (!fs.existsSync(QA_RUNNER_CATALOG_JSON_PATH)) {
+    return [];
+  }
+  return JSON.parse(fs.readFileSync(QA_RUNNER_CATALOG_JSON_PATH, "utf8")) as QaRunnerCatalogEntry[];
+}
+
+export function collectBundledQaRunnerCatalog(params?: {
+  rootDir?: string;
+}): readonly QaRunnerCatalogEntry[] {
+  const catalog: QaRunnerCatalogEntry[] = [];
+  const seenCommandNames = new Map<string, string>();
+
+  for (const entry of listBundledPluginMetadata({
+    rootDir: params?.rootDir,
+    includeChannelConfigs: false,
+  })) {
+    const qaRunners = entry.manifest.qaRunners ?? [];
+    const npmSpec = entry.packageManifest?.install?.npmSpec?.trim() || entry.packageName?.trim();
+    if (!npmSpec) {
+      continue;
+    }
+    for (const runner of qaRunners) {
+      const previousOwner = seenCommandNames.get(runner.commandName);
+      if (previousOwner) {
+        throw new Error(
+          `QA runner command "${runner.commandName}" declared by both "${previousOwner}" and "${entry.manifest.id}"`,
+        );
+      }
+      seenCommandNames.set(runner.commandName, entry.manifest.id);
+      catalog.push({
+        pluginId: entry.manifest.id,
+        commandName: runner.commandName,
+        ...(runner.description ? { description: runner.description } : {}),
+        npmSpec,
+      });
+    }
+  }
+
+  return catalog.toSorted((left, right) => left.commandName.localeCompare(right.commandName));
+}
+
+export async function writeBundledQaRunnerCatalog(params: {
+  repoRoot: string;
+  check: boolean;
+}): Promise<{ changed: boolean; jsonPath: string }> {
+  const jsonPath = path.join(params.repoRoot, "scripts", "lib", "qa-runner-catalog.json");
+  const expectedJson = `${JSON.stringify(collectBundledQaRunnerCatalog({ rootDir: params.repoRoot }), null, 2)}\n`;
+  const currentJson = fs.existsSync(jsonPath) ? fs.readFileSync(jsonPath, "utf8") : "";
+  const changed = currentJson !== expectedJson;
+
+  if (!params.check && changed) {
+    fs.mkdirSync(path.dirname(jsonPath), { recursive: true });
+    fs.writeFileSync(jsonPath, expectedJson, "utf8");
+  }
+
+  return { changed, jsonPath };
+}

src/plugins/runtime-sidecar-paths-baseline.ts

@@ -2,7 +2,7 @@ import fs from "node:fs";
 import path from "node:path";
 import { listBundledPluginMetadata } from "./bundled-plugin-metadata.js";
 
-const NON_PACKAGED_RUNTIME_SIDECAR_PLUGIN_DIRS = new Set(["qa-channel", "qa-lab"]);
+const NON_PACKAGED_RUNTIME_SIDECAR_PLUGIN_DIRS = new Set(["qa-channel", "qa-lab", "qa-matrix"]);
 
 function buildBundledDistArtifactPath(dirName: string, artifact: string): string {
   return ["dist", "extensions", dirName, artifact].join("/");

test/openclaw-npm-postpublish-verify.test.ts

@@ -301,7 +301,7 @@ describe("collectInstalledMirroredRootDependencyManifestErrors", () => {
     }
   });
 
-  it("allows npm update compatibility sidecar directories without package.json", () => {
+  it("rejects private qa sidecar directories that are missing package.json", () => {
     const packageRoot = makeInstalledPackageRoot();
 
     try {
@@ -322,7 +322,10 @@ describe("collectInstalledMirroredRootDependencyManifestErrors", () => {
         "utf8",
       );
 
-      expect(collectInstalledMirroredRootDependencyManifestErrors(packageRoot)).toEqual([]);
+      expect(collectInstalledMirroredRootDependencyManifestErrors(packageRoot)).toEqual([
+        `installed bundled extension manifest missing: ${join(packageRoot, "dist/extensions/qa-channel/package.json")}.`,
+        `installed bundled extension manifest missing: ${join(packageRoot, "dist/extensions/qa-lab/package.json")}.`,
+      ]);
     } finally {
       rmSync(packageRoot, { recursive: true, force: true });
     }

test/openclaw-npm-release-check.test.ts

@@ -325,6 +325,8 @@ describe("collectForbiddenPackedPathErrors", () => {
       ]),
     ).toEqual([
       'npm package must not include private QA channel artifact "dist/extensions/qa-channel/package.json".',
+      'npm package must not include private QA channel artifact "dist/extensions/qa-channel/runtime-api.js".',
+      'npm package must not include private QA lab artifact "dist/extensions/qa-lab/runtime-api.js".',
       'npm package must not include private QA lab artifact "dist/extensions/qa-lab/src/cli.js".',
     ]);
   });

test/scripts/bundled-plugin-build-entries.test.ts

@@ -82,6 +82,9 @@ describe("bundled plugin build entries", () => {
     expect(artifacts.some((artifact) => artifact.startsWith("dist/extensions/qa-lab/"))).toBe(
       false,
     );
+    expect(artifacts.some((artifact) => artifact.startsWith("dist/extensions/qa-matrix/"))).toBe(
+      false,
+    );
   });
 
   it("keeps bundled channel secret contracts on packed top-level sidecars", () => {