feat(qa): add mantis Slack desktop smoke

This commit is contained in:
Peter Steinberger
2026-05-04 03:47:19 +01:00
parent 471489159b
commit f632f5e60b
8 changed files with 1215 additions and 20 deletions

View File

@@ -21,6 +21,7 @@ Docs: https://docs.openclaw.ai
- Gateway/config: stop Gateway startup and hot reload from auto-restoring invalid config; invalid config now fails closed and `openclaw doctor --fix` owns last-known-good repair.
- Gateway/performance: lazy-load early runtime discovery and shutdown-hook helpers, defer maintenance timers until after readiness, and trim duplicate plugin auto-enable work during Gateway startup.
- QA/Mantis: add a `pnpm openclaw qa mantis discord-smoke` runner and manual GitHub workflow that verify the Mantis Discord bot can see the configured guild/channel, post a smoke message, add a reaction, and upload artifacts.
- QA/Mantis: add `pnpm openclaw qa mantis slack-desktop-smoke` to run Slack live QA inside a Crabbox VNC desktop, open Slack Web, and capture desktop screenshots beside the Slack QA artifacts.
- QA/Mantis: pass the runtime env through desktop-browser Crabbox and artifact-copy child commands, so embedded Mantis callers can provide Crabbox credentials without mutating the parent process. Thanks @vincentkoc.
- QA/Slack: add a Slack live transport QA runner with canary and mention-gating coverage for the private bot-to-bot harness. Thanks @vincentkoc.
- Gateway/performance: lazy-load the heavy cron runtime after the rest of Gateway startup, defer restart-sentinel refresh after readiness, and let the Gateway startup benchmark write per-run V8 CPU profiles with `--cpu-prof-dir`.

View File

@@ -111,6 +111,51 @@ Useful desktop smoke flags:
- `--keep-lease` or `OPENCLAW_MANTIS_KEEP_VM=1` keeps a newly created passing lease open for VNC inspection. Failed runs keep the lease by default when one was created so an operator can reconnect.
- `--class`, `--idle-timeout`, and `--ttl` tune machine size and lease lifetime.
The first full desktop transport primitive is the Slack desktop smoke:
```bash
pnpm openclaw qa mantis slack-desktop-smoke \
--output-dir .artifacts/qa-e2e/mantis/slack-desktop \
--gateway-setup \
--scenario slack-canary \
--keep-lease
```
It leases or reuses a Crabbox desktop machine, syncs the current checkout into
the VM, runs `pnpm openclaw qa slack` inside that VM, opens Slack Web in the VNC
browser, captures the visible desktop, and copies both the Slack QA artifacts and
the VNC screenshot back to the local output directory. This is the first Mantis
shape where the SUT OpenClaw gateway and the browser both live inside the same
Linux desktop VM.
With `--gateway-setup`, the command prepares a persistent disposable OpenClaw
home at `$HOME/.openclaw-mantis/slack-openclaw`, patches Slack Socket Mode
configuration for the selected channel, starts `openclaw gateway run` on port
`38973`, and keeps Chrome running in the VNC session. This is the "leave me a
Linux desktop with Slack and a claw running" mode; the bot-to-bot Slack QA lane
remains the default when `--gateway-setup` is omitted.
Required inputs for `--credential-source env`:
- `OPENCLAW_QA_SLACK_CHANNEL_ID`
- `OPENCLAW_QA_SLACK_DRIVER_BOT_TOKEN`
- `OPENCLAW_QA_SLACK_SUT_BOT_TOKEN`
- `OPENCLAW_QA_SLACK_SUT_APP_TOKEN`
- `OPENCLAW_LIVE_OPENAI_KEY` for the remote model lane. If only
`OPENAI_API_KEY` is set locally, Mantis maps it to `OPENCLAW_LIVE_OPENAI_KEY`
before invoking Crabbox so Crabbox's `OPENCLAW_*` env forwarding can carry it
into the VM.
Useful Slack desktop flags:
- `--lease-id <cbx_...>` reruns against a machine where an operator already logged in to Slack Web through VNC.
- `--gateway-setup` starts a persistent OpenClaw Slack gateway in the VM instead of only running the bot-to-bot QA lane.
- `--slack-url <url>` opens a specific Slack Web URL. Without it, Mantis derives `https://app.slack.com/client/<team>/<channel>` from Slack `auth.test` when the SUT bot token is available.
- `--slack-channel-id <id>` controls the Slack channel allowlist used by gateway setup.
- `OPENCLAW_MANTIS_SLACK_BROWSER_PROFILE_DIR` controls the persistent Chrome profile inside the VM. The default is `$HOME/.config/openclaw-mantis/slack-chrome-profile`, so a manual Slack Web login survives reruns on the same lease.
- `--credential-source convex --credential-role ci` uses the shared credential pool instead of direct Slack env tokens.
- `--provider-mode`, `--model`, `--alt-model`, and `--fast` pass through to the Slack live lane.
The GitHub smoke workflow is `Mantis Discord Smoke`. The before and after GitHub
workflow for the first real scenario is `Mantis Discord Status Reactions`. It
accepts:

View File

@@ -29,26 +29,26 @@ Current pieces:
Every QA flow runs under `pnpm openclaw qa <subcommand>`. Many have `pnpm qa:*`
script aliases; both forms are supported.
| Command | Purpose |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qa run` | Bundled QA self-check; writes a Markdown report. |
| `qa suite` | Run repo-backed scenarios against the QA gateway lane. Aliases: `pnpm openclaw qa suite --runner multipass` for a disposable Linux VM. |
| `qa coverage` | Print the markdown scenario-coverage inventory (`--json` for machine output). |
| `qa parity-report` | Compare two `qa-suite-summary.json` files and write the agentic parity report. |
| `qa character-eval` | Run the character QA scenario across multiple live models with a judged report. See [Reporting](#reporting). |
| `qa manual` | Run a one-off prompt against the selected provider/model lane. |
| `qa ui` | Start the QA debugger UI and local QA bus (alias: `pnpm qa:lab:ui`). |
| `qa docker-build-image` | Build the prebaked QA Docker image. |
| `qa docker-scaffold` | Write a docker-compose scaffold for the QA dashboard + gateway lane. |
| `qa up` | Build the QA site, start the Docker-backed stack, print the URL (alias: `pnpm qa:lab:up`; `:fast` variant adds `--use-prebuilt-image --bind-ui-dist --skip-ui-build`). |
| `qa aimock` | Start only the AIMock provider server. |
| `qa mock-openai` | Start only the scenario-aware `mock-openai` provider server. |
| `qa credentials doctor` / `add` / `list` / `remove` | Manage the shared Convex credential pool. |
| `qa matrix` | Live transport lane against a disposable Tuwunel homeserver. See [Matrix QA](/concepts/qa-matrix). |
| `qa telegram` | Live transport lane against a real private Telegram group. |
| `qa discord` | Live transport lane against a real private Discord guild channel. |
| `qa slack` | Live transport lane against a real private Slack channel. |
| `qa mantis` | Before and after verification runner for live transport bugs, with Discord status-reactions evidence and a Crabbox desktop/browser smoke. See [Mantis](/concepts/mantis). |
| Command | Purpose |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `qa run` | Bundled QA self-check; writes a Markdown report. |
| `qa suite` | Run repo-backed scenarios against the QA gateway lane. Aliases: `pnpm openclaw qa suite --runner multipass` for a disposable Linux VM. |
| `qa coverage` | Print the markdown scenario-coverage inventory (`--json` for machine output). |
| `qa parity-report` | Compare two `qa-suite-summary.json` files and write the agentic parity report. |
| `qa character-eval` | Run the character QA scenario across multiple live models with a judged report. See [Reporting](#reporting). |
| `qa manual` | Run a one-off prompt against the selected provider/model lane. |
| `qa ui` | Start the QA debugger UI and local QA bus (alias: `pnpm qa:lab:ui`). |
| `qa docker-build-image` | Build the prebaked QA Docker image. |
| `qa docker-scaffold` | Write a docker-compose scaffold for the QA dashboard + gateway lane. |
| `qa up` | Build the QA site, start the Docker-backed stack, print the URL (alias: `pnpm qa:lab:up`; `:fast` variant adds `--use-prebuilt-image --bind-ui-dist --skip-ui-build`). |
| `qa aimock` | Start only the AIMock provider server. |
| `qa mock-openai` | Start only the scenario-aware `mock-openai` provider server. |
| `qa credentials doctor` / `add` / `list` / `remove` | Manage the shared Convex credential pool. |
| `qa matrix` | Live transport lane against a disposable Tuwunel homeserver. See [Matrix QA](/concepts/qa-matrix). |
| `qa telegram` | Live transport lane against a real private Telegram group. |
| `qa discord` | Live transport lane against a real private Discord guild channel. |
| `qa slack` | Live transport lane against a real private Slack channel. |
| `qa mantis` | Before and after verification runner for live transport bugs, with Discord status-reactions evidence, Crabbox desktop/browser smoke, and Slack-in-VNC smoke. See [Mantis](/concepts/mantis). |
## Operator flow
@@ -121,6 +121,23 @@ pnpm openclaw qa slack
They target a pre-existing real channel with two bots (driver + SUT). Required env vars, scenario lists, output artifacts, and the Convex credential pool are documented in [Telegram, Discord, and Slack QA reference](#telegram-discord-and-slack-qa-reference) below.
For a full Slack desktop VM run with VNC rescue, run:
```bash
pnpm openclaw qa mantis slack-desktop-smoke \
--gateway-setup \
--scenario slack-canary \
--keep-lease
```
That command leases a Crabbox desktop/browser machine, runs the Slack live lane
inside the VM, opens Slack Web in the VNC browser, captures the desktop, and
copies `slack-qa/` plus `slack-desktop-smoke.png` back to the Mantis artifact
directory. Reuse `--lease-id <cbx_...>` after logging in to Slack Web manually
through VNC. With `--gateway-setup`, Mantis leaves a persistent OpenClaw Slack
gateway running inside the VM on port `38973`; without it, the command runs the
normal bot-to-bot Slack QA lane and exits after artifact capture.
Before using pooled live credentials, run:
```bash

View File

@@ -51,6 +51,7 @@ const {
runMantisBeforeAfterCommand,
runMantisDesktopBrowserSmokeCommand,
runMantisDiscordSmokeCommand,
runMantisSlackDesktopSmokeCommand,
} = vi.hoisted(() => ({
runQaCredentialsAddCommand: vi.fn(),
runQaCredentialsListCommand: vi.fn(),
@@ -62,6 +63,7 @@ const {
runMantisBeforeAfterCommand: vi.fn(),
runMantisDesktopBrowserSmokeCommand: vi.fn(),
runMantisDiscordSmokeCommand: vi.fn(),
runMantisSlackDesktopSmokeCommand: vi.fn(),
}));
const { listQaRunnerCliContributions } = vi.hoisted(() => ({
@@ -82,6 +84,7 @@ vi.mock("./mantis/cli.runtime.js", () => ({
runMantisBeforeAfterCommand,
runMantisDesktopBrowserSmokeCommand,
runMantisDiscordSmokeCommand,
runMantisSlackDesktopSmokeCommand,
}));
vi.mock("./cli.runtime.js", () => ({
@@ -110,6 +113,7 @@ describe("qa cli registration", () => {
runMantisBeforeAfterCommand.mockReset();
runMantisDesktopBrowserSmokeCommand.mockReset();
runMantisDiscordSmokeCommand.mockReset();
runMantisSlackDesktopSmokeCommand.mockReset();
listQaRunnerCliContributions
.mockReset()
.mockReturnValue([createAvailableQaRunnerContribution()]);
@@ -283,6 +287,70 @@ describe("qa cli registration", () => {
});
});
it("routes mantis Slack desktop smoke flags into the mantis runtime command", async () => {
await program.parseAsync([
"node",
"openclaw",
"qa",
"mantis",
"slack-desktop-smoke",
"--repo-root",
"/tmp/openclaw-repo",
"--output-dir",
".artifacts/qa-e2e/mantis/slack-desktop",
"--crabbox-bin",
"/tmp/crabbox",
"--provider",
"hetzner",
"--machine-class",
"beast",
"--lease-id",
"cbx_123abc",
"--idle-timeout",
"45m",
"--ttl",
"120m",
"--slack-url",
"https://app.slack.com/client/T123/C123",
"--provider-mode",
"live-frontier",
"--model",
"openai/gpt-5.4",
"--alt-model",
"openai/gpt-5.4",
"--scenario",
"slack-canary",
"--credential-source",
"env",
"--credential-role",
"maintainer",
"--fast",
"--keep-lease",
]);
expect(runMantisSlackDesktopSmokeCommand).toHaveBeenCalledWith({
alternateModel: "openai/gpt-5.4",
crabboxBin: "/tmp/crabbox",
credentialRole: "maintainer",
credentialSource: "env",
fastMode: true,
gatewaySetup: undefined,
idleTimeout: "45m",
keepLease: true,
leaseId: "cbx_123abc",
machineClass: "beast",
outputDir: ".artifacts/qa-e2e/mantis/slack-desktop",
primaryModel: "openai/gpt-5.4",
provider: "hetzner",
providerMode: "live-frontier",
repoRoot: "/tmp/openclaw-repo",
scenarioIds: ["slack-canary"],
slackChannelId: undefined,
slackUrl: "https://app.slack.com/client/T123/C123",
ttl: "120m",
});
});
it("routes coverage report flags into the qa runtime command", async () => {
await program.parseAsync([
"node",

View File

@@ -4,6 +4,10 @@ import {
} from "./desktop-browser-smoke.runtime.js";
import { runMantisDiscordSmoke, type MantisDiscordSmokeOptions } from "./discord-smoke.runtime.js";
import { runMantisBeforeAfter, type MantisBeforeAfterOptions } from "./run.runtime.js";
import {
runMantisSlackDesktopSmoke,
type MantisSlackDesktopSmokeOptions,
} from "./slack-desktop-smoke.runtime.js";
export async function runMantisDiscordSmokeCommand(opts: MantisDiscordSmokeOptions) {
const result = await runMantisDiscordSmoke(opts);
@@ -34,3 +38,15 @@ export async function runMantisDesktopBrowserSmokeCommand(opts: MantisDesktopBro
process.exitCode = 1;
}
}
export async function runMantisSlackDesktopSmokeCommand(opts: MantisSlackDesktopSmokeOptions) {
const result = await runMantisSlackDesktopSmoke(opts);
process.stdout.write(`Mantis Slack desktop report: ${result.reportPath}\n`);
process.stdout.write(`Mantis Slack desktop summary: ${result.summaryPath}\n`);
if (result.screenshotPath) {
process.stdout.write(`Mantis Slack desktop screenshot: ${result.screenshotPath}\n`);
}
if (result.status === "fail") {
process.exitCode = 1;
}
}

View File

@@ -3,6 +3,7 @@ import { createLazyCliRuntimeLoader } from "../live-transports/shared/live-trans
import type { MantisDesktopBrowserSmokeOptions } from "./desktop-browser-smoke.runtime.js";
import type { MantisDiscordSmokeOptions } from "./discord-smoke.runtime.js";
import type { MantisBeforeAfterOptions } from "./run.runtime.js";
import type { MantisSlackDesktopSmokeOptions } from "./slack-desktop-smoke.runtime.js";
type MantisCliRuntime = typeof import("./cli.runtime.js");
@@ -25,6 +26,11 @@ async function runDesktopBrowserSmoke(opts: MantisDesktopBrowserSmokeOptions) {
await runtime.runMantisDesktopBrowserSmokeCommand(opts);
}
async function runSlackDesktopSmoke(opts: MantisSlackDesktopSmokeOptions) {
const runtime = await loadMantisCliRuntime();
await runtime.runMantisSlackDesktopSmokeCommand(opts);
}
type MantisDiscordSmokeCommanderOptions = {
channelId?: string;
guildId?: string;
@@ -67,6 +73,33 @@ type MantisDesktopBrowserSmokeCommanderOptions = {
ttl?: string;
};
type MantisSlackDesktopSmokeCommanderOptions = {
altModel?: string;
class?: string;
crabboxBin?: string;
credentialRole?: string;
credentialSource?: string;
fast?: boolean;
gatewaySetup?: boolean;
idleTimeout?: string;
keepLease?: boolean;
leaseId?: string;
machineClass?: string;
model?: string;
outputDir?: string;
provider?: string;
providerMode?: string;
repoRoot?: string;
scenario?: string[];
slackChannelId?: string;
slackUrl?: string;
ttl?: string;
};
function collectString(value: string, previous: string[] = []) {
return [...previous, value];
}
export function registerMantisCli(qa: Command) {
const mantis = qa
.command("mantis")
@@ -162,4 +195,58 @@ export function registerMantisCli(qa: Command) {
ttl: opts.ttl,
});
});
mantis
.command("slack-desktop-smoke")
.description(
"Lease or reuse a Crabbox VNC desktop, run Slack QA inside it, open Slack in the browser, and capture a screenshot",
)
.option("--repo-root <path>", "Repository root to target when running from a neutral cwd")
.option("--output-dir <path>", "Mantis Slack desktop artifact directory")
.option("--crabbox-bin <path>", "Crabbox binary path")
.option("--provider <provider>", "Crabbox provider")
.option("--machine-class <class>", "Crabbox machine class")
.option("--class <class>", "Alias for --machine-class")
.option("--lease-id <id>", "Reuse an existing Crabbox lease")
.option("--idle-timeout <duration>", "Crabbox idle timeout")
.option("--ttl <duration>", "Crabbox maximum lease lifetime")
.option("--keep-lease", "Keep a lease created by this run after a passing smoke")
.option("--gateway-setup", "Start a persistent OpenClaw Slack gateway inside the VNC VM")
.option("--slack-url <url>", "Slack web URL to open in the visible browser")
.option("--slack-channel-id <id>", "Slack channel id for gateway setup allowlist")
.option("--provider-mode <mode>", "QA provider mode")
.option("--model <ref>", "Primary provider/model ref")
.option("--alt-model <ref>", "Alternate provider/model ref")
.option(
"--scenario <id>",
"Run only the named Slack QA scenario (repeatable)",
collectString,
[],
)
.option("--credential-source <source>", "Credential source for Slack QA: env or convex")
.option("--credential-role <role>", "Credential role for convex auth")
.option("--fast", "Enable provider fast mode where supported")
.action(async (opts: MantisSlackDesktopSmokeCommanderOptions) => {
await runSlackDesktopSmoke({
alternateModel: opts.altModel,
crabboxBin: opts.crabboxBin,
credentialRole: opts.credentialRole,
credentialSource: opts.credentialSource,
fastMode: opts.fast,
gatewaySetup: opts.gatewaySetup,
idleTimeout: opts.idleTimeout,
keepLease: opts.keepLease,
leaseId: opts.leaseId,
machineClass: opts.machineClass ?? opts.class,
outputDir: opts.outputDir,
primaryModel: opts.model,
provider: opts.provider,
providerMode: opts.providerMode,
repoRoot: opts.repoRoot,
scenarioIds: opts.scenario,
slackChannelId: opts.slackChannelId,
slackUrl: opts.slackUrl,
ttl: opts.ttl,
});
});
}

View File

@@ -0,0 +1,177 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { runMantisSlackDesktopSmoke } from "./slack-desktop-smoke.runtime.js";
describe("mantis Slack desktop smoke runtime", () => {
let repoRoot: string;
beforeEach(async () => {
repoRoot = await fs.mkdtemp(path.join(os.tmpdir(), "mantis-slack-desktop-smoke-"));
});
afterEach(async () => {
await fs.rm(repoRoot, { force: true, recursive: true });
});
it("leases a desktop box, runs Slack QA inside it, copies artifacts, and stops on pass", async () => {
const commands: { args: readonly string[]; command: string; env?: NodeJS.ProcessEnv }[] = [];
const runtimeEnv = {
PATH: process.env.PATH,
OPENAI_API_KEY: "openai-runtime-key",
OPENCLAW_QA_SLACK_CHANNEL_ID: "C123",
OPENCLAW_QA_SLACK_DRIVER_BOT_TOKEN: "driver-token",
OPENCLAW_QA_SLACK_SUT_APP_TOKEN: "app-token",
OPENCLAW_QA_SLACK_SUT_BOT_TOKEN: "sut-token",
};
const runner = vi.fn(
async (command: string, args: readonly string[], options: { env?: NodeJS.ProcessEnv }) => {
commands.push({ command, args, env: options.env });
if (command === "/tmp/crabbox" && args[0] === "warmup") {
return { stdout: "ready lease cbx_abc123\n", stderr: "" };
}
if (command === "/tmp/crabbox" && args[0] === "inspect") {
return {
stdout: `${JSON.stringify({
host: "203.0.113.10",
id: "cbx_abc123",
provider: "hetzner",
slug: "bright-mantis",
sshKey: "/tmp/key",
sshPort: "2222",
sshUser: "crabbox",
state: "active",
})}\n`,
stderr: "",
};
}
if (command === "rsync") {
const outputDir = args.at(-1);
expect(outputDir).toBeTypeOf("string");
await fs.mkdir(outputDir as string, { recursive: true });
if (String(outputDir).endsWith("slack-qa/")) {
await fs.writeFile(path.join(outputDir as string, "slack-qa-report.md"), "# Slack\n");
} else {
await fs.writeFile(path.join(outputDir as string, "slack-desktop-smoke.png"), "png");
await fs.writeFile(path.join(outputDir as string, "remote-metadata.json"), "{}\n");
await fs.writeFile(path.join(outputDir as string, "chrome.log"), "chrome\n");
await fs.writeFile(path.join(outputDir as string, "slack-desktop-command.log"), "qa\n");
}
return { stdout: "", stderr: "" };
}
return { stdout: "", stderr: "" };
},
);
const result = await runMantisSlackDesktopSmoke({
commandRunner: runner,
crabboxBin: "/tmp/crabbox",
env: runtimeEnv,
now: () => new Date("2026-05-04T13:00:00.000Z"),
outputDir: ".artifacts/qa-e2e/mantis/slack-desktop-test",
primaryModel: "openai/gpt-5.4",
repoRoot,
scenarioIds: ["slack-canary"],
slackUrl: "https://app.slack.com/client/T123/C123",
});
expect(result.status).toBe("pass");
expect(commands.map((entry) => [entry.command, entry.args[0]])).toEqual([
["/tmp/crabbox", "warmup"],
["/tmp/crabbox", "inspect"],
["/tmp/crabbox", "run"],
["rsync", "-az"],
["rsync", "-az"],
["/tmp/crabbox", "stop"],
]);
expect(
commands.every((entry) => entry.env?.OPENCLAW_LIVE_OPENAI_KEY === "openai-runtime-key"),
).toBe(true);
const runArgs = commands.find(
(entry) => entry.command === "/tmp/crabbox" && entry.args[0] === "run",
)?.args;
expect(runArgs).not.toContain("--no-sync");
const remoteScript = runArgs?.at(-1);
expect(remoteScript).toContain("${BROWSER:-}");
expect(remoteScript).toContain("${CHROME_BIN:-}");
expect(remoteScript).toContain("pnpm install --frozen-lockfile");
expect(remoteScript).toContain("pnpm build");
expect(remoteScript).toContain("openclaw qa slack");
expect(remoteScript).toContain("--scenario 'slack-canary'");
expect(remoteScript).toContain("OPENCLAW_MANTIS_SLACK_BROWSER_PROFILE_DIR");
const rsyncArgs = commands
.filter((entry) => entry.command === "rsync")
.flatMap((entry) => [...entry.args]);
expect(rsyncArgs).not.toContain("--delete");
expect(rsyncArgs).toEqual(
expect.arrayContaining([
"crabbox@203.0.113.10:/tmp/openclaw-mantis-slack-desktop-2026-05-04T13-00-00-000Z/slack-desktop-smoke.png",
"crabbox@203.0.113.10:/tmp/openclaw-mantis-slack-desktop-2026-05-04T13-00-00-000Z/slack-qa/",
]),
);
await expect(fs.readFile(result.screenshotPath ?? "", "utf8")).resolves.toBe("png");
const summary = JSON.parse(await fs.readFile(result.summaryPath, "utf8")) as {
crabbox: { id: string; vncCommand: string };
status: string;
};
expect(summary).toMatchObject({
crabbox: {
id: "cbx_abc123",
vncCommand: "/tmp/crabbox vnc --provider hetzner --id cbx_abc123 --open",
},
status: "pass",
});
});
it("copies the screenshot before reporting a failed remote Slack QA run", async () => {
const runner = vi.fn(async (command: string, args: readonly string[]) => {
if (command === "/tmp/crabbox" && args[0] === "inspect") {
return {
stdout: `${JSON.stringify({
host: "203.0.113.10",
id: "cbx_existing",
provider: "hetzner",
sshKey: "/tmp/key",
sshPort: "2222",
sshUser: "crabbox",
})}\n`,
stderr: "",
};
}
if (command === "/tmp/crabbox" && args[0] === "run") {
throw new Error("remote Slack QA failed");
}
if (command === "rsync") {
const outputDir = args.at(-1);
await fs.mkdir(outputDir as string, { recursive: true });
await fs.writeFile(path.join(outputDir as string, "slack-desktop-smoke.png"), "png");
await fs.writeFile(path.join(outputDir as string, "remote-metadata.json"), "{}\n");
await fs.writeFile(path.join(outputDir as string, "chrome.log"), "chrome\n");
await fs.writeFile(path.join(outputDir as string, "slack-desktop-command.log"), "qa\n");
}
return { stdout: "", stderr: "" };
});
const result = await runMantisSlackDesktopSmoke({
commandRunner: runner,
crabboxBin: "/tmp/crabbox",
leaseId: "cbx_existing",
outputDir: ".artifacts/qa-e2e/mantis/slack-desktop-fail",
repoRoot,
});
expect(result.status).toBe("fail");
await expect(
fs.readFile(path.join(result.outputDir, "slack-desktop-smoke.png"), "utf8"),
).resolves.toBe("png");
const summary = JSON.parse(await fs.readFile(result.summaryPath, "utf8")) as {
artifacts: { screenshotPath?: string };
error?: string;
status: string;
};
expect(summary.status).toBe("fail");
expect(summary.error).toContain("remote Slack QA failed");
expect(summary.artifacts.screenshotPath).toContain("slack-desktop-smoke.png");
});
});

View File

@@ -0,0 +1,784 @@
import { spawn, type SpawnOptions } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { ensureRepoBoundDirectory, resolveRepoRelativeOutputDir } from "../cli-paths.js";
export type MantisSlackDesktopSmokeOptions = {
alternateModel?: string;
commandRunner?: CommandRunner;
crabboxBin?: string;
credentialRole?: string;
credentialSource?: string;
env?: NodeJS.ProcessEnv;
fastMode?: boolean;
gatewaySetup?: boolean;
idleTimeout?: string;
keepLease?: boolean;
leaseId?: string;
machineClass?: string;
now?: () => Date;
outputDir?: string;
primaryModel?: string;
provider?: string;
providerMode?: string;
repoRoot?: string;
scenarioIds?: string[];
slackChannelId?: string;
slackUrl?: string;
ttl?: string;
};
export type MantisSlackDesktopSmokeResult = {
outputDir: string;
reportPath: string;
screenshotPath?: string;
status: "pass" | "fail";
summaryPath: string;
};
type CommandResult = {
stderr: string;
stdout: string;
};
type CommandRunner = (
command: string,
args: readonly string[],
options: SpawnOptions,
) => Promise<CommandResult>;
type CrabboxInspect = {
host?: string;
id?: string;
provider?: string;
ready?: boolean;
slug?: string;
sshKey?: string;
sshPort?: string;
sshUser?: string;
state?: string;
};
type MantisSlackDesktopSmokeSummary = {
artifacts: {
reportPath: string;
screenshotPath?: string;
slackQaDir?: string;
summaryPath: string;
};
crabbox: {
bin: string;
createdLease: boolean;
id: string;
provider: string;
slug?: string;
state?: string;
vncCommand: string;
};
error?: string;
finishedAt: string;
outputDir: string;
remoteOutputDir: string;
slackUrl?: string;
startedAt: string;
status: "pass" | "fail";
};
const DEFAULT_PROVIDER = "hetzner";
const DEFAULT_CLASS = "beast";
const DEFAULT_IDLE_TIMEOUT = "90m";
const DEFAULT_TTL = "180m";
const DEFAULT_CREDENTIAL_SOURCE = "env";
const DEFAULT_CREDENTIAL_ROLE = "maintainer";
const DEFAULT_PROVIDER_MODE = "live-frontier";
const DEFAULT_MODEL = "openai/gpt-5.4";
const DEFAULT_SLACK_CHANNEL_ID = "C0AUXUC5AGN";
const CRABBOX_BIN_ENV = "OPENCLAW_MANTIS_CRABBOX_BIN";
const CRABBOX_PROVIDER_ENV = "OPENCLAW_MANTIS_CRABBOX_PROVIDER";
const CRABBOX_CLASS_ENV = "OPENCLAW_MANTIS_CRABBOX_CLASS";
const CRABBOX_LEASE_ID_ENV = "OPENCLAW_MANTIS_CRABBOX_LEASE_ID";
const CRABBOX_KEEP_ENV = "OPENCLAW_MANTIS_KEEP_VM";
const CRABBOX_IDLE_TIMEOUT_ENV = "OPENCLAW_MANTIS_CRABBOX_IDLE_TIMEOUT";
const CRABBOX_TTL_ENV = "OPENCLAW_MANTIS_CRABBOX_TTL";
const SLACK_URL_ENV = "OPENCLAW_MANTIS_SLACK_URL";
const SLACK_CHANNEL_ID_ENV = "OPENCLAW_MANTIS_SLACK_CHANNEL_ID";
function trimToValue(value: string | undefined) {
const trimmed = value?.trim();
return trimmed && trimmed.length > 0 ? trimmed : undefined;
}
function isTruthyOptIn(value: string | undefined) {
const normalized = value?.trim().toLowerCase();
return normalized === "1" || normalized === "true" || normalized === "yes";
}
function defaultOutputDir(repoRoot: string, startedAt: Date) {
const stamp = startedAt.toISOString().replace(/[:.]/gu, "-");
return path.join(repoRoot, ".artifacts", "qa-e2e", "mantis", `slack-desktop-${stamp}`);
}
async function defaultCommandRunner(
command: string,
args: readonly string[],
options: SpawnOptions,
): Promise<CommandResult> {
return new Promise((resolve, reject) => {
const child = spawn(command, args, {
...options,
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout?.on("data", (chunk: Buffer) => {
const text = chunk.toString();
stdout += text;
if (options.stdio === "inherit") {
process.stdout.write(text);
}
});
child.stderr?.on("data", (chunk: Buffer) => {
const text = chunk.toString();
stderr += text;
if (options.stdio === "inherit") {
process.stderr.write(text);
}
});
child.on("error", reject);
child.on("close", (code, signal) => {
if (code === 0) {
resolve({ stdout, stderr });
return;
}
const detail = signal ? `signal ${signal}` : `exit code ${code ?? "unknown"}`;
reject(new Error(`${command} ${args.join(" ")} failed with ${detail}`));
});
});
}
async function pathExists(filePath: string) {
try {
await fs.access(filePath);
return true;
} catch {
return false;
}
}
async function resolveCrabboxBin(params: {
env: NodeJS.ProcessEnv;
explicit?: string;
repoRoot: string;
}) {
const configured = trimToValue(params.explicit) ?? trimToValue(params.env[CRABBOX_BIN_ENV]);
if (configured) {
return configured;
}
const sibling = path.resolve(params.repoRoot, "../crabbox/bin/crabbox");
if (await pathExists(sibling)) {
return sibling;
}
return "crabbox";
}
function buildCrabboxEnv(env: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
const next = {
...env,
};
if (!trimToValue(next.OPENCLAW_LIVE_OPENAI_KEY) && trimToValue(next.OPENAI_API_KEY)) {
next.OPENCLAW_LIVE_OPENAI_KEY = next.OPENAI_API_KEY;
}
if (!trimToValue(next.OPENCLAW_MANTIS_SLACK_BOT_TOKEN) && trimToValue(next.SLACK_BOT_TOKEN)) {
next.OPENCLAW_MANTIS_SLACK_BOT_TOKEN = next.SLACK_BOT_TOKEN;
}
if (!trimToValue(next.OPENCLAW_MANTIS_SLACK_APP_TOKEN) && trimToValue(next.SLACK_APP_TOKEN)) {
next.OPENCLAW_MANTIS_SLACK_APP_TOKEN = next.SLACK_APP_TOKEN;
}
return next;
}
function extractLeaseId(output: string) {
return output.match(/\bcbx_[a-f0-9]+\b/u)?.[0];
}
function shellQuote(value: string) {
return `'${value.replaceAll("'", "'\\''")}'`;
}
function renderRemoteScript(params: {
alternateModel: string;
credentialRole: string;
credentialSource: string;
fastMode: boolean;
primaryModel: string;
providerMode: string;
remoteOutputDir: string;
scenarioIds: readonly string[];
setupGateway: boolean;
slackChannelId: string;
slackUrl?: string;
}) {
const shellOutputDir = shellQuote(params.remoteOutputDir);
const slackUrl = shellQuote(params.slackUrl ?? "");
const credentialSource = shellQuote(params.credentialSource);
const credentialRole = shellQuote(params.credentialRole);
const providerMode = shellQuote(params.providerMode);
const primaryModel = shellQuote(params.primaryModel);
const alternateModel = shellQuote(params.alternateModel);
const fastMode = params.fastMode ? "1" : "0";
const setupGateway = params.setupGateway ? "1" : "0";
const slackChannelId = shellQuote(params.slackChannelId);
const scenarioArgs = params.scenarioIds.flatMap((id) => ["--scenario", shellQuote(id)]).join(" ");
return `set -euo pipefail
out=${shellOutputDir}
slack_url_override=${slackUrl}
credential_source=${credentialSource}
credential_role=${credentialRole}
provider_mode=${providerMode}
primary_model=${primaryModel}
alternate_model=${alternateModel}
fast_mode=${fastMode}
setup_gateway=${setupGateway}
slack_channel_id=${slackChannelId}
rm -rf "$out"
mkdir -p "$out"
export DISPLAY="\${DISPLAY:-:99}"
if [ -n "\${OPENCLAW_LIVE_OPENAI_KEY:-}" ] && [ -z "\${OPENAI_API_KEY:-}" ]; then
export OPENAI_API_KEY="$OPENCLAW_LIVE_OPENAI_KEY"
fi
if ! command -v node >/dev/null 2>&1; then
sudo apt-get update -y >"$out/node-apt.log" 2>&1
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash - >>"$out/node-apt.log" 2>&1
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y nodejs >>"$out/node-apt.log" 2>&1
fi
if ! command -v scrot >/dev/null 2>&1; then
sudo apt-get update -y >"$out/apt.log" 2>&1
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y scrot >>"$out/apt.log" 2>&1
fi
browser_bin=""
for candidate in "\${BROWSER:-}" "\${CHROME_BIN:-}" google-chrome chromium chromium-browser; do
if [ -n "$candidate" ] && command -v "$candidate" >/dev/null 2>&1; then
browser_bin="$(command -v "$candidate")"
break
fi
done
if [ -z "$browser_bin" ]; then
echo "No browser binary found. Checked BROWSER, CHROME_BIN, google-chrome, chromium, chromium-browser." >&2
exit 127
fi
team_id="\${OPENCLAW_QA_SLACK_TEAM_ID:-}"
auth_test_token="\${OPENCLAW_QA_SLACK_SUT_BOT_TOKEN:-\${OPENCLAW_MANTIS_SLACK_BOT_TOKEN:-}}"
if [ -z "$slack_url_override" ] && [ -z "$team_id" ] && [ -n "$auth_test_token" ]; then
node --input-type=module >"$out/slack-auth-test.json" 2>"$out/slack-auth-test.err" <<'MANTIS_SLACK_AUTH'
const token = process.env.OPENCLAW_QA_SLACK_SUT_BOT_TOKEN || process.env.OPENCLAW_MANTIS_SLACK_BOT_TOKEN;
const response = await fetch("https://slack.com/api/auth.test", {
method: "POST",
headers: { authorization: \`Bearer \${token}\` },
});
const body = await response.json();
process.stdout.write(JSON.stringify({ ok: body.ok, team_id: body.team_id, user_id: body.user_id }));
if (!body.ok) process.exit(1);
MANTIS_SLACK_AUTH
team_id="$(node --input-type=module -e 'import fs from "node:fs"; const value = JSON.parse(fs.readFileSync(process.argv[1], "utf8")); process.stdout.write(value.team_id || "");' "$out/slack-auth-test.json" || true)"
fi
slack_url="$slack_url_override"
if [ -z "$slack_url" ] && [ -n "$team_id" ] && [ -n "\${OPENCLAW_QA_SLACK_CHANNEL_ID:-}" ]; then
slack_url="https://app.slack.com/client/$team_id/$OPENCLAW_QA_SLACK_CHANNEL_ID"
fi
profile="\${OPENCLAW_MANTIS_SLACK_BROWSER_PROFILE_DIR:-$HOME/.config/openclaw-mantis/slack-chrome-profile}"
mkdir -p "$profile"
if [ "$setup_gateway" = "1" ]; then
export SLACK_BOT_TOKEN="\${OPENCLAW_MANTIS_SLACK_BOT_TOKEN:-\${SLACK_BOT_TOKEN:-}}"
export SLACK_APP_TOKEN="\${OPENCLAW_MANTIS_SLACK_APP_TOKEN:-\${SLACK_APP_TOKEN:-}}"
if [ -z "$SLACK_BOT_TOKEN" ] || [ -z "$SLACK_APP_TOKEN" ]; then
echo "Gateway setup requires OPENCLAW_MANTIS_SLACK_BOT_TOKEN and OPENCLAW_MANTIS_SLACK_APP_TOKEN." >&2
exit 2
fi
if [ -z "$slack_url" ] && [ -n "$team_id" ]; then
slack_url="https://app.slack.com/client/$team_id/$slack_channel_id"
fi
fi
if [ -z "$slack_url" ]; then
slack_url="https://app.slack.com/client"
fi
if [ "$setup_gateway" = "1" ]; then
nohup "$browser_bin" \
--user-data-dir="$profile" \
--no-first-run \
--no-default-browser-check \
--disable-dev-shm-usage \
--window-size=1440,1000 \
--window-position=0,0 \
--class=mantis-slack-desktop-smoke \
"$slack_url" >"$out/chrome.log" 2>&1 &
else
"$browser_bin" \
--user-data-dir="$profile" \
--no-first-run \
--no-default-browser-check \
--disable-dev-shm-usage \
--window-size=1440,1000 \
--window-position=0,0 \
--class=mantis-slack-desktop-smoke \
"$slack_url" >"$out/chrome.log" 2>&1 &
fi
chrome_pid=$!
qa_status=0
{
set -e
echo "remote pwd: $(pwd)"
sudo corepack enable || sudo npm install -g pnpm@10.33.2
pnpm install --frozen-lockfile
pnpm build
if [ "$setup_gateway" = "1" ]; then
export OPENCLAW_HOME="$HOME/.openclaw-mantis/slack-openclaw"
mkdir -p "$OPENCLAW_HOME"
cat >"$out/slack.socket.patch.json5" <<MANTIS_SLACK_PATCH
{
gateway: {
port: 38973,
auth: { mode: "none" },
},
channels: {
slack: {
enabled: true,
mode: "socket",
webhookPath: "/slack/events",
userTokenReadOnly: true,
appToken: { source: "env", provider: "default", id: "SLACK_APP_TOKEN" },
botToken: { source: "env", provider: "default", id: "SLACK_BOT_TOKEN" },
groupPolicy: "allowlist",
channels: {
"$slack_channel_id": {
enabled: true,
requireMention: true,
allowBots: true,
users: ["*"],
},
},
},
},
}
MANTIS_SLACK_PATCH
pnpm openclaw config patch --file "$out/slack.socket.patch.json5" --dry-run
pnpm openclaw config patch --file "$out/slack.socket.patch.json5"
nohup pnpm openclaw gateway run --dev --allow-unconfigured --port 38973 --cli-backend-logs >"$out/openclaw-gateway.log" 2>&1 &
echo "$!" >"$out/openclaw-gateway.pid"
sleep 12
else
qa_args=(openclaw qa slack --repo-root . --output-dir "$out/slack-qa" --provider-mode "$provider_mode" --model "$primary_model" --alt-model "$alternate_model" --credential-source "$credential_source" --credential-role "$credential_role")
if [ "$fast_mode" = "1" ]; then
qa_args+=(--fast)
fi
pnpm "\${qa_args[@]}" ${scenarioArgs}
fi
} >"$out/slack-desktop-command.log" 2>&1 || qa_status=$?
sleep 5
scrot "$out/slack-desktop-smoke.png" || true
if [ "$setup_gateway" != "1" ]; then
kill "$chrome_pid" >/dev/null 2>&1 || true
fi
cat >"$out/remote-metadata.json" <<MANTIS_REMOTE_METADATA
{
"browserBinary": "$browser_bin",
"browserProfile": "$profile",
"display": "$DISPLAY",
"openedUrl": "$slack_url",
"gatewaySetup": $setup_gateway,
"gatewayPort": 38973,
"qaExitCode": $qa_status,
"credentialSource": "$credential_source",
"credentialRole": "$credential_role",
"providerMode": "$provider_mode",
"capturedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
MANTIS_REMOTE_METADATA
test -s "$out/slack-desktop-smoke.png"
exit "$qa_status"
`;
}
function renderReport(summary: MantisSlackDesktopSmokeSummary) {
const lines = [
"# Mantis Slack Desktop Smoke",
"",
`Status: ${summary.status}`,
summary.slackUrl ? `Slack URL: ${summary.slackUrl}` : undefined,
`Output: ${summary.outputDir}`,
`Started: ${summary.startedAt}`,
`Finished: ${summary.finishedAt}`,
"",
"## Crabbox",
"",
`- Provider: ${summary.crabbox.provider}`,
`- Lease: ${summary.crabbox.id}${summary.crabbox.slug ? ` (${summary.crabbox.slug})` : ""}`,
`- Created by run: ${summary.crabbox.createdLease}`,
`- State: ${summary.crabbox.state ?? "unknown"}`,
`- VNC: \`${summary.crabbox.vncCommand}\``,
"",
"## Artifacts",
"",
summary.artifacts.screenshotPath
? `- Screenshot: \`${path.basename(summary.artifacts.screenshotPath)}\``
: "- Screenshot: missing",
summary.artifacts.slackQaDir ? "- Slack QA artifacts: `slack-qa/`" : undefined,
"- Remote metadata: `remote-metadata.json`",
"- Remote command log: `slack-desktop-command.log`",
"- Chrome log: `chrome.log`",
summary.error ? `- Error: ${summary.error}` : undefined,
"",
].filter((line) => line !== undefined);
return `${lines.join("\n")}\n`;
}
async function runCommand(params: {
args: readonly string[];
command: string;
cwd: string;
env: NodeJS.ProcessEnv;
runner: CommandRunner;
stdio?: "inherit" | "pipe";
}) {
return params.runner(params.command, params.args, {
cwd: params.cwd,
env: params.env,
stdio: params.stdio ?? "pipe",
});
}
async function warmupCrabbox(params: {
crabboxBin: string;
cwd: string;
env: NodeJS.ProcessEnv;
idleTimeout: string;
machineClass: string;
provider: string;
runner: CommandRunner;
ttl: string;
}) {
const result = await runCommand({
command: params.crabboxBin,
args: [
"warmup",
"--provider",
params.provider,
"--desktop",
"--browser",
"--class",
params.machineClass,
"--idle-timeout",
params.idleTimeout,
"--ttl",
params.ttl,
],
cwd: params.cwd,
env: params.env,
runner: params.runner,
stdio: "inherit",
});
const leaseId = extractLeaseId(`${result.stdout}\n${result.stderr}`);
if (!leaseId) {
throw new Error("Crabbox warmup did not print a cbx_ lease id.");
}
return leaseId;
}
async function inspectCrabbox(params: {
crabboxBin: string;
cwd: string;
env: NodeJS.ProcessEnv;
leaseId: string;
provider: string;
runner: CommandRunner;
}) {
const result = await runCommand({
command: params.crabboxBin,
args: ["inspect", "--provider", params.provider, "--id", params.leaseId, "--json"],
cwd: params.cwd,
env: params.env,
runner: params.runner,
});
return JSON.parse(result.stdout) as CrabboxInspect;
}
function sshCommand(params: { inspect: CrabboxInspect }) {
const { host, sshKey, sshPort, sshUser } = params.inspect;
if (!host || !sshKey || !sshUser) {
throw new Error("Crabbox inspect output is missing SSH copy details.");
}
return {
host,
sshUser,
sshArgs: [
"ssh",
"-i",
shellQuote(sshKey),
"-p",
sshPort ?? "22",
"-o",
"BatchMode=yes",
"-o",
"ConnectTimeout=15",
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
].join(" "),
};
}
async function copyRemoteArtifacts(params: {
cwd: string;
env: NodeJS.ProcessEnv;
inspect: CrabboxInspect;
outputDir: string;
remoteOutputDir: string;
runner: CommandRunner;
}) {
const { host, sshArgs, sshUser } = sshCommand({ inspect: params.inspect });
await fs.mkdir(path.join(params.outputDir, "slack-qa"), { recursive: true });
await runCommand({
command: "rsync",
args: [
"-az",
"-e",
sshArgs,
`${sshUser}@${host}:${params.remoteOutputDir}/slack-desktop-smoke.png`,
`${sshUser}@${host}:${params.remoteOutputDir}/remote-metadata.json`,
`${sshUser}@${host}:${params.remoteOutputDir}/chrome.log`,
`${sshUser}@${host}:${params.remoteOutputDir}/slack-desktop-command.log`,
`${params.outputDir}/`,
],
cwd: params.cwd,
env: params.env,
runner: params.runner,
});
await runCommand({
command: "rsync",
args: [
"-az",
"-e",
sshArgs,
`${sshUser}@${host}:${params.remoteOutputDir}/slack-qa/`,
`${path.join(params.outputDir, "slack-qa")}/`,
],
cwd: params.cwd,
env: params.env,
runner: params.runner,
}).catch(() => ({ stdout: "", stderr: "" }));
}
async function stopCrabbox(params: {
crabboxBin: string;
cwd: string;
env: NodeJS.ProcessEnv;
leaseId: string;
provider: string;
runner: CommandRunner;
}) {
await runCommand({
command: params.crabboxBin,
args: ["stop", "--provider", params.provider, params.leaseId],
cwd: params.cwd,
env: params.env,
runner: params.runner,
stdio: "inherit",
});
}
export async function runMantisSlackDesktopSmoke(
opts: MantisSlackDesktopSmokeOptions = {},
): Promise<MantisSlackDesktopSmokeResult> {
const env = buildCrabboxEnv(opts.env ?? process.env);
const startedAt = (opts.now ?? (() => new Date()))();
const repoRoot = path.resolve(opts.repoRoot ?? process.cwd());
const outputDir = await ensureRepoBoundDirectory(
repoRoot,
resolveRepoRelativeOutputDir(repoRoot, opts.outputDir) ?? defaultOutputDir(repoRoot, startedAt),
"Mantis Slack desktop smoke output directory",
{ mode: 0o755 },
);
const summaryPath = path.join(outputDir, "mantis-slack-desktop-smoke-summary.json");
const reportPath = path.join(outputDir, "mantis-slack-desktop-smoke-report.md");
const crabboxBin = await resolveCrabboxBin({ env, explicit: opts.crabboxBin, repoRoot });
const provider =
trimToValue(opts.provider) ?? trimToValue(env[CRABBOX_PROVIDER_ENV]) ?? DEFAULT_PROVIDER;
const machineClass =
trimToValue(opts.machineClass) ?? trimToValue(env[CRABBOX_CLASS_ENV]) ?? DEFAULT_CLASS;
const idleTimeout =
trimToValue(opts.idleTimeout) ??
trimToValue(env[CRABBOX_IDLE_TIMEOUT_ENV]) ??
DEFAULT_IDLE_TIMEOUT;
const ttl = trimToValue(opts.ttl) ?? trimToValue(env[CRABBOX_TTL_ENV]) ?? DEFAULT_TTL;
const credentialSource = trimToValue(opts.credentialSource) ?? DEFAULT_CREDENTIAL_SOURCE;
const credentialRole = trimToValue(opts.credentialRole) ?? DEFAULT_CREDENTIAL_ROLE;
const providerMode = trimToValue(opts.providerMode) ?? DEFAULT_PROVIDER_MODE;
const primaryModel = trimToValue(opts.primaryModel) ?? DEFAULT_MODEL;
const alternateModel = trimToValue(opts.alternateModel) ?? primaryModel;
const fastMode = opts.fastMode ?? true;
const gatewaySetup = opts.gatewaySetup ?? false;
const scenarioIds = opts.scenarioIds ?? [];
const slackChannelId =
trimToValue(opts.slackChannelId) ??
trimToValue(env[SLACK_CHANNEL_ID_ENV]) ??
trimToValue(env.OPENCLAW_QA_SLACK_CHANNEL_ID) ??
DEFAULT_SLACK_CHANNEL_ID;
const slackUrl = trimToValue(opts.slackUrl) ?? trimToValue(env[SLACK_URL_ENV]);
const runner = opts.commandRunner ?? defaultCommandRunner;
const explicitLeaseId = trimToValue(opts.leaseId) ?? trimToValue(env[CRABBOX_LEASE_ID_ENV]);
const keepLease = opts.keepLease ?? (gatewaySetup || isTruthyOptIn(env[CRABBOX_KEEP_ENV]));
const createdLease = explicitLeaseId === undefined;
const remoteOutputDir = `/tmp/openclaw-mantis-slack-desktop-${startedAt
.toISOString()
.replace(/[^0-9A-Za-z]/gu, "-")}`;
let leaseId = explicitLeaseId;
let summary: MantisSlackDesktopSmokeSummary | undefined;
let screenshotPath: string | undefined;
let slackQaDir: string | undefined;
try {
leaseId =
leaseId ??
(await warmupCrabbox({
crabboxBin,
cwd: repoRoot,
env,
idleTimeout,
machineClass,
provider,
runner,
ttl,
}));
const inspected = await inspectCrabbox({
crabboxBin,
cwd: repoRoot,
env,
leaseId,
provider,
runner,
});
let remoteRunError: unknown;
await runCommand({
command: crabboxBin,
args: [
"run",
"--provider",
provider,
"--id",
leaseId,
"--desktop",
"--browser",
"--shell",
"--",
renderRemoteScript({
alternateModel,
credentialRole,
credentialSource,
fastMode,
primaryModel,
providerMode,
remoteOutputDir,
scenarioIds,
setupGateway: gatewaySetup,
slackChannelId,
slackUrl,
}),
],
cwd: repoRoot,
env,
runner,
stdio: "inherit",
}).catch((error: unknown) => {
remoteRunError = error;
return { stdout: "", stderr: "" };
});
await copyRemoteArtifacts({
cwd: repoRoot,
env,
inspect: inspected,
outputDir,
remoteOutputDir,
runner,
});
screenshotPath = path.join(outputDir, "slack-desktop-smoke.png");
slackQaDir = path.join(outputDir, "slack-qa");
if (!(await pathExists(screenshotPath))) {
throw new Error("Slack desktop screenshot was not copied back from Crabbox.");
}
if (remoteRunError) {
throw remoteRunError;
}
summary = {
artifacts: {
reportPath,
screenshotPath,
slackQaDir,
summaryPath,
},
crabbox: {
bin: crabboxBin,
createdLease,
id: leaseId,
provider,
slug: inspected.slug,
state: inspected.state,
vncCommand: `${crabboxBin} vnc --provider ${provider} --id ${leaseId} --open`,
},
finishedAt: new Date().toISOString(),
outputDir,
remoteOutputDir,
slackUrl,
startedAt: startedAt.toISOString(),
status: "pass",
};
return {
outputDir,
reportPath,
screenshotPath,
status: "pass",
summaryPath,
};
} catch (error) {
summary = {
artifacts: {
reportPath,
screenshotPath,
slackQaDir,
summaryPath,
},
crabbox: {
bin: crabboxBin,
createdLease,
id: leaseId ?? "unallocated",
provider,
vncCommand: leaseId
? `${crabboxBin} vnc --provider ${provider} --id ${leaseId} --open`
: "unallocated",
},
error: formatErrorMessage(error),
finishedAt: new Date().toISOString(),
outputDir,
remoteOutputDir,
slackUrl,
startedAt: startedAt.toISOString(),
status: "fail",
};
await fs.writeFile(path.join(outputDir, "error.txt"), `${summary.error}\n`, "utf8");
return {
outputDir,
reportPath,
status: "fail",
summaryPath,
};
} finally {
if (summary) {
summary.finishedAt = new Date().toISOString();
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, "utf8");
await fs.writeFile(reportPath, renderReport(summary), "utf8");
}
if (summary?.status === "pass" && createdLease && leaseId && !keepLease) {
await stopCrabbox({ crabboxBin, cwd: repoRoot, env, leaseId, provider, runner });
}
}
}