mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 07:30:43 +00:00
feat: add proxy validation command
Adds `openclaw proxy validate` for operator-managed proxy preflight checks, including allowed/denied destination validation, CLI output, tests, docs, and changelog coverage. Maintainer follow-ups before landing: - validate custom allowed URLs before probing; - use a temporary loopback canary for default denied checks and fail custom denied transport errors as unverifiable; - redact proxy URL userinfo, query strings, and fragments from text/JSON validation output. Validation: - `pnpm test src/infra/net/proxy/proxy-validation.test.ts src/cli/proxy-cli.runtime.test.ts src/cli/proxy-cli.test.ts -- --reporter=verbose` - `pnpm exec oxfmt --check --threads=1 CHANGELOG.md src/cli/proxy-cli.ts src/cli/proxy-cli.runtime.ts src/cli/proxy-cli.test.ts src/cli/proxy-cli.runtime.test.ts src/infra/net/proxy/proxy-validation.ts src/infra/net/proxy/proxy-validation.test.ts docs/cli/proxy.md docs/security/network-proxy.md` - `pnpm exec oxlint src/cli/proxy-cli.runtime.ts src/cli/proxy-cli.runtime.test.ts` - `git diff --check` - Testbox `pnpm install && OPENCLAW_TESTBOX=1 pnpm check:changed` on `tbx_01kqgz68ff20n3dtrgq0j1mykt` - GitHub CI success on `321b3aaf2b8be27dec6ce2ac5e4007ed064218b5`
This commit is contained in:
Jesse Merhi
@@ -9,6 +9,7 @@ Docs: https://docs.openclaw.ai
|
||||
- Messages/docs: clarify that `BodyForAgent` is the primary inbound model text while `Body` is the legacy envelope fallback, and add Signal coverage so channel hardening patches target the real prompt path. Refs #66198. Thanks @defonota3box.
|
||||
- Control UI/Usage: add UTC quarter-hour token buckets for the Usage Mosaic and reuse them for hour filtering, keeping the legacy session-span fallback for older summaries. (#74337) Thanks @konanok.
|
||||
- BlueBubbles: add opt-in `channels.bluebubbles.replyContextApiFallback` that fetches the original message from the BlueBubbles HTTP API when the in-memory reply-context cache misses (multi-instance deployments sharing one BB account, post-restart, after long-lived TTL/LRU eviction). Off by default; channel-level setting propagates to accounts that omit the flag through `mergeAccountConfig`; routed through the typed `BlueBubblesClient` so every fetch is SSRF-guarded by the same three-mode policy as every other BB client request; reply-id shape is validated and part-index prefixes (`p:0/<guid>`) are stripped before the request; concurrent webhooks for the same `replyToId` coalesce into one fetch and successful responses populate the reply cache for subsequent hits. Also promotes BlueBubbles attachment download failures from verbose to runtime error so silently-dropped inbound images are visible at default log level, and extends `sanitizeForLog` to redact `?password=…`/`?token=…` query params and `Authorization:` headers before they reach the log sink (CWE-532). (#71820) Thanks @coletebou and @zqchris.
|
||||
- CLI/proxy: add `openclaw proxy validate` so operators can verify effective proxy configuration, proxy reachability, and expected allow/deny destination behavior before deploying proxy-routed OpenClaw commands. (#73438) Thanks @jesse-merhi.
|
||||
|
||||
### Fixes
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
summary: "CLI reference for `openclaw proxy`, the local debug proxy and capture inspector"
|
||||
summary: "CLI reference for `openclaw proxy`, including operator-managed proxy validation and the local debug proxy capture inspector"
|
||||
read_when:
|
||||
- You need to validate operator-managed proxy routing before deployment
|
||||
- You need to capture OpenClaw transport traffic locally for debugging
|
||||
- You want to inspect debug proxy sessions, blobs, or built-in query presets
|
||||
title: "Proxy"
|
||||
@@ -8,18 +9,21 @@ title: "Proxy"
|
||||
|
||||
# `openclaw proxy`
|
||||
|
||||
Run the local explicit debug proxy and inspect captured traffic.
|
||||
Validate operator-managed proxy routing, or run the local explicit debug proxy
|
||||
and inspect captured traffic.
|
||||
|
||||
This is a debugging command for transport-level investigation. It can start a
|
||||
local proxy, run a child command with capture enabled, list capture sessions,
|
||||
query common traffic patterns, read captured blobs, and purge local capture
|
||||
data.
|
||||
Use `validate` to preflight an operator-managed forward proxy before enabling
|
||||
OpenClaw proxy routing. The other commands are debugging tools for
|
||||
transport-level investigation: they can start a local proxy, run a child command
|
||||
with capture enabled, list capture sessions, query common traffic patterns, read
|
||||
captured blobs, and purge local capture data.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
openclaw proxy start [--host <host>] [--port <port>]
|
||||
openclaw proxy run [--host <host>] [--port <port>] -- <cmd...>
|
||||
openclaw proxy validate [--json] [--proxy-url <url>] [--allowed-url <url>] [--denied-url <url>] [--timeout-ms <ms>]
|
||||
openclaw proxy coverage
|
||||
openclaw proxy sessions [--limit <count>]
|
||||
openclaw proxy query --preset <name> [--session <id>]
|
||||
@@ -27,6 +31,28 @@ openclaw proxy blob --id <blobId>
|
||||
openclaw proxy purge
|
||||
```
|
||||
|
||||
## Validate
|
||||
|
||||
`openclaw proxy validate` checks the effective operator-managed proxy URL from
|
||||
`--proxy-url`, config, or `OPENCLAW_PROXY_URL`. It reports a config problem when
|
||||
no proxy is enabled and configured; use `--proxy-url` for a one-off preflight
|
||||
before changing config. By default it verifies that a public destination succeeds
|
||||
through the proxy and that the proxy cannot reach a temporary loopback canary.
|
||||
Custom denied destinations are fail-closed: HTTP responses and ambiguous
|
||||
transport failures both fail unless you can verify a deployment-specific denial
|
||||
signal separately.
|
||||
|
||||
Options:
|
||||
|
||||
- `--json`: print machine-readable JSON.
|
||||
- `--proxy-url <url>`: validate this proxy URL instead of config or env.
|
||||
- `--allowed-url <url>`: add a destination expected to succeed through the proxy. Repeat to check multiple destinations.
|
||||
- `--denied-url <url>`: add a destination expected to be blocked by the proxy. Repeat to check multiple destinations.
|
||||
- `--timeout-ms <ms>`: per-request timeout in milliseconds.
|
||||
|
||||
See [Network Proxy](/security/network-proxy) for deployment guidance and denial
|
||||
semantics.
|
||||
|
||||
## Query presets
|
||||
|
||||
`openclaw proxy query --preset <name>` accepts:
|
||||
@@ -42,9 +68,11 @@ openclaw proxy purge
|
||||
|
||||
- `start` defaults to `127.0.0.1` unless `--host` is set.
|
||||
- `run` starts a local debug proxy and then runs the command after `--`.
|
||||
- `validate` exits with code 1 when proxy config or destination checks fail.
|
||||
- Captures are local debugging data; use `openclaw proxy purge` when finished.
|
||||
|
||||
## Related
|
||||
|
||||
- [CLI reference](/cli)
|
||||
- [Network Proxy](/security/network-proxy)
|
||||
- [Trusted proxy auth](/gateway/trusted-proxy-auth)
|
||||
|
||||
@@ -135,13 +135,43 @@ If your cloud provider or network platform documents additional metadata hosts o
|
||||
|
||||
Validate the proxy from the same host, container, or service account that runs OpenClaw:
|
||||
|
||||
```bash
|
||||
openclaw proxy validate --proxy-url http://127.0.0.1:3128
|
||||
```
|
||||
|
||||
By default, when no custom destinations are provided, the command checks that `https://example.com/` succeeds and starts a temporary loopback canary that the proxy must not reach. The default denied check passes when the proxy returns a non-2xx denial response or blocks the canary with a transport failure; it fails if a successful response reaches the canary. If no proxy is enabled and configured, validation reports a config problem; use `--proxy-url` for a one-off preflight before changing config. Use `--allowed-url` and `--denied-url` to test deployment-specific expectations. Custom denied destinations are fail-closed: any HTTP response means the destination was reachable through the proxy, and any transport error is reported as inconclusive because OpenClaw cannot prove the proxy blocked a reachable origin. On validation failure, the command exits with code 1.
|
||||
|
||||
Use `--json` for automation. The JSON output contains the overall result, the effective proxy config source, any config errors, and each destination check. Proxy URL credentials are redacted in text and JSON output:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"config": {
|
||||
"enabled": true,
|
||||
"proxyUrl": "http://127.0.0.1:3128/",
|
||||
"source": "override",
|
||||
"errors": []
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"kind": "allowed",
|
||||
"url": "https://example.com/",
|
||||
"ok": true,
|
||||
"status": 200
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
You can also validate manually with `curl`:
|
||||
|
||||
```bash
|
||||
curl -x http://127.0.0.1:3128 https://example.com/
|
||||
curl -x http://127.0.0.1:3128 http://127.0.0.1/
|
||||
curl -x http://127.0.0.1:3128 http://169.254.169.254/
|
||||
```
|
||||
|
||||
The public request should succeed. The loopback and metadata requests should fail at the proxy.
|
||||
The public request should succeed. The loopback and metadata requests should be blocked by the proxy. For `openclaw proxy validate`, the built-in loopback canary can distinguish a proxy denial from a reachable origin. Custom `--denied-url` checks do not have that canary, so treat both HTTP responses and ambiguous transport failures as validation failures unless your proxy exposes a deployment-specific denial signal you can verify separately.
|
||||
|
||||
Then enable OpenClaw proxy routing:
|
||||
|
||||
|
||||
@@ -4,10 +4,14 @@ import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const { serverStopSpy, spawnMock } = vi.hoisted(() => ({
|
||||
serverStopSpy: vi.fn(async () => undefined),
|
||||
spawnMock: vi.fn(),
|
||||
}));
|
||||
const { getRuntimeConfigMock, runProxyValidationMock, serverStopSpy, spawnMock } = vi.hoisted(
|
||||
() => ({
|
||||
getRuntimeConfigMock: vi.fn(),
|
||||
runProxyValidationMock: vi.fn(),
|
||||
serverStopSpy: vi.fn(async () => undefined),
|
||||
spawnMock: vi.fn(),
|
||||
}),
|
||||
);
|
||||
|
||||
vi.mock("node:child_process", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("node:child_process")>();
|
||||
@@ -24,6 +28,14 @@ vi.mock("../proxy-capture/proxy-server.js", () => ({
|
||||
})),
|
||||
}));
|
||||
|
||||
vi.mock("../config/config.js", () => ({
|
||||
getRuntimeConfig: getRuntimeConfigMock,
|
||||
}));
|
||||
|
||||
vi.mock("../infra/net/proxy/proxy-validation.js", () => ({
|
||||
runProxyValidation: runProxyValidationMock,
|
||||
}));
|
||||
|
||||
describe("proxy cli runtime", () => {
|
||||
const envKeys = [
|
||||
"OPENCLAW_DEBUG_PROXY_DB_PATH",
|
||||
@@ -42,6 +54,33 @@ describe("proxy cli runtime", () => {
|
||||
process.env.OPENCLAW_DEBUG_PROXY_CERT_DIR = path.join(tempDir, "certs");
|
||||
delete process.env.OPENCLAW_DEBUG_PROXY_ENABLED;
|
||||
delete process.env.OPENCLAW_DEBUG_PROXY_SESSION_ID;
|
||||
getRuntimeConfigMock.mockReset();
|
||||
getRuntimeConfigMock.mockReturnValue({
|
||||
proxy: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
},
|
||||
});
|
||||
runProxyValidationMock.mockReset();
|
||||
runProxyValidationMock.mockResolvedValue({
|
||||
ok: true,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
source: "config",
|
||||
errors: [],
|
||||
},
|
||||
checks: [
|
||||
{
|
||||
kind: "allowed",
|
||||
url: "https://example.com/",
|
||||
ok: true,
|
||||
status: 200,
|
||||
},
|
||||
],
|
||||
});
|
||||
process.exitCode = undefined;
|
||||
vi.spyOn(process.stdout, "write").mockImplementation(() => true);
|
||||
serverStopSpy.mockClear();
|
||||
spawnMock.mockReset();
|
||||
});
|
||||
@@ -49,7 +88,9 @@ describe("proxy cli runtime", () => {
|
||||
afterEach(async () => {
|
||||
const { closeDebugProxyCaptureStore } = await import("../proxy-capture/store.sqlite.js");
|
||||
closeDebugProxyCaptureStore();
|
||||
vi.restoreAllMocks();
|
||||
vi.resetModules();
|
||||
process.exitCode = undefined;
|
||||
for (const key of envKeys) {
|
||||
const value = savedEnv[key];
|
||||
if (value == null) {
|
||||
@@ -61,6 +102,289 @@ describe("proxy cli runtime", () => {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("prints proxy validation text and leaves exit code unset on success", async () => {
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({
|
||||
proxyUrl: "http://override.example:3128",
|
||||
allowedUrls: ["https://allowed.example/"],
|
||||
deniedUrls: ["http://127.0.0.1/"],
|
||||
timeoutMs: 1234,
|
||||
});
|
||||
|
||||
expect(getRuntimeConfigMock).toHaveBeenCalledOnce();
|
||||
expect(runProxyValidationMock).toHaveBeenCalledWith({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
},
|
||||
env: process.env,
|
||||
proxyUrlOverride: "http://override.example:3128",
|
||||
allowedUrls: ["https://allowed.example/"],
|
||||
deniedUrls: ["http://127.0.0.1/"],
|
||||
timeoutMs: 1234,
|
||||
});
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
"Proxy validation passed\n\n" +
|
||||
"Proxy\n" +
|
||||
" Source: config\n" +
|
||||
" URL: http://config-proxy.example:3128/\n\n" +
|
||||
"Checks\n" +
|
||||
" ✓ allowed https://example.com/ HTTP 200\n",
|
||||
);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
});
|
||||
|
||||
it("redacts proxy credentials in text output", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://user:secret@proxy.example:3128?token=secret#fragment",
|
||||
source: "config",
|
||||
errors: [],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({});
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
"Proxy validation passed\n\n" +
|
||||
"Proxy\n" +
|
||||
" Source: config\n" +
|
||||
" URL: http://redacted:redacted@proxy.example:3128/\n",
|
||||
);
|
||||
});
|
||||
|
||||
it("redacts proxy credentials in JSON output", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://user:secret@proxy.example:3128?token=secret#fragment",
|
||||
source: "config",
|
||||
errors: [],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({ json: true });
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
`${JSON.stringify(
|
||||
{
|
||||
ok: true,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://redacted:redacted@proxy.example:3128/",
|
||||
source: "config",
|
||||
errors: [],
|
||||
},
|
||||
checks: [],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
});
|
||||
|
||||
it("prints actionable disabled proxy config output", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: false,
|
||||
proxyUrl: "http://proxy.example:3128",
|
||||
source: "config",
|
||||
errors: ["proxy validation requires proxy.enabled to be true for configured proxy URLs"],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({});
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
"Proxy validation failed\n\n" +
|
||||
"Proxy\n" +
|
||||
" Source: config\n" +
|
||||
" URL: http://proxy.example:3128/\n\n" +
|
||||
"Problems\n" +
|
||||
" - proxy validation requires proxy.enabled to be true for configured proxy URLs\n\n" +
|
||||
"Next steps\n" +
|
||||
" Enable proxy.enabled with proxy.proxyUrl or OPENCLAW_PROXY_URL, or pass --proxy-url for an explicit one-off validation.\n",
|
||||
);
|
||||
});
|
||||
|
||||
it("prints actionable output when proxy config is disabled and missing", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: false,
|
||||
source: "disabled",
|
||||
errors: [
|
||||
"proxy validation requires proxy.enabled=true with proxy.proxyUrl or OPENCLAW_PROXY_URL, or --proxy-url",
|
||||
],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({});
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
"Proxy validation failed\n\n" +
|
||||
"Proxy\n" +
|
||||
" Source: disabled\n" +
|
||||
" URL: not configured\n\n" +
|
||||
"Problems\n" +
|
||||
" - proxy validation requires proxy.enabled=true with proxy.proxyUrl or OPENCLAW_PROXY_URL, or --proxy-url\n\n" +
|
||||
"Next steps\n" +
|
||||
" Enable proxy.enabled with proxy.proxyUrl or OPENCLAW_PROXY_URL, or pass --proxy-url for an explicit one-off validation.\n",
|
||||
);
|
||||
expect(process.exitCode).toBe(1);
|
||||
});
|
||||
|
||||
it("redacts malformed proxy URLs in text output", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://user:secret@",
|
||||
source: "env",
|
||||
errors: ["proxyUrl must use http://"],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({});
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
"Proxy validation failed\n\n" +
|
||||
"Proxy\n" +
|
||||
" Source: env\n" +
|
||||
" URL: <invalid proxy URL>\n\n" +
|
||||
"Problems\n" +
|
||||
" - proxyUrl must use http://\n\n" +
|
||||
"Next steps\n" +
|
||||
" Fix proxy.proxyUrl, OPENCLAW_PROXY_URL, or --proxy-url so it uses a reachable http:// proxy.\n",
|
||||
);
|
||||
});
|
||||
|
||||
it("redacts malformed proxy URLs in JSON output", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://user:secret@",
|
||||
source: "override",
|
||||
errors: ["proxyUrl must use http://"],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({ json: true });
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
`${JSON.stringify(
|
||||
{
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "<invalid proxy URL>",
|
||||
source: "override",
|
||||
errors: ["proxyUrl must use http://"],
|
||||
},
|
||||
checks: [],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
});
|
||||
|
||||
it("prints actionable check failure output", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://proxy.example:3128",
|
||||
source: "config",
|
||||
errors: [],
|
||||
},
|
||||
checks: [
|
||||
{
|
||||
kind: "allowed",
|
||||
url: "http://target.example/allowed",
|
||||
ok: true,
|
||||
status: 200,
|
||||
},
|
||||
{
|
||||
kind: "denied",
|
||||
url: "http://target.example/allowed",
|
||||
ok: false,
|
||||
status: 200,
|
||||
error: "Denied destination was reachable through the proxy",
|
||||
},
|
||||
],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({});
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
"Proxy validation failed\n\n" +
|
||||
"Proxy\n" +
|
||||
" Source: config\n" +
|
||||
" URL: http://proxy.example:3128/\n\n" +
|
||||
"Checks\n" +
|
||||
" ✓ allowed http://target.example/allowed HTTP 200\n" +
|
||||
" ✗ denied http://target.example/allowed HTTP 200\n" +
|
||||
" Denied destination was reachable through the proxy\n\n" +
|
||||
"Next steps\n" +
|
||||
" Update the proxy ACL so denied destinations are blocked, or pass the expected --denied-url values.\n",
|
||||
);
|
||||
expect(process.exitCode).toBe(1);
|
||||
});
|
||||
|
||||
it("prints proxy validation JSON and sets exit code on failure", async () => {
|
||||
runProxyValidationMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: true,
|
||||
source: "missing",
|
||||
errors: ["proxy validation requires proxy.proxyUrl, --proxy-url, or OPENCLAW_PROXY_URL"],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
const { runProxyValidateCommand } = await import("./proxy-cli.runtime.js");
|
||||
|
||||
await runProxyValidateCommand({ json: true });
|
||||
|
||||
expect(process.stdout.write).toHaveBeenCalledWith(
|
||||
`${JSON.stringify(
|
||||
{
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: true,
|
||||
source: "missing",
|
||||
errors: [
|
||||
"proxy validation requires proxy.proxyUrl, --proxy-url, or OPENCLAW_PROXY_URL",
|
||||
],
|
||||
},
|
||||
checks: [],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
expect(process.exitCode).toBe(1);
|
||||
});
|
||||
|
||||
it("stops the proxy server and ends the session when child spawn fails", async () => {
|
||||
spawnMock.mockImplementation(() => {
|
||||
const child = new EventEmitter();
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import process from "node:process";
|
||||
import { getRuntimeConfig } from "../config/config.js";
|
||||
import {
|
||||
runProxyValidation,
|
||||
type ProxyValidationResult,
|
||||
} from "../infra/net/proxy/proxy-validation.js";
|
||||
import { ensureDebugProxyCa } from "../proxy-capture/ca.js";
|
||||
import { buildDebugProxyCoverageReport } from "../proxy-capture/coverage.js";
|
||||
import { resolveDebugProxySettings, applyDebugProxyEnv } from "../proxy-capture/env.js";
|
||||
@@ -115,6 +120,135 @@ export async function runDebugProxyRunCommand(opts: {
|
||||
}
|
||||
}
|
||||
|
||||
function redactProxyUrl(value: string | undefined): string | undefined {
|
||||
if (!value) {
|
||||
return undefined;
|
||||
}
|
||||
try {
|
||||
const url = new URL(value);
|
||||
if (url.username || url.password) {
|
||||
url.username = "redacted";
|
||||
url.password = "redacted";
|
||||
}
|
||||
url.search = "";
|
||||
url.hash = "";
|
||||
return url.toString();
|
||||
} catch {
|
||||
return "<invalid proxy URL>";
|
||||
}
|
||||
}
|
||||
|
||||
function redactProxyValidationResult(result: ProxyValidationResult): ProxyValidationResult {
|
||||
return {
|
||||
...result,
|
||||
config: {
|
||||
...result.config,
|
||||
proxyUrl: redactProxyUrl(result.config.proxyUrl),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function formatProxyCheckLine(check: ProxyValidationResult["checks"][number]): string {
|
||||
const icon = check.ok ? "✓" : "✗";
|
||||
const paddedKind = check.kind.padEnd(7, " ");
|
||||
const status = check.status === undefined ? "" : ` HTTP ${check.status}`;
|
||||
return ` ${icon} ${paddedKind} ${check.url}${status}`;
|
||||
}
|
||||
|
||||
function formatProxyValidationNextSteps(result: ProxyValidationResult): string[] {
|
||||
if (result.ok) {
|
||||
return [];
|
||||
}
|
||||
if (result.config.errors.some((error) => error.includes("proxy.enabled"))) {
|
||||
return [
|
||||
"Enable proxy.enabled with proxy.proxyUrl or OPENCLAW_PROXY_URL, or pass --proxy-url for an explicit one-off validation.",
|
||||
];
|
||||
}
|
||||
if (result.config.errors.length > 0) {
|
||||
return [
|
||||
"Fix proxy.proxyUrl, OPENCLAW_PROXY_URL, or --proxy-url so it uses a reachable http:// proxy.",
|
||||
];
|
||||
}
|
||||
if (result.checks.some((check) => !check.ok && check.kind === "allowed")) {
|
||||
return [
|
||||
"Confirm the proxy is reachable from this deployment context and permits the allowed destinations.",
|
||||
];
|
||||
}
|
||||
if (result.checks.some((check) => !check.ok && check.kind === "denied")) {
|
||||
return [
|
||||
"Update the proxy ACL so denied destinations are blocked, or pass the expected --denied-url values.",
|
||||
];
|
||||
}
|
||||
return [
|
||||
"Review the failed checks above and update proxy configuration or validation destinations.",
|
||||
];
|
||||
}
|
||||
|
||||
function formatProxyValidationText(result: ProxyValidationResult): string {
|
||||
const redactedProxyUrl = redactProxyUrl(result.config.proxyUrl);
|
||||
const lines = [
|
||||
`Proxy validation ${result.ok ? "passed" : "failed"}`,
|
||||
"",
|
||||
"Proxy",
|
||||
` Source: ${result.config.source}`,
|
||||
` URL: ${redactedProxyUrl ?? "not configured"}`,
|
||||
];
|
||||
|
||||
if (result.config.errors.length > 0) {
|
||||
lines.push("", "Problems");
|
||||
for (const error of result.config.errors) {
|
||||
lines.push(` - ${error}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (result.checks.length > 0) {
|
||||
lines.push("", "Checks");
|
||||
for (const check of result.checks) {
|
||||
lines.push(formatProxyCheckLine(check));
|
||||
if (check.error) {
|
||||
lines.push(` ${check.error}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const nextSteps = formatProxyValidationNextSteps(result);
|
||||
if (nextSteps.length > 0) {
|
||||
lines.push("", "Next steps");
|
||||
for (const nextStep of nextSteps) {
|
||||
lines.push(` ${nextStep}`);
|
||||
}
|
||||
}
|
||||
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
export async function runProxyValidateCommand(opts: {
|
||||
json?: boolean;
|
||||
proxyUrl?: string;
|
||||
allowedUrls?: string[];
|
||||
deniedUrls?: string[];
|
||||
timeoutMs?: number;
|
||||
}) {
|
||||
const config = getRuntimeConfig();
|
||||
const result = await runProxyValidation({
|
||||
config: config?.proxy,
|
||||
env: process.env,
|
||||
proxyUrlOverride: opts.proxyUrl,
|
||||
allowedUrls: opts.allowedUrls,
|
||||
deniedUrls: opts.deniedUrls,
|
||||
timeoutMs: opts.timeoutMs,
|
||||
});
|
||||
const outputResult = redactProxyValidationResult(result);
|
||||
process.stdout.write(
|
||||
opts.json === true
|
||||
? `${JSON.stringify(outputResult, null, 2)}\n`
|
||||
: formatProxyValidationText(outputResult),
|
||||
);
|
||||
if (!result.ok) {
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
export async function runDebugProxySessionsCommand(opts: { limit?: number }) {
|
||||
const settings = resolveDebugProxySettings();
|
||||
const sessions = getDebugProxyCaptureStore(settings.dbPath, settings.blobDir).listSessions(
|
||||
|
||||
@@ -11,11 +11,22 @@ describe("proxy cli", () => {
|
||||
expect(proxy?.commands.map((command) => command.name())).toEqual([
|
||||
"start",
|
||||
"run",
|
||||
"validate",
|
||||
"coverage",
|
||||
"sessions",
|
||||
"query",
|
||||
"blob",
|
||||
"purge",
|
||||
]);
|
||||
|
||||
const validate = proxy?.commands.find((command) => command.name() === "validate");
|
||||
expect(validate?.description()).toBe("Validate the operator-managed network proxy");
|
||||
expect(validate?.options.map((option) => option.long)).toEqual([
|
||||
"--json",
|
||||
"--proxy-url",
|
||||
"--allowed-url",
|
||||
"--denied-url",
|
||||
"--timeout-ms",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -18,6 +18,10 @@ function parseOptionalNumber(value: string | undefined): number | undefined {
|
||||
return Number.isFinite(parsed) ? parsed : undefined;
|
||||
}
|
||||
|
||||
function collectOption(value: string, previous: string[] | undefined): string[] {
|
||||
return [...(previous ?? []), value];
|
||||
}
|
||||
|
||||
export function registerProxyCli(program: Command) {
|
||||
const proxy = program
|
||||
.command("proxy")
|
||||
@@ -50,6 +54,37 @@ export function registerProxyCli(program: Command) {
|
||||
});
|
||||
});
|
||||
|
||||
proxy
|
||||
.command("validate")
|
||||
.description("Validate the operator-managed network proxy")
|
||||
.option("--json", "Print machine-readable JSON")
|
||||
.option("--proxy-url <url>", "Proxy URL to validate instead of config/env")
|
||||
.option(
|
||||
"--allowed-url <url>",
|
||||
"Destination expected to succeed through the proxy",
|
||||
collectOption,
|
||||
)
|
||||
.option("--denied-url <url>", "Destination expected to be blocked by the proxy", collectOption)
|
||||
.option("--timeout-ms <ms>", "Per-request timeout in milliseconds", parseOptionalNumber)
|
||||
.action(
|
||||
async (opts: {
|
||||
json?: boolean;
|
||||
proxyUrl?: string;
|
||||
allowedUrl?: string[];
|
||||
deniedUrl?: string[];
|
||||
timeoutMs?: number;
|
||||
}) => {
|
||||
const runtime = await loadProxyCliRuntime();
|
||||
await runtime.runProxyValidateCommand({
|
||||
json: opts.json,
|
||||
proxyUrl: opts.proxyUrl,
|
||||
allowedUrls: opts.allowedUrl,
|
||||
deniedUrls: opts.deniedUrl,
|
||||
timeoutMs: opts.timeoutMs,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
proxy
|
||||
.command("coverage")
|
||||
.description("Report current debug proxy transport coverage and remaining gaps")
|
||||
|
||||
423
src/infra/net/proxy/proxy-validation.test.ts
Normal file
423
src/infra/net/proxy/proxy-validation.test.ts
Normal file
@@ -0,0 +1,423 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
DEFAULT_PROXY_VALIDATION_ALLOWED_URLS,
|
||||
resolveProxyValidationConfig,
|
||||
runProxyValidation,
|
||||
} from "./proxy-validation.js";
|
||||
|
||||
describe("proxy validation", () => {
|
||||
it("resolves proxy URL overrides before config and OPENCLAW_PROXY_URL", () => {
|
||||
const result = resolveProxyValidationConfig({
|
||||
proxyUrlOverride: "http://override-proxy.example:3128",
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
},
|
||||
env: {
|
||||
OPENCLAW_PROXY_URL: "http://env-proxy.example:3128",
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toEqual({
|
||||
enabled: true,
|
||||
proxyUrl: "http://override-proxy.example:3128",
|
||||
source: "override",
|
||||
errors: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("resolves config proxy URLs before OPENCLAW_PROXY_URL", () => {
|
||||
const result = resolveProxyValidationConfig({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
},
|
||||
env: {
|
||||
OPENCLAW_PROXY_URL: "http://env-proxy.example:3128",
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toEqual({
|
||||
enabled: true,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
source: "config",
|
||||
errors: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("uses OPENCLAW_PROXY_URL when enabled config has no URL", () => {
|
||||
const result = resolveProxyValidationConfig({
|
||||
config: { enabled: true },
|
||||
env: {
|
||||
OPENCLAW_PROXY_URL: "http://env-proxy.example:3128",
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toEqual({
|
||||
enabled: true,
|
||||
proxyUrl: "http://env-proxy.example:3128",
|
||||
source: "env",
|
||||
errors: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("reports disabled proxy config when a config URL is present but proxy routing is disabled", async () => {
|
||||
const fetchCheck = vi.fn();
|
||||
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: false,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
},
|
||||
env: {},
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(fetchCheck).not.toHaveBeenCalled();
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: false,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
source: "config",
|
||||
errors: ["proxy validation requires proxy.enabled to be true for configured proxy URLs"],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("reports disabled proxy config when only OPENCLAW_PROXY_URL is present", async () => {
|
||||
const fetchCheck = vi.fn();
|
||||
|
||||
const result = await runProxyValidation({
|
||||
config: {},
|
||||
env: {
|
||||
OPENCLAW_PROXY_URL: "http://env-proxy.example:3128",
|
||||
},
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(fetchCheck).not.toHaveBeenCalled();
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: false,
|
||||
proxyUrl: "http://env-proxy.example:3128",
|
||||
source: "env",
|
||||
errors: ["proxy validation requires proxy.enabled to be true for OPENCLAW_PROXY_URL"],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("allows explicit proxy URL overrides even when config proxy routing is disabled", async () => {
|
||||
const fetchCheck = vi.fn().mockResolvedValueOnce({ ok: true, status: 200 });
|
||||
|
||||
const result = await runProxyValidation({
|
||||
proxyUrlOverride: "http://override-proxy.example:3128",
|
||||
config: {
|
||||
enabled: false,
|
||||
proxyUrl: "http://config-proxy.example:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: ["https://example.com/"],
|
||||
deniedUrls: [],
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(fetchCheck).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reports missing URL when proxy validation is enabled without an effective URL", () => {
|
||||
const result = resolveProxyValidationConfig({
|
||||
config: { enabled: true },
|
||||
env: {},
|
||||
});
|
||||
|
||||
expect(result.enabled).toBe(true);
|
||||
expect(result.proxyUrl).toBeUndefined();
|
||||
expect(result.source).toBe("missing");
|
||||
expect(result.errors).toEqual([
|
||||
"proxy validation requires proxy.proxyUrl, --proxy-url, or OPENCLAW_PROXY_URL",
|
||||
]);
|
||||
});
|
||||
|
||||
it("reports disabled proxy config as an actionable validation problem", async () => {
|
||||
const fetchCheck = vi.fn();
|
||||
|
||||
const result = await runProxyValidation({
|
||||
config: {},
|
||||
env: {},
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(fetchCheck).not.toHaveBeenCalled();
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
config: {
|
||||
enabled: false,
|
||||
source: "disabled",
|
||||
errors: [
|
||||
"proxy validation requires proxy.enabled=true with proxy.proxyUrl or OPENCLAW_PROXY_URL, or --proxy-url",
|
||||
],
|
||||
},
|
||||
checks: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects non-http proxy URLs", () => {
|
||||
const result = resolveProxyValidationConfig({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "https://proxy.example:3128",
|
||||
},
|
||||
env: {},
|
||||
});
|
||||
|
||||
expect(result.errors).toEqual(["proxyUrl must use http://"]);
|
||||
});
|
||||
|
||||
it("checks default allowed and denied destinations through the proxy", async () => {
|
||||
const fetchCheck = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({ ok: true, status: 200 })
|
||||
.mockRejectedValueOnce(new Error("loopback blocked"));
|
||||
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(fetchCheck).toHaveBeenCalledTimes(2);
|
||||
expect(fetchCheck).toHaveBeenNthCalledWith(1, {
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
targetUrl: DEFAULT_PROXY_VALIDATION_ALLOWED_URLS[0],
|
||||
timeoutMs: 5000,
|
||||
});
|
||||
const deniedCall = fetchCheck.mock.calls[1]?.[0];
|
||||
expect(deniedCall).toMatchObject({
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
timeoutMs: 5000,
|
||||
});
|
||||
expect(deniedCall?.targetUrl).toMatch(/^http:\/\/127\.0\.0\.1:\d+\/$/);
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.checks[0]).toMatchObject({
|
||||
kind: "allowed",
|
||||
url: DEFAULT_PROXY_VALIDATION_ALLOWED_URLS[0],
|
||||
ok: true,
|
||||
});
|
||||
expect(result.checks[1]).toMatchObject({
|
||||
kind: "denied",
|
||||
ok: true,
|
||||
error: "loopback blocked",
|
||||
});
|
||||
expect(result.checks[1]?.url).toMatch(/^http:\/\/127\.0\.0\.1:\d+\/$/);
|
||||
});
|
||||
|
||||
it("fails the default loopback denied canary on successful ambiguous responses", async () => {
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: [],
|
||||
fetchCheck: vi.fn().mockImplementation(async ({ targetUrl }) => {
|
||||
return {
|
||||
ok: true,
|
||||
status: 204,
|
||||
deniedCanaryToken: targetUrl.includes("127.0.0.1:") ? undefined : "unexpected",
|
||||
};
|
||||
}),
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toHaveLength(1);
|
||||
expect(result.checks[0]).toMatchObject({
|
||||
kind: "denied",
|
||||
ok: false,
|
||||
status: 204,
|
||||
error: "Denied loopback canary returned HTTP 204 without the validation token",
|
||||
});
|
||||
expect(result.checks[0]?.url).toMatch(/^http:\/\/127\.0\.0\.1:\d+\/$/);
|
||||
});
|
||||
|
||||
it("passes the default loopback denied canary when the proxy returns a denial response", async () => {
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: [],
|
||||
fetchCheck: vi.fn().mockResolvedValue({ ok: false, status: 403 }),
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.checks).toHaveLength(1);
|
||||
expect(result.checks[0]).toMatchObject({
|
||||
kind: "denied",
|
||||
ok: true,
|
||||
status: 403,
|
||||
});
|
||||
expect(result.checks[0]?.url).toMatch(/^http:\/\/127\.0\.0\.1:\d+\/$/);
|
||||
});
|
||||
|
||||
it("fails denied checks when the destination returns HTTP 403", async () => {
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: [],
|
||||
deniedUrls: ["http://127.0.0.1/"],
|
||||
fetchCheck: vi.fn().mockResolvedValue({ ok: false, status: 403 }),
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toEqual([
|
||||
{
|
||||
kind: "denied",
|
||||
url: "http://127.0.0.1/",
|
||||
ok: false,
|
||||
status: 403,
|
||||
error: "Denied destination returned HTTP 403; expected the proxy to block the connection",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails denied checks when the destination returns a non-2xx HTTP status", async () => {
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: [],
|
||||
deniedUrls: ["https://example.com/not-found"],
|
||||
fetchCheck: vi.fn().mockResolvedValue({ ok: false, status: 404 }),
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toEqual([
|
||||
{
|
||||
kind: "denied",
|
||||
url: "https://example.com/not-found",
|
||||
ok: false,
|
||||
status: 404,
|
||||
error: "Denied destination returned HTTP 404; expected the proxy to block the connection",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails custom denied checks on ambiguous transport errors", async () => {
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: [],
|
||||
deniedUrls: ["https://example.com/closed"],
|
||||
fetchCheck: vi.fn().mockRejectedValue(new Error("ECONNREFUSED")),
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toEqual([
|
||||
{
|
||||
kind: "denied",
|
||||
url: "https://example.com/closed",
|
||||
ok: false,
|
||||
error: "Denied destination failed without a verifiable proxy-deny signal: ECONNREFUSED",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails invalid custom denied URLs before probing", async () => {
|
||||
const fetchCheck = vi.fn();
|
||||
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: [],
|
||||
deniedUrls: ["not a url"],
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(fetchCheck).not.toHaveBeenCalled();
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toEqual([
|
||||
{
|
||||
kind: "denied",
|
||||
url: "not a url",
|
||||
ok: false,
|
||||
error: "Invalid denied destination URL",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails invalid custom allowed URLs before probing", async () => {
|
||||
const fetchCheck = vi.fn();
|
||||
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: ["not a url"],
|
||||
deniedUrls: [],
|
||||
fetchCheck,
|
||||
});
|
||||
|
||||
expect(fetchCheck).not.toHaveBeenCalled();
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toEqual([
|
||||
{
|
||||
kind: "allowed",
|
||||
url: "not a url",
|
||||
ok: false,
|
||||
error: "Invalid allowed destination URL",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails validation when a denied destination succeeds", async () => {
|
||||
const result = await runProxyValidation({
|
||||
config: {
|
||||
enabled: true,
|
||||
proxyUrl: "http://127.0.0.1:3128",
|
||||
},
|
||||
env: {},
|
||||
allowedUrls: ["https://example.com/"],
|
||||
deniedUrls: ["http://127.0.0.1/"],
|
||||
fetchCheck: vi.fn().mockResolvedValue({ ok: true, status: 200 }),
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.checks).toEqual([
|
||||
{
|
||||
kind: "allowed",
|
||||
url: "https://example.com/",
|
||||
ok: true,
|
||||
status: 200,
|
||||
},
|
||||
{
|
||||
kind: "denied",
|
||||
url: "http://127.0.0.1/",
|
||||
ok: false,
|
||||
status: 200,
|
||||
error: "Denied destination returned HTTP 200; expected the proxy to block the connection",
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
431
src/infra/net/proxy/proxy-validation.ts
Normal file
431
src/infra/net/proxy/proxy-validation.ts
Normal file
@@ -0,0 +1,431 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { createServer, type Server } from "node:http";
|
||||
import type { ProxyConfig } from "../../../config/zod-schema.proxy.js";
|
||||
import { fetchWithRuntimeDispatcher } from "../runtime-fetch.js";
|
||||
import { createHttp1ProxyAgent } from "../undici-runtime.js";
|
||||
|
||||
export const DEFAULT_PROXY_VALIDATION_ALLOWED_URLS = ["https://example.com/"] as const;
|
||||
export const DEFAULT_PROXY_VALIDATION_DENIED_URLS = ["http://127.0.0.1/"] as const;
|
||||
|
||||
export const DEFAULT_PROXY_VALIDATION_TIMEOUT_MS = 5000;
|
||||
const DENIED_CANARY_HEADER = "x-openclaw-proxy-validation-canary";
|
||||
|
||||
export type ProxyValidationConfigSource = "override" | "config" | "env" | "missing" | "disabled";
|
||||
|
||||
export type ProxyValidationResolvedConfig = {
|
||||
enabled: boolean;
|
||||
proxyUrl?: string;
|
||||
source: ProxyValidationConfigSource;
|
||||
errors: string[];
|
||||
};
|
||||
|
||||
export type ProxyValidationCheckKind = "allowed" | "denied";
|
||||
|
||||
export type ProxyValidationCheck = {
|
||||
kind: ProxyValidationCheckKind;
|
||||
url: string;
|
||||
ok: boolean;
|
||||
status?: number;
|
||||
error?: string;
|
||||
};
|
||||
|
||||
export type ProxyValidationResult = {
|
||||
ok: boolean;
|
||||
config: ProxyValidationResolvedConfig;
|
||||
checks: ProxyValidationCheck[];
|
||||
};
|
||||
|
||||
export type ProxyValidationFetchCheckParams = {
|
||||
proxyUrl: string;
|
||||
targetUrl: string;
|
||||
timeoutMs: number;
|
||||
};
|
||||
|
||||
export type ProxyValidationFetchCheckResult = {
|
||||
ok: boolean;
|
||||
status: number;
|
||||
deniedCanaryToken?: string;
|
||||
};
|
||||
|
||||
export type ProxyValidationFetchCheck = (
|
||||
params: ProxyValidationFetchCheckParams,
|
||||
) => Promise<ProxyValidationFetchCheckResult>;
|
||||
|
||||
export type ResolveProxyValidationConfigOptions = {
|
||||
config?: ProxyConfig;
|
||||
env?: NodeJS.ProcessEnv | Partial<Record<"OPENCLAW_PROXY_URL", string | undefined>>;
|
||||
proxyUrlOverride?: string;
|
||||
};
|
||||
|
||||
export type RunProxyValidationOptions = ResolveProxyValidationConfigOptions & {
|
||||
allowedUrls?: readonly string[];
|
||||
deniedUrls?: readonly string[];
|
||||
timeoutMs?: number;
|
||||
fetchCheck?: ProxyValidationFetchCheck;
|
||||
};
|
||||
|
||||
function normalizeProxyUrl(value: string | undefined): string | undefined {
|
||||
const trimmed = value?.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function isHttpProxyUrl(value: string): boolean {
|
||||
try {
|
||||
return new URL(value).protocol === "http:";
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function validateProxyUrl(value: string | undefined): string[] {
|
||||
if (!value) {
|
||||
return ["proxy validation requires proxy.proxyUrl, --proxy-url, or OPENCLAW_PROXY_URL"];
|
||||
}
|
||||
if (!isHttpProxyUrl(value)) {
|
||||
return ["proxyUrl must use http://"];
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
function validateProxyEnabled(source: ProxyValidationConfigSource, enabled: boolean): string[] {
|
||||
if (enabled || source === "override" || source === "missing" || source === "disabled") {
|
||||
return [];
|
||||
}
|
||||
if (source === "env") {
|
||||
return ["proxy validation requires proxy.enabled to be true for OPENCLAW_PROXY_URL"];
|
||||
}
|
||||
return ["proxy validation requires proxy.enabled to be true for configured proxy URLs"];
|
||||
}
|
||||
|
||||
function validateResolvedProxy(
|
||||
source: ProxyValidationConfigSource,
|
||||
enabled: boolean,
|
||||
value: string | undefined,
|
||||
): string[] {
|
||||
return [...validateProxyUrl(value), ...validateProxyEnabled(source, enabled)];
|
||||
}
|
||||
|
||||
export function resolveProxyValidationConfig(
|
||||
options: ResolveProxyValidationConfigOptions,
|
||||
): ProxyValidationResolvedConfig {
|
||||
const overrideUrl = normalizeProxyUrl(options.proxyUrlOverride);
|
||||
if (overrideUrl) {
|
||||
return {
|
||||
enabled: true,
|
||||
proxyUrl: overrideUrl,
|
||||
source: "override",
|
||||
errors: validateResolvedProxy("override", true, overrideUrl),
|
||||
};
|
||||
}
|
||||
|
||||
const configUrl = normalizeProxyUrl(options.config?.proxyUrl);
|
||||
if (configUrl) {
|
||||
return {
|
||||
enabled: options.config?.enabled === true,
|
||||
proxyUrl: configUrl,
|
||||
source: "config",
|
||||
errors: validateResolvedProxy("config", options.config?.enabled === true, configUrl),
|
||||
};
|
||||
}
|
||||
|
||||
const envUrl = normalizeProxyUrl(options.env?.OPENCLAW_PROXY_URL);
|
||||
if (envUrl) {
|
||||
return {
|
||||
enabled: options.config?.enabled === true,
|
||||
proxyUrl: envUrl,
|
||||
source: "env",
|
||||
errors: validateResolvedProxy("env", options.config?.enabled === true, envUrl),
|
||||
};
|
||||
}
|
||||
|
||||
if (options.config?.enabled === true) {
|
||||
return {
|
||||
enabled: true,
|
||||
source: "missing",
|
||||
errors: validateProxyUrl(undefined),
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
enabled: false,
|
||||
source: "disabled",
|
||||
errors: [
|
||||
"proxy validation requires proxy.enabled=true with proxy.proxyUrl or OPENCLAW_PROXY_URL, or --proxy-url",
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async function defaultProxyValidationFetchCheck({
|
||||
proxyUrl,
|
||||
targetUrl,
|
||||
timeoutMs,
|
||||
}: ProxyValidationFetchCheckParams): Promise<ProxyValidationFetchCheckResult> {
|
||||
const dispatcher = createHttp1ProxyAgent({ uri: proxyUrl }, timeoutMs);
|
||||
try {
|
||||
const response = await fetchWithRuntimeDispatcher(targetUrl, {
|
||||
dispatcher,
|
||||
redirect: "manual",
|
||||
});
|
||||
void response.body?.cancel();
|
||||
return {
|
||||
ok: response.ok,
|
||||
status: response.status,
|
||||
deniedCanaryToken: response.headers.get(DENIED_CANARY_HEADER) ?? undefined,
|
||||
};
|
||||
} finally {
|
||||
await dispatcher.close();
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeTimeoutMs(value: number | undefined): number {
|
||||
if (value === undefined || !Number.isFinite(value) || value <= 0) {
|
||||
return DEFAULT_PROXY_VALIDATION_TIMEOUT_MS;
|
||||
}
|
||||
return Math.floor(value);
|
||||
}
|
||||
|
||||
function isValidHttpTargetUrl(value: string): boolean {
|
||||
try {
|
||||
const url = new URL(value);
|
||||
return url.protocol === "http:" || url.protocol === "https:";
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
type ProxyValidationDeniedTarget = {
|
||||
url: string;
|
||||
expectedCanaryToken?: string;
|
||||
transportErrorMeansBlocked: boolean;
|
||||
};
|
||||
|
||||
type DeniedCanary = {
|
||||
target: ProxyValidationDeniedTarget;
|
||||
close: () => Promise<void>;
|
||||
};
|
||||
|
||||
function closeServer(server: Server): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
server.close((err) => {
|
||||
if (err) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function createLoopbackDeniedCanary(): Promise<DeniedCanary> {
|
||||
const token = randomUUID();
|
||||
const server = createServer((_request, response) => {
|
||||
response.writeHead(204, {
|
||||
[DENIED_CANARY_HEADER]: token,
|
||||
"cache-control": "no-store",
|
||||
});
|
||||
response.end();
|
||||
});
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
server.off("error", reject);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
|
||||
const address = server.address();
|
||||
if (typeof address === "string" || address === null) {
|
||||
await closeServer(server);
|
||||
throw new Error("Unable to start loopback proxy validation canary");
|
||||
}
|
||||
|
||||
return {
|
||||
target: {
|
||||
url: `http://127.0.0.1:${address.port}/`,
|
||||
expectedCanaryToken: token,
|
||||
transportErrorMeansBlocked: true,
|
||||
},
|
||||
close: () => closeServer(server),
|
||||
};
|
||||
}
|
||||
|
||||
async function resolveDeniedTargets(
|
||||
deniedUrls: readonly string[] | undefined,
|
||||
): Promise<{ targets: ProxyValidationDeniedTarget[]; close: () => Promise<void> }> {
|
||||
if (deniedUrls !== undefined) {
|
||||
return {
|
||||
targets: deniedUrls.map((url) => ({
|
||||
url,
|
||||
transportErrorMeansBlocked: false,
|
||||
})),
|
||||
close: async () => undefined,
|
||||
};
|
||||
}
|
||||
|
||||
const canary = await createLoopbackDeniedCanary();
|
||||
return {
|
||||
targets: [canary.target],
|
||||
close: canary.close,
|
||||
};
|
||||
}
|
||||
|
||||
async function runAllowedCheck(params: {
|
||||
url: string;
|
||||
proxyUrl: string;
|
||||
timeoutMs: number;
|
||||
fetchCheck: ProxyValidationFetchCheck;
|
||||
}): Promise<ProxyValidationCheck> {
|
||||
if (!isValidHttpTargetUrl(params.url)) {
|
||||
return {
|
||||
kind: "allowed",
|
||||
url: params.url,
|
||||
ok: false,
|
||||
error: "Invalid allowed destination URL",
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await params.fetchCheck({
|
||||
proxyUrl: params.proxyUrl,
|
||||
targetUrl: params.url,
|
||||
timeoutMs: params.timeoutMs,
|
||||
});
|
||||
if (!result.ok) {
|
||||
return {
|
||||
kind: "allowed",
|
||||
url: params.url,
|
||||
ok: false,
|
||||
status: result.status,
|
||||
error: `Allowed destination returned HTTP ${result.status}`,
|
||||
};
|
||||
}
|
||||
return { kind: "allowed", url: params.url, ok: true, status: result.status };
|
||||
} catch (err) {
|
||||
return {
|
||||
kind: "allowed",
|
||||
url: params.url,
|
||||
ok: false,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
async function runDeniedCheck(params: {
|
||||
target: ProxyValidationDeniedTarget;
|
||||
proxyUrl: string;
|
||||
timeoutMs: number;
|
||||
fetchCheck: ProxyValidationFetchCheck;
|
||||
}): Promise<ProxyValidationCheck> {
|
||||
if (!isValidHttpTargetUrl(params.target.url)) {
|
||||
return {
|
||||
kind: "denied",
|
||||
url: params.target.url,
|
||||
ok: false,
|
||||
error: "Invalid denied destination URL",
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await params.fetchCheck({
|
||||
proxyUrl: params.proxyUrl,
|
||||
targetUrl: params.target.url,
|
||||
timeoutMs: params.timeoutMs,
|
||||
});
|
||||
if (
|
||||
params.target.expectedCanaryToken !== undefined &&
|
||||
result.deniedCanaryToken !== params.target.expectedCanaryToken
|
||||
) {
|
||||
if (result.ok) {
|
||||
return {
|
||||
kind: "denied",
|
||||
url: params.target.url,
|
||||
ok: false,
|
||||
status: result.status,
|
||||
error: `Denied loopback canary returned HTTP ${result.status} without the validation token`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
kind: "denied",
|
||||
url: params.target.url,
|
||||
ok: true,
|
||||
status: result.status,
|
||||
};
|
||||
}
|
||||
return {
|
||||
kind: "denied",
|
||||
url: params.target.url,
|
||||
ok: false,
|
||||
status: result.status,
|
||||
error:
|
||||
params.target.expectedCanaryToken === undefined
|
||||
? `Denied destination returned HTTP ${result.status}; expected the proxy to block the connection`
|
||||
: `Denied loopback canary was reachable through the proxy with HTTP ${result.status}`,
|
||||
};
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (params.target.transportErrorMeansBlocked) {
|
||||
return {
|
||||
kind: "denied",
|
||||
url: params.target.url,
|
||||
ok: true,
|
||||
error: message,
|
||||
};
|
||||
}
|
||||
return {
|
||||
kind: "denied",
|
||||
url: params.target.url,
|
||||
ok: false,
|
||||
error: `Denied destination failed without a verifiable proxy-deny signal: ${message}`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export async function runProxyValidation(
|
||||
options: RunProxyValidationOptions,
|
||||
): Promise<ProxyValidationResult> {
|
||||
const config = resolveProxyValidationConfig(options);
|
||||
if (config.errors.length > 0) {
|
||||
return { ok: false, config, checks: [] };
|
||||
}
|
||||
if (!config.proxyUrl) {
|
||||
if (!config.enabled && config.source === "disabled") {
|
||||
return {
|
||||
ok: false,
|
||||
config: {
|
||||
...config,
|
||||
errors: [
|
||||
"Proxy validation is disabled. Set proxy.enabled=true or pass --proxy-url to run validation.",
|
||||
],
|
||||
},
|
||||
checks: [],
|
||||
};
|
||||
}
|
||||
return { ok: false, config, checks: [] };
|
||||
}
|
||||
|
||||
const timeoutMs = normalizeTimeoutMs(options.timeoutMs);
|
||||
const fetchCheck = options.fetchCheck ?? defaultProxyValidationFetchCheck;
|
||||
const allowedUrls = options.allowedUrls ?? DEFAULT_PROXY_VALIDATION_ALLOWED_URLS;
|
||||
const deniedTargets = await resolveDeniedTargets(options.deniedUrls);
|
||||
const checks: ProxyValidationCheck[] = [];
|
||||
|
||||
try {
|
||||
for (const url of allowedUrls) {
|
||||
checks.push(await runAllowedCheck({ url, proxyUrl: config.proxyUrl, timeoutMs, fetchCheck }));
|
||||
}
|
||||
for (const target of deniedTargets.targets) {
|
||||
checks.push(
|
||||
await runDeniedCheck({ target, proxyUrl: config.proxyUrl, timeoutMs, fetchCheck }),
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
await deniedTargets.close();
|
||||
}
|
||||
|
||||
return {
|
||||
ok: checks.every((check) => check.ok),
|
||||
config,
|
||||
checks,
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user