feat(gateway): advertise chat attachment limits on hello-ok

Clients had no way to learn the gateway attachment ceilings, so external
clients hardcoded guesses that drifted from server enforcement. Publish the
two unconditional decoded-size ceilings on hello-ok policy.attachments from
one shared resolver so advertised values cannot drift from the parser.

MIME acceptance and per-message counts stay server-side: they depend on the
entrypoint, the resolved model, and payload sniffing, so they cannot be stated
once per connection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 17d6c355-8948-4b48-a936-e08b1c8806ef
This commit is contained in:
Omar Shahine
2026-07-30 02:41:25 +00:00
committed by clawsweeper
parent e07d75dc09
commit b6c4b2d55b
12 changed files with 286 additions and 39 deletions

View File

@@ -111,6 +111,27 @@ Capability-gated agent tools are a separate use of the same declaration. If an
agent tool requires a client capability, the Gateway omits that tool unless the
originating client advertised every required capability.
## Validate attachments before sending
Attachment limits are operator-tunable, so do not hardcode them. Read
`hello-ok.policy.attachments` and validate locally before uploading:
```ts
const attachments = hello.policy.attachments;
if (attachments) {
const ceiling = isImage ? attachments.maxImageBytes : attachments.maxBytes;
if (file.byteLength > ceiling) rejectLocally();
}
```
Both values are decoded per-attachment ceilings. Still check the serialized
request against `policy.maxPayload`: attachments travel as base64, so a file near
`maxBytes` can exceed the frame limit on its own. Older gateways omit
`policy.attachments`; when it is absent, send and handle the server's typed
rejection. Accepted MIME types and per-message counts are not advertised because
they depend on the entrypoint and the resolved model, and the values are a
connection-time snapshot, so re-read them on every reconnect.
## Recover state after reconnect
Treat every successful reconnect as a new projection over durable history and

View File

@@ -153,7 +153,8 @@ Gateway responds with `hello-ok`:
"policy": {
"maxPayload": 26214400,
"maxBufferedBytes": 52428800,
"tickIntervalMs": 15000
"tickIntervalMs": 15000,
"attachments": { "maxBytes": 20971520, "maxImageBytes": 6291456 }
}
}
}
@@ -162,7 +163,31 @@ Gateway responds with `hello-ok`:
`server`, `features`, `snapshot`, `policy`, and `auth` are all required by
`HelloOkSchema` (`packages/gateway-protocol/src/schema/frames.ts`). `auth`
reports the negotiated role/scopes even when no device token is issued (shape
above). `pluginSurfaceUrls` is optional and maps plugin surface names (e.g.
above). `policy.attachments` is optional (older gateways omit it) and advertises
the decoded-size ceilings chat attachments face on `chat.send`, `sessions.send`,
and session-creation initial turns:
| Field | Meaning |
| --------------- | --------------------------------------------------------------------------------------------------- |
| `maxBytes` | Largest decoded size accepted for a single attachment (`agents.defaults.mediaMaxMb`, default 20 MB) |
| `maxImageBytes` | Largest decoded size accepted for a single image: `min(maxBytes, 6 MB agent-hydration cap)` |
Validating before send:
1. Check each file's decoded size against `maxImageBytes` for images and
`maxBytes` for everything else.
2. Serialize the whole request and check its encoded size against
`policy.maxPayload`. `policy.attachments` is a per-attachment ceiling, never a
promise the frame fits: attachments travel as base64, so a 20 MB file is about
26.7 MB on the wire and exceeds the default 25 MiB frame limit on its own.
3. Treat the server as authoritative for everything else. Accepted MIME types and
per-message counts are deliberately not advertised because they depend on the
entrypoint, the resolved model, and payload sniffing; the gateway still returns
a typed rejection.
4. Re-read the values on every reconnect. They are a connection-time snapshot, so
a live `mediaMaxMb` edit reaches existing connections only after they reconnect.
`pluginSurfaceUrls` is optional and maps plugin surface names (e.g.
`canvas`) to scoped hosted URLs; it may expire, so nodes call
`node.pluginSurface.refresh` with `{ "surface": "canvas" }` for a fresh entry.
The deprecated `canvasHostUrl` / `canvasCapability` / `node.canvas.capability.refresh`
@@ -1034,10 +1059,13 @@ third-party clients.
| Default tick interval (pre `hello-ok`) | `30_000` ms | `packages/gateway-client/src/client.ts` |
| Tick-timeout close | code `4000` when silence exceeds `tickIntervalMs * 2` | `packages/gateway-client/src/client.ts` |
| `MAX_PAYLOAD_BYTES` | `25 * 1024 * 1024` (25 MB) | `src/gateway/server-constants.ts` |
| Chat attachment ceiling | `agents.defaults.mediaMaxMb`, default 20 MB decoded | `src/gateway/chat-attachment-policy.ts` |
| Chat attachment image ceiling | `min(attachment ceiling, 6 MB)` | `src/gateway/chat-attachment-policy.ts`, `packages/media-core/src/constants.ts` |
The server advertises the effective `policy.tickIntervalMs`,
`policy.maxPayload`, and `policy.maxBufferedBytes` in `hello-ok`; clients
should honor those values rather than the pre-handshake defaults.
`policy.maxPayload`, `policy.maxBufferedBytes`, and `policy.attachments` in
`hello-ok`; clients should honor those values rather than the pre-handshake
defaults or hardcoded attachment sizes.
The reference client lets finite requests own their configured deadline when
every pending request has one. An `expectFinal` request without a finite

View File

@@ -138,6 +138,17 @@ export const HelloOkSchema = closedObject({
maxPayload: Type.Integer({ minimum: 1 }),
maxBufferedBytes: Type.Integer({ minimum: 1 }),
tickIntervalMs: Type.Integer({ minimum: 1 }),
// Additive: unconditional decoded-size ceilings for chat attachments, so
// clients can validate a file before sending instead of hardcoding guesses.
// Per attachment, not per frame: the encoded request must still fit
// `maxPayload`. MIME acceptance and per-message counts stay server-side
// because they depend on the entrypoint, resolved model, and payload sniffing.
attachments: Type.Optional(
closedObject({
maxBytes: Type.Integer({ minimum: 1 }),
maxImageBytes: Type.Integer({ minimum: 1 }),
}),
),
allowedSessionVisibilities: Type.Optional(Type.Array(SessionVisibilitySchema)),
hasMultipleSessionSharingIdentities: Type.Optional(Type.Boolean()),
}),

View File

@@ -0,0 +1,69 @@
// Attachment policy tests guard the numbers advertised on `hello-ok` against the
// ceilings the parser actually enforces.
import { MAX_IMAGE_BYTES } from "@openclaw/media-core/constants";
import { describe, expect, it } from "vitest";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import {
DEFAULT_CHAT_ATTACHMENT_MAX_BYTES,
resolveChatAttachmentMaxBytes,
resolveChatAttachmentPolicy,
} from "./chat-attachment-policy.js";
const MB = 1024 * 1024;
const cfgWithMediaMaxMb = (value: unknown): OpenClawConfig =>
({ agents: { defaults: { mediaMaxMb: value } } }) as unknown as OpenClawConfig;
describe("resolveChatAttachmentMaxBytes", () => {
it("honours a configured agents.defaults.mediaMaxMb", () => {
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(10))).toBe(10 * MB);
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(50))).toBe(50 * MB);
});
it("falls back to the default ceiling when unset", () => {
expect(resolveChatAttachmentMaxBytes({} as OpenClawConfig)).toBe(
DEFAULT_CHAT_ATTACHMENT_MAX_BYTES,
);
expect(resolveChatAttachmentMaxBytes({ agents: {} } as unknown as OpenClawConfig)).toBe(
DEFAULT_CHAT_ATTACHMENT_MAX_BYTES,
);
});
it("rejects non-positive, non-finite, or non-number values", () => {
for (const bad of [0, -5, Number.NaN, Number.POSITIVE_INFINITY, "50", null, undefined]) {
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(bad))).toBe(
DEFAULT_CHAT_ATTACHMENT_MAX_BYTES,
);
}
});
it("never floors a legal sub-byte mediaMaxMb to zero", () => {
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(0.0000001))).toBe(1);
});
it("keeps an enormous mediaMaxMb representable instead of overflowing", () => {
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(1e308))).toBe(Number.MAX_SAFE_INTEGER);
});
});
describe("resolveChatAttachmentPolicy", () => {
it("advertises the configured ceiling with the image hydration cap applied", () => {
expect(resolveChatAttachmentPolicy(cfgWithMediaMaxMb(20))).toEqual({
maxBytes: 20 * MB,
maxImageBytes: MAX_IMAGE_BYTES,
});
});
it("clamps maxImageBytes to the configured ceiling when it is the smaller limit", () => {
expect(resolveChatAttachmentPolicy(cfgWithMediaMaxMb(1))).toEqual({
maxBytes: MB,
maxImageBytes: MB,
});
});
it("keeps both ceilings positive so the hello-ok schema stays satisfiable", () => {
const policy = resolveChatAttachmentPolicy(cfgWithMediaMaxMb(0.0000001));
expect(policy.maxBytes).toBeGreaterThanOrEqual(1);
expect(policy.maxImageBytes).toBeGreaterThanOrEqual(1);
});
});

View File

@@ -0,0 +1,42 @@
// Connection-level chat attachment ceilings shared by the parser and the
// `hello-ok` handshake. Kept out of chat-attachments.ts so the handshake path
// does not pull the media probe/store graph in just to read two numbers.
import { MAX_IMAGE_BYTES } from "@openclaw/media-core/constants";
import type { OpenClawConfig } from "../config/types.openclaw.js";
const DEFAULT_CHAT_ATTACHMENT_MAX_MB = 20;
/** Default decoded-size ceiling when `agents.defaults.mediaMaxMb` is unset or invalid. */
export const DEFAULT_CHAT_ATTACHMENT_MAX_BYTES = DEFAULT_CHAT_ATTACHMENT_MAX_MB * 1024 * 1024;
/** Resolve the maximum decoded attachment size accepted for chat inputs. */
export function resolveChatAttachmentMaxBytes(cfg: OpenClawConfig): number {
const configured = cfg.agents?.defaults?.mediaMaxMb;
const mb =
typeof configured === "number" && Number.isFinite(configured) && configured > 0
? configured
: DEFAULT_CHAT_ATTACHMENT_MAX_MB;
// mediaMaxMb only has to be positive, so a sub-byte value would floor to 0 and
// a huge one overflows to Infinity, which serializes as null on the handshake
// frame and fails its integer schema. Both ends have to stay representable.
return Math.min(Number.MAX_SAFE_INTEGER, Math.max(1, Math.floor(mb * 1024 * 1024)));
}
/** Unconditional decoded-size ceilings advertised on `hello-ok.policy.attachments`. */
type ChatAttachmentPolicy = {
maxBytes: number;
maxImageBytes: number;
};
/**
* Resolve the decoded-size ceilings every chat attachment faces regardless of
* entrypoint or model. Images are checked against the configured ceiling first
* and the agent-hydration cap second, so their effective limit is the smaller of
* the two. MIME acceptance and per-message counts are deliberately absent: they
* depend on the entrypoint, the resolved model, and payload sniffing, so they
* cannot be stated once per connection.
*/
export function resolveChatAttachmentPolicy(cfg: OpenClawConfig): ChatAttachmentPolicy {
const maxBytes = resolveChatAttachmentMaxBytes(cfg);
return { maxBytes, maxImageBytes: Math.min(maxBytes, MAX_IMAGE_BYTES) };
}

View File

@@ -33,11 +33,14 @@ vi.mock("../media/media-probe.js", () => ({
import { MAX_IMAGE_BYTES } from "@openclaw/media-core/constants";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import {
resolveChatAttachmentMaxBytes,
resolveChatAttachmentPolicy,
} from "./chat-attachment-policy.js";
import {
type ChatAttachment,
parseMessageWithAttachments,
persistInboundImagesForTranscript,
resolveChatAttachmentMaxBytes,
stripImageMediaMarkers,
UnsupportedAttachmentError,
} from "./chat-attachments.js";
@@ -67,13 +70,17 @@ function pdfAttachment(overrides: Partial<ChatAttachment> = {}): ChatAttachment
};
}
function oversizedPngBase64(): string {
function pngBase64OfBytes(bytes: number): string {
const pngHeader = PNG_1x1.slice(0, 64);
let base64Length = Math.ceil(((MAX_IMAGE_BYTES + 1) * 4) / 3);
let base64Length = Math.ceil((bytes * 4) / 3);
base64Length += (4 - (base64Length % 4)) % 4;
return `${pngHeader}${"A".repeat(base64Length - pngHeader.length)}`;
}
function oversizedPngBase64(): string {
return pngBase64OfBytes(MAX_IMAGE_BYTES + 1);
}
async function parseWithWarnings(
message: string,
attachments: ChatAttachment[],
@@ -623,30 +630,47 @@ describe("parseMessageWithAttachments validation errors", () => {
});
});
describe("resolveChatAttachmentMaxBytes", () => {
describe("advertised attachment policy matches enforcement", () => {
const MB = 1024 * 1024;
const DEFAULT_BYTES = 20 * MB;
const cfgWithMediaMaxMb = (value: unknown): OpenClawConfig =>
const cfgWithMediaMaxMb = (value: number): OpenClawConfig =>
({ agents: { defaults: { mediaMaxMb: value } } }) as unknown as OpenClawConfig;
it("honours a configured agents.defaults.mediaMaxMb", () => {
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(10))).toBe(10 * MB);
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(50))).toBe(50 * MB);
});
it("falls back to DEFAULT_CHAT_ATTACHMENT_MAX_MB when unset", () => {
expect(resolveChatAttachmentMaxBytes({} as OpenClawConfig)).toBe(DEFAULT_BYTES);
expect(resolveChatAttachmentMaxBytes({ agents: {} } as unknown as OpenClawConfig)).toBe(
DEFAULT_BYTES,
async function parseImageWithPolicy(cfg: OpenClawConfig, imageBytes: number) {
const policy = resolveChatAttachmentPolicy(cfg);
const parse = parseMessageWithAttachments(
"x",
[pngAttachment({ fileName: "big.png", content: pngBase64OfBytes(imageBytes) })],
{ maxBytes: policy.maxBytes, log: { warn: () => {} } },
);
return { policy, parse };
}
it("rejects images above the advertised maxImageBytes when the config ceiling is the smaller limit", async () => {
const { policy, parse } = await parseImageWithPolicy(cfgWithMediaMaxMb(1), 3 * MB);
expect(policy.maxImageBytes).toBe(MB);
await expect(parse).rejects.toThrow(/exceeds size limit/i);
});
it("rejects non-positive, non-finite, or non-number values", () => {
for (const bad of [0, -5, Number.NaN, Number.POSITIVE_INFINITY, "50", null, undefined]) {
expect(resolveChatAttachmentMaxBytes(cfgWithMediaMaxMb(bad))).toBe(DEFAULT_BYTES);
it("accepts images under the advertised maxImageBytes", async () => {
const { policy, parse } = await parseImageWithPolicy(cfgWithMediaMaxMb(20), 3 * MB);
expect(policy.maxImageBytes).toBe(MAX_IMAGE_BYTES);
const parsed = await parse;
try {
expect(parsed.offloadedRefs).toHaveLength(1);
} finally {
await cleanupOffloadedRefs(parsed.offloadedRefs);
}
});
it("rejects images above the advertised maxImageBytes when the hydration cap is the smaller limit", async () => {
const { policy, parse } = await parseImageWithPolicy(
cfgWithMediaMaxMb(20),
MAX_IMAGE_BYTES + 3,
);
expect(policy.maxImageBytes).toBe(MAX_IMAGE_BYTES);
await expect(parse).rejects.toThrow(/image exceeds size limit/i);
});
});
describe("attachment validation", () => {

View File

@@ -5,7 +5,6 @@ import { MAX_IMAGE_BYTES, type MediaKind } from "@openclaw/media-core/constants"
import { extensionForMime, kindFromMime, mimeTypeFromFilePath } from "@openclaw/media-core/mime";
import { expectDefined } from "@openclaw/normalization-core";
import { normalizeOptionalLowercaseString } from "@openclaw/normalization-core/string-coerce";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { formatErrorMessage, formatUncaughtError } from "../infra/errors.js";
import type { SubsystemLogger } from "../logging/subsystem.js";
import type { MediaFact } from "../media/media-facts.js";
@@ -13,6 +12,7 @@ import { probeMediaFilesWithinBudget } from "../media/media-probe.js";
import type { PromptImageOrderEntry } from "../media/prompt-image-order.js";
import { sniffMimeFromBase64 } from "../media/sniff-mime-from-base64.js";
import { deleteMediaBuffer, saveMediaBuffer, type SavedMedia } from "../media/store.js";
import { DEFAULT_CHAT_ATTACHMENT_MAX_BYTES } from "./chat-attachment-policy.js";
import { formatForLog } from "./ws-log.js";
export type ChatAttachment = {
@@ -78,8 +78,6 @@ const MAX_CHAT_ATTACHMENT_MEDIA_PROBES = 8;
const CHAT_ATTACHMENT_MEDIA_PROBE_CONCURRENCY = 2;
const CHAT_ATTACHMENT_MEDIA_PROBE_BUDGET_MS = 3000;
const DEFAULT_CHAT_ATTACHMENT_MAX_MB = 20;
async function enrichOffloadedMediaMetadata(refs: OffloadedRef[]): Promise<void> {
const candidates = refs.flatMap((ref) => {
const kind = kindFromMime(ref.mimeType);
@@ -174,16 +172,6 @@ export async function persistInboundImagesForTranscript(params: {
return ordered;
}
/** Resolve the maximum decoded attachment size accepted for chat image inputs. */
export function resolveChatAttachmentMaxBytes(cfg: OpenClawConfig): number {
const configured = cfg.agents?.defaults?.mediaMaxMb;
const mb =
typeof configured === "number" && Number.isFinite(configured) && configured > 0
? configured
: DEFAULT_CHAT_ATTACHMENT_MAX_MB;
return Math.floor(mb * 1024 * 1024);
}
type UnsupportedAttachmentReason =
| "empty-payload"
| "text-only-image"
@@ -374,7 +362,7 @@ export async function parseMessageWithAttachments(
acceptNonImage?: boolean;
},
): Promise<ParsedMessageWithImages> {
const maxBytes = opts?.maxBytes ?? DEFAULT_CHAT_ATTACHMENT_MAX_MB * 1024 * 1024;
const maxBytes = opts?.maxBytes ?? DEFAULT_CHAT_ATTACHMENT_MAX_BYTES;
const log = opts?.log;
const shouldForceImageOffload = opts?.supportsImages === false;
const supportsInlineImages = opts?.supportsInlineImages !== false;

View File

@@ -29,11 +29,11 @@ import {
isInternalNonDeliveryChannel,
normalizeMessageChannel,
} from "../../utils/message-channel.js";
import { resolveChatAttachmentMaxBytes } from "../chat-attachment-policy.js";
import {
MediaOffloadError,
logAttachmentFailure,
parseMessageWithAttachments,
resolveChatAttachmentMaxBytes,
type ChatAttachment,
} from "../chat-attachments.js";
import {

View File

@@ -14,12 +14,12 @@ import { measureDiagnosticsTimelineSpan } from "../../infra/diagnostics-timeline
import { formatErrorMessage } from "../../infra/errors.js";
import { parseInboundMediaUri } from "../../media/media-reference.js";
import { deleteMediaBuffer, MEDIA_MAX_BYTES } from "../../media/store.js";
import { resolveChatAttachmentMaxBytes } from "../chat-attachment-policy.js";
import {
MediaOffloadError,
type OffloadedRef,
logAttachmentFailure,
parseMessageWithAttachments,
resolveChatAttachmentMaxBytes,
stripImageMediaMarkers,
UnsupportedAttachmentError,
} from "../chat-attachments.js";

View File

@@ -20,10 +20,10 @@ export { enqueueSystemEvent } from "../infra/system-events.js";
export { deleteMediaBuffer } from "../media/store.js";
export { normalizeMainKey } from "../routing/session-key.js";
export { defaultRuntime } from "../runtime.js";
export { resolveChatAttachmentMaxBytes } from "./chat-attachment-policy.js";
export {
parseMessageWithAttachments,
persistInboundImagesForTranscript,
resolveChatAttachmentMaxBytes,
} from "./chat-attachments.js";
export { normalizeRpcAttachmentsToChatAttachments } from "./server-methods/attachment-normalize.js";
export {

View File

@@ -0,0 +1,62 @@
// Handshake coverage for the additive chat-attachment limits on `hello-ok`, so
// clients can validate a file before sending instead of hardcoding guesses.
import { MAX_IMAGE_BYTES } from "@openclaw/media-core/constants";
import { afterAll, beforeAll, describe, expect, test } from "vitest";
import { clearConfigCache, clearRuntimeConfigSnapshot } from "../config/config.js";
import {
connectOk,
createGatewaySuiteHarness,
installGatewayTestHooks,
testState,
} from "./test-helpers.js";
installGatewayTestHooks({ scope: "suite" });
type GatewayHarness = Awaited<ReturnType<typeof createGatewaySuiteHarness>>;
let gateway: GatewayHarness;
type HelloPolicy = {
policy?: { attachments?: { maxBytes?: number; maxImageBytes?: number } };
};
async function readAdvertisedAttachments(mediaMaxMb: number): Promise<unknown> {
testState.agentConfig = { mediaMaxMb };
// The handshake reads the pinned runtime snapshot, so drop it to pick the
// override up on the next connection.
clearConfigCache();
clearRuntimeConfigSnapshot();
const socket = await gateway.openWs();
try {
const hello = (await connectOk(socket)) as HelloPolicy;
return hello.policy?.attachments;
} finally {
socket.close();
}
}
describe("hello-ok attachment limits", () => {
beforeAll(async () => {
gateway = await createGatewaySuiteHarness();
});
afterAll(async () => {
await gateway.close();
testState.agentConfig = undefined;
clearConfigCache();
clearRuntimeConfigSnapshot();
});
test("advertises the configured ceiling and the image hydration cap", async () => {
expect(await readAdvertisedAttachments(7)).toEqual({
maxBytes: 7 * 1024 * 1024,
maxImageBytes: MAX_IMAGE_BYTES,
});
});
test("clamps the advertised image ceiling to a smaller configured ceiling", async () => {
expect(await readAdvertisedAttachments(2)).toEqual({
maxBytes: 2 * 1024 * 1024,
maxImageBytes: 2 * 1024 * 1024,
});
});
});

View File

@@ -14,6 +14,7 @@ import {
} from "../../../infra/node-pairing.js";
import { listProfiles } from "../../../state/user-profiles.js";
import { resolveRuntimeServiceVersion } from "../../../version.js";
import { resolveChatAttachmentPolicy } from "../../chat-attachment-policy.js";
import {
listControlUiPluginTabs,
listControlUiPluginWidgetKinds,
@@ -124,6 +125,7 @@ export async function sendGatewayHello(
maxPayload: MAX_PAYLOAD_BYTES,
maxBufferedBytes: MAX_BUFFERED_BYTES,
tickIntervalMs: TICK_INTERVAL_MS,
attachments: resolveChatAttachmentPolicy(context.configSnapshot),
allowedSessionVisibilities: allowedSessionVisibilities(context.configSnapshot),
hasMultipleSessionSharingIdentities:
listProfiles().filter((profile) => !profile.mergedInto).length >= 2,