Replace the bare onTurnAdopted callback and queuedFollowupLifecycle with one
turnAdoptionLifecycle surface (onSettled guaranteed via finally; adoption-loss
aborts queued steering turns without transcript replay), threaded through the
turn kernel and agent runner, exposed to plugins via
runtime.state.openChannelIngressDrain and the channel-outbound SDK barrel,
with surface budgets re-measured against the narrowed baselines (#108656).
Embedded runs targeting a CLI runtime provider fall through to the openclaw
harness and call the provider API directly with the runtime's credentials
(cli_runtime_passthrough_openclaw). Anthropic routes direct anthropic-messages
calls on subscription OAuth tokens to metered extra-usage billing — this is
long-standing behavior, not a recent change. Without extra-usage balance,
every such run (e.g. active-memory recall) fails with a billing error; with
extra-usage enabled, the run silently draws paid metered usage instead of the
plan limits the CLI runtime was configured for. Only CLI-backed execution
runs on plan limits for those credentials.
Add an opt-in RunEmbeddedAgentParams.cliBackendDispatch: "subscription-auth"
that dispatches the run through runCliAgent as a one-shot turn when the
provider is claude-cli, a CLI backend is registered, and the ordered auth
profile selection for the passthrough resolves to a subscription (oauth/token)
credential or nothing rather than an API key; resolution stays on stored
credential metadata, with no credential materialization or refresh on the
per-turn path. The dispatch translates toolsAllow into the selectable-backend
surface (native: [], allowlisted loopback MCP tools; wildcard allowlists stay
MCP-only), runs with a fresh CLI process (no live-session reuse; session-
scoped bundle-MCP retirement on run end rather than the process-wide loopback
close), bridges CLI tool result events to onAgentToolResult with native-path
semantics (normalizeToolName + isToolResultError), and drops CLI session
bindings from the result.
The selectable-backend MCP list now also bounds the loopback MCP grant
server-side: the grant carries a per-run gateway tool allowlist enforced in
scoped tool resolution, so tools outside the run's allowlist can be neither
listed nor called even under CLI bypass permission modes where
--allowedTools is advisory.
active-memory recall opts in so recall works on claude-cli subscription-only
instances and stops drawing metered extra usage where it previously could.
Scoped to claude-cli; other CLI runtimes keep the passthrough until their
direct-API contract is verified.
The dispatch also mirrors the run into the run's session transcript through the session
accessor (user turn, tool call/result records as they stream, final assistant
snapshot at run end) so transcript consumers keep parity with embedded runs:
active-memory's persistTranscripts, timeout partial-text salvage, and the
live terminal-search watcher that polls the session file mid-run.
Post-review hardening: canonical anthropic/<model> refs whose configured
agentRuntime is claude-cli resolve through the runtime policy before the
dispatch gate (they previously stayed on the failing passthrough); restricted
dispatches serve an exclusive loopback-only MCP bundle so user/plugin MCP
servers stay outside the run's tool universe; and the transcript recorder
flushes the latest assistant snapshot the moment the run aborts, so timeout
salvage sees partial text even while the killed CLI child is still settling.
Recalls routed to the claude-cli runtime default to a 45s budget (measured
CLI-dispatched runs take 14-20s, over the plain 15s default); explicit
timeoutMs config always wins.
Transcript mirror keeps bare-array tool_result content (claude stream-json
echoes MCP results without a {content} wrapper); dropping it classified every
successful recall as no_relevant_memory.
CLI dispatch resolves inside session/global lane admission so dispatched
runs obey the same lifecycle, placement, and concurrency gates as native
embedded runs.
LOC-ratchet offsets move the loopback grant-context builders to
cli-runner/mcp-grant-context.ts and dedupe the run/prep stage-summary
emitters into attempt-stage-timing.ts; unused type exports dropped and the
now-used stream-message baseline entry removed.
The recall timeout default now consumes the runner's own dispatch
eligibility through a new plugin-runtime seam
(agent.resolveCliBackendDispatchEligibility): API-key and missing-backend
routes keep the passthrough and its plain 15s default.
Eligibility honors an explicitly pinned authProfileId (the credential the
run executes on) before ordered profile selection, in both directions.
Transcript mirror composes buildAssistantMessage + buildUsageWithNoCost
directly; main trimmed the zero-usage wrapper export (ab0ccc244b) before
this branch's usage landed.
Dispatch eligibility is provider-owned: the anthropic plugin's claude-cli
backend declares CliBackendPlugin.subscriptionAuthDispatch and core reads
the registered descriptor instead of a core provider allowlist.
Dispatch fails closed on tool policy (only non-empty named allowlists are
expressible on the CLI surface; deny-all, wildcards, absent allowlists, and
disableTools/modelRun keep the passthrough), threads the pinned
authProfileId into CLI runtime resolution, and emits onExecutionStarted at
the admitted dispatch boundary.
* fix(infra): guard channel ingress queue parseJson against corrupted JSON
* fix(infra): fix type assertion in ingress queue test
* fix(infra): use tagged parse result and validate payload before claiming
* fix(infra): remove unnecessary non-null assertion in ingress queue test
* fix(infra): scan corrupt ingress rows in claimNext
* test(gateway): avoid typed empty mock call tuple access
* fix(infra): tombstone corrupt ingress rows on duplicate enqueue and stale recovery
Two P1 gaps: enqueue() threw on duplicate when the existing row had corrupt
payload_json, and recoverStaleClaims() silently skipped corrupt claimed rows,
leaving them invisible to recovery. Both paths now tombstone the unrecoverable
row as failed with reason "corrupt_payload" and return a proper result.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(infra): resolve lint shadow and await-thenable in recoverStaleClaims
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(channels): tombstone corrupt ingress rows
* test(channels): use explicit placeholder claim tokens
* refactor(channels): name claim token values
* refactor(channels): keep claim projection direct
* fix(channels): preserve active ingress claims
* fix(channels): make corrupt recovery policy-aware
* refactor(channels): name corrupt claim token
* fix(channels): bound corrupt ingress reconciliation
* fix(channels): paginate pending ingress by key
* refactor(telegram): return live owner check directly
* build(plugin-sdk): refresh public export budget
---------
Co-authored-by: Pick-cat <huang.ting3@xydigit.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(sessions): migrate runtime storage to sqlite
* test(sessions): fix sqlite CI regressions
* test(sessions): align remaining sqlite fixtures
* fix(codex): require sqlite trajectory recorder
* test(sessions): align orphan recovery sqlite fixture
* test(sessions): align sqlite rebase fixtures
* fix(sessions): finish current-main integration of the sqlite flip
Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.
* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs
Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.
* feat(sessions): keep archived transcripts by default with zstd cold storage
Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.
* fix(state): converge agent DB migration lineages and bound database growth
Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.
Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.
* fix(codex): resolve the store path for thread-history import via the SDK
The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.
* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors
A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.
* fix(sessions): complete sqlite conflict translations
* test(sqlite): align hardening checks with maintenance
* test(sessions): inspect compressed transcript archives
* fix(tests): await session seeds and drop unused helpers flagged by CI lint
The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.
* test(sessions): type legacy proof events directly
* test(sessions): align hardening contracts
* perf(sessions): read usage transcript sizes from SQL aggregates
Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.
* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir
Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.
* test(agents): seed harness admission through sqlite
* fix(sqlite): close agent db on pragma setup failure
* fix(doctor): compact and retrofit incremental auto-vacuum after session import
The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat: node-hosted plugins — dynamic tools, MCP servers, and skills
Nodes become declarative plugin hosts:
- node.pluginTools.update: node hosts publish plugin-registered agent tool
descriptors; gateway materializes them as agent tools executing via
node.invoke under the node command allowlist, with tools.effective
invalidation and node online/offline removal.
- Trusted paired-node descriptors: no gateway-side plugin registration
required; gateway.nodes.pluginTools.enabled off-switch (default on);
description/count caps; deterministic node-prefixed collision names.
- Declarative node-hosted MCP: nodeHost.mcp.servers (McpServerConfig shape)
starts MCP clients on the node host, publishes tools as pluginId node-mcp,
executes via built-in mcp.tools.call.v1 with per-layer timeouts, failure
isolation, and orphan-safe shutdown. No re-pairing when servers change.
- Node-hosted skills: node.skills.update publishes ~/.openclaw/skills
content (64 skills/64KB/512KB caps both sides); gateway merges them into
the skills snapshot while connected and exec host=node is available, with
node:// locators, node-prefixed collisions, disabled command dispatch,
and gateway.nodes.skills.enabled + nodeHost.skills.enabled switches.
- Security: node-supplied pluginIds cannot satisfy pluginId-scoped tool
allowlists unless gateway-registered; reserved node-mcp id requires the
core MCP descriptor shape; protocol registry kept out of public
plugin-sdk dts.
- E2E: pond harness proves publication, MCP round-trip, skills locator, and
disconnect/reconnect for all three surfaces.
* style: format node-plugin-tools test
* fix(skills): keep status loader unfiltered when eligibility is passed
skills.status started passing eligibility for the node-skill merge, which
flipped loadWorkspaceSkillEntries into filtered mode and dropped disabled
skills from status reports (QA plugin-lifecycle-hot-reload timeout). Status
now merges node skills explicitly around an unfiltered load. Also: regen
docs_map for new node docs sections; add the intentional node-host MCP
onclose suppression to the lint-suppression allowlist.
* feat: add session thread management
Squash of codex/thread-management (025aefc3ad1) onto origin/main:
pin/archive/rename sessions via sessions.patch, archived-aware
sessions.list, lifecycle fencing, read-only archived chat, SDK +
Swift protocol support, Control UI session management.
* refactor(ui): minimal session rows with hover-revealed management
Chat picker and sidebar recents share session-row primitives: single-line
rows, relative timestamps, rename/archive/pin revealed on hover or focus,
accent pin badge for pinned rows, and an active-run spinner in the trail
slot. Sidebar floats pinned sessions above recency via the shared
comparator and gains archive/pin actions through the unified sessions-view
patch fallback. Archive eligibility is one shared policy
(canArchiveSessionRow); the sidebar/picker active-run tooltip now uses the
real sessionsView.activeRun locale key.
* fix: align session admission with mailbox-era main
Integration fixes after rebasing onto current main: sessions_list mailbox
test expectations learn the archived/pinned row fields and archived:false
list param; gateway agent admission treats a session as deleted only when
both the requested and canonical alias sets miss it (legacy bare-main
stores and exec-approval followups read under different spellings); cron
persist tests keep a consistent store across claim-guarded persist calls;
the ACP abort hook test asserts abort propagation instead of signal
identity; drop dead lifecycle writes flagged by no-useless-assignment and
fix the promise-executor return in the codex compact test.
* fix(qa): align UI e2e and shard fixtures with redesigned session rows
Sidebar session rows are wrapper divs with an inner link now: update the
navigation browser tests and chat-flow Playwright selectors. Seed a real
per-test session store for the auto-fallback admission guard instead of
depending on leftover host files at /tmp/sessions.json. Teach the
test-projects routing fixture about the suites that newly import the
shared temp-dir helper. Document the Codex thread-format contract for
archivedAt/pinnedAt (flag derived from server-stamped timestamp, epoch ms
here vs Codex epoch seconds) at the type and in the session docs.
* test: route auto-fallback suite through temp-dir helper plans
The auto-fallback suite now imports the shared temp-dir helper for its
seeded session store, so the top-level helper routing fixture must list
it in the auto-reply plan.
Make plugin-state enforce the plugin-wide live-row fuse by evicting only from the namespace currently being written, preserving sibling namespace rows and still failing atomically when the current namespace cannot free enough rows.
Raise the plugin-wide cap to 6,000 rows, keep Telegram's persistent message-cache namespace at 3,000 entries, and document the updated SDK runtime contract. Harden legacy plugin-state import so capacity pressure cannot archive a source after losing imported keys, with focused regression coverage for Telegram-shaped namespaces and migration rollback.
Also restore the Docker runtime-assets preflight step in full release validation so release workflow contract tests stay aligned.
Verification: focused plugin-state, migration, Telegram, workflow-contract, lint, deprecated-API, diff-check, Blacksmith Testbox, CI, CodeQL, Workflow Sanity, OpenGrep, and autoreview all passed on PR head fee021cfa6.
Co-authored-by: Keshav's Bot <keshavbotagent@gmail.com>
* feat(file-transfer): add bundled plugin for binary file ops on nodes
New extensions/file-transfer/ plugin exposing four agent tools
(file_fetch, dir_list, dir_fetch, file_write) and four matching
node-host commands (file.fetch, dir.list, dir.fetch, file.write).
Lets agents read and write files on paired nodes by absolute path,
bypassing the bash output cap (200KB) and the live tool-result
text cap that would otherwise truncate base64 payloads.
Public surface
--------------
- file_fetch({ node, path, maxBytes? })
Image MIMEs return image content blocks; small text (<=8 KB) inlines
as text content; everything else returns a saved-media-path text
block. sha256-verified end-to-end.
- dir_list({ node, path, pageToken?, maxEntries? })
Structured directory listing — name, path, size, mimeType, isDir,
mtime. Paginated. No content transfer.
- dir_fetch({ node, path, maxBytes?, includeDotfiles? })
Server-side tar -czf streamed back, unpacked into the gateway media
store, returns a manifest of saved paths. Single round-trip.
60s wall-clock timeouts on tar create/unpack. tar -xzf without -P
rejects absolute paths in archive entries.
- file_write({ node, path, contentBase64, mimeType?, overwrite?,
createParents? })
Atomic write (temp + rename). Refuses to overwrite by default.
Refuses to write through symlinks (lstat check). Buffer-side
sha256 (no read-back race). Pair with file_fetch to round-trip
files between nodes — DO NOT use exec/cp for file copies.
All four commands gated by:
- dangerous-by-default node command policy
(gateway.nodes.allowCommands opt-in)
- per-node path policy (gateway.nodes.fileTransfer)
- optional operator approval prompt (ask: off | on-miss | always)
16 MB raw byte ceiling per single-frame round-trip (25 MB WS frame
with ~33% base64 overhead and JSON envelope). 8 MB defaults.
Path policy and approvals
-------------------------
Default behavior is DENY. The operator must explicitly opt in:
{
"gateway": {
"nodes": {
"fileTransfer": {
"<nodeId-or-displayName>": {
"ask": "off" | "on-miss" | "always",
"allowReadPaths": ["~/Screenshots/**", "/tmp/**"],
"allowWritePaths": ["~/Downloads/**"],
"denyPaths": ["**/.ssh/**", "**/.aws/**"],
"maxBytes": 16777216
},
"*": { "ask": "on-miss" }
}
}
}
}
ask modes:
off — silent: allow if matched, deny if not (default)
on-miss — silent allow if matched; prompt on miss
always — prompt every call (denyPaths still hard-deny)
denyPaths always wins. allow-always from the prompt persists the
exact path back into allowReadPaths/allowWritePaths via
mutateConfigFile so subsequent matching calls go silent.
Reuses existing primitives — no new gateway methods:
plugin.approval.request / plugin.approval.waitDecision
decision: allow-once | allow-always | deny
Pre-flight against requested path AND post-flight against the
canonicalPath returned by the node — closes symlink-escape attacks
where the requested path matched policy but realpath resolves
somewhere else.
Audit log
---------
JSONL at ~/.openclaw/audit/file-transfer.jsonl. Records every
decision (allow/allowed-once/allowed-always/denied/error) with
timestamp, op, nodeId, displayName, requestedPath, canonicalPath,
decision, error code, sizeBytes, sha256, durationMs. Best-effort
writes; never propagates failure.
Plugin layout
-------------
extensions/file-transfer/
index.ts definePluginEntry, nodeHostCommands
openclaw.plugin.json contracts.tools registration
package.json
src/node-host/{file-fetch,dir-list,dir-fetch,file-write}.ts
src/tools/{file-fetch,dir-list,dir-fetch,file-write}-tool.ts
src/shared/
mime.ts single-source extension->MIME map + image/text sets
errors.ts shared error code enum and helpers
params.ts shared param-validation helpers + GatewayCallOptions
policy.ts evaluateFilePolicy, persistAllowAlways
approval.ts plugin.approval.request wrapper
gatekeep.ts one-stop policy + approval + audit orchestrator
audit.ts JSONL audit sink
Core touch points
-----------------
- src/infra/node-commands.ts: NODE_FILE_FETCH_COMMAND,
NODE_DIR_LIST_COMMAND, NODE_DIR_FETCH_COMMAND,
NODE_FILE_WRITE_COMMAND, NODE_FILE_COMMANDS array
- src/gateway/node-command-policy.ts: all four added to
DEFAULT_DANGEROUS_NODE_COMMANDS
- src/security/audit-extra.sync.ts: audit detail mentions file ops
- src/agents/tools/nodes-tool-media.ts: MEDIA_INVOKE_ACTIONS entry
for file.fetch redirects raw nodes(action=invoke) callers to the
dedicated file_fetch tool to prevent base64 context bloat
- src/agents/tools/nodes-tool.ts: nodes tool description points to
the dedicated file_fetch tool
Known limitations / follow-ups
------------------------------
- No tests in this PR. For a security-sensitive surface this is a
gap; will follow up with a test pass.
- Direct CLI invocation (openclaw nodes invoke --command file.fetch)
bypasses the plugin policy entirely. Plugin-side gating is the
realistic threat model (agent on iMessage requesting paths it
shouldn't), but for true defense-in-depth, policy belongs in the
gateway-side node.invoke dispatch. Move-policy-to-core is a
separate PR.
- file_watch (long-lived filesystem event subscription) is not
included; it needs a new node-protocol primitive for streaming
event channels and was descoped from this PR.
- dir_fetch includeDotfiles: true is the only supported mode;
BSD tar exclude patterns reliably collapse dotfile filtering
to an empty archive. Reliable filtering needs a
`find ! -name ".*" | tar -T -` pipeline; deferred.
- dir_fetch du -sk preflight is a heuristic (du * 4 vs maxBytes);
the mid-stream byte cap is the actual safety net.
* test(file-transfer): add unit tests for handlers, policy, and shared utilities
Adds 77 tests covering:
- handleFileFetch: validation, fs errors, sha256, size cap, symlink canonicalization
- handleFileWrite: validation, atomic write, overwrite policy, parent dir handling, symlink refusal, integrity check, size cap
- handleDirList: validation, fs errors, sorted listing, dotfile inclusion, pagination
- handleDirFetch: validation, fs errors, gzipped tar with sha256, mid-stream byte cap
- evaluateFilePolicy: default-deny, denyPaths-wins, allow matching, ask modes (off/on-miss/always), node-id/displayName/'*' resolution
- persistAllowAlways: append, dedupe, create-on-missing
- shared/mime: extension lookup, image/text inline sets
- shared/errors: err helper, classifyFsError, throwFromNodePayload
Also fixes accumulated lint regressions in the prod source flagged once these
files moved into the changed-gate scope (parseInt -> Number.parseInt, redundant
type casts removed, single-statement if bodies wrapped in braces).
* fix(file-transfer): address PR review feedback (security + availability)
Reviewer findings addressed (greptile + aisle):
- policy: persistAllowAlways no longer escalates per-node approvals to the
'*' wildcard entry; allow-always now writes under the specific node's
own entry, never the wildcard (greptile P1 SECURITY).
- policy: add literal '..' segment short-circuit in evaluateFilePolicy,
raised before glob match. Stops "/allowed/../etc/passwd" from passing
preflight against "/allowed/**" globs (aisle MEDIUM CWE-22).
- file-write: replace no-op base64 try/catch with actual round-trip
validation. Buffer.from(s, "base64") never throws — invalid input
silently decoded to garbage bytes. Now re-encodes and compares
modulo padding/url-variant chars (greptile P1 SECURITY).
- file-write: document the parent-symlink residual risk and rely on the
existing gateway-side post-flight policy check; full rollback requires
a node-side file.unlink which is deferred to a follow-up. Initial
segment-walk attempt was reverted because it false-positives on system
symlinks like macOS /var → /private/var (aisle HIGH CWE-59).
- dir-fetch tool: add preValidateTarball pass that runs `tar -tzvf` and
rejects symlinks, hardlinks, absolute paths, '..' traversal,
uncompressed sizes >64MB, and entry counts >5000 — before any
extraction. Drops --no-overwrite-dir (GNU-only flag rejected by BSD
tar on macOS) (aisle HIGH x2 CWE-22 + CWE-409, greptile P2).
- dir-fetch tool: stream-hash files via fs.open + read loop instead of
fs.readFile to avoid full-buffer reads on large extracted entries.
- dir-fetch handler: replace spawnSync in countTarEntries with async
spawn + bounded buffer so tar -tzf can't park the node-host event
loop for up to 10s on a slow filesystem (greptile P1 AVAIL).
- audit: clear auditDirPromise on rejection so a transient mkdir
failure doesn't permanently silence the audit log (greptile P2).
New tests: wildcard escalation rejection, base64 malformed/url-variant,
'..' traversal short-circuit (3 cases). 84/84 passing.
* fix(file-transfer): CI failures + second-round PR review feedback
CI failures on previous push:
- Declare runtime deps (minimatch, typebox) in package.json — failed the
extension-runtime-dependencies contract test that scans imports.
- Switch policy.ts and policy.test.ts off the broad
openclaw/plugin-sdk/config-runtime barrel and onto the narrow
openclaw/plugin-sdk/config-mutation + runtime-config-snapshot subpaths.
This satisfies the deprecated-internal-config-api architecture guard.
Second-round Aisle findings:
- policy: traversal-segment check now treats backslash and forward slash
as equivalent, so a Windows node can't be hit with mixed-separator
"C:\\allowed\\..\\Windows\\system.ini" (Aisle HIGH CWE-22).
- dir-fetch tool: replace the single fragile `tar -tvzf` parser pass
(which broke for filenames containing whitespace) with two robust
passes: `tar -tzf` for paths only (one per line, no parsing of
fixed columns) and `tar -tzvf` for type chars only (FIRST CHAR of each
line, never the path column). Also reject backslash-containing entry
names. Drops the in-process uncompressed-size cap because reliably
parsing sizes from tar output is fragile and Aisle flagged it as a
bypass primitive — entry-count cap stays (Aisle HIGH CWE-22, MED).
Tests still 84/84 passing.
* fix(file-transfer): third-round PR review feedback
Aisle's re-analysis on b63daa6a05 surfaced 3 actionable findings:
- nodes.invoke bypass (HIGH CWE-285): generic nodes.action="invoke" let
agents call dir.list/dir.fetch/file.write directly, skipping the
file-transfer plugin's gatekeep + policy + approval flow. Only file.fetch
was redirected to its dedicated tool. Add the other three to
MEDIA_INVOKE_ACTIONS so the redirect-or-deny logic in
nodes-tool-commands fires for all four. The dedicated tools enforce
policy; the generic invoke surface no longer has a way to skip them
without an explicit allowMediaInvokeCommands opt-in.
- prototype pollution in persistAllowAlways (MED CWE-1321): a paired
node with displayName "__proto__" / "prototype" / "constructor" would
mutate the fileTransfer object's prototype when persisting allow-always.
Reject those keys explicitly. Switch the existing-key lookup to
Object.prototype.hasOwnProperty.call so a key like "constructor"
doesn't accidentally match Object.prototype.constructor.
- decompression-bomb cap in dir_fetch (MED CWE-409): compressed tar is
bounded upstream, but a highly compressible bomb can still expand to
gigabytes. Enforce DIR_FETCH_MAX_UNCOMPRESSED_BYTES (64MB) summed
across extracted files and DIR_FETCH_MAX_SINGLE_FILE_BYTES (16MB) per
entry, both checked during the post-extract walk. On bust, rm -rf the
rootDir and audit-log + throw UNCOMPRESSED_TOO_LARGE.
Tests: 85/85 passing (added prototype-pollution rejection test).
Aisle's HIGH parent-symlink finding remains documented as deferred — full
rollback requires a node-side file.unlink command which is out of scope
for this PR. The gateway-side post-flight policy check still detects and
loudly errors on canonical-path mismatches.
* fix(file-transfer): refuse symlink traversal by default with followSymlinks opt-in
Closes the deferred Aisle HIGH parent-symlink finding. Instead of
detecting the escape in a post-flight gateway check after the file is
already written, the node-side handler now refuses pre-flight if any
component of the requested path resolves through a symlink.
Behavior:
- Reads (file.fetch / dir.list / dir.fetch): node realpath()s the
requested path. If canonical != requested AND followSymlinks=false,
return SYMLINK_REDIRECT { canonicalPath } — no I/O happens.
- Writes (file.write): node realpath()s the parent dir. Same refusal
rule. The lstat-on-final check is kept to catch the case where the
target file itself is an existing symlink.
- Opt-in: set gateway.nodes.fileTransfer.<node>.followSymlinks=true to
bring back the previous "follow + post-flight check" behavior.
Operator UX: the SYMLINK_REDIRECT response includes the canonical path
so the operator can either update their allow list to the canonical form
or set followSymlinks=true on that node. On macOS, /var → /private/var
and /tmp → /private/tmp are system aliases that trip the new check, so
operators using those paths need followSymlinks=true OR canonical-path
allowlists.
Wiring:
- Add followSymlinks?: boolean to NodeFilePolicyConfig.
- evaluateFilePolicy returns followSymlinks (default false) on its
ok=true branches.
- gatekeep propagates it via GatekeepOutcome.
- Each tool passes it as a node.invoke param.
- Each handler honors it pre-flight before any read/write.
Tests updated: 89/89 passing.
- realpath(mkdtemp()) so existing happy-path tests don't trip the new
default on macOS where mkdtemp lands under symlinked /var/folders.
- New tests: SYMLINK_REDIRECT refusal for file.fetch and file.write
parent traversal; opt-in passthrough when followSymlinks=true.
- New policy test: followSymlinks propagation default false / true.
* fix(file-transfer): close two more aisle findings on 069bd66
Aisle re-analysis on 069bd66 surfaced two issues my earlier round-three
fix missed:
- HIGH (CWE-284): file.fetch / dir.fetch / dir.list / file.write were
still bypassable via the generic nodes.action="invoke" surface when
the operator had set allowMediaInvokeCommands=true. That flag was
meant to opt in to base64-bloat for camera/screen, not to disable
path policy on file-transfer. Split the redirect map: introduce
POLICY_REDIRECT_INVOKE_COMMANDS (file-transfer only) which ALWAYS
rerouts to its dedicated tool regardless of the bloat flag. Camera
and screen continue to use the bloat-only redirect (suppressed by
allowMediaInvokeCommands=true). Confirmed by clawsweeper P1.
- MED (CWE-276): tar -xzf in dir_fetch unpack preserved archive
ownership and permissions, so a malicious node could plant
setuid/setgid or world-writable files on a gateway running with
elevated privileges. Add --no-same-owner --no-same-permissions
(both flags are portable across BSD tar / GNU tar).
Tests: 89/89 passing.
* chore(file-transfer): drop file_watch from plugin description
Phase 5 (file_watch) was deferred earlier in this PR. Strip the watch
mention from the plugin description in package.json,
openclaw.plugin.json, and index.ts so the metadata reflects what's
actually shipped (file_fetch, dir_list, dir_fetch, file_write).
Closes clawsweeper P3.
* fix(file-transfer): hash before rename and allow zero-byte round-trip
Two of Peter's review findings on PR #74134:
- P2 (file-write integrity): hash the decoded buffer + compare against
expectedSha256 BEFORE temp+rename. Previously the rename happened
first, then the sha check unlinked the target on mismatch — with
overwrite=true a bad caller hash could replace + delete the original.
Now a hash mismatch returns INTEGRITY_FAILURE without touching disk.
Added a regression test that asserts the original file survives.
- P2/P3 (zero-byte round-trip): the tool layer's truthy checks on
contentBase64 and base64 rejected the empty string, blocking zero-byte
files from round-tripping through file_fetch -> file_write. Switched
to type-checks (typeof === "string") and added zero-byte tests at the
handler layer for both fetch and write (sha matches the known empty
digest).
Tests: 92/92 passing.
* fix(file-transfer): declare gateway.nodes.fileTransfer in core config schema
Peter's P1/P2 finding: the plugin reads/writes gateway.nodes.fileTransfer
via casts through unknown because the strict zod schema and OpenClawConfig
type didn't declare it. That meant `openclaw config validate` would
reject the very examples in the plugin's own documentation.
- Add fileTransfer block to gateway.nodes in src/config/zod-schema.ts
with the full per-node entry shape (ask, allowReadPaths,
allowWritePaths, denyPaths, maxBytes, followSymlinks).
- Add GatewayNodeFileTransferEntry + the fileTransfer field on
GatewayNodesConfig in src/config/types.gateway.ts.
- Drop the `as unknown` casts in the extension's policy.ts now that
gateway.nodes.fileTransfer is properly typed end-to-end.
- Regenerate docs/.generated/config-baseline.sha256.
Tests: 92/92 passing. pnpm config:docs:check OK.
* fix(file-transfer): enforce path policy at gateway dispatch
Closes Peter's P1 review finding on PR #74134.
The agent-tool-only redirect added in earlier commits left CLI
(`openclaw nodes invoke`), plugin-runtime, and raw `node.invoke` callers
able to skip the file-transfer path policy entirely. The fix moves the
security boundary down to the gateway: every code path that reaches
`node.invoke` for file.fetch / dir.list / dir.fetch / file.write now
runs the same allow/deny check.
- New: src/gateway/file-transfer-dispatch.ts with
`evaluateFileTransferDispatchPolicy` and `isFileTransferCommand`. Same
semantics as the extension-side `evaluateFilePolicy` minus the
operator-prompt flow (prompts stay at the agent-tool layer; the
gateway is silent enforcement).
- src/gateway/server-methods/nodes.ts: after the existing command
allowlist check, run the new gate before forwarding. Denies emit
INVALID_REQUEST with a structured `{ command, code, reason }`.
- Decision matrix mirrors the extension: NO_POLICY (no entry for
this node) deny, denyPaths-wins, '..' traversal short-circuit
(with backslash separator handling), allowPaths match → allow,
no allow match → deny.
- 19 new unit tests covering each branch including identity
resolution (nodeId/displayName/'*'), prototype-pollution-safe lookup,
and read-vs-write allow-list separation.
Note on allow-once approvals: the agent tool's interactive
`allow-once` decision now has to flow through the dedicated tool's
pre-flight (which forwards an approved request); raw `nodes.invoke`
callers cannot benefit from one-time approvals because the gateway is
silent. allow-always (which persists to allowReadPaths/allowWritePaths)
continues to work transparently because by the time the next request
hits the gateway the path is in the persisted allow list.
Tests: 92 extension + 19 gateway = 111 total, all passing.
* fix(file-transfer): enforce node policy in gateway
* fix(file-transfer): use plugin node policy only
* fix(file-transfer): harden node policy edge cases
* fix(file-transfer): close review hardening gaps
* fix(file-transfer): harden node invoke policy
* fix(file-transfer): align runtime dependency versions
* fix(file-transfer): keep minimatch extension-owned
* refactor(file-transfer): remove unused approval gate
* fix(file-transfer): require canonical node policy authorization
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
* fix(clawsweeper): address review for automerge-openclaw-openclaw-74134 (1)
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
* fix(file-transfer): recheck dir fetch archive policy after fetch
* fix(file-transfer): name file-transfer tool in invoke redirect
---------
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: clawsweeper-repair <clawsweeper-repair@users.noreply.github.com>
