Follow-up delta to the merged #103579 head, rebased onto current main:
- gateway-protocol wire types derive from owner-module schema consts
(types.ts tombstone) and ProtocolSchemas leaves the package index so the
public plugin-sdk d.ts graph tree-shakes the registry declaration
- approval access authority follows the operator.approvals scope tier with
reviewerDeviceIds as the opt-in restriction (cross-surface
first-answer-wins; requester identity gates only legacy adapters)
- plugin node.invoke approvals register directly so unrenderable
presentations fail closed before request routing
- exec-approval manager reconciliation with #103515 revocation hardening
(resolution source attribution, one-shot ask-fallback consumption)
- surface-report pins and plugin-sdk API baseline refreshed; Swift models
regenerated
* fix(sessions): read zstd transcript archives through a materialized cache
Compressed archives were discovered but unreadable: the reset-archive
header probe read raw zstd bytes and every JSONL reader parsed them as
malformed lines, so compressed archives silently contributed zero
history and zero usage. Archives are write-once, so they materialize
once into a plain JSONL cache under the OpenClaw tmp dir and every
downstream reader (index, tail chunks, probes, usage scans) works
unchanged; the usage scanner also decompresses directly for whole-file
iteration.
* fix(sessions): keep usage byte-range semantics for compressed archives
Route zstd archive usage reads through the materialized plain-JSONL
cache so persisted incremental-scan offsets always measure decompressed
bytes; the whole-file decompress branch ignored requested ranges and
could overcount archived usage.
* fix(sessions): normalize compressed archives to decompressed space at usage discovery
Sizes, incremental offsets, and cache signatures for zstd archives now
all measure the materialized plain-JSONL cache, closing the truncation
window when persisted offsets from the compressed path met decompressed
reads.
* fix(sessions): gate the archive read cache on source identity
Every materialized-cache hit now stats the source archive: a deleted or
budget-evicted archive scrubs its cache entries instead of serving
stale plaintext, and the cache name carries the source size so a
same-path rewrite can never alias.
* fix(sessions): include source mtime in the archive cache identity
Same-path same-size rewrites can no longer alias a stale plaintext
cache entry; any source change mints a new cache name and the previous
entries are scrubbed on the next materialization.
* fix(sessions): bound and isolate the materialized archive cache
Cache entries expire on a 24h TTL sweep so budget-evicted archives
cannot leave plaintext copies behind, stale-identity scrubbing skips
the live identity and in-flight temp files, and unique temp names plus
last-rename-wins make concurrent materialization race-free.
* fix(sessions): drop useless probe-path initializer flagged by lint
* fix(agents): surface utility-model narration failures and show the utility model in models status
Narration failures were verbose-only, so a dead utility-model credential
silently degraded channel progress drafts back to raw tool lines. The
narrator now warns once per turn when it disables after consecutive
failures, naming the utility model + auth profile, and openclaw models
status renders the resolved utility model (config / provider default /
disabled) in human and JSON output.
Formatting verified remotely via Testbox format:check (hook bypassed:
no node_modules in this worktree).
Fixes#104764
* fix(models): include the utility model in models status auth analysis
The utility ref was rendered but not registered as a provider use, so
missing utility-provider auth kept --check green (Codex review P2).
Utility completions ride the plain API path like image models.
Formatting verified remotely via Testbox format:check.
* fix(models): register the utility model only for otherwise-unused providers
main's route analysis reports per-model issues; a same-provider utility
ref duplicated route reports (CI: incompatible-routes test). Shared
providers are already analyzed via their configured uses; utility-only
providers still enter the analysis so they stay visible to status.
Formatting verified remotely via Testbox format:check.
* fix(models): always analyze the utility model route in models status
Same-provider dedupe hid the documented incident class: an OAuth-healthy
primary says nothing about the utility model's plain API route (Codex
review round 3). The openai route-test fixture now pins utilityModel off
so route tests stay focused on their configured models, and a dedicated
test covers the OAuth-primary/api-key-utility scenario.
Formatting verified remotely via Testbox format:check.
* fix(models): give the utility model full text-route analysis in models status
Uses without codex fallback previously skipped per-model route
resolution (image-auth scoping), so the utility ref never produced
missing-auth route issues. Route scope is now explicit: text uses get
evaluateModelAuth, image uses keep provider-wide availability.
Formatting verified remotely via Testbox format:check.
* fix(models): probe the configured utility model and document source labels
--probe candidates now include the utility ref so utility-only providers
probe the actual configured model instead of an arbitrary catalog entry
(Codex review round 4). Inline comment records that the status resolver
never falls back to the primary, so 'provider default' labels are exact.
Formatting verified remotely via Testbox format:check.
* fix(models): canonicalize the reported utility ref and dedupe route diagnostics
utilityModel accepts aliases, so status now reports the resolved
provider/model instead of the raw alias, and identical route issues from
overlapping primary/utility refs collapse to one entry (Codex round 5).
Formatting verified remotely via Testbox format:check.
zod@4 declares sideEffects:false, so bundlers tree-shake the classic entry's
implicit config(en()) locale registration and every issue in a built dist
falls back to the bare "Invalid input" message. Users hit unactionable
errors like "agents.defaults: Invalid input" (#89445, #103956) and
message-text-keyed issue filtering in validation.ts diverges from source
behavior. Register the English locale explicitly next to the config schemas;
zod keeps its config on globalThis, so one call covers the whole process.
Fixes#104014
* feat(gateway): persist operator approvals
* fix(gateway): preserve approval ids exactly
* fix(gateway): enforce approval reviewer bindings
* fix(gateway): reconcile durable approvals with exec revocation hardening
Map #103515 semantics onto the durable lifecycle: resolutionSource and
one-shot consumeAskFallback stay process-local record facts, trusted
auto-review resolves through the durable CAS as a runtime resolver, and
lost races settle with the winner's operator source. Also: derive the
audience walker cap from the store cap, report APPROVAL_ALREADY_RESOLVED
for a resolve that loses the CAS race, document approval.get/resolve in
the protocol docs, and regenerate Swift models on the new base.
* fix(approvals): validate resolver kind
* docs(approvals): explain malformed verdict denial
* fix(protocol): regenerate approval models after rebase
* chore: leave changelog entry to release generation
* fix(gateway): preserve approval registration boundaries
* test(gateway): prove multi-device approval races
* test(gateway): keep approval order assertion stable
* docs: index operator approval architecture
* fix(protocol): bound approval declaration exports
* feat(ui): detect background tasks eagerly and show live tool activity in the chat rail
Track tool-start counts and the last tool name per task run in the task
registry, persist them on task_runs, and expose them as additive optional
TaskSummary fields. The chat background-tasks rail now loads its snapshot
eagerly so the collapsed toggle badge detects running work immediately,
and rows show a live elapsed timer, tool-use count, and the tool
currently in use (run duration for finished rows).
Issue: #104775
* fix(ui): translate background-task activity strings and refresh generated protocol models
The singular tool-use label is literal ("1 tool use") because several
locales legitimately drop the count placeholder in singular forms, which
fail-closes the placeholder guard in the i18n sync. Locale bundles are
the authenticated ui:i18n:sync output (fallbacks=0); GatewayModels.swift
is protocol:gen:swift output for the new TaskSummary fields.
* chore(ui): refresh i18n raw-copy baseline after rebase
* fix(cron): delete successful on-exit deleteAfterRun jobs
deleteAfterRun finalization was restricted to at schedules, so a successful
on-exit job with deleteAfterRun=true was retained as a disabled job instead
of being removed (the gateway exit watcher durably disables on-exit jobs
BEFORE firing as replay protection, and no later transition deleted them).
Successful delete-after-run now covers every one-shot schedule kind; the
pre-fire disable ordering is unchanged. Regression covers deletion on ok,
retention without the flag, and retention on failure through the same force
path the exit watcher fires.
Fixes#104518
* test(cron): prove durable on-exit finalization
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(agents): lease managed worktrees across a run with an exclusive removal claim
A resumed session's managed worktree could be removed mid-run by idle GC, a
separate remover, or a manual worktrees.gc, destroying in-flight work; a session
whose worktree was already removed was admitted anyway and ran without its
checkout.
Runs now take a shared run-lease and removal takes an exclusive claim, both held
as rows in the shared state_leases table so admission and removal serialize
through one BEGIN IMMEDIATE transaction. Lease rows carry an opaque token plus
pid and process start time; dead or reused-pid owners are pruned inside the same
transaction. Removal claims before any cleanliness or snapshot work and rejects
while a live run lease exists; a run releases only its own token. The worktree id
is resolved from the session binding or by containment over both the run cwd and
workspace dir, so a workspace-only child still leases. The git worktree lock is
retained as a secondary raw-Git guard and refcounted per process so a parent and
a same-process child do not unlock each other.
* fix(agents): serialize worktree git lock cleanup
* test(agents): seed worktree races in sqlite
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(agents): keep failover detail truncation UTF-16 safe
Replace raw .slice(0, MAX_FAILOVER_DETAIL_CHARS) with truncateUtf16Safe
so provider error details containing emoji or other non-BMP characters
are not split at a surrogate-pair boundary before failover classification.
Adds a regression test via extractFailoverSignalDetails.
* test(agents): tighten failover UTF-16 coverage
---------
Co-authored-by: chengzhichao-xydt <chengzhichao-xydt@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* perf(acp): maintain ledger byte budgets at insert time
estimateSqliteLedgerBytes ran a full SUM(length(...)) scan over every
replay event on every append, and once the 16MB budget was hit the trim
loop rescanned after each single-row delete (#100622). Event rows now
carry estimated_bytes computed at insert, session rows keep a running
footprint aggregate, budget checks sum over at most maxSessions rows,
and eviction deletes bounded batches with RETURNING so aggregates stay
exact. Additive columns backfill once via ensureAdditiveStateColumns.
Fixes#100622
* fix(acp): keep ledger footprint aggregates exact across key changes and conflicts
Session-key/cwd length changes now adjust the aggregate via SET
expressions that read the pre-update row, and the conflict-upsert
recomputes from surviving event rows instead of clobbering to row
overhead. Invariant test now rebinds a provisional key to a longer
canonical key mid-stream.
* refactor(sessions): migrate runtime storage to sqlite
* test(sessions): fix sqlite CI regressions
* test(sessions): align remaining sqlite fixtures
* fix(codex): require sqlite trajectory recorder
* test(sessions): align orphan recovery sqlite fixture
* test(sessions): align sqlite rebase fixtures
* fix(sessions): finish current-main integration of the sqlite flip
Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.
* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs
Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.
* feat(sessions): keep archived transcripts by default with zstd cold storage
Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.
* fix(state): converge agent DB migration lineages and bound database growth
Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.
Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.
* fix(codex): resolve the store path for thread-history import via the SDK
The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.
* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors
A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.
* fix(sessions): complete sqlite conflict translations
* test(sqlite): align hardening checks with maintenance
* test(sessions): inspect compressed transcript archives
* fix(tests): await session seeds and drop unused helpers flagged by CI lint
The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.
* test(sessions): type legacy proof events directly
* test(sessions): align hardening contracts
* perf(sessions): read usage transcript sizes from SQL aggregates
Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.
* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir
Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.
* test(agents): seed harness admission through sqlite
* fix(sqlite): close agent db on pragma setup failure
* fix(doctor): compact and retrofit incremental auto-vacuum after session import
The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(write): report actual UTF-8 byte count in success message
Replace string.length (UTF-16 code units) with Buffer.byteLength(content, 'utf8')
so the reported byte count matches the actual bytes written to disk.
For ASCII content the values are identical. For non-ASCII content (CJK, emoji),
string.length underreports — e.g. '你好' reports as 2 bytes but is actually 6.
* refactor(write): centralize success reporting
* test(write): avoid shadowing recovery fixture
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
