Commit Graph

49 Commits

Author SHA1 Message Date
Sally O'Malley
b81666ca6a Fix container image upgrade migrations before gateway readiness (#101881)
* run all 'openclaw upgrade' migrations with container image upgrades

Signed-off-by: sallyom <somalley@redhat.com>

* fix: block gateway startup on plugin repair warnings

Signed-off-by: sallyom <somalley@redhat.com>

---------

Signed-off-by: sallyom <somalley@redhat.com>
2026-07-08 14:19:05 -04:00
Alex Knight
bd4d4c0d31 fix(state): close agent-db and proxy-capture SQLite handles on exit; rebind stale proxy stores after shared-state close (#100827)
* fix(state): close cached SQLite state and agent databases on process exit

* refactor(state): pair SQLite exit-close with cache lifecycle and rebind stale proxy-capture stores

* fix(cli): register debug-proxy finalize before the state DB exit hook

* refactor(state): narrow to agent-db and proxy-capture exit lifecycle after #100691 landed shared-state close

---------

Co-authored-by: Alex Knight <15041791+amknight@users.noreply.github.com>
2026-07-08 21:48:18 +10:00
Peter Steinberger
a6768d9de5 feat(cron): event triggers — polled condition-watcher scripts via code mode (#101195)
* feat(cron): add headless code-mode driver and trigger-script evaluator

Part A of cron event triggers: runCodeModeScriptHeadless runs a script
to completion (exec/settle/resume, no snapshots, no session), plus the
cron-facing evaluator with per-job tool catalog cache, semaphore, 16KB
state cap, and closed failure taxonomy.

* fix(cron): correct trigger-script bootstrap flags and cache narrowing

* feat(cron): add event triggers (polled condition-watcher scripts)

Part B: trigger field on cron jobs gated by cron.triggers.enabled;
timer evaluates the script each due tick, quiet ticks leave no run
history, fired runs append the script message to the payload; once
semantics, min-interval floor, SQLite columns, RPC/CLI/agent-tool
surfaces, and docs.

* fix(cron): propagate triggerEval through startup catch-up outcomes

* fix(cron): honor cron staggering on quiet trigger ticks; fix trigger test types

* fix(cron): reject with Error reason in trigger-script abort test

* fix(cron): regenerate protocol/docs/snapshot artifacts and break madge cycle for trigger types

CI fixes for #101195: trigger evaluator result types move to the cron
types leaf (madge cycle), cron tool schema inventory gains trigger,
Swift protocol bindings + docs map + Linux prompt snapshots regenerated.

* fix(cron): drop underscore-dangle names in trigger code sync assert

* fix(cron): adopt registerHeadlessToolSearchCatalog after tool-search symbol localization

Upstream #101831 made registerToolSearchCatalog module-private; fold the
headless ref-only catalog registration into one public seam.
2026-07-07 19:12:38 +01:00
Jason (Json)
f7e962d6e3 feat(cli): add openclaw promos to discover and claim ClawHub promotional model offers (#100236)
* feat: add openclaw promos CLI for ClawHub promotional model offers

* fix: harden promos claim auth validation and sanitize remote promo text

* fix: enforce promo window client-side and validate slug contract

* fix: shell-safe model ref contract and explicit --api-key override

* fix: hold promotion aliases to the models alias contract

* docs: document argv-credential contract and env alternative for promos claim

* fix: enforce provider plugin enablement on the credential-reuse claim path

* fix: require plugin install before credential-reuse shortcut in promos claim

* fix: run runtime plugin repair on promo defaults and harden identifier parsing

* fix: distinct message for contract-invalid promo aliases

* fix(cli): validate promotion plugin contracts

* fix(cli): recheck promotion window before claim

* feat(cli): surface ClawHub promotions in models list via the hosted feed

Passive discovery for promotional model offers. A cadence-gated (24h),
fail-silent conditional GET of ClawHub's immutable promotions feed
snapshot (If-None-Match -> 304, short 2.5s timeout, unauthenticated so
CDN caches stay unfragmented) is cached in two new shared-state-DB
tables, fully separate from update_check_state. models list renders an
'Available via promotion' group for live offers whose models are not in
the user's configured set (including the zero-row fresh-install path),
tags claimed models promo / promo ended from provenance recorded at
claim time, and prints a one-time notice per newly seen offer; promos
list and claim mark offers as seen. Machine outputs stay clean, snapshot
sequence is monotonic against stale edges, and claims still revalidate
against the live API so the kill switch always wins.

* style: satisfy lint on promotions feed additions

no-useless-fallback-in-spread on the optional request headers and
no-map-spread in claim-provenance row mapping.

* fix(cli): harden promotions feed cache

* fix(cli): honor live promotion validity
2026-07-07 01:10:42 -06:00
Peter Steinberger
1a307be2e0 feat(skills): add lifecycle curator for workshop-created skills (#101214)
Tracks per-skill usage from the skill.used diagnostic event (trusted-only
delivery, file-scoped identity), sweeps workshop-created skills daily from
gateway maintenance (active -> stale 30d -> archived 90d, pinned bypass,
restore-only unarchive, files never touched), filters archived skills from
snapshots fail-open, reports workspace-scoped overlap candidates, and adds
openclaw skills curator CLI, additive gateway methods, and a warn-only
doctor finding. Zero new config keys; SQLite/Kysely state only.
2026-07-07 03:25:44 +01:00
Peter Steinberger
f53346944d feat: correlate native search outcomes in audit history (#98704)
* feat: correlate native search outcomes in audit history

Metadata-only audit ledger for agent runs and tool actions in the shared
state DB: stable event identity, closed action/status/error vocabularies,
one-way-hashed tool-call ids, never-inferred terminal outcomes for native
web-search (explicit completed/failed only; otherwise unknown), bounded
retention, audit.list gateway RPC and openclaw audit CLI. Squashed from
the 82-commit audit stack for replay onto current main.

* feat(audit): add audit.enabled config gate (default on)

The metadata-only audit ledger records by default: an audit trail enabled
only after an incident cannot explain the incident, and the rows are
strictly less sensitive than the transcripts every install already
stores. audit.enabled=false stops new writes at the gateway subscription
seam; audit.list and openclaw audit keep serving existing records until
they expire. Documented in the configuration reference, protocol page,
and CLI reference.

* fix: repair full-matrix CI findings after rebase

- break the dynamic-tools/dynamic-tool-execution import cycle by
  extracting resolveCodexToolAbortTerminalReason into a leaf module
- restore main's session-worktree protocol exports lost in the
  index.ts auto-merge
- register the audit event writer worker as a knip entry point
- docs table formatting; subagent wait-cancellation test scoped to its
  audit intent (outcome + timing) and advanced past main's new
  lifecycle-timeout retry grace
2026-07-06 12:30:12 +01:00
Peter Steinberger
62d5961b5d fix(ci): restore gateway architecture and lint gates (#100703)
* fix(ci): restore gateway architecture and lint gates

* fix(state): refresh gateway boot Kysely type
2026-07-06 08:19:04 +01:00
Ayaan Zaidi
a18708c5c1 feat(gateway): add persisted crash-loop breaker and fatal-config exit contract
Gateways that crash-loop under systemd/launchd previously flapped forever
with no persisted state and no supervisor signal. The gateway now records
every boot outcome in the shared state DB (gateway_boot_lifecycle); three
unclean boots within five minutes trip a breaker that boots the gateway in
safe mode: full control plane available, channel/provider auto-start
suppressed at the channel-manager seam (startup, config hot-reload,
secrets.reload) with manual channels.start override, one stability bundle
per trip. The breaker re-evaluates each boot and logs recovery when the
window drains. Slow shutdowns record forced_stop and never count as
crashes; /readyz stays ready and reports suppressed channels; the health
monitor treats suppressed accounts as expected-stopped. Fatal invalid-config
errors now exit 78 (EX_CONFIG) on both the startup and unhandled-rejection
paths, engaging the systemd unit's pre-existing RestartPreventExitStatus=78
so supervisors stop relaunching until the config is fixed.
2026-07-06 06:45:55 +00:00
Peter Steinberger
0acd851a3b feat(agents): add managed git worktree lifecycle (create/provision/snapshot/restore/GC) (#100535)
Centralized managed worktrees under <state-dir>/worktrees/<repo-fingerprint>/<name>
with branch-per-task (openclaw/<name>), .worktreeinclude provisioning, an optional
.openclaw/worktree-setup.sh repo hook, and a SQLite registry in the shared state DB.
Removal always snapshots the tree (untracked included, gitignored excluded) to
refs/openclaw/snapshots/<id>; restore rebuilds the branch at the original commit with
the snapshot content as uncommitted state. Lossless run-end cleanup, 7-day idle GC for
run-owned worktrees (manual exempt), orphan reconciliation, 30-day snapshot retention.
Surfaces: worktrees.* gateway RPC (operator.admin mutations), openclaw worktrees CLI,
Control UI page, plugin-SDK facade + Workboard kind:"worktree" materialization.

E2E-verified on Testbox: full create->work->remove->restore->gc lifecycle.
2026-07-06 05:24:58 +01:00
Peter Steinberger
fdc9aa82d7 feat(cron): declarative jobs with owner attribution and richer status (#100480)
Adds scope-local declaration-key convergence, immutable owner attribution, richer status projection, and delivery validation to canonical cron jobs.

Supersedes the #98727 routines registry approach; no parallel registry, RPC namespace, or CLI noun.
2026-07-06 03:35:26 +01:00
Yuval Dinodia
4b9c51c5c5 fix(cron): preserve delivery thread id type across SQLite round-trip (#98699)
* fix(cron): preserve delivery thread id type across SQLite reload

A numeric delivery threadId such as a Telegram forum topic id reloaded as
a string after a gateway restart because the split delivery_thread_id
column stores TEXT. resolveCronDeliveryPlan and channel delivery are type
sensitive, so the numeric topic id was forwarded as a string and dropped
the configured topic.

Source the delivery threadId type from the canonical job_json config on
read, falling back to the raw column text. Both freshly written rows and
legacy bare-text rows keep string versus number identity without any
column format change or migration.

* fix(cron): recover typed delivery thread id when split column is null

* fix(cron): persist delivery thread ID type

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-05 12:53:02 -07:00
Peter Steinberger
614e87cce1 chore: update dependencies (#100027) 2026-07-04 14:56:50 -04:00
Peter Lee
b9ae0a76fa test: prefer shared temp dir helpers in config, gateway, cron, crestodian, and state tests (#96711)
* fix: add temp dir cleanup to 14 P0 test files (Group A)

* fix: restore os import and fix fs.rmSync call in temp dir cleanup

* fix: use shared temp dir helpers in config, gateway, cron, crestodian, and state tests

* fix(test): rename underscore-prefixed temp dir vars and remove unused imports

* fix(test): remove unused path import in crestodian audit test

* fix(test): reorder expected plans to match buildVitestRunPlans output after rebase
2026-07-01 01:24:21 -07:00
Gio Della-Libera
d1b917120a Persist hosted catalog snapshots in state (#95964)
Merged via squash.

Prepared head SHA: 372ec63c99
Co-authored-by: giodl73-repo <235387111+giodl73-repo@users.noreply.github.com>
Co-authored-by: giodl73-repo <235387111+giodl73-repo@users.noreply.github.com>
Reviewed-by: @giodl73-repo
2026-06-27 19:01:22 -07:00
Cameron Beeley
9aea104cc8 fix(cron): stop stamping an unenforceable default toolsAllow cap on CLI runs
#91499 auto-stamps the creator's tool surface as a default toolsAllow cap
on agentTurn cron payloads whenever the creating session is tool-restricted
(a narrowing allow-policy or an explicit deny). CLI backends cannot enforce
a runtime toolsAllow — cli-runner/prepare.ts rejects any defined allow-list
— so every scheduled agentTurn that resolves to a CLI backend (e.g.
claude-cli) fails to start. This silently broke per-thread scheduled
continuations on CLI backends.

A CLI backend is not a runtime tool-policy boundary: it runs with its own
configured tool set, as the operator, on the local machine, and refuses a
runtime allow-list outright. An inherited default cap is therefore
unenforceable on a CLI backend. Decide at run time, where the backend is
known:

- Flag the default. capCronAgentTurnToolsAllow stamps toolsAllowIsDefault
  when it fills in the creator surface because the cron requested nothing
  (or a bare "*"). An explicit narrowing or empty allow-list is a real
  per-cron restriction and carries no flag.
- Drop only the default, only on CLI. The run-executor drops a flagged
  default in the CLI branch and lets the run proceed. An explicit per-cron
  restriction (no flag) is deliberately passed through, so prepare.ts still
  fails it closed and surfaces that the requested policy needs an embedded
  runtime. Embedded runs are untouched and keep the full cap enforced.
- Persist the flag. New nullable cron_jobs.payload_tools_allow_is_default
  column (additive ensureColumn migration + codec read/write) so the
  decision survives a gateway restart, plus toolsAllowIsDefault on the
  gateway-protocol agentTurn payload schema — the stamped payload is
  otherwise rejected by the contract's additionalProperties:false.
- Preserve the flag across updates. A no-toolsAllow update (reschedule,
  prompt edit) no longer carries the stored default forward as a literal
  value — that routed it through the explicit-narrowing branch, stripped the
  flag, and re-broke the job on CLI after the next restart. The default is
  re-derived (flag intact); an explicit restriction is still carried forward
  unflagged.

Net policy: on CLI only the unenforceable inherited default is relaxed;
explicit per-cron restrictions still fail closed; embedded backends are
unchanged.

Tests: run-executor drops the flagged default but propagates an explicit
restriction on CLI; cron-tool stamps/clears the flag across create and
update and preserves it across a no-toolsAllow update; store round-trips the
flag (and its absence) through SQLite.

Not covered: agentTurn crons created during the regression window carry a
flagless toolsAllow and remain fail-closed on CLI until recreated or updated
with an explicit toolsAllow.
2026-06-24 06:26:52 -07:00
Vincent Koc
858fd2c5a2 refactor(sqlite): share user version probe 2026-06-23 03:19:45 +08:00
Vincent Koc
179eb15554 chore(deadcode): remove unused state path readers 2026-06-22 04:50:54 +08:00
Vincent Koc
1609365b3e test(state): canonicalize sqlite volume assertions 2026-06-19 19:45:40 +02:00
Josh Lehman
9cfd1cd287 fix(memory): key index sources by path and source 2026-06-19 01:20:06 +08:00
Vincent Koc
88bc08c124 refactor(memory): canonicalize agent database tables 2026-06-19 01:20:06 +08:00
Vincent Koc
f324f7e281 refactor(memory): use the per-agent sqlite database 2026-06-19 01:20:06 +08:00
Vincent Koc
45f9086d29 refactor(state): remove unused audit writers 2026-06-17 20:52:28 +08:00
Vincent Koc
4e2351dd4d refactor(runtime): remove unused internal wrappers 2026-06-17 20:11:16 +08:00
Vincent Koc
aeb5b794c9 fix(sqlite): tolerate unsupported private modes 2026-06-16 09:22:48 +02:00
Vincent Koc
e51c0c8cea fix(sqlite): include rollback journals in security paths 2026-06-16 09:22:48 +02:00
Vincent Koc
ea346f4361 fix(sqlite): close databases after failed initialization 2026-06-16 03:00:23 +02:00
Peter Steinberger
0bbac63d00 fix(tasks): migrate legacy agent attribution 2026-06-15 07:09:01 -07:00
Peter Steinberger
1fef20c96b fix(tasks): preserve requester agent attribution 2026-06-15 07:09:01 -07:00
Vincent Koc
b470316fc0 fix(state): harden sqlite path caching
Resolve explicit relative SQLite DB paths before caching handles and centralize durable SQLite connection pragmas so busy_timeout is applied before WAL/NFS negotiation.
2026-06-15 01:04:35 +08:00
Yzx
5b21a0337b fix(state): avoid sqlite wal on nfs state volumes
Detects NFS-backed SQLite database paths in the shared WAL helper and uses rollback journaling for those paths while preserving WAL/checkpoint maintenance on local filesystems. The NFS path now verifies SQLite's effective journal mode before disabling WAL maintenance, and core/memory/proxy-capture callers pass database path context into the centralized helper.

Fixes #90491.

Proof: local focused Vitest/format/lint; autoreview clean after fixing the journal-mode verification finding; Crabbox AWS focused test run `run_2ea7014350da`; Crabbox AWS changed gate `run_c828bbfe7d23`; exact-head GitHub CI green on `59674305ecd863d4815eec6098ccd3daab79ca4f`.
2026-06-14 03:32:13 +08:00
Galin Iliev
301213a05f test(sqlite): add state perf query plan harness
Adds a SQLite state query-plan regression test and smoke benchmark, wires the smoke artifact into source performance evidence, validates SQLite smoke output in the performance summary, and removes a retired ClawHub nav entry that broke docs link checks.

Fixes #91616
2026-06-11 14:49:26 -07:00
Truffle
13dce48321 fix(state): tolerate chmod-less state volumes
Keep state database startup working on filesystems without POSIX chmod support while failing closed for ordinary EPERM ownership failures. EPERM is tolerated only for already-private targets or when a disposable same-directory probe confirms chmod is unsupported.\n\nFixes #91919
2026-06-11 08:20:42 +09:00
Vincent Koc
7f1d82ab25 revert(sessions): defer session metadata sqlite
Reverts 538d36eaaa while preserving subsequent main changes. The beta-only SQLite downgrade rescue and reverse migration remain excluded.
2026-06-10 16:34:06 +09:00
Peter Steinberger
538d36eaaa refactor: move session metadata to SQLite (#91322)
* refactor: move session metadata to sqlite

* test: seed session stores with sqlite fixtures

* test: seed remaining session stores with sqlite fixtures

* fix: stabilize sqlite session cache freshness

* test: seed cli transcript metadata in sqlite
2026-06-07 23:17:35 -07:00
Peter Steinberger
c2d825ae53 fix: migrate legacy agent registry schema via doctor
Move the shipped legacy shared-state agent database registry repair into doctor. Runtime now fails fast with a doctor repair hint when the old primary-key shape remains.
2026-06-07 02:23:51 -07:00
Peter Steinberger
b2e320dfb1 docs: document support test files 2026-06-04 20:31:55 -04:00
Peter Steinberger
e16ac04330 refactor(auth): store auth profiles in sqlite (#89102) 2026-06-03 16:14:15 -07:00
Peter Steinberger
0b8aabe864 docs: document auth profile failure policy contract (#89613)
* docs: document markdown marker renderer

* docs: document rendered markdown chunking

* docs: document markdown text chunking

* docs: document shared text chunking

* docs: document plugin text chunking exports

* docs: document avatar policy constants

* docs: document node match candidates

* docs: document scoped expiring id cache

* docs: document runtime import normalization

* docs: document string sample summaries

* docs: document session usage timeseries types

* docs: document session usage response types

* docs: document manifest frontmatter shapes

* docs: document channel route input metadata

* docs: document pair loop guard settings

* docs: document migration config patch helpers

* docs: document api provider registry

* docs: document tool call repair payloads

* docs: document plugin tool payload helpers

* docs: document lazy promise loader

* docs: document store writer queue state

* docs: document thread binding lifecycle

* docs: document concurrency helper contract

* docs: document gateway client info contract

* docs: document delivery context contracts

* docs: document secret ref defaults contract

* docs: document command gating contract

* docs: document avatar policy contract

* docs: document node match policy

* docs: document message channel normalization

* docs: document boolean parsing contract

* docs: document zod parse helpers

* docs: document direct dm guard policy

* docs: document fixed window limiter contract

* docs: document node presence event contract

* docs: document secret normalization contract

* docs: document progress draft line removal

* docs: document usage formatting contracts

* docs: document agent run status contract

* docs: document runtime import helpers

* docs: document provider utility ownership

* docs: document invalid config helpers

* docs: document json compat parser

* docs: document channel config metadata ownership

* docs: document channel logging helpers

* docs: document sender identity validation ownership

* docs: document string sampling helper

* docs: document global singleton helpers

* docs: document transcript tool helpers

* docs: document exec safe-bin normalization

* docs: document reaction level resolver

* docs: document account snapshot redaction boundary

* docs: document messaging target helpers

* docs: document thread binding messages

* docs: document conversation binding context

* docs: document conversation resolution helper

* docs: document owner display secret retention

* docs: document provider request config types

* docs: document skills config types

* docs: document memory config types

* docs: document imessage config types

* docs: document crestodian config types

* docs: document tools config policies

* docs: document shared config base types

* docs: document channel config contracts

* docs: document openclaw config state types

* docs: document model config contracts

* docs: document shared agent config types

* docs: document agent defaults config types

* docs: document secret input contracts

* docs: document auth config contracts

* docs: document gateway config contracts

* docs: document tool call stream repair contracts

* docs: document memory host facades

* docs: document llm core contracts

* docs: document markdown core contracts

* docs: document gateway connect error contracts

* docs: document gateway protocol primitives

* docs: document gateway frame schemas

* docs: document gateway device schemas

* docs: document gateway environment schemas

* docs: document gateway push schemas

* docs: document gateway plugin schemas

* docs: document gateway artifact schemas

* docs: document gateway command schemas

* docs: document gateway task schemas

* docs: document gateway exec approval schemas

* docs: document gateway secret schemas

* docs: document gateway config schemas

* docs: document gateway snapshot schemas

* docs: document gateway chat schemas

* docs: document gateway wizard schemas

* docs: document gateway node schemas

* docs: document gateway plugin approval schemas

* docs: document gateway talk schemas

* docs: document gateway agent schemas

* docs: document gateway session schemas

* docs: document gateway cron schemas

* docs: document gateway agent model skill schemas

* docs: document gateway skill proposal tool schemas

* docs: document gateway protocol registry

* docs: document gateway channel status schemas

* docs: document gateway schema regression tests

* docs: document gateway schema barrel

* docs: document gateway validator tests

* docs: document gateway primitive push tests

* docs: document gateway contract tests

* docs: document native protocol guard

* docs: document channel schema tests

* docs: document gateway protocol smoke tests

* docs: document gateway protocol entrypoint

* docs: document gateway protocol type exports

* docs: document gateway error codes

* docs: document protocol schema registry

* docs: document talk audio codec

* docs: document talk activation names

* docs: document talk consult questions

* docs: document talk consult tool

* docs: document talk run control contracts

* docs: document talk run control adapter

* docs: document talkback consult queue

* docs: document talk consult transcript guard

* docs: document talk fast context runtime

* docs: document forced talk consult coordinator

* docs: document talk output activity tracker

* docs: document talk event metrics

* docs: document talk diagnostics

* docs: document talk observability hook

* docs: document talk provider resolver

* docs: document talk provider registry

* docs: document talk runtime primitives

* docs: document talk consult controller logs

* docs: document channel identity helpers

* docs: document channel account allowlist helpers

* docs: document channel metadata draft controls

* docs: document channel ingress policy

* docs: document channel sender access gates

* docs: document channel catalog message contracts

* docs: document channel account plugin helpers

* docs: document configured binding helpers

* docs: document channel acp approval config helpers

* docs: document channel bundled config write helpers

* docs: document channel plugin utility contracts

* docs: document channel config access helpers

* docs: document channel message action helpers

* docs: document channel outbound runtime helpers

* docs: document channel pairing promotion helpers

* docs: document channel registry helpers

* docs: document channel setup wizard helpers

* docs: document channel lifecycle status helpers

* docs: document channel target thread helpers

* docs: document channel session binding helpers

* docs: document channel package module probes

* docs: document channel setup wizard contracts

* docs: document channel plugin API barrels

* docs: document channel contract test helpers

* docs: document channel core helpers

* docs: document small core facades

* docs: document provider runtime helpers

* docs: document persistence and realtime helpers

* docs: document mcp and state helpers

* docs: document tool planner contracts

* docs: document music generation runtime

* docs: document crestodian command flow

* docs: document utility helpers

* docs: document node host helpers

* docs: document transcript contracts

* docs: document trajectory export contracts

* docs: document image generation contracts

* docs: document routing helper contracts

* docs: document session helper contracts

* docs: document video generation contracts

* docs: document model catalog contracts

* docs: document proxy capture contracts

* docs: document status rendering contracts

* docs: document test helper contracts

* docs: document wizard setup contracts

* docs: document process contracts

* docs: document memory host sdk contracts

* docs: document tts contracts

* docs: document secrets runtime contracts

* docs: document shared helper contracts

* docs: document hook runtime contracts

* docs: document security audit contracts

* docs: document flow contracts

* docs: document media understanding contracts

* docs: document tui contracts

* docs: document logging contracts

* docs: document llm contracts

* docs: document cron contracts

* docs: document daemon contracts

* docs: document task contracts

* docs: document acp contracts

* docs: document test utility contracts

* docs: document skill contracts

* docs: document config contracts

* docs: document outbound infra contracts

* docs: document command analysis contracts

* docs: document provider usage infra contracts

* docs: document file safety infra contracts

* docs: document exec approval infra contracts

* docs: document gateway runtime infra contracts

* docs: document infra utility contracts

* docs: document infra queue storage contracts

* docs: document heartbeat infra contracts

* docs: document remaining infra contracts

* docs: document gateway auth contracts

* docs: document gateway display helpers

* docs: document gateway http helpers

* docs: document gateway node helpers

* docs: document gateway mcp helpers

* docs: document gateway support helpers

* docs: document gateway server runtime helpers

* docs: document gateway runtime bootstrap helpers

* docs: document gateway session events

* docs: document gateway utility helpers

* docs: document gateway talk helpers

* docs: document gateway helper contracts

* docs: document gateway server method helpers

* docs: document gateway server auth helpers

* docs: document gateway server tests

* docs: document gateway test helpers

* docs: document gateway node tests

* docs: document gateway channel tests

* docs: document gateway session tests

* docs: document gateway server startup tests

* docs: document gateway tool test helpers

* docs: document gateway server test helpers

* docs: document gateway server method tests

* docs: document remaining gateway tests

* docs: document plugin sdk public subpaths

* docs: document plugin sdk runtime helpers

* docs: document plugin sdk memory provider helpers

* docs: document plugin sdk runtime facades

* docs: document plugin sdk command approval helpers

* docs: document plugin sdk runtime types

* docs: document plugin sdk browser account helpers

* docs: document plugin sdk media memory helpers

* docs: document plugin sdk core tests

* docs: document plugin sdk contract helpers

* docs: document plugin sdk test helpers

* docs: document remaining plugin sdk tests

* docs: document cli utility helpers

* docs: document cli runtime helpers

* docs: document cli command registration helpers

* docs: document node cli helpers

* docs: document cli program registration

* docs: document message cli registration

* docs: document daemon cli helpers

* docs: document cli route parsers
2026-06-03 15:20:39 -07:00
Peter Steinberger
732d6972d7 fix: repair model provider edge cases
Repairs a batch of narrow model/provider edge cases:

- honor OpenAI and Anthropic base URL environment overrides when provider config does not set an explicit base URL
- preserve OpenRouter Anthropic cache retention while stripping unsupported transport options
- allow apply_patch for non-OpenAI providers when the tool config otherwise permits it
- prune stale same-provider model selections from configure/model picker state
- expose GitHub Copilot bundled thinking policy metadata to offline/provider-policy lookups
- repair additive SQLite shared-state upgrades for existing databases
- keep same-size rotated log readers from reusing stale content in CI tooling

Proof:

- GitHub PR checks green on exact head 46514909b0
- Crabbox delegated Blacksmith Testbox tbx_01kt3em5r9vd7g0bnykrff6jdk exited 0
- Focused local Vitest/oxlint/format proof recorded in PR body and land-ready comment

Fixes #80347.
Fixes #88357.
Fixes #45269.
Supersedes #74427, #74432, #79370, #79894, #80366, and #88359.
2026-06-02 02:35:12 -04:00
Peter Steinberger
b0679d1f13 refactor(channels): store inbound queues in SQLite 2026-05-31 21:15:29 +01:00
Peter Steinberger
db40fde88c fix: persist ACP metadata in SQLite (#88724)
* fix: persist acp metadata in sqlite

* test: align session store acp expectations
2026-05-31 18:38:51 +01:00
Peter Steinberger
a84819a639 refactor(cron): keep runtime on canonical sqlite rows 2026-05-31 16:02:06 +01:00
Peter Steinberger
a825b5576b refactor: simplify sqlite cron persistence 2026-05-30 22:11:17 +01:00
Peter Steinberger
005da57957 Move cron persistence to SQLite (#88285)
* refactor: move cron persistence to sqlite

* fix: repair sqlite cron migration regressions

* fix: move cron legacy migration to doctor

* test: align cron sqlite migration fixtures

* test: fix cron sqlite rebase gates

* test: align cron sqlite runtime tests

* test: fix doctor e2e migration mock

* test: fix doctor shard e2e isolation

* test: fix infra child-process mocks
2026-05-30 21:03:41 +01:00
Peter Steinberger
fc50f949d4 Add per-agent SQLite cache store (#88349)
* feat: add per-agent sqlite cache store

* fix: preserve sqlite cache adapter scope

* chore: mark sqlite cache scaffold intentional
2026-05-30 17:00:24 +01:00
Peter Steinberger
d115fb4cf9 refactor: move task state to shared sqlite
Move task run, delivery, and flow registry persistence onto the shared OpenClaw state SQLite database.

Summary:
- Store task runs, delivery state, and flow runs in state/openclaw.sqlite via the generated Kysely schema.
- Migrate shipped task sidecars into the shared state DB and archive old sidecars, including invalid-config/read-only CLI paths.
- Keep startup migration lightweight for read-only status/tasks paths while still detecting known legacy state markers and custom session stores.

Verification:
- .agents/skills/autoreview/scripts/autoreview --mode local: clean after final fix
- pnpm test src/tasks/task-registry.store.test.ts src/tasks/task-flow-registry.store.test.ts src/commands/doctor-state-migrations.test.ts -- --reporter=verbose
- pnpm test src/commands/doctor-state-migrations.test.ts src/cli/program/config-guard.test.ts src/cli/route.test.ts src/cli/command-path-policy.test.ts -- --reporter=verbose
- pnpm test src/cli/program/config-guard.test.ts src/cli/route.test.ts src/cli/command-startup-policy.test.ts src/cli/command-path-policy.test.ts src/cli/command-execution-startup.test.ts -- --reporter=verbose
- pnpm test src/cli/program/config-guard.test.ts src/cli/argv.test.ts src/cli/route.test.ts src/commands/doctor-config-preflight.state-migration.test.ts -- --reporter=verbose
- pnpm test src/tasks/task-flow-registry.store.test.ts -- --reporter=verbose
- pnpm test test/scripts/lint-suppressions.test.ts -- --reporter=verbose
- pnpm db:kysely:check
- pnpm lint:kysely
- git diff --check HEAD
- pnpm test:startup:memory
- PR CI green on 2f7d76f0d5
2026-05-30 04:54:37 +02:00
Peter Steinberger
bc848b367f refactor: add shared sqlite state database
Adds the shared SQLite state database base, moves plugin keyed state into it with doctor migration coverage, and keeps generated Kysely guardrails aligned. Proof: focused SQLite/plugin-state tests, db:kysely:check, lint:kysely, architecture/dependency guards, autoreview, and PR CI all clean.
2026-05-30 00:52:23 +02:00
Peter Steinberger
694ca50e97 Revert "refactor: move runtime state to SQLite"
This reverts commit f91de52f0d.
2026-05-13 13:33:38 +01:00
Peter Steinberger
f91de52f0d refactor: move runtime state to SQLite
* refactor: remove stale file-backed shims

* fix: harden sqlite state ci boundaries

* refactor: store matrix idb snapshots in sqlite

* fix: satisfy rebased CI guardrails

* refactor: store current conversation bindings in sqlite table

* refactor: store tui last sessions in sqlite table

* refactor: reset sqlite schema history

* refactor: drop unshipped sqlite table migration

* refactor: remove plugin index file rollback

* refactor: drop unshipped sqlite sidecar migrations

* refactor: remove runtime commitments kv migration

* refactor: preserve kysely sync result types

* refactor: drop unshipped sqlite schema migration table

* test: keep session usage coverage sqlite-backed

* refactor: keep sqlite migration doctor-only

* refactor: isolate device legacy imports

* refactor: isolate push voicewake legacy imports

* refactor: isolate remaining runtime legacy imports

* refactor: tighten sqlite migration guardrails

* test: cover sqlite persisted enum parsing

* refactor: isolate legacy update and tui imports

* refactor: tighten sqlite state ownership

* refactor: move legacy imports behind doctor

* refactor: remove legacy session row lookup

* refactor: canonicalize memory transcript locators

* refactor: drop transcript path scope fallbacks

* refactor: drop runtime legacy session delivery pruning

* refactor: store tts prefs only in sqlite

* refactor: remove cron store path runtime

* refactor: use cron sqlite store keys

* refactor: rename telegram message cache scope

* refactor: read memory dreaming status from sqlite

* refactor: rename cron status store key

* refactor: stop remembering transcript file paths

* test: use sqlite locators in agent fixtures

* refactor: remove file-shaped commitments and cron store surfaces

* refactor: keep compaction transcript handles out of session rows

* refactor: derive transcript handles from session identity

* refactor: derive runtime transcript handles

* refactor: remove gateway session locator reads

* refactor: remove transcript locator from session rows

* refactor: store raw stream diagnostics in sqlite

* refactor: remove file-shaped transcript rotation

* refactor: hide legacy trajectory paths from runtime

* refactor: remove runtime transcript file bridges

* refactor: repair database-first rebase fallout

* refactor: align tests with database-first state

* refactor: remove transcript file handoffs

* refactor: sync post-compaction memory by transcript scope

* refactor: run codex app-server sessions by id

* refactor: bind codex runtime state by session id

* refactor: pass memory transcripts by sqlite scope

* refactor: remove transcript locator cleanup leftovers

* test: remove stale transcript file fixtures

* refactor: remove transcript locator test helper

* test: make cron sqlite keys explicit

* test: remove cron runtime store paths

* test: remove stale session file fixtures

* test: use sqlite cron keys in diagnostics

* refactor: remove runtime delivery queue backfill

* test: drop fake export session file mocks

* refactor: rename acp session read failure flag

* refactor: rename acp row session key

* refactor: remove session store test seams

* refactor: move legacy session parser tests to doctor

* refactor: reindex managed memory in place

* refactor: drop stale session store wording

* refactor: rename session row helpers

* refactor: rename sqlite session entry modules

* refactor: remove transcript locator leftovers

* refactor: trim file-era audit wording

* refactor: clean managed media through sqlite

* fix: prefer explicit agent for exports

* fix: use prepared agent for session resets

* fix: canonicalize legacy codex binding import

* test: rename state cleanup helper

* docs: align backup docs with sqlite state

* refactor: drop legacy Pi usage auth fallback

* refactor: move legacy auth profile imports to doctor

* refactor: keep Pi model discovery auth in memory

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

* refactor: remove model json compatibility aliases

* refactor: store auth profiles in sqlite

* refactor: seed copied auth profiles in sqlite

* refactor: make auth profile runtime sqlite-addressed

* refactor: migrate hermes secrets into sqlite auth store

* refactor: move plugin install config migration to doctor

* refactor: rename plugin index audit checks

* test: drop auth file assumptions

* test: remove legacy transcript file assertions

* refactor: drop legacy cli session aliases

* refactor: store skill uploads in sqlite

* refactor: keep subagent attachments in sqlite vfs

* refactor: drop subagent attachment cleanup state

* refactor: move legacy session aliases to doctor

* refactor: require node 24 for sqlite state runtime

* refactor: move provider caches into sqlite state

* fix: harden virtual agent filesystem

* refactor: enforce database-first runtime state

* refactor: rename compaction transcript rotation setting

* test: clean sqlite refactor test types

* refactor: consolidate sqlite runtime state

* refactor: model session conversations in sqlite

* refactor: stop deriving cron delivery from session keys

* refactor: stop classifying sessions from key shape

* refactor: hydrate announce targets from typed delivery

* refactor: route heartbeat delivery from typed sqlite context

* refactor: tighten typed sqlite session routing

* refactor: remove session origin routing shadow

* refactor: drop session origin shadow fixtures

* perf: query sqlite vfs paths by prefix

* refactor: use typed conversation metadata for sessions

* refactor: prefer typed session routing metadata

* refactor: require typed session routing metadata

* refactor: resolve group tool policy from typed sessions

* refactor: delete dead session thread info bridge

* Show Codex subscription reset times in channel errors (#80456)

* feat(plugin-sdk): consolidate session workflow APIs

* fix(agents): allow read-only agent mount reads

* [codex] refresh plugin regression fixtures

* fix(agents): restore compaction gateway logs

* test: tighten gateway startup assertions

* Redact persisted secret-shaped payloads [AI] (#79006)

* test: tighten device pair notify assertions

* test: tighten hermes secret assertions

* test: assert matrix client error shapes

* test: assert config compat warnings

* fix(heartbeat): remap cron-run exec events to session keys (#80214)

* fix(codex): route btw through native side threads

* fix(auth): accept friendly OpenAI order for Codex profiles

* fix(codex): rotate auth profiles inside harness

* fix: keep browser status page probe within timeout

* test: assert agents add outputs

* test: pin cron read status

* fix(agents): avoid Pi resource discovery stalls

Co-authored-by: dataCenter430 <titan032000@gmail.com>

* fix: retire timed-out codex app-server clients

* test: tighten qa lab runtime assertions

* test: check security fix outputs

* test: verify extension runtime messages

* feat(wake): expose typed sessionKey on wake protocol + system event CLI

* fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790)

* test: guard talk consult call helper

* fix(codex): scale context engine projection (#80761)

* fix(codex): scale context engine projection

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* chore: align Codex projection changelog

* chore: realign Codex projection changelog

* fix: isolate Codex projection patch

---------

Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>

* refactor: move agent runtime state toward piless

* refactor: remove cron session reaper

* refactor: move session management to sqlite

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: remove stale file-backed shims

* test: harden kysely type coverage

# Conflicts:
#	.agents/skills/kysely-database-access/SKILL.md
#	src/infra/kysely-sync.types.test.ts
#	src/proxy-capture/store.sqlite.test.ts
#	src/state/openclaw-agent-db.test.ts
#	src/state/openclaw-state-db.test.ts

* refactor: remove cron store path runtime

* refactor: keep compaction transcript handles out of session rows

* refactor: derive embedded transcripts from sqlite identity

* refactor: remove embedded transcript locator handoff

* refactor: remove runtime transcript file bridges

* refactor: remove transcript file handoffs

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

# Conflicts:
#	docs/cli/secrets.md
#	docs/gateway/authentication.md
#	docs/gateway/secrets.md

* fix: keep oauth sibling sync sqlite-local

# Conflicts:
#	src/commands/onboard-auth.test.ts

* refactor: remove task session store maintenance

# Conflicts:
#	src/commands/tasks.ts

* refactor: keep diagnostics in state sqlite

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* Show Codex subscription reset times in channel errors (#80456)

* fix(codex): refresh subscription limit resets

* fix(codex): format reset times for channels

* Update CHANGELOG with latest changes and fixes

Updated CHANGELOG with recent fixes and improvements.

* fix(codex): keep command load failures on codex surface

* fix(codex): format account rate limits as rows

* fix(codex): summarize account limits as usage status

* fix(codex): simplify account limit status

* test: tighten subagent announce queue assertion

* test: tighten session delete lifecycle assertions

* test: tighten cron ops assertions

* fix: track cron execution milestones

* test: tighten hermes secret assertions

* test: assert matrix sync store payloads

* test: assert config compat warnings

* fix(codex): align btw side thread semantics

* fix(codex): honor codex fallback blocking

* fix(agents): avoid Pi resource discovery stalls

* test: tighten codex event assertions

* test: tighten cron assertions

* Fix Codex app-server OAuth harness auth

* refactor: move agent runtime state toward piless

* refactor: move device and push state to sqlite

* refactor: move runtime json state imports to doctor

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: clarify cron sqlite store keys

* refactor: remove stale file-backed shims

* refactor: bind codex runtime state by session id

* test: expect sqlite trajectory branch export

* refactor: rename session row helpers

* fix: keep legacy device identity import in doctor

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* build: align pi contract wrappers

* chore: repair database-first rebase

* refactor: remove session file test contracts

* test: update gateway session expectations

* refactor: stop routing from session compatibility shadows

* refactor: stop persisting session route shadows

* refactor: use typed delivery context in clients

* refactor: stop echoing session route shadows

* refactor: repair embedded runner rebase imports

# Conflicts:
#	src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts

* refactor: align pi contract imports

* refactor: satisfy kysely sync helper guard

* refactor: remove file transcript bridge remnants

* refactor: remove session locator compatibility

* refactor: remove session file test contracts

* refactor: keep rebase database-first clean

* refactor: remove session file assumptions from e2e

* docs: clarify database-first goal state

* test: remove legacy store markers from sqlite runtime tests

* refactor: remove legacy store assumptions from runtime seams

* refactor: align sqlite runtime helper seams

* test: update memory recall sqlite audit mock

* refactor: align database-first runtime type seams

* test: clarify doctor cron legacy store names

* fix: preserve sqlite session route projections

* test: fix copilot token cache test syntax

* docs: update database-first proof status

* test: align database-first test fixtures

* docs: update database-first proof status

* refactor: clean extension database-first drift

* test: align agent session route proof

* test: clarify doctor legacy path fixtures

* chore: clean database-first changed checks

* chore: repair database-first rebase markers

* build: allow baileys git subdependency

* chore: repair exp-vfs rebase drift

* chore: finish exp-vfs rebase cleanup

* chore: satisfy rebase lint drift

* chore: fix qqbot rebase type seam

* chore: fix rebase drift leftovers

* fix: keep auth profile oauth secrets out of sqlite

* fix: repair rebase drift tests

* test: stabilize pairing request ordering

* test: use source manifests in plugin contract checks

* fix: restore gateway session metadata after rebase

* fix: repair database-first rebase drift

* fix: clean up database-first rebase fallout

* test: stabilize line quick reply receipt time

* fix: repair extension rebase drift

* test: keep transcript redaction tests sqlite-backed

* fix: carry injected transcript redaction through sqlite

* chore: clean database branch rebase residue

* fix: repair database branch CI drift

* fix: repair database branch CI guard drift

* fix: stabilize oauth tls preflight test

* test: align database branch fast guards

* test: repair build artifact boundary guards

* chore: clean changelog rebase markers

---------

Co-authored-by: pashpashpash <nik@vault77.ai>
Co-authored-by: Eva <eva@100yen.org>
Co-authored-by: stainlu <stainlu@newtype-ai.org>
Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com>
Co-authored-by: Ruben Cuevas <hi@rubencu.com>
Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>
Co-authored-by: Shakker <shakkerdroid@gmail.com>
Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com>
Co-authored-by: dataCenter430 <titan032000@gmail.com>
Co-authored-by: Kaspre <kaspre@gmail.com>
Co-authored-by: pandadev66 <nova.full.stack@outlook.com>
Co-authored-by: Eva <admin@100yen.org>
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>
Co-authored-by: jeffjhunter <support@aipersonamethod.com>
2026-05-13 13:15:12 +01:00