Commit Graph

191 Commits

Author SHA1 Message Date
Peter Steinberger
694ca50e97 Revert "refactor: move runtime state to SQLite"
This reverts commit f91de52f0d.
2026-05-13 13:33:38 +01:00
Peter Steinberger
f91de52f0d refactor: move runtime state to SQLite
* refactor: remove stale file-backed shims

* fix: harden sqlite state ci boundaries

* refactor: store matrix idb snapshots in sqlite

* fix: satisfy rebased CI guardrails

* refactor: store current conversation bindings in sqlite table

* refactor: store tui last sessions in sqlite table

* refactor: reset sqlite schema history

* refactor: drop unshipped sqlite table migration

* refactor: remove plugin index file rollback

* refactor: drop unshipped sqlite sidecar migrations

* refactor: remove runtime commitments kv migration

* refactor: preserve kysely sync result types

* refactor: drop unshipped sqlite schema migration table

* test: keep session usage coverage sqlite-backed

* refactor: keep sqlite migration doctor-only

* refactor: isolate device legacy imports

* refactor: isolate push voicewake legacy imports

* refactor: isolate remaining runtime legacy imports

* refactor: tighten sqlite migration guardrails

* test: cover sqlite persisted enum parsing

* refactor: isolate legacy update and tui imports

* refactor: tighten sqlite state ownership

* refactor: move legacy imports behind doctor

* refactor: remove legacy session row lookup

* refactor: canonicalize memory transcript locators

* refactor: drop transcript path scope fallbacks

* refactor: drop runtime legacy session delivery pruning

* refactor: store tts prefs only in sqlite

* refactor: remove cron store path runtime

* refactor: use cron sqlite store keys

* refactor: rename telegram message cache scope

* refactor: read memory dreaming status from sqlite

* refactor: rename cron status store key

* refactor: stop remembering transcript file paths

* test: use sqlite locators in agent fixtures

* refactor: remove file-shaped commitments and cron store surfaces

* refactor: keep compaction transcript handles out of session rows

* refactor: derive transcript handles from session identity

* refactor: derive runtime transcript handles

* refactor: remove gateway session locator reads

* refactor: remove transcript locator from session rows

* refactor: store raw stream diagnostics in sqlite

* refactor: remove file-shaped transcript rotation

* refactor: hide legacy trajectory paths from runtime

* refactor: remove runtime transcript file bridges

* refactor: repair database-first rebase fallout

* refactor: align tests with database-first state

* refactor: remove transcript file handoffs

* refactor: sync post-compaction memory by transcript scope

* refactor: run codex app-server sessions by id

* refactor: bind codex runtime state by session id

* refactor: pass memory transcripts by sqlite scope

* refactor: remove transcript locator cleanup leftovers

* test: remove stale transcript file fixtures

* refactor: remove transcript locator test helper

* test: make cron sqlite keys explicit

* test: remove cron runtime store paths

* test: remove stale session file fixtures

* test: use sqlite cron keys in diagnostics

* refactor: remove runtime delivery queue backfill

* test: drop fake export session file mocks

* refactor: rename acp session read failure flag

* refactor: rename acp row session key

* refactor: remove session store test seams

* refactor: move legacy session parser tests to doctor

* refactor: reindex managed memory in place

* refactor: drop stale session store wording

* refactor: rename session row helpers

* refactor: rename sqlite session entry modules

* refactor: remove transcript locator leftovers

* refactor: trim file-era audit wording

* refactor: clean managed media through sqlite

* fix: prefer explicit agent for exports

* fix: use prepared agent for session resets

* fix: canonicalize legacy codex binding import

* test: rename state cleanup helper

* docs: align backup docs with sqlite state

* refactor: drop legacy Pi usage auth fallback

* refactor: move legacy auth profile imports to doctor

* refactor: keep Pi model discovery auth in memory

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

* refactor: remove model json compatibility aliases

* refactor: store auth profiles in sqlite

* refactor: seed copied auth profiles in sqlite

* refactor: make auth profile runtime sqlite-addressed

* refactor: migrate hermes secrets into sqlite auth store

* refactor: move plugin install config migration to doctor

* refactor: rename plugin index audit checks

* test: drop auth file assumptions

* test: remove legacy transcript file assertions

* refactor: drop legacy cli session aliases

* refactor: store skill uploads in sqlite

* refactor: keep subagent attachments in sqlite vfs

* refactor: drop subagent attachment cleanup state

* refactor: move legacy session aliases to doctor

* refactor: require node 24 for sqlite state runtime

* refactor: move provider caches into sqlite state

* fix: harden virtual agent filesystem

* refactor: enforce database-first runtime state

* refactor: rename compaction transcript rotation setting

* test: clean sqlite refactor test types

* refactor: consolidate sqlite runtime state

* refactor: model session conversations in sqlite

* refactor: stop deriving cron delivery from session keys

* refactor: stop classifying sessions from key shape

* refactor: hydrate announce targets from typed delivery

* refactor: route heartbeat delivery from typed sqlite context

* refactor: tighten typed sqlite session routing

* refactor: remove session origin routing shadow

* refactor: drop session origin shadow fixtures

* perf: query sqlite vfs paths by prefix

* refactor: use typed conversation metadata for sessions

* refactor: prefer typed session routing metadata

* refactor: require typed session routing metadata

* refactor: resolve group tool policy from typed sessions

* refactor: delete dead session thread info bridge

* Show Codex subscription reset times in channel errors (#80456)

* feat(plugin-sdk): consolidate session workflow APIs

* fix(agents): allow read-only agent mount reads

* [codex] refresh plugin regression fixtures

* fix(agents): restore compaction gateway logs

* test: tighten gateway startup assertions

* Redact persisted secret-shaped payloads [AI] (#79006)

* test: tighten device pair notify assertions

* test: tighten hermes secret assertions

* test: assert matrix client error shapes

* test: assert config compat warnings

* fix(heartbeat): remap cron-run exec events to session keys (#80214)

* fix(codex): route btw through native side threads

* fix(auth): accept friendly OpenAI order for Codex profiles

* fix(codex): rotate auth profiles inside harness

* fix: keep browser status page probe within timeout

* test: assert agents add outputs

* test: pin cron read status

* fix(agents): avoid Pi resource discovery stalls

Co-authored-by: dataCenter430 <titan032000@gmail.com>

* fix: retire timed-out codex app-server clients

* test: tighten qa lab runtime assertions

* test: check security fix outputs

* test: verify extension runtime messages

* feat(wake): expose typed sessionKey on wake protocol + system event CLI

* fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790)

* test: guard talk consult call helper

* fix(codex): scale context engine projection (#80761)

* fix(codex): scale context engine projection

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* chore: align Codex projection changelog

* chore: realign Codex projection changelog

* fix: isolate Codex projection patch

---------

Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>

* refactor: move agent runtime state toward piless

* refactor: remove cron session reaper

* refactor: move session management to sqlite

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: remove stale file-backed shims

* test: harden kysely type coverage

# Conflicts:
#	.agents/skills/kysely-database-access/SKILL.md
#	src/infra/kysely-sync.types.test.ts
#	src/proxy-capture/store.sqlite.test.ts
#	src/state/openclaw-agent-db.test.ts
#	src/state/openclaw-state-db.test.ts

* refactor: remove cron store path runtime

* refactor: keep compaction transcript handles out of session rows

* refactor: derive embedded transcripts from sqlite identity

* refactor: remove embedded transcript locator handoff

* refactor: remove runtime transcript file bridges

* refactor: remove transcript file handoffs

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

# Conflicts:
#	docs/cli/secrets.md
#	docs/gateway/authentication.md
#	docs/gateway/secrets.md

* fix: keep oauth sibling sync sqlite-local

# Conflicts:
#	src/commands/onboard-auth.test.ts

* refactor: remove task session store maintenance

# Conflicts:
#	src/commands/tasks.ts

* refactor: keep diagnostics in state sqlite

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* Show Codex subscription reset times in channel errors (#80456)

* fix(codex): refresh subscription limit resets

* fix(codex): format reset times for channels

* Update CHANGELOG with latest changes and fixes

Updated CHANGELOG with recent fixes and improvements.

* fix(codex): keep command load failures on codex surface

* fix(codex): format account rate limits as rows

* fix(codex): summarize account limits as usage status

* fix(codex): simplify account limit status

* test: tighten subagent announce queue assertion

* test: tighten session delete lifecycle assertions

* test: tighten cron ops assertions

* fix: track cron execution milestones

* test: tighten hermes secret assertions

* test: assert matrix sync store payloads

* test: assert config compat warnings

* fix(codex): align btw side thread semantics

* fix(codex): honor codex fallback blocking

* fix(agents): avoid Pi resource discovery stalls

* test: tighten codex event assertions

* test: tighten cron assertions

* Fix Codex app-server OAuth harness auth

* refactor: move agent runtime state toward piless

* refactor: move device and push state to sqlite

* refactor: move runtime json state imports to doctor

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: clarify cron sqlite store keys

* refactor: remove stale file-backed shims

* refactor: bind codex runtime state by session id

* test: expect sqlite trajectory branch export

* refactor: rename session row helpers

* fix: keep legacy device identity import in doctor

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* build: align pi contract wrappers

* chore: repair database-first rebase

* refactor: remove session file test contracts

* test: update gateway session expectations

* refactor: stop routing from session compatibility shadows

* refactor: stop persisting session route shadows

* refactor: use typed delivery context in clients

* refactor: stop echoing session route shadows

* refactor: repair embedded runner rebase imports

# Conflicts:
#	src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts

* refactor: align pi contract imports

* refactor: satisfy kysely sync helper guard

* refactor: remove file transcript bridge remnants

* refactor: remove session locator compatibility

* refactor: remove session file test contracts

* refactor: keep rebase database-first clean

* refactor: remove session file assumptions from e2e

* docs: clarify database-first goal state

* test: remove legacy store markers from sqlite runtime tests

* refactor: remove legacy store assumptions from runtime seams

* refactor: align sqlite runtime helper seams

* test: update memory recall sqlite audit mock

* refactor: align database-first runtime type seams

* test: clarify doctor cron legacy store names

* fix: preserve sqlite session route projections

* test: fix copilot token cache test syntax

* docs: update database-first proof status

* test: align database-first test fixtures

* docs: update database-first proof status

* refactor: clean extension database-first drift

* test: align agent session route proof

* test: clarify doctor legacy path fixtures

* chore: clean database-first changed checks

* chore: repair database-first rebase markers

* build: allow baileys git subdependency

* chore: repair exp-vfs rebase drift

* chore: finish exp-vfs rebase cleanup

* chore: satisfy rebase lint drift

* chore: fix qqbot rebase type seam

* chore: fix rebase drift leftovers

* fix: keep auth profile oauth secrets out of sqlite

* fix: repair rebase drift tests

* test: stabilize pairing request ordering

* test: use source manifests in plugin contract checks

* fix: restore gateway session metadata after rebase

* fix: repair database-first rebase drift

* fix: clean up database-first rebase fallout

* test: stabilize line quick reply receipt time

* fix: repair extension rebase drift

* test: keep transcript redaction tests sqlite-backed

* fix: carry injected transcript redaction through sqlite

* chore: clean database branch rebase residue

* fix: repair database branch CI drift

* fix: repair database branch CI guard drift

* fix: stabilize oauth tls preflight test

* test: align database branch fast guards

* test: repair build artifact boundary guards

* chore: clean changelog rebase markers

---------

Co-authored-by: pashpashpash <nik@vault77.ai>
Co-authored-by: Eva <eva@100yen.org>
Co-authored-by: stainlu <stainlu@newtype-ai.org>
Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com>
Co-authored-by: Ruben Cuevas <hi@rubencu.com>
Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>
Co-authored-by: Shakker <shakkerdroid@gmail.com>
Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com>
Co-authored-by: dataCenter430 <titan032000@gmail.com>
Co-authored-by: Kaspre <kaspre@gmail.com>
Co-authored-by: pandadev66 <nova.full.stack@outlook.com>
Co-authored-by: Eva <admin@100yen.org>
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>
Co-authored-by: jeffjhunter <support@aipersonamethod.com>
2026-05-13 13:15:12 +01:00
Peter Steinberger
060768ef75 test: dedupe gateway run loop mock read 2026-05-13 10:33:35 +01:00
Peter Steinberger
f96cfeeb73 test: dedupe gateway cli mock reads 2026-05-13 00:02:45 +01:00
Shakker
7bb2d20303 test: remove duplicate gateway startup case 2026-05-12 11:09:49 +01:00
Peter Steinberger
3d93d127f3 test: guard gateway run mock calls 2026-05-12 07:28:41 +01:00
Peter Steinberger
dad240eecd test: guard gateway run loop mock calls 2026-05-12 07:20:43 +01:00
Peter Steinberger
5a432aacd7 test: guard gateway register mock calls 2026-05-12 07:19:14 +01:00
scoootscooob
294cdf973b fix: reread config on in-process gateway restart (#80161)
* fix: reread config on in-process gateway restart

* fix: refresh swift protocol model

(cherry picked from commit 537bcada37)
2026-05-12 06:21:09 +01:00
Shakker
5c65a22c8c test: count gateway lock release 2026-05-12 04:58:02 +01:00
Peter Steinberger
6b32f6f283 test: guard cli helper assertions 2026-05-11 20:11:18 +01:00
Shakker
1d5785ba85 test: assert gateway restart handoffs 2026-05-11 13:02:47 +01:00
Shakker
5860df1dd1 test: tighten gateway option collision assertions 2026-05-11 04:49:46 +01:00
brokemac79
a67753cc25 fix(agents): clarify subagent spawn wait guidance (#79051)
Summary:
- Replace the subagent spawn accepted-note yield guidance with push-based completion-event guidance.
- Cover the prompt with regression assertions that keep sessions_yield out of the note.
- Keep current rebased lint/type test helpers green.

Verification:
- pnpm lint
- pnpm check:test-types
- env -u OPENCLAW_TESTBOX -u OPENCLAW_TESTBOX_ID pnpm check:changed

Co-authored-by: brokemac79 <martin_cleary@yahoo.co.uk>
2026-05-10 17:11:32 +01:00
Peter Steinberger
dee3d58c8b test: clear gateway run option broad matchers 2026-05-10 17:08:40 +01:00
scoootscooob
e5fe9bdef0 fix: reread config on in-process gateway restart (#80161)
* fix: reread config on in-process gateway restart

* fix: refresh swift protocol model
2026-05-10 04:01:19 -04:00
Vincent Koc
be1c38e692 fix(cli): improve terminal error guidance 2026-05-10 10:28:47 +08:00
Vincent Koc
346e327586 fix(cli): guide onboarding option errors 2026-05-10 05:41:28 +08:00
Val Alexander
fa79e9754e fix(gateway): harden macOS update restart lifecycle
Summary:
- Clear stale SIGUSR1 restart state before rejected or externally allowed restart handling can leave an in-flight token stuck.
- Verify the live gateway version after macOS package-update service refreshes and skip redundant restarts when the refreshed LaunchAgent already serves the expected version.
- Set generated LaunchAgents to a 10s throttle plus 20s shutdown window and widen gateway bind retries around supervisor-owned restarts.

Fixes #79577. Refs #78699 and #60885.

Verification:
- pnpm test src/cli/gateway-cli/run-loop.test.ts src/infra/infra-runtime.test.ts
- pnpm test src/cli/update-cli.test.ts src/daemon/launchd.test.ts src/gateway/server/http-listen.test.ts
- pnpm exec oxfmt --check --threads=1 src/cli/gateway-cli/run-loop.ts src/cli/gateway-cli/run-loop.test.ts
- pnpm check:changed
- Crabbox/Blacksmith wrapper smoke passed focused tests plus pnpm check:changed: https://github.com/openclaw/openclaw/actions/runs/25595985603
- PR CI was green before upstream main advanced; the latest rebased heads hit unrelated broad lint failures also reproduced on current main CI (for example https://github.com/openclaw/openclaw/actions/runs/25598671666). No failing lint diagnostics referenced this gateway/update diff.
2026-05-09 05:21:17 -05:00
Shakker
6369349484 test: update gateway auth assertions 2026-05-09 02:26:32 +01:00
Vincent Koc
ad943ec30c fix(cli): guide auth and gateway setup errors 2026-05-09 09:20:16 +08:00
Peter Steinberger
627b0073f2 test: remove gateway restart delay wait 2026-05-06 07:02:27 +01:00
Shakker
acb0acd8dd fix: add gateway supervisor restart handoff 2026-05-05 08:38:00 +01:00
Peter Steinberger
d6917edc53 fix: preserve gateway watch trace overrides 2026-05-04 23:26:25 +01:00
Peter Steinberger
e84d4b27f4 feat: add gateway stall diagnostics 2026-05-04 23:26:25 +01:00
Satoshi F.
103cdd9d96 fix(gateway): add safe restart coordinator (#76923)
Add a safe restart coordinator that preflights active Gateway work before restart.

- expose gateway.restart.preflight and gateway.restart.request RPC methods
- add explicit openclaw gateway restart --safe / openclaw daemon restart --safe path
- narrow restart blockers to running non-ended tasks so queued records no longer block indefinitely
- keep existing restart behavior unchanged; --force remains the immediate override

Co-authored-by: NikolaFC <54186359+NikolaFC@users.noreply.github.com>
Co-authored-by: galiniliev <5711535+galiniliev@users.noreply.github.com>
2026-05-04 10:58:36 -07:00
Peter Steinberger
0ee52e9405 fix: keep config recovery in doctor 2026-05-03 18:04:50 +01:00
Vincent Koc
f6f8d74419 fix(gateway): expose restart drain controls 2026-05-02 14:43:59 -07:00
Peter Steinberger
5980040894 fix: avoid duplicate gateway config loads 2026-05-02 15:49:16 +01:00
Peter Steinberger
aebac43d97 fix: stop orphaned QA gateway children 2026-05-02 11:15:16 +01:00
Peter Steinberger
59fb9e5ca7 refactor: unify lazy import loaders 2026-05-02 10:55:59 +01:00
Peter Steinberger
960fabdaef refactor: trim cli helper exports 2026-05-02 07:36:54 +01:00
Peter Steinberger
ed8f50f240 refactor: simplify plugin dependency handling
Simplify plugin installation and runtime loading around package-manager-owned dependencies, with Jiti reserved for local/TS fallback paths.

Also scans npm plugin install roots so hoisted transitive dependencies are covered by dependency denylist and node_modules symlink checks.
2026-05-01 21:32:22 +01:00
Peter Steinberger
250376f885 fix: simplify bundled runtime dependency repair (#75183)
Summary:
- Merged fix: simplify bundled runtime dependency repair after ClawSweeper review.

ClawSweeper fixups:
- Included follow-up commit: fix: verify cached bundled runtime roots
- Included follow-up commit: refactor: simplify plugin runtime startup paths
- Included follow-up commit: refactor: trim plugin startup policy helpers
- Included follow-up commit: refactor: trust package manager runtime deps materialization
- Included follow-up commit: fix: narrow channel runtime deps skip policy
- Included follow-up commit: refactor: defer startup plugin runtime deps
- Ran the ClawSweeper repair loop before final review.

Validation:
- ClawSweeper review passed for head 04dc566534.
- Required merge gates passed before the squash merge.

Prepared head SHA: 04dc566534
Review: https://github.com/openclaw/openclaw/pull/75183#issuecomment-4358383786

Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: Shakker <shakkerdroid@gmail.com>
Co-authored-by: clawsweeper-repair <clawsweeper-repair@users.noreply.github.com>
2026-05-01 07:49:02 +00:00
Peter Steinberger
a379ac0562 fix: guard plugin HTTP calls in CI 2026-05-01 08:26:29 +01:00
Peter Steinberger
42d73fd955 refactor: remove dead private helpers 2026-05-01 06:55:26 +01:00
Alex Knight
aa9db998f7 fix(gateway): show config recovery validation details (#75081)
* fix(gateway): show config recovery validation details

* fix(cli): let gateway recovery run before proxy bootstrap
2026-05-01 07:14:33 +10:00
Peter Steinberger
46888f5afb test(gateway): align lock conflict success expectation 2026-04-30 16:39:12 +01:00
Peter Steinberger
9289a502bb fix(gateway): stop systemd EADDRINUSE restart loops 2026-04-30 16:30:56 +01:00
Peter Steinberger
616f24fd49 refactor(gateway): consolidate lifecycle lazy boundary (#74105)
* refactor(gateway): consolidate lifecycle lazy boundary

* test(gateway): cover quoted lifecycle imports
2026-04-29 16:45:51 +01:00
Vincent Koc
4eba70b532 docs(types): mark legacy aliases deprecated 2026-04-28 23:10:10 -07:00
Vincent Koc
1f41b8b44b fix(gateway): bound default restart deferral 2026-04-28 18:42:49 -07:00
Peter Steinberger
a0c850d188 test: stabilize gateway restart loop signals 2026-04-28 02:37:24 +01:00
Peter Steinberger
1cea04ba0f fix(ci): mock gateway run path helpers 2026-04-28 02:30:39 +01:00
Peter Steinberger
e651809084 perf: slim gateway startup imports 2026-04-28 02:26:27 +01:00
Peter Steinberger
24b45a038c fix(gateway): bound supervised lock recovery 2026-04-27 22:44:37 +01:00
Peter Steinberger
59faa023fe fix(gateway): unblock sidecar startup 2026-04-27 21:34:44 +01:00
Peter Steinberger
7f3f108521 refactor(config): migrate plugin config access 2026-04-27 12:35:58 +01:00
Samuel Rodda
6c252cc54c fix(update): require applied gateway restarts
Require Control UI updates to observe a real gateway process replacement, surface skipped/error update outcomes, and verify the running gateway version after restart.\n\nAdds update.status restart-sentinel plumbing, docs, generated protocol model updates, and changelog attribution.\n\nLocal verification:\n- pnpm test src/gateway/server-methods/update.test.ts src/cli/gateway-cli/run-loop.test.ts src/infra/restart-sentinel.test.ts src/infra/process-respawn.test.ts src/infra/update-runner.test.ts ui/src/ui/app-gateway.node.test.ts ui/src/ui/controllers/config.test.ts\n- git diff --check\n- pnpm exec oxfmt --check --threads=1 CHANGELOG.md docs/gateway/protocol.md docs/gateway/configuration.md docs/web/control-ui.md\n- pnpm docs:check-mdx
2026-04-27 04:07:43 -05:00
Peter Steinberger
1ea12fe3e2 fix: stage bundled plugin runtime deps safely 2026-04-27 06:16:26 +01:00