Commit Graph

39 Commits

Author SHA1 Message Date
brokemac79
3cf296185f fix(docker): pin setup cli container paths 2026-05-13 14:13:57 +01:00
Peter Steinberger
694ca50e97 Revert "refactor: move runtime state to SQLite"
This reverts commit f91de52f0d.
2026-05-13 13:33:38 +01:00
Peter Steinberger
f91de52f0d refactor: move runtime state to SQLite
* refactor: remove stale file-backed shims

* fix: harden sqlite state ci boundaries

* refactor: store matrix idb snapshots in sqlite

* fix: satisfy rebased CI guardrails

* refactor: store current conversation bindings in sqlite table

* refactor: store tui last sessions in sqlite table

* refactor: reset sqlite schema history

* refactor: drop unshipped sqlite table migration

* refactor: remove plugin index file rollback

* refactor: drop unshipped sqlite sidecar migrations

* refactor: remove runtime commitments kv migration

* refactor: preserve kysely sync result types

* refactor: drop unshipped sqlite schema migration table

* test: keep session usage coverage sqlite-backed

* refactor: keep sqlite migration doctor-only

* refactor: isolate device legacy imports

* refactor: isolate push voicewake legacy imports

* refactor: isolate remaining runtime legacy imports

* refactor: tighten sqlite migration guardrails

* test: cover sqlite persisted enum parsing

* refactor: isolate legacy update and tui imports

* refactor: tighten sqlite state ownership

* refactor: move legacy imports behind doctor

* refactor: remove legacy session row lookup

* refactor: canonicalize memory transcript locators

* refactor: drop transcript path scope fallbacks

* refactor: drop runtime legacy session delivery pruning

* refactor: store tts prefs only in sqlite

* refactor: remove cron store path runtime

* refactor: use cron sqlite store keys

* refactor: rename telegram message cache scope

* refactor: read memory dreaming status from sqlite

* refactor: rename cron status store key

* refactor: stop remembering transcript file paths

* test: use sqlite locators in agent fixtures

* refactor: remove file-shaped commitments and cron store surfaces

* refactor: keep compaction transcript handles out of session rows

* refactor: derive transcript handles from session identity

* refactor: derive runtime transcript handles

* refactor: remove gateway session locator reads

* refactor: remove transcript locator from session rows

* refactor: store raw stream diagnostics in sqlite

* refactor: remove file-shaped transcript rotation

* refactor: hide legacy trajectory paths from runtime

* refactor: remove runtime transcript file bridges

* refactor: repair database-first rebase fallout

* refactor: align tests with database-first state

* refactor: remove transcript file handoffs

* refactor: sync post-compaction memory by transcript scope

* refactor: run codex app-server sessions by id

* refactor: bind codex runtime state by session id

* refactor: pass memory transcripts by sqlite scope

* refactor: remove transcript locator cleanup leftovers

* test: remove stale transcript file fixtures

* refactor: remove transcript locator test helper

* test: make cron sqlite keys explicit

* test: remove cron runtime store paths

* test: remove stale session file fixtures

* test: use sqlite cron keys in diagnostics

* refactor: remove runtime delivery queue backfill

* test: drop fake export session file mocks

* refactor: rename acp session read failure flag

* refactor: rename acp row session key

* refactor: remove session store test seams

* refactor: move legacy session parser tests to doctor

* refactor: reindex managed memory in place

* refactor: drop stale session store wording

* refactor: rename session row helpers

* refactor: rename sqlite session entry modules

* refactor: remove transcript locator leftovers

* refactor: trim file-era audit wording

* refactor: clean managed media through sqlite

* fix: prefer explicit agent for exports

* fix: use prepared agent for session resets

* fix: canonicalize legacy codex binding import

* test: rename state cleanup helper

* docs: align backup docs with sqlite state

* refactor: drop legacy Pi usage auth fallback

* refactor: move legacy auth profile imports to doctor

* refactor: keep Pi model discovery auth in memory

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

* refactor: remove model json compatibility aliases

* refactor: store auth profiles in sqlite

* refactor: seed copied auth profiles in sqlite

* refactor: make auth profile runtime sqlite-addressed

* refactor: migrate hermes secrets into sqlite auth store

* refactor: move plugin install config migration to doctor

* refactor: rename plugin index audit checks

* test: drop auth file assumptions

* test: remove legacy transcript file assertions

* refactor: drop legacy cli session aliases

* refactor: store skill uploads in sqlite

* refactor: keep subagent attachments in sqlite vfs

* refactor: drop subagent attachment cleanup state

* refactor: move legacy session aliases to doctor

* refactor: require node 24 for sqlite state runtime

* refactor: move provider caches into sqlite state

* fix: harden virtual agent filesystem

* refactor: enforce database-first runtime state

* refactor: rename compaction transcript rotation setting

* test: clean sqlite refactor test types

* refactor: consolidate sqlite runtime state

* refactor: model session conversations in sqlite

* refactor: stop deriving cron delivery from session keys

* refactor: stop classifying sessions from key shape

* refactor: hydrate announce targets from typed delivery

* refactor: route heartbeat delivery from typed sqlite context

* refactor: tighten typed sqlite session routing

* refactor: remove session origin routing shadow

* refactor: drop session origin shadow fixtures

* perf: query sqlite vfs paths by prefix

* refactor: use typed conversation metadata for sessions

* refactor: prefer typed session routing metadata

* refactor: require typed session routing metadata

* refactor: resolve group tool policy from typed sessions

* refactor: delete dead session thread info bridge

* Show Codex subscription reset times in channel errors (#80456)

* feat(plugin-sdk): consolidate session workflow APIs

* fix(agents): allow read-only agent mount reads

* [codex] refresh plugin regression fixtures

* fix(agents): restore compaction gateway logs

* test: tighten gateway startup assertions

* Redact persisted secret-shaped payloads [AI] (#79006)

* test: tighten device pair notify assertions

* test: tighten hermes secret assertions

* test: assert matrix client error shapes

* test: assert config compat warnings

* fix(heartbeat): remap cron-run exec events to session keys (#80214)

* fix(codex): route btw through native side threads

* fix(auth): accept friendly OpenAI order for Codex profiles

* fix(codex): rotate auth profiles inside harness

* fix: keep browser status page probe within timeout

* test: assert agents add outputs

* test: pin cron read status

* fix(agents): avoid Pi resource discovery stalls

Co-authored-by: dataCenter430 <titan032000@gmail.com>

* fix: retire timed-out codex app-server clients

* test: tighten qa lab runtime assertions

* test: check security fix outputs

* test: verify extension runtime messages

* feat(wake): expose typed sessionKey on wake protocol + system event CLI

* fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790)

* test: guard talk consult call helper

* fix(codex): scale context engine projection (#80761)

* fix(codex): scale context engine projection

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* chore: align Codex projection changelog

* chore: realign Codex projection changelog

* fix: isolate Codex projection patch

---------

Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>

* refactor: move agent runtime state toward piless

* refactor: remove cron session reaper

* refactor: move session management to sqlite

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: remove stale file-backed shims

* test: harden kysely type coverage

# Conflicts:
#	.agents/skills/kysely-database-access/SKILL.md
#	src/infra/kysely-sync.types.test.ts
#	src/proxy-capture/store.sqlite.test.ts
#	src/state/openclaw-agent-db.test.ts
#	src/state/openclaw-state-db.test.ts

* refactor: remove cron store path runtime

* refactor: keep compaction transcript handles out of session rows

* refactor: derive embedded transcripts from sqlite identity

* refactor: remove embedded transcript locator handoff

* refactor: remove runtime transcript file bridges

* refactor: remove transcript file handoffs

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

# Conflicts:
#	docs/cli/secrets.md
#	docs/gateway/authentication.md
#	docs/gateway/secrets.md

* fix: keep oauth sibling sync sqlite-local

# Conflicts:
#	src/commands/onboard-auth.test.ts

* refactor: remove task session store maintenance

# Conflicts:
#	src/commands/tasks.ts

* refactor: keep diagnostics in state sqlite

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* Show Codex subscription reset times in channel errors (#80456)

* fix(codex): refresh subscription limit resets

* fix(codex): format reset times for channels

* Update CHANGELOG with latest changes and fixes

Updated CHANGELOG with recent fixes and improvements.

* fix(codex): keep command load failures on codex surface

* fix(codex): format account rate limits as rows

* fix(codex): summarize account limits as usage status

* fix(codex): simplify account limit status

* test: tighten subagent announce queue assertion

* test: tighten session delete lifecycle assertions

* test: tighten cron ops assertions

* fix: track cron execution milestones

* test: tighten hermes secret assertions

* test: assert matrix sync store payloads

* test: assert config compat warnings

* fix(codex): align btw side thread semantics

* fix(codex): honor codex fallback blocking

* fix(agents): avoid Pi resource discovery stalls

* test: tighten codex event assertions

* test: tighten cron assertions

* Fix Codex app-server OAuth harness auth

* refactor: move agent runtime state toward piless

* refactor: move device and push state to sqlite

* refactor: move runtime json state imports to doctor

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: clarify cron sqlite store keys

* refactor: remove stale file-backed shims

* refactor: bind codex runtime state by session id

* test: expect sqlite trajectory branch export

* refactor: rename session row helpers

* fix: keep legacy device identity import in doctor

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* build: align pi contract wrappers

* chore: repair database-first rebase

* refactor: remove session file test contracts

* test: update gateway session expectations

* refactor: stop routing from session compatibility shadows

* refactor: stop persisting session route shadows

* refactor: use typed delivery context in clients

* refactor: stop echoing session route shadows

* refactor: repair embedded runner rebase imports

# Conflicts:
#	src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts

* refactor: align pi contract imports

* refactor: satisfy kysely sync helper guard

* refactor: remove file transcript bridge remnants

* refactor: remove session locator compatibility

* refactor: remove session file test contracts

* refactor: keep rebase database-first clean

* refactor: remove session file assumptions from e2e

* docs: clarify database-first goal state

* test: remove legacy store markers from sqlite runtime tests

* refactor: remove legacy store assumptions from runtime seams

* refactor: align sqlite runtime helper seams

* test: update memory recall sqlite audit mock

* refactor: align database-first runtime type seams

* test: clarify doctor cron legacy store names

* fix: preserve sqlite session route projections

* test: fix copilot token cache test syntax

* docs: update database-first proof status

* test: align database-first test fixtures

* docs: update database-first proof status

* refactor: clean extension database-first drift

* test: align agent session route proof

* test: clarify doctor legacy path fixtures

* chore: clean database-first changed checks

* chore: repair database-first rebase markers

* build: allow baileys git subdependency

* chore: repair exp-vfs rebase drift

* chore: finish exp-vfs rebase cleanup

* chore: satisfy rebase lint drift

* chore: fix qqbot rebase type seam

* chore: fix rebase drift leftovers

* fix: keep auth profile oauth secrets out of sqlite

* fix: repair rebase drift tests

* test: stabilize pairing request ordering

* test: use source manifests in plugin contract checks

* fix: restore gateway session metadata after rebase

* fix: repair database-first rebase drift

* fix: clean up database-first rebase fallout

* test: stabilize line quick reply receipt time

* fix: repair extension rebase drift

* test: keep transcript redaction tests sqlite-backed

* fix: carry injected transcript redaction through sqlite

* chore: clean database branch rebase residue

* fix: repair database branch CI drift

* fix: repair database branch CI guard drift

* fix: stabilize oauth tls preflight test

* test: align database branch fast guards

* test: repair build artifact boundary guards

* chore: clean changelog rebase markers

---------

Co-authored-by: pashpashpash <nik@vault77.ai>
Co-authored-by: Eva <eva@100yen.org>
Co-authored-by: stainlu <stainlu@newtype-ai.org>
Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com>
Co-authored-by: Ruben Cuevas <hi@rubencu.com>
Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>
Co-authored-by: Shakker <shakkerdroid@gmail.com>
Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com>
Co-authored-by: dataCenter430 <titan032000@gmail.com>
Co-authored-by: Kaspre <kaspre@gmail.com>
Co-authored-by: pandadev66 <nova.full.stack@outlook.com>
Co-authored-by: Eva <admin@100yen.org>
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>
Co-authored-by: jeffjhunter <support@aipersonamethod.com>
2026-05-13 13:15:12 +01:00
Ayaan Zaidi
c3e3146639 fix(docker): persist auth profile key mount 2026-05-12 16:35:53 +05:30
Shakker
8f79e34cbe test: verify install and runtime messages 2026-05-11 17:16:25 +01:00
Peter Steinberger
20837afb21 test: tighten docker setup missing path assertions 2026-05-09 13:20:59 +01:00
Shakker
5c9755d347 test: tighten e2e array assertions 2026-05-09 06:03:18 +01:00
Peter Steinberger
30e870e424 test: tighten e2e absence assertions 2026-05-09 02:02:28 +01:00
Peter Steinberger
359115f0d9 test: simplify docker setup log collectors 2026-05-08 23:42:28 +01:00
Peter Steinberger
dd271968df test: clarify docker setup line assertions 2026-05-08 10:58:51 +01:00
Peter Steinberger
9ef37d1907 test: tighten assertions and harness coverage 2026-05-08 05:28:12 +01:00
Shubhankar Tripathy
34f805a012 fix(docker): pin container-side workspace and config dirs in compose
Fixes #77436
2026-05-04 16:03:51 -07:00
Peter Steinberger
02c2160478 chore: clean up root clutter 2026-05-03 12:20:52 +01:00
Vincent Koc
b258c3fc65 fix(secretrefs): preserve exec resolver env 2026-05-02 21:21:51 -07:00
Peter Steinberger
ed97d62868 test: drop stale Docker plugin deps guard 2026-05-01 22:19:48 +01:00
Peter Steinberger
ed8f50f240 refactor: simplify plugin dependency handling
Simplify plugin installation and runtime loading around package-manager-owned dependencies, with Jiti reserved for local/TS fallback paths.

Also scans npm plugin install roots so hoisted transitive dependencies are covered by dependency denylist and node_modules symlink checks.
2026-05-01 21:32:22 +01:00
jinjim
490e6d6dc5 feat(docker): add OPENCLAW_SKIP_ONBOARDING env to skip onboarding during Docker setup (#55518)
Merged via squash.

Prepared head SHA: 2744ed8b53
Co-authored-by: jinjimz <201528812+jinjimz@users.noreply.github.com>
Co-authored-by: sallyom <11166065+sallyom@users.noreply.github.com>
Reviewed-by: @sallyom
2026-04-28 18:50:51 -04:00
Peter Steinberger
a3bbcf2792 fix(docker): keep plugin runtime deps off bind mounts 2026-04-28 09:37:59 +01:00
Peter Steinberger
6018f29dbf ci: keep docker bonjour setting automatic 2026-04-27 00:00:22 +01:00
Peter Steinberger
3a8961af0f test: copy docker build helper in setup e2e 2026-04-26 22:54:27 +01:00
Peter Steinberger
d1a5ea2024 fix(docker): disable bonjour by default for compose 2026-04-26 05:51:05 +01:00
Vincent Koc
e7de00c363 test(e2e): reuse suite temp root tracker in docker setup tests 2026-04-06 06:27:48 +01:00
Peter Steinberger
2176b68e50 fix: batch docker config writes 2026-04-05 22:31:11 +01:00
Vignesh Natarajan
6883f688e8 Docker setup: force BuildKit for local builds 2026-03-28 20:46:35 -07:00
Peter Steinberger
33e9e485b8 refactor: clarify docker setup cli phases 2026-03-24 16:46:12 -07:00
김채완
d21ecd7642 Tests: match any pre-start openclaw-cli run 2026-03-24 16:35:42 -07:00
김채완
3ce09bd071 Tests: reset docker setup log before isolated assert 2026-03-24 16:35:42 -07:00
김채완
81be4b45a6 Docker: seed localhost control UI origin 2026-03-24 16:35:42 -07:00
김채완
dbb806d257 Docker: avoid setup CLI namespace loop 2026-03-24 16:35:42 -07:00
Vincent Koc
46ccbacbd9 refactor(scripts): move container setup entrypoints 2026-03-19 13:40:26 -07:00
Peter Steinberger
5c120cb36c refactor: make setup the primary wizard surface 2026-03-15 22:01:04 -07:00
Peter Steinberger
94531fa237 test: reduce docker setup e2e duplication 2026-03-13 21:40:53 +00:00
Jealous
4d3a2f674b Docker: add OPENCLAW_TZ timezone support (#34119)
* Docker: add OPENCLAW_TZ timezone support

* fix: validate docker timezone names

* fix: support Docker timezone override (#34119) (thanks @Lanfei)

---------

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
2026-03-13 14:51:55 +05:30
Vincent Koc
3f3f66a5f7 Docker: trim runtime image payload (#40307)
* Docker: shrink runtime image payload

* Docker: add runtime pnpm opt-in

* Docker: collapse helper entrypoint chmod layers

* Docker: restore bundled pnpm runtime

* Update CHANGELOG.md
2026-03-08 16:07:04 -07:00
Vincent Koc
475b0cb49a Docker Setup: allowlist dotenv token fixtures 2026-03-07 16:27:00 -08:00
Peter Steinberger
ada4ee08d9 fix(docker): land #33097 from @chengzhichao-xydt
Landed from contributor PR #33097 by @chengzhichao-xydt.

Co-authored-by: Zhichao Cheng <cheng.zhichao@xydigit.com>
2026-03-07 23:41:57 +00:00
Vincent Koc
e4d80ed556 CI: restore main detect-secrets scan (#38438)
* Tests: stabilize detect-secrets fixtures

* Tests: fix rebased detect-secrets false positives

* Docs: keep snippets valid under detect-secrets

* Tests: finalize detect-secrets false-positive fixes

* Tests: reduce detect-secrets false positives

* Tests: keep detect-secrets pragmas inline

* Tests: remediate next detect-secrets batch

* Tests: tighten detect-secrets allowlists

* Tests: stabilize detect-secrets formatter drift
2026-03-07 10:06:35 -08:00
jamtujest
cb491dfde5 feat(docker): add opt-in sandbox support for Docker deployments (#29974)
* feat(docker): add opt-in sandbox support for Docker deployments

Enable Docker-based sandbox isolation via OPENCLAW_SANDBOX=1 env var
in docker-setup.sh. This is a prerequisite for agents.defaults.sandbox
to function in any Docker deployment (self-hosted, Hostinger, DigitalOcean).

Changes:
- Dockerfile: add OPENCLAW_INSTALL_DOCKER_CLI build arg (~50MB, opt-in)
- docker-compose.yml: add commented-out docker.sock mount with docs
- docker-setup.sh: auto-detect Docker socket, inject mount, detect GID,
  build sandbox image, configure sandbox defaults, add group_add

All changes are opt-in. Zero impact on existing deployments.

Usage: OPENCLAW_SANDBOX=1 ./docker-setup.sh

Closes #29933
Related: #7575, #7827, #28401, #10361, #12505, #28326

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address code review feedback on sandbox support

- Persist OPENCLAW_SANDBOX, DOCKER_GID, OPENCLAW_INSTALL_DOCKER_CLI
  to .env via upsert_env so group_add survives re-runs
- Show config set errors instead of swallowing them silently;
  report partial failure when sandbox config is incomplete
- Warn when Dockerfile.sandbox is missing but sandbox config
  is still applied (sandbox image won't exist)
- Fix non-canonical whitespace in apt sources.list entry
  by using printf instead of echo with line continuation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove `local` outside function and guard sandbox behind Docker CLI check

- Remove `local` keyword from top-level `sandbox_config_ok` assignment
  which caused script exit under `set -euo pipefail` (bash `local`
  outside a function is an error)
- Add Docker CLI prerequisite check for pre-built (non-local) images:
  runs `docker --version` inside the container and skips sandbox setup
  with a clear warning if the CLI is missing
- Split sandbox block so config is only applied after prerequisites pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: defer docker.sock mount until sandbox prerequisites pass

Move Docker socket mounting from the early setup phase (before image
build/pull) to a dedicated compose overlay created only after:
1. Docker CLI is verified inside the container image
2. /var/run/docker.sock exists on the host

Previously the socket was mounted optimistically at startup, leaving
the host Docker daemon exposed even when sandbox setup was later
skipped due to missing Docker CLI. Now the gateway starts without
the socket, and a docker-compose.sandbox.yml overlay is generated
only when all prerequisites pass. The gateway restart at the end of
sandbox setup picks up both the socket mount and sandbox config.

Also moves group_add from write_extra_compose() into the sandbox
overlay, keeping all sandbox-specific compose configuration together.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(docker): fix sandbox docs URL in setup output

* Docker: harden sandbox setup fallback behavior

* Tests: cover docker-setup sandbox edge paths

* Docker: roll back sandbox mode on partial config failure

* Tests: assert sandbox mode rollback on partial setup

* Docs: document Docker sandbox bootstrap env controls

* Changelog: credit Docker sandbox bootstrap hardening

* Update CHANGELOG.md

* Docker: verify Docker apt signing key fingerprint

* Docker: avoid sandbox overlay deps during policy writes

* Tests: assert no-deps sandbox rollback gateway recreate

* Docs: mention OPENCLAW_INSTALL_DOCKER_CLI in Docker env vars

---------

Co-authored-by: Jakub Karwowski <jakubkarwowski@Mac.lan>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-03-01 23:06:10 -08:00
Peter Steinberger
a13586619b test: move integration-heavy suites to e2e lane 2026-03-02 05:33:07 +00:00