vultr/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-07-14 23:16:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
d287c9b414a03f665ea86016fd018b416ef07e94
openclaw/docs/start/setup.md
Josh Lehman 0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236) 
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00

6.4 KiB
Raw Blame History

summary, read_when, title
summary read_when title
Advanced setup and development workflows for OpenClaw
Setting up a new machine
You want "latest + greatest" without breaking your personal setup
Setup
If you are setting up for the first time, start with [Getting Started](/start/getting-started). For onboarding details, see [Onboarding (CLI)](/start/wizard).

TL;DR

Pick a setup workflow based on how often you want updates and whether you want to run the Gateway yourself:

  • Tailoring lives outside the repo: keep your config and workspace in ~/.openclaw/openclaw.json and ~/.openclaw/workspace/ so repo updates don't touch them.
  • Stable workflow (recommended for most): install the macOS app and let it run the bundled Gateway.
  • Bleeding edge workflow (dev): run the Gateway yourself via pnpm gateway:watch, then let the macOS app attach in Local mode.

Prereqs (from source)

  • Node 24 recommended (Node 22 LTS, currently 22.19+, still supported)
  • pnpm required for source checkouts. OpenClaw loads bundled plugins from the extensions/* pnpm workspace packages in dev mode, so root npm install does not prepare the full source tree.
  • Docker (optional; only for containerized setup/e2e - see Docker)

Tailoring strategy (so updates do not hurt)

If you want "100% tailored to me" and easy updates, keep your customization in:

  • Config: ~/.openclaw/openclaw.json (JSON/JSON5-ish)
  • Workspace: ~/.openclaw/workspace (skills, prompts, memories; make it a private git repo)

Bootstrap the config/workspace folders once, without running the full onboarding wizard:

openclaw setup --baseline

No global install yet? Run it from this repo instead:

pnpm openclaw setup --baseline

(Bare openclaw setup, without --baseline, is an alias for openclaw onboard and runs the full interactive wizard.)

Run the Gateway from this repo

After pnpm build, you can run the packaged CLI directly:

node openclaw.mjs gateway --port 18789 --verbose

Stable workflow (macOS app first)

  1. Install + launch OpenClaw.app (menu bar).
  2. Complete the onboarding/permissions checklist (TCC prompts).
  3. Ensure Gateway is Local and running (the app manages it).
  4. Link surfaces (example: WhatsApp):
openclaw channels login
  1. Sanity check:
openclaw health

If onboarding is not available in your build:

  • Run openclaw setup, then openclaw channels login, then start the Gateway manually (openclaw gateway).

Bleeding edge workflow (Gateway in a terminal)

Goal: work on the TypeScript Gateway, get hot reload, keep the macOS app UI attached.

0) (Optional) Run the macOS app from source too

If you also want the macOS app on the bleeding edge:

./scripts/restart-mac.sh

1) Start the dev Gateway

pnpm install
# First run only (or after resetting local OpenClaw config/workspace)
pnpm openclaw setup
pnpm gateway:watch

gateway:watch starts or restarts the Gateway watch process in a named tmux session (openclaw-gateway-watch-main) and auto-attaches from interactive terminals. Non-interactive shells stay detached and print tmux attach -t openclaw-gateway-watch-main; use OPENCLAW_GATEWAY_WATCH_ATTACH=0 pnpm gateway:watch to keep an interactive run detached, or pnpm gateway:watch:raw for foreground watch mode. The watcher reloads on relevant source, config, and bundled-plugin metadata changes. If the watched Gateway exits during startup, gateway:watch runs openclaw doctor --fix --non-interactive once and retries; set OPENCLAW_GATEWAY_WATCH_AUTO_DOCTOR=0 to disable that dev-only repair pass. pnpm gateway:watch does not rebuild dist/control-ui, so rerun pnpm ui:build after ui/ changes or use pnpm ui:dev while developing the Control UI.

2) Point the macOS app at your running Gateway

In OpenClaw.app:

  • Connection Mode: Local The app will attach to the running gateway on the configured port.

3) Verify

  • In-app Gateway status should read "Using existing gateway …"
  • Or via CLI:
openclaw health

Common footguns

  • Wrong port: Gateway WS defaults to ws://127.0.0.1:18789; keep app + CLI on the same port.
  • Where state lives:
    • Channel/provider state: ~/.openclaw/credentials/
    • Model auth profiles: ~/.openclaw/agents/<agentId>/agent/auth-profiles.json
    • Sessions and transcripts: ~/.openclaw/agents/<agentId>/agent/openclaw-agent.sqlite
    • Legacy/archive session artifacts: ~/.openclaw/agents/<agentId>/sessions/
    • Logs: /tmp/openclaw/

Credential storage map

Use this when debugging auth or deciding what to back up:

  • WhatsApp: ~/.openclaw/credentials/whatsapp/<accountId>/creds.json
  • Telegram bot token: config/env or channels.telegram.tokenFile (regular file only; symlinks rejected)
  • Discord bot token: config/env or SecretRef (env/file/exec providers)
  • Slack tokens: config/env (channels.slack.*)
  • Pairing allowlists:
    • ~/.openclaw/credentials/<channel>-allowFrom.json (default account)
    • ~/.openclaw/credentials/<channel>-<accountId>-allowFrom.json (non-default accounts)
  • Model auth profiles: ~/.openclaw/agents/<agentId>/agent/auth-profiles.json
  • File-backed secrets payload (optional): ~/.openclaw/secrets.json
  • Legacy OAuth import: ~/.openclaw/credentials/oauth.json More detail: Security.

Updating (without wrecking your setup)

  • Keep ~/.openclaw/workspace and ~/.openclaw/ as "your stuff"; don't put personal prompts/config into the openclaw repo.
  • Updating source: git pull + pnpm install + keep using pnpm gateway:watch.

Linux (systemd user service)

Linux installs use a systemd user service. By default, systemd stops user services on logout/idle, which kills the Gateway. Onboarding attempts to enable lingering for you (may prompt for sudo). If it's still off, run:

sudo loginctl enable-linger $USER

For always-on or multi-user servers, consider a system service instead of a user service (no lingering needed). See Gateway runbook for the systemd notes.

Related docs

Reference in New Issue View Git Blame Copy Permalink