mirror of
https://github.com/openclaw/openclaw.git
synced 2026-07-15 05:36:10 +00:00
0a8e3604ba
* refactor(sessions): migrate runtime storage to sqlite * test(sessions): fix sqlite CI regressions * test(sessions): align remaining sqlite fixtures * fix(codex): require sqlite trajectory recorder * test(sessions): align orphan recovery sqlite fixture * test(sessions): align sqlite rebase fixtures * fix(sessions): finish current-main integration of the sqlite flip Resolve the whole-store SDK removal across its owner boundary: drop the loadSessionStore re-export and the registry whole-store wrappers, wire hasTrackedActiveSessionRun into gateway chat, complete the preserveLockedHarnessIds cleanup contract, flip the codex thread-history import to storePath targets, and port remaining main-side tests from file-store helpers to session accessor reads. * chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain bump gets its own review, and regenerate docs_map, the plugin SDK API baseline, and the export-surface ratchet for the merged tree. * feat(sessions): keep archived transcripts by default with zstd cold storage Codex-style retention: deleting or resetting a session archives its transcript as a zstd-compressed JSONL artifact (plain when the runtime lacks node:zlib zstd) and keeps it until the disk budget evicts oldest first. resetArchiveRetention now governs both deleted and reset archives and defaults to keep; maxDiskBytes defaults to 2gb so retention stays bounded, with archives evicted before live sessions. The cron reaper follows the same knob instead of deleting archives on its own timer. * fix(state): converge agent DB migration lineages and bound database growth Merge coherence: run both structure-gated legacy memory-schema repairs (flip-lineage drop, main-lineage identity rebuild) before the flip migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all converge, and hoist foreign_keys=OFF outside the schema transaction where the pragma was silently ignored and the v1 sessions rebuild cascade-deleted session_entries. Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL maintenance releases freed pages in bounded passes (never a blocking full VACUUM), and doctor reports state/agent DB bloat from freelist stats. * fix(codex): resolve the store path for thread-history import via the SDK The supervision catalog passed the legacy sessionFile locator to the storePath-targeted transcript mirror; resolve the agent store path with the session-store SDK helper instead of a runtime-object seam so test fakes and headless callers need no extra surface. Drop the obsolete missing-session-id preprocessing case: sessions rows are NOT NULL on session_id and upsert repairs id-less patches at write time. * fix(sessions): fail safe on malformed disk-budget config and doctor stat errors A malformed explicit maxDiskBytes disables the budget instead of falling back to the destructive 2gb default the user never chose, and the doctor bloat check skips databases whose paths stat-fail instead of aborting doctor. * fix(sessions): complete sqlite conflict translations * test(sqlite): align hardening checks with maintenance * test(sessions): inspect compressed transcript archives * fix(tests): await session seeds and drop unused helpers flagged by CI lint The five unawaited writeSessionStoreSeed calls raced their SQLite seeds against the assertions, failing compact shards; the bloat probe drops a useless initializer and the merged tests drop now-unused helpers. * test(sessions): type legacy proof events directly * test(sessions): align hardening contracts * perf(sessions): read usage transcript sizes from SQL aggregates Usage/cost scans walked every session and materialized every transcript event just to re-stringify it for a byte estimate — the #86718 stall class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes in SQLite without loading a single row. * fix(sessions): re-root foreign-root transcript paths onto the current sessions dir Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry absolute sessionFile paths from the old root; the containment fallback kept those foreign paths, so migration read (and would archive) files in the original root and reported local copies missing. Re-root the canonical agents/<id>/sessions suffix onto the current dir when the file exists there; genuine cross-root layouts still fall through unchanged. * test(agents): seed harness admission through sqlite * fix(sqlite): close agent db on pragma setup failure * fix(doctor): compact and retrofit incremental auto-vacuum after session import The migration is the sanctioned offline window: post-import compact reclaims import churn and applies auto_vacuum=INCREMENTAL to databases created before the fresh-DB pragma existed, so runtime maintenance can release pages in bounded passes on every install. --------- Co-authored-by: Peter Steinberger <steipete@gmail.com>
OpenClaw docs i18n assets
This folder stores translation config for the source docs repo.
Generated locale trees and live translation memory now live in the publish repo:
- repo:
openclaw/docs - local checkout:
~/Projects/openclaw-docs
Source of truth
- English docs are authored in
openclaw/openclaw. - The source docs tree lives under
docs/. - The source repo no longer keeps committed generated locale trees such as
docs/zh-CN/**,docs/zh-TW/**,docs/ja-JP/**,docs/es/**,docs/pt-BR/**,docs/ko/**,docs/de/**,docs/fr/**,docs/hi/**,docs/ar/**,docs/it/**,docs/vi/**,docs/nl/**,docs/fa/**,docs/ru/**,docs/tr/**,docs/uk/**,docs/id/**,docs/pl/**, ordocs/th/**.
End-to-end flow
- Edit English docs in
openclaw/openclaw. - Push to
main. openclaw/openclaw/.github/workflows/docs-sync-publish.ymlmirrors the docs tree intoopenclaw/docs.- The sync script rewrites the publish
docs/docs.jsonso the generated locale picker blocks exist there even though they are no longer committed in the source repo. openclaw/docs/.github/workflows/translate-all.ymlwaits formainto settle, translates only stale or missing locale pages, and uploads per-locale artifacts.- The publish repo finalizer applies successful locale artifacts and pushes one aggregate
chore(i18n): refresh translationscommit. - A weekly
fullrun reconciles every locale/page path so flaky model failures are retried without making hot docs commits wait.
Why the split exists
- Keep generated locale output out of the main product repo.
- Keep Mintlify on a single published docs tree.
- Preserve the built-in language switcher for Mintlify-supported generated locales by letting the publish repo own generated locale trees.
- Keep generated Thai (
th) and Persian (fa) docs plus translation memory even though Mintlify does not currently accept those codes innavigation.languages. Their absence from the built-in docs language picker is a host limitation, not a failed translation run.
Locale visibility
- Control UI supports
en,zh-CN,zh-TW,pt-BR,de,es,ja-JP,ko,fr,hi,ar,it,vi,nl,fa,ru,tr,uk,id,pl, andth. - Docs translation workflows generate the same non-English locale set in
openclaw/docs. - The Mintlify docs language picker can expose only the locales accepted by Mintlify
navigation.languages; Russian (ru) and Hindi (hi) are now included in the publish configuration. - Do not treat locale visibility in generated
docs/docs.jsonas proof that translation artifacts exist. Verify each generated locale folder and its translation memory inopenclaw/docs.
Files in this folder
glossary.<lang>.json— preferred term mappings used as prompt guidance.zh-Hans-navigation.json— curated zh-Hans Mintlify locale navigation reinserted into the publish repo during sync.ar-navigation.json,de-navigation.json,es-navigation.json,fr-navigation.json,id-navigation.json,it-navigation.json,ja-navigation.json,ko-navigation.json,pl-navigation.json,pt-BR-navigation.json, andtr-navigation.json— starter locale metadata kept alongside the source repo, but the publish sync now clones the full English nav tree for clone-en locales so translated pages are visible in Mintlify without hand-maintaining per-locale nav JSON.<lang>.tm.jsonl— translation memory keyed by workflow + model + text hash.
In this repo, generated locale TM files such as docs/.i18n/zh-CN.tm.jsonl, docs/.i18n/zh-TW.tm.jsonl, docs/.i18n/ja-JP.tm.jsonl, docs/.i18n/es.tm.jsonl, docs/.i18n/pt-BR.tm.jsonl, docs/.i18n/ko.tm.jsonl, docs/.i18n/de.tm.jsonl, docs/.i18n/fr.tm.jsonl, docs/.i18n/ar.tm.jsonl, docs/.i18n/it.tm.jsonl, docs/.i18n/vi.tm.jsonl, docs/.i18n/nl.tm.jsonl, docs/.i18n/fa.tm.jsonl, docs/.i18n/tr.tm.jsonl, docs/.i18n/uk.tm.jsonl, docs/.i18n/id.tm.jsonl, docs/.i18n/pl.tm.jsonl, and docs/.i18n/th.tm.jsonl are intentionally no longer committed.
Glossary format
glossary.<lang>.json is an array of entries:
{
"source": "troubleshooting",
"target": "故障排除"
}
Fields:
source: English (or source) phrase to prefer.target: preferred translation output.
Translation mechanics
scripts/docs-i18nstill owns translation generation.- Doc mode writes
x-i18n.source_hashinto each translated page. - The publish workflow precomputes a pending file list by comparing the current English source hash to the stored locale
x-i18n.source_hash. - If the pending count is
0, the expensive translation step is skipped entirely. - If there are pending files, the workflow translates only those files.
- Locale workers retry transient model-format failures, but unchanged files stay skipped because the same hash check runs on each retry.
- Locale workers upload artifacts; the publish repo finalizer commits all successful locale outputs together.
- Published GitHub releases dispatch one aggregate translation refresh so release docs can catch up without waiting for the weekly reconciliation.
Operational notes
- Sync metadata is written to
.openclaw-sync/source.jsonin the publish repo. - Source repo secret:
OPENCLAW_DOCS_SYNC_TOKEN - Publish repo secret:
OPENCLAW_DOCS_I18N_OPENAI_API_KEY - If locale output looks stale, check the
Translate Allworkflow inopenclaw/docsfirst.