Files
openclaw/docs/concepts/compaction.md
Josh Lehman 0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00

9.0 KiB

summary, read_when, title
summary read_when title
How OpenClaw summarizes long conversations to stay within model limits
You want to understand auto-compaction and /compact
You are debugging long sessions hitting context limits
Compaction

Every model has a context window: the maximum number of tokens it can process. When a conversation approaches that limit, OpenClaw compacts older messages into a summary so the chat can continue.

How it works

  1. Older conversation turns are summarized into a compact entry.
  2. The summary is saved in the session transcript.
  3. Recent messages are kept intact.

OpenClaw keeps assistant tool calls paired with their matching toolResult entries when it picks a compaction split point. If the point lands inside a tool block, OpenClaw moves the boundary so the pair stays together and the current unsummarized tail is preserved.

The full conversation history stays on disk. Compaction only changes what the model sees on the next turn.

New configs default `agents.defaults.compaction.mode` to `"safeguard"` (stricter guardrails, summary quality audits). Set `mode: "default"` explicitly to opt out.

Auto-compaction

Auto-compaction is on by default. It runs when the session nears the context limit, or when the model returns a context-overflow error (in which case OpenClaw compacts and retries).

You will see:

  • embedded run auto-compaction start / complete in normal Gateway logs.
  • 🧹 Auto-compaction complete in verbose mode.
  • /status showing 🧹 Compactions: <count>.
Before compacting, OpenClaw automatically reminds the agent to save important notes to [memory](/concepts/memory) files. This prevents context loss. OpenClaw matches dozens of provider-specific overflow error strings (Anthropic, OpenAI, Bedrock, Gemini, Ollama, OpenRouter, and more). Common examples:
- `request_too_large`
- `context length exceeded`
- `input exceeds the maximum number of tokens`
- `input token count exceeds the maximum number of input tokens` (Bedrock)
- `input is too long for the model`
- `ollama error: context length exceeded`

Manual compaction

Type /compact in any chat to force a compaction. Add instructions to guide the summary:

/compact Focus on the API design decisions

When agents.defaults.compaction.keepRecentTokens is set (default: 20,000), manual compaction honors that cut-point and keeps the recent tail in rebuilt context. Without an explicit keep budget, manual compaction behaves as a hard checkpoint and continues from the new summary alone.

Configuration

Configure compaction under agents.defaults.compaction in your openclaw.json. The most common knobs are listed below; for the full reference, see Session management deep dive.

Using a different model

By default, compaction uses the agent's primary model. Set agents.defaults.compaction.model to delegate summarization to a more capable or specialized model. The override accepts a provider/model-id string or a bare alias configured under agents.defaults.models:

{
  "agents": {
    "defaults": {
      "compaction": {
        "model": "openrouter/anthropic/claude-sonnet-4-6"
      }
    }
  }
}

Bare configured aliases resolve to their canonical provider and model before compaction starts. If a bare value matches both an alias and a configured literal model ID, the literal model ID wins. An unmatched bare value remains a model ID on the active provider.

This works with local models too, for example a second Ollama model dedicated to summarization:

{
  "agents": {
    "defaults": {
      "compaction": {
        "model": "ollama/llama3.1:8b"
      }
    }
  }
}

When unset, compaction starts with the active session model. If summarization fails with a model-fallback-eligible provider error, OpenClaw retries that compaction attempt through the session's existing model fallback chain. The fallback choice is temporary and is not written back to session state. An explicit agents.defaults.compaction.model override remains exact and does not inherit the session fallback chain.

Identifier preservation

Compaction summarization preserves opaque identifiers by default (identifierPolicy: "strict"). Override with identifierPolicy: "off" to disable, or identifierPolicy: "custom" plus identifierInstructions for custom guidance.

Active transcript byte guard

When agents.defaults.compaction.maxActiveTranscriptBytes is set, OpenClaw triggers normal local compaction before a run if transcript history reaches that size. This is useful for long-running sessions where provider-side context management may keep model context healthy while persisted transcript history keeps growing. It does not split raw bytes; it asks the normal compaction pipeline to create a semantic summary.

The byte guard applies to the active SQLite transcript history. Legacy JSONL checkpoint artifacts are not the active compaction target.

Successor transcripts

When agents.defaults.compaction.truncateAfterCompaction is enabled, OpenClaw does not rewrite the existing transcript in place. It creates a new active successor transcript from the compaction summary, preserved state, and unsummarized tail, then records checkpoint metadata that points branch/restore flows at that compacted successor. Successor transcripts also drop exact duplicate long user turns that arrive inside a short retry window, so channel retry storms are not carried into the next active transcript after compaction.

OpenClaw no longer writes separate .checkpoint.*.jsonl copies for new compactions. Existing legacy checkpoint files can still be used while referenced and are pruned by normal session cleanup.

Compaction notices

By default, compaction runs silently. Set notifyUser to show brief status messages when compaction starts and completes, and to surface a degraded notice when a pre-compaction memory flush is exhausted but the reply still continues:

{
  agents: {
    defaults: {
      compaction: {
        notifyUser: true,
      },
    },
  },
}

Memory flush

Before compaction, OpenClaw can run a silent memory flush turn to store durable notes to disk. Set agents.defaults.compaction.memoryFlush.model when this housekeeping turn should use a local model instead of the active conversation model:

{
  "agents": {
    "defaults": {
      "compaction": {
        "memoryFlush": {
          "model": "ollama/qwen3:8b"
        }
      }
    }
  }
}

The memory-flush model override is exact and does not inherit the active session fallback chain. See Memory for details and config.

Pluggable compaction providers

Plugins can register a custom compaction provider via registerCompactionProvider() on the plugin API. When a provider is registered and configured, OpenClaw delegates summarization to it instead of the built-in LLM pipeline.

To use a registered provider, set its id in your config:

{
  "agents": {
    "defaults": {
      "compaction": {
        "provider": "my-provider"
      }
    }
  }
}

Setting a provider automatically forces mode: "safeguard". Providers receive the same compaction instructions and identifier-preservation policy as the built-in path, and OpenClaw still preserves recent-turn and split-turn suffix context after provider output.

If the provider fails or returns an empty result, OpenClaw falls back to built-in LLM summarization.

Compaction vs pruning

Compaction Pruning
What it does Summarizes older conversation Trims old tool results
Saved? Yes (in session transcript) No (in-memory only, per request)
Scope Entire conversation Tool results only

Session pruning is a lighter-weight complement that trims tool output without summarizing.

Troubleshooting

Compacting too often? The model's context window may be small, or tool outputs may be large. Try enabling session pruning.

Context feels stale after compaction? Use /compact Focus on <topic> to guide the summary, or enable the memory flush so notes survive.

Need a clean slate? /new starts a fresh session without compacting.

For advanced configuration (reserve tokens, identifier preservation, custom context engines, OpenAI server-side compaction), see the Session management deep dive.

  • Session: session management and lifecycle.
  • Session pruning: trimming tool results.
  • Context: how context is built for agent turns.
  • Hooks: compaction lifecycle hooks (before_compaction, after_compaction).