openclaw/docs/reference/session-management-compaction.md

---
summary: "Deep dive: session store + transcripts, lifecycle, and (auto)compaction internals"
read_when:
  - You need to debug session ids, SQLite session rows/events, or doctor migration of legacy sessions.json/JSONL files
  - You are changing auto-compaction behavior or adding "pre-compaction" housekeeping
  - You want to implement memory flushes or silent system turns
title: "Session management deep dive"
---

OpenClaw manages sessions end-to-end across these areas:

- **Session routing** (how inbound messages map to a `sessionKey`)
- **Session store** and what it tracks
- **Transcript persistence** (SQLite event streams, doctor-only JSONL import, explicit debug export) and its structure
- **Transcript hygiene** (provider-specific fixups before runs)
- **Context limits** (context window vs tracked tokens)
- **Compaction** (manual and auto-compaction) and where to hook pre-compaction work
- **Silent housekeeping** (memory writes that should not produce user-visible output)

If you want a higher-level overview first, start with:

- [Session management](/concepts/session)
- [Compaction](/concepts/compaction)
- [Memory overview](/concepts/memory)
- [Memory search](/concepts/memory-search)
- [Session pruning](/concepts/session-pruning)
- [Transcript hygiene](/reference/transcript-hygiene)

---

## Source of truth: the Gateway

OpenClaw is designed around a single **Gateway process** that owns session state.

- UIs (macOS app, web Control UI, TUI) should query the Gateway for session lists and token counts.
- In remote mode, session databases are on the remote host; "checking your local Mac files" won't reflect what the Gateway is using.

---

## Two persistence layers

OpenClaw persists sessions in two layers:

1. **Session store**
   - Key/value map: `sessionKey -> SessionEntry`
   - SQLite-backed by default; legacy JSON import is doctor-only and support export is explicit
   - Tracks session metadata (current session id, last activity, toggles, token counters, etc.)

2. **Transcript (`agentId`, `sessionId`)**
   - SQLite-backed transcript event stream with tree structure (entries have `id` + `parentId`)
   - Stores the actual conversation + tool calls + compaction summaries
   - Used to rebuild the model context for future turns
   - Stored in SQLite for OpenClaw-owned runtime paths; JSONL files are legacy
     doctor-import inputs or explicit support artifacts, not runtime
     compatibility sidecars

- Runtime code passes structured agent/session scope. There is no active
  transcript file, URI, or locator layer.
- Scoped latest/tail assistant-text lookups, session exports, `before_reset`
  hook payloads, silent session rotations, chat history, TUI history,
  recovery, managed media indexing, token estimation, title/preview/usage
  helpers, and bounded session inspection read the scoped SQLite transcript.
- Pre-compaction checkpoints are SQLite transcript snapshots. OpenClaw does
  not create `.checkpoint.*.jsonl` copies on the runtime path.

Gateway history readers should avoid materializing the whole transcript unless
the surface explicitly needs arbitrary historical access. First-page history,
embedded chat history, restart recovery, and token/usage checks use bounded tail
reads. Full transcript scans are keyed by SQLite agent/session scope, not by a
file path.

---

## On-disk locations

Per agent, on the Gateway host:

- Global store: `~/.openclaw/state/openclaw.sqlite` by default. It stores
  shared registry, migration, plugin, task, and backup metadata.
- Agent store: `~/.openclaw/agents/<agentId>/agent/openclaw-agent.sqlite`. It
  stores canonical session rows, transcript events, snapshots, VFS entries,
  artifacts, and agent-local cache rows.
- Legacy imports: `openclaw doctor --fix` imports
  `~/.openclaw/agents/<agentId>/sessions/sessions.json` indexes and JSONL
  transcripts into the agent SQLite database, then removes imported legacy
  sources after durable verification. Gateway startup leaves legacy indexes
  alone.
- Transcripts: runtime transcript events live in the per-agent database
  (`transcript_events` and `transcript_event_identities`). The canonical
  identity is structured scope: `agentId` plus `sessionId`. Legacy JSONL files
  are doctor migration inputs or explicit support artifacts, never runtime
  sidecars or compatibility handles.

OpenClaw resolves these via `src/config/sessions/*`.

---

## Store Cleanup

SQLite is the canonical per-agent session backend. `sessions.json` is a legacy
doctor-import input, not a parallel export/debug store. Runtime code should
read and write explicit `{ agentId, sessionKey }` rows.

Runtime writes normalize and persist only; they do not prune, cap, import,
archive, or run disk-budget cleanup. Session store reads also do not import,
prune, or cap entries during Gateway startup. Use `openclaw doctor --fix` for
legacy JSON/JSONL import.

OpenClaw no longer creates automatic `sessions.json.bak.*` rotation backups
during Gateway writes. Legacy `session.maintenance.*` and `session.writeLock.*`
settings are doctor-migrated raw config only, and `openclaw doctor --fix`
removes them from older configs.

Transcript mutations are serialized through SQLite transactions plus the
per-session append queue. Runtime bootstrap and manual compaction repair write
SQLite transcript rows directly. Any retained JSONL shape is an explicit
doctor/import/export/debug boundary, not a runtime lookup or persistence path.

Legacy session import belongs to `openclaw doctor --fix`. Runtime no longer has
a session cleanup command that prunes missing transcript rows; after doctor
runs, reset or delete any intentionally stale session explicitly.

---

## Cron sessions and run logs

Isolated cron runs also create session entries/transcripts. Session rows use the
same SQLite session tables as other rows:

- Legacy cron session imports happen through `openclaw doctor --fix`.
- `cron.runLog.maxBytes` + `cron.runLog.keepLines` prune SQLite cron run history (defaults: `2_000_000` approximate serialized bytes and `2000` rows per job).

When cron force-creates a new isolated run session, it sanitizes the previous
`cron:<jobId>` session entry before writing the new row. It carries safe
preferences such as thinking/fast/verbose settings, labels, and explicit
user-selected model/auth overrides. It drops ambient conversation context such
as channel/group routing, send or queue policy, elevation, origin, and ACP
runtime binding so a fresh isolated run cannot inherit stale delivery or
runtime authority from an older run.

---

## Session keys (`sessionKey`)

A `sessionKey` identifies _which conversation bucket_ you're in (routing + isolation).

Common patterns:

- Main/direct chat (per agent): `agent:<agentId>:<mainKey>` (default `main`)
- Group: `agent:<agentId>:<channel>:group:<id>`
- Room/channel (Discord/Slack): `agent:<agentId>:<channel>:channel:<id>` or `...:room:<id>`
- Cron: `cron:<job.id>`
- Webhook: `hook:<uuid>` (unless overridden)

The canonical rules are documented at [/concepts/session](/concepts/session).

---

## Session ids (`sessionId`)

Each `sessionKey` points at a current `sessionId` (the SQLite transcript identity
that continues the conversation).

Rules of thumb:

- **Reset** (`/new`, `/reset`) creates a new `sessionId` for that `sessionKey`.
- **Daily reset** (default 4:00 AM local time on the gateway host) creates a new `sessionId` on the next message after the reset boundary.
- **Idle expiry** (`session.reset.idleMinutes`) creates a new `sessionId` when a message arrives after the idle window. When daily + idle are both configured, whichever expires first wins. `openclaw doctor --fix` migrates old `session.idleMinutes` configs into `session.reset.idleMinutes`.
- **System events** (heartbeat, cron wakeups, exec notifications, gateway bookkeeping) may mutate the session row but do not extend daily/idle reset freshness. Reset rollover discards queued system-event notices for the previous session before the fresh prompt is built.
- **Parent fork policy** uses PI's active branch when creating a thread or subagent fork. If that branch is too large, OpenClaw starts the child with isolated context instead of failing or inheriting unusable history. The sizing policy is automatic; legacy `session.parentForkMaxTokens` config is removed by `openclaw doctor --fix`.

Implementation detail: the decision happens in `initSessionState()` in `src/auto-reply/reply/session.ts`.

---

## Session store schema

The store's value type is `SessionEntry` in `src/config/sessions/types.ts`.

Key fields (not exhaustive):

- `sessionStartedAt`: start timestamp for the current `sessionId`; daily reset
  freshness uses this. Legacy rows may derive it from the JSONL session header.
- `lastInteractionAt`: last real user/channel interaction timestamp; idle reset
  freshness uses this so heartbeat, cron, and exec events do not keep sessions
  alive. Legacy rows without this field fall back to the recovered session start
  time for idle freshness.
- `updatedAt`: last store-row mutation timestamp, used for listing and
  bookkeeping. It is not the authority for daily/idle reset freshness.
- `sessionId`: current SQLite transcript id; callers pass structured
  `{ agentId, sessionId }` scope instead of a transcript path override
- `chatType`: `direct | group | room` (helps UIs and send policy)
- `provider`, `subject`, `room`, `space`, `displayName`: metadata for group/channel labeling
- Toggles:
  - `thinkingLevel`, `verboseLevel`, `reasoningLevel`, `elevatedLevel`
  - `sendPolicy` (per-session override)
- Model selection:
  - `providerOverride`, `modelOverride`, `authProfileOverride`
- Token counters (best-effort / provider-dependent):
  - `inputTokens`, `outputTokens`, `totalTokens`, `contextTokens`
- `compactionCount`: how often auto-compaction completed for this session key
- `memoryFlushAt`: timestamp for the last pre-compaction memory flush
- `memoryFlushCompactionCount`: compaction count when the last flush ran

The store is safe to edit, but the Gateway is the authority: it may rewrite or rehydrate entries as sessions run.

---

## Transcript structure

Transcripts are stored as SQLite rows and opened by `{agentId, sessionId}`.

The event stream is stored in the per-agent `transcript_events` table:

- First event: session header (`type: "session"`, includes `id`, `cwd`,
  `timestamp`, optional `parentSession`)
- Then: session entries with `id` + `parentId` (tree)

Doctor JSONL import uses the same event shape, one JSON object per line.
User-facing exports may materialize support-bundle JSONL from SQLite rows, but
runtime code does not read or write transcript JSONL files.

Notable entry types:

- `message`: user/assistant/toolResult messages
- `custom_message`: extension-injected messages that _do_ enter model context (can be hidden from UI)
- `custom`: extension state that does _not_ enter model context
- `compaction`: persisted compaction summary with `firstKeptEntryId` and `tokensBefore`
- `branch_summary`: persisted summary when navigating a tree branch

Runtime transcript repair and compaction mutate SQLite rows through scoped
transcript APIs. Legacy JSONL shape upgrades happen only in doctor import before
rows are written.

---

## Context windows vs tracked tokens

Two different concepts matter:

1. **Model context window**: hard cap per model (tokens visible to the model)
2. **Session store counters**: rolling stats written into the session store (used for /status and dashboards)

If you're tuning limits:

- The context window comes from the model catalog (and can be overridden via config).
- `contextTokens` in the store is a runtime estimate/reporting value; don't treat it as a strict guarantee.

For more, see [/token-use](/reference/token-use).

---

## Compaction: what it is

Compaction summarizes older conversation into a persisted `compaction` entry in the transcript and keeps recent messages intact.

After compaction, future turns see:

- The compaction summary
- Messages after `firstKeptEntryId`

Compaction is **persistent** (unlike session pruning). See [/concepts/session-pruning](/concepts/session-pruning).

## Compaction chunk boundaries and tool pairing

When OpenClaw splits a long transcript into compaction chunks, it keeps
assistant tool calls paired with their matching `toolResult` entries.

- If the token-share split lands between a tool call and its result, OpenClaw
  shifts the boundary to the assistant tool-call message instead of separating
  the pair.
- If a trailing tool-result block would otherwise push the chunk over target,
  OpenClaw preserves that pending tool block and keeps the unsummarized tail
  intact.
- Aborted/error tool-call blocks do not hold a pending split open.

---

## When auto-compaction happens (Pi runtime)

In the embedded Pi agent, auto-compaction triggers in two cases:

1. **Overflow recovery**: the model returns a context overflow error
   (`request_too_large`, `context length exceeded`, `input exceeds the maximum
number of tokens`, `input token count exceeds the maximum number of input
tokens`, `input is too long for the model`, `ollama error: context length
exceeded`, and similar provider-shaped variants) → compact → retry.
2. **Threshold maintenance**: after a successful turn, when:

`contextTokens > contextWindow - reserveTokens`

Where:

- `contextWindow` is the model's context window
- `reserveTokens` is headroom reserved for prompts + the next model output

These are Pi runtime semantics (OpenClaw consumes the events, but Pi decides when to compact).

OpenClaw can also trigger a preflight local compaction before opening the next
run when `agents.defaults.compaction.maxActiveTranscriptBytes` is set and the
active SQLite transcript reaches that size. This is a transcript-size guard for
local reopen cost, not raw archival: OpenClaw still runs normal semantic
compaction, and it requires `rotateAfterCompaction` so the compacted summary
can become a new successor transcript.

For embedded Pi runs, `agents.defaults.compaction.midTurnPrecheck.enabled: true`
adds an opt-in tool-loop guard. After a tool result is appended and before the
next model call, OpenClaw estimates the prompt pressure using the same preflight
budget logic used at turn start. If the context no longer fits, the guard does
not compact inside Pi's `transformContext` hook. It raises a structured
mid-turn precheck signal, stops the current prompt submission, and lets the
outer run loop use the existing recovery path: truncate oversized tool results
when that is enough, or trigger the configured compaction mode and retry. The
option is disabled by default and works with both `default` and `safeguard`
compaction modes, including provider-backed safeguard compaction.
This is independent of `maxActiveTranscriptBytes`: the byte-size guard runs
before a turn opens, while mid-turn precheck runs later in the embedded Pi tool
loop after new tool results have been appended.

---

## Compaction settings (`reserveTokens`, `keepRecentTokens`)

Pi's compaction settings live in Pi settings:

```json5
{
  compaction: {
    enabled: true,
    reserveTokens: 16384,
    keepRecentTokens: 20000,
  },
}
```

OpenClaw also enforces a safety floor for embedded runs:

- If `compaction.reserveTokens < reserveTokensFloor`, OpenClaw bumps it.
- Default floor is `20000` tokens.
- Set `agents.defaults.compaction.reserveTokensFloor: 0` to disable the floor.
- If it's already higher, OpenClaw leaves it alone.
- Manual `/compact` honors an explicit `agents.defaults.compaction.keepRecentTokens`
  and keeps Pi's recent-tail cut point. Without an explicit keep budget,
  manual compaction remains a hard checkpoint and rebuilt context starts from
  the new summary.
- Set `agents.defaults.compaction.midTurnPrecheck.enabled: true` to run the
  optional tool-loop precheck after new tool results and before the next model
  call. This is a trigger only; summary generation still uses the configured
  compaction path. It is independent of `maxActiveTranscriptBytes`, which is a
  turn-start active-transcript byte-size guard.
- Set `agents.defaults.compaction.maxActiveTranscriptBytes` to a byte value or
  string such as `"20mb"` to run local compaction before a turn when the active
  transcript gets large. This guard is active only when
  `rotateAfterCompaction` is also enabled. Leave it unset or set `0` to
  disable.
- When `agents.defaults.compaction.rotateAfterCompaction` is enabled,
  OpenClaw rewrites the active SQLite transcript to the compacted successor
  after compaction. The old full transcript is available only through the
  SQLite pre-compaction checkpoint snapshot while retained.

Why: leave enough headroom for multi-turn "housekeeping" (like memory writes) before compaction becomes unavoidable.

Implementation: `ensurePiCompactionReserveTokens()` in `src/agents/pi-settings.ts`
(called from `src/agents/pi-embedded-runner.ts`).

---

## Pluggable compaction providers

Plugins can register a compaction provider via `registerCompactionProvider()` on the plugin API. When `agents.defaults.compaction.provider` is set to a registered provider id, the safeguard extension delegates summarization to that provider instead of the built-in `summarizeInStages` pipeline.

- `provider`: id of a registered compaction provider plugin. Leave unset for default LLM summarization.
- Setting a `provider` forces `mode: "safeguard"`.
- Providers receive the same compaction instructions and identifier-preservation policy as the built-in path.
- The safeguard still preserves recent-turn and split-turn suffix context after provider output.
- Built-in safeguard summarization re-distills prior summaries with new messages
  instead of preserving the full previous summary verbatim.
- Safeguard mode enables summary quality audits by default; set
  `qualityGuard.enabled: false` to skip retry-on-malformed-output behavior.
- If the provider fails or returns an empty result, OpenClaw falls back to built-in LLM summarization automatically.
- Abort/timeout signals are re-thrown (not swallowed) to respect caller cancellation.

Source: `src/plugins/compaction-provider.ts`, `src/agents/pi-hooks/compaction-safeguard.ts`.

---

## User-visible surfaces

You can observe compaction and session state via:

- `/status` (in any chat session)
- `openclaw status` (CLI)
- `openclaw sessions` / `sessions --json`
- Gateway logs (`pnpm gateway:watch` or `openclaw logs --follow`): `embedded run auto-compaction start` + `complete`
- Verbose mode: `🧹 Auto-compaction complete` + compaction count

---

## Silent housekeeping (`NO_REPLY`)

OpenClaw supports "silent" turns for background tasks where the user should not see intermediate output.

Convention:

- The assistant starts its output with the exact silent token `NO_REPLY` /
  `no_reply` to indicate "do not deliver a reply to the user".
- OpenClaw strips/suppresses this in the delivery layer.
- Exact silent-token suppression is case-insensitive, so `NO_REPLY` and
  `no_reply` both count when the whole payload is just the silent token.
- This is for true background/no-delivery turns only; it is not a shortcut for
  ordinary actionable user requests.

As of `2026.1.10`, OpenClaw also suppresses **draft/typing streaming** when a
partial chunk begins with `NO_REPLY`, so silent operations don't leak partial
output mid-turn.

---

## Pre-compaction "memory flush" (implemented)

Goal: before auto-compaction happens, run a silent agentic turn that writes durable
state to disk (e.g. `memory/YYYY-MM-DD.md` in the agent workspace) so compaction can't
erase critical context.

OpenClaw uses the **pre-threshold flush** approach:

1. Monitor session context usage.
2. When it crosses a "soft threshold" (below Pi's compaction threshold), run a silent
   "write memory now" directive to the agent.
3. Use the exact silent token `NO_REPLY` / `no_reply` so the user sees
   nothing.

Config (`agents.defaults.compaction.memoryFlush`):

- `enabled` (default: `true`)
- `model` (optional exact provider/model override for the flush turn, for example `ollama/qwen3:8b`)
- `softThresholdTokens` (default: `4000`)
- `prompt` (user message for the flush turn)
- `systemPrompt` (extra system prompt appended for the flush turn)

Notes:

- The default prompt/system prompt include a `NO_REPLY` hint to suppress
  delivery.
- When `model` is set, the flush turn uses that model without inheriting the
  active session fallback chain, so local-only housekeeping does not silently
  fall back to a paid conversation model.
- The flush runs once per compaction cycle (tracked in the session store).
- The flush runs only for embedded Pi sessions (CLI backends skip it).
- The flush is skipped when the session workspace is read-only (`workspaceAccess: "ro"` or `"none"`).
- See [Memory](/concepts/memory) for the workspace file layout and write patterns.

Pi also exposes a `session_before_compact` hook in the extension API, but OpenClaw's
flush logic lives on the Gateway side today.

---

## Troubleshooting checklist

- Session key wrong? Start with [/concepts/session](/concepts/session) and confirm the `sessionKey` in `/status`.
- Session metadata vs transcript mismatch? Confirm the Gateway host and agent database from `openclaw status`.
- Compaction spam? Check:
  - model context window (too small)
  - compaction settings (`reserveTokens` too high for the model window can cause earlier compaction)
  - tool-result bloat: review compaction thresholds and tool-result persistence
- Silent turns leaking? Confirm the reply starts with `NO_REPLY` (case-insensitive exact token) and you're on a build that includes the streaming suppression fix.

## Related

- [Session management](/concepts/session)
- [Session pruning](/concepts/session-pruning)
- [Context engine](/concepts/context-engine)