Files
openclaw/docs/reference/transcript-hygiene.md
Peter Steinberger 4273ca9dbd refactor(sessions): remove file-era transcript runtime (#113233)
* refactor(sessions): keep helper transcripts in memory

* refactor(sessions): remove file-era transcript storage

* test(sessions): use SQLite identity in attempt persistence

* test(codex): isolate legacy transcript fixtures

* fix(sessions): preserve SQLite transcript identity

* fix(sessions): harden transcript lifecycle invariants

* fix(sessions): validate transcript identities

* fix(sessions): close identity compatibility gaps

* fix(sessions): preserve leaf and plugin identities

* fix(sessions): retain dispatch transcript targets

* fix(sessions): preserve active transcript context

* fix(sessions): isolate artifact accounting

* fix(sessions): bound SQLite usage accounting

* fix(sessions): retain bounded latest usage

* fix(sessions): align rebased transcript targets

* test(sessions): align accessor scope fixture

* fix(telegram): derive SQLite transcript identity

* refactor(sessions): remove file-era compaction residue

* chore(sessions): lower max-lines baseline

* fix(sessions): preserve structured transcript identity

* test(sessions): align doctor identity assertions

* fix(sessions): isolate default SDK database

* refactor(sessions): remove dead file-era exports

* fix(sessions): reconcile SQLite transcript identity

* fix(sessions): pass checkpoint identity explicitly

* test(sessions): make entry field probe explicit

* test(sessions): satisfy transcript cleanup lint

* test(sessions): align diagnostics identity proof

* fix(sessions): finish transcript runtime teardown

* fix(sessions): preserve transcript identity invariants

* fix(sessions): harden transcript compatibility edges

* fix(sessions): preserve checkpoint transcript anchors

* fix(sessions): preserve SQLite lifecycle invariants

* fix(sessions): retarget compaction successors

* test(sessions): preserve transcript fixture semantics

* feat(plugin-sdk): add command transcript targets

* fix(sessions): serialize transcript rewrites

* fix(sessions): validate legacy successor identity

* fix(sessions): normalize compaction ownership

* fix(sessions): validate successor identity before adoption

* fix(sessions): preserve plugin transcript ownership

* fix(sessions): carry transcript identity through commands

* fix(sessions): import legacy checkpoint artifacts into SQLite

* fix(sessions): preserve successor transcript ownership

* fix(sessions): align transcript consumers with target identity

* fix(sessions): scope transcript token estimates

* fix(sessions): retain agent identity across lifecycle hooks

* fix(sessions): resolve scoped SQLite targets

* fix(sessions): isolate lifecycle transcript targets

* fix(sessions): validate compaction agent ownership

* fix(sessions): preserve reset and cleanup lifecycle

* fix(sessions): serialize prompt cleanup lifecycle

* fix(sessions): remove stale lock import

* fix(sessions): preserve reset target context

* fix(sessions): fence prompt reload takeover

* fix(sessions): unblock abort and default lifecycle reads

* fix(sessions): validate legacy successor scope

* fix(sessions): reject metadata-only runtime rows

* fix(sessions): propagate custom transcript stores

* fix(sessions): preserve adopted retry targets

* fix(sessions): allow unkeyed usage reads

* fix(sessions): harden runtime target boundaries

* fix(sessions): serialize retry transcript writes

* fix(sessions): bound prompt reload disposal

* fix(sessions): complete retry marker identity

* fix(sessions): keep legacy marker identity minimal

* test(sessions): tighten teardown fixture types

* fix(sessions): preserve compatibility target identity

* test(sessions): persist post-checkpoint boundary turn

* test(sessions): align runtime store mock contracts

* style(sessions): simplify persisted identity guard

* fix(sessions): prefer complete typed targets

* fix(sessions): recover legacy marker targets

* test(sessions): align marker lookup fixture scope

* fix(sessions): validate partial transcript targets

* fix(sessions): reconcile partial transcript identities

* fix(sessions): canonicalize compatibility identities

* test(sessions): cover compatibility aliases

* fix(sessions): adopt legacy successor identity

* fix(sessions): preserve usage read identity

* fix(sessions): preserve partial marker compatibility

* fix(sessions): validate legacy successor mappings

* fix(sessions): reconcile marker store mappings

* fix(sessions): preserve legacy fallback identity

* fix(sessions): harden marker alias resolution

* fix(sessions): prefer verified successor aliases

* fix(sessions): resolve preferred marker aliases

* fix(sessions): serialize cleanup admission

* fix(sessions): align marker lookup scopes

* fix(codex): type marker alias summaries

* style(sessions): satisfy changed lint

* test(sessions): align structured target assertions

* fix(sessions): reconcile latest identity contracts

* fix(sessions): validate transcript identity boundaries

* docs(sessions): explain stable registry keys

* fix(sessions): harden compatibility target round trips

* fix(sessions): port usage identity to split modules

* test(sessions): align subagent transcript identity

* fix(sessions): finish transcript identity migration

* fix(agents): route subagent completion capture through transcript targets

* fix(agents): settle SQLite prompt handoff during cleanup

* chore: shrink max-lines baseline after teardown

* fix(sessions): port teardown across split runtime owners

* fix(sessions): carry transcript targets through split owners

* test(agents): use SQLite compaction target in abort coverage

* chore: retain unrelated max-lines suppressions

* chore: shrink max-lines baseline after main splits

* style(agents): const compaction checkpoint locals

* fix(sessions): harden SQLite teardown boundaries

* test(sessions): use typed metadata in predicate isolation fixture

* test(agents): cover malformed settlement rejections lint-safely

* fix(sessions): close remaining SQLite identity races

* fix(agents): fail closed on incomplete successor targets

* fix(sessions): preserve transcript identity fallbacks

* fix(agents): preserve session-key abort admission

* fix(trajectory): validate incomplete export targets

* test(sessions): drop retired pricing cache imports

* fix(sessions): validate partial transcript identities

* fix(sessions): close transcript identity edge cases

* fix(plugins): reserve retired transcript locator slot

* fix(sessions): scope transcript locks by target

* style(sessions): simplify SDK initialization error

* fix(sessions): preserve initialized transcript state

* fix(codex): verify mirrored history session keys

* fix(sessions): reject stale transcript ownership

* fix(sessions): anchor asynchronous transcript ownership

* fix(sessions): measure active transcript state

* fix(sessions): preserve scoped transcript compaction

* fix(sessions): harden transcript identity and lifecycle

* fix(sessions): resolve scoped command transcript stores

* fix(sessions): make transcript appends failure-atomic

* fix(sessions): enforce scoped transcript ownership

* fix(sessions): reject cross-owner transcript handoffs

* fix(sessions): fence cleanup transcript ownership

* fix(sessions): retire stale write ownership contexts

* fix(sessions): preserve pending session migration state

* fix(sessions): validate migrated transcript ownership

* fix(sessions): validate usage transcript targets

* fix(sessions): clear predecessor transcript metadata

* fix(sessions): align durable session event targets

* fix(sessions): fence late prompt handoffs

* fix(sessions): fence lifecycle transcript fallbacks

* fix(sessions): bound zero-length memory capture

* fix(sessions): preserve transcript teardown ownership

* fix(sessions): reject duplicate cleanup ownership

* fix(sessions): serialize runtime writes with sqlite leases

* fix(sessions): close sqlite teardown concurrency gaps

* fix(sessions): preserve nested lifecycle failures

* fix(sessions): canonicalize sqlite transcript ownership

* fix(sessions): settle disposed prompt handoffs

* fix(sessions): resolve canonical attempt lock targets

* test(sessions): align canonical target fixtures

* test(sessions): retire redundant jsonl parser coverage

* refactor(sessions): split active transcript cursors

* test(memory): retire legacy marker fixture

* fix(sessions): preserve canonical transcript access after rebase

* fix(sessions): fence prompt lease and return transcript targets

* fix(sessions): colocate transcript leases with target store

* fix(sessions): canonicalize transcript lease and worker targets

* fix(sessions): preserve plugin and fork identity markers

* fix(sessions): complete sqlite transcript target migration

* fix(sessions): integrate canonical followup identity

* fix(sessions): preserve bounded transcript topology

* fix(sessions): validate transcript identity boundaries

* fix(context): separate caller and successor targets

* test(sessions): split persistence compatibility coverage

* test(sessions): preserve fixture topology efficiently

* chore(sdk): refresh plugin api baseline

* test(agents): align compaction lock target mocks

* test(sessions): seed malformed transcript fixtures directly

* fix(agents): canonicalize transcript compatibility inputs

* fix(agents): type optional tool result ids

* test(ci): stabilize loaded process timing

* test(tui): wait for collect queue admission
2026-07-27 22:33:24 -04:00

286 lines
12 KiB
Markdown

---
summary: "Reference: provider-specific transcript sanitization and repair rules"
read_when:
- You are debugging provider request rejections tied to transcript shape
- You are changing transcript sanitization or tool-call repair logic
- You are investigating tool-call id mismatches across providers
title: "Transcript hygiene"
---
OpenClaw applies **provider-specific fixes** to transcripts before a run
(building model context). These are **in-memory** adjustments used to satisfy
strict provider requirements. Runtime transcript state stays in SQLite;
provider-specific
assistant-prefill stripping happens only while constructing outbound
payloads.
Scope includes:
- Runtime-only prompt context staying out of user-visible transcript turns
- Tool call id sanitization
- Tool call input validation
- Tool result pairing repair
- Turn validation / ordering
- Thought signature cleanup
- Thinking signature cleanup
- Image payload sanitization
- Blank text-block cleanup before provider replay
- Incomplete reasoning-only length-turn cleanup before provider replay
- User-input provenance tagging (for inter-session routed prompts)
- Empty assistant error-turn repair for Bedrock Converse replay
If you need transcript storage details, see
[Session management deep dive](/reference/session-management-compaction).
---
## Global rule: runtime context is not user transcript
Runtime/system context can be added to the model prompt for a turn, but it is
not end-user-authored content. OpenClaw keeps a separate transcript-facing
prompt body for Gateway replies, queued followups, ACP, CLI, and embedded
OpenClaw runs. Stored visible user turns use that transcript body instead of
the runtime-enriched prompt.
For legacy sessions that already persisted runtime wrappers, Gateway history
surfaces apply a display projection before returning messages to WebChat,
TUI, REST, or SSE clients.
---
## Where this runs
All transcript hygiene is centralized in the embedded runner:
- Policy selection: `src/agents/transcript-policy.ts`
(`resolveTranscriptPolicy`, keyed on `provider`, `modelApi`, and `modelId`)
- Sanitization/repair application: `sanitizeSessionHistory` in
`src/agents/embedded-agent-runner/replay-history.ts`
Legacy JSONL validation and import belong to `openclaw doctor --fix`; the
embedded runner does not repair or reopen file-backed runtime transcripts.
---
## Global rule: image sanitization
Image payloads are always sanitized to prevent provider-side rejection due to
size limits (downscale/recompress oversized base64 images). This also helps
control image-driven token pressure for vision-capable models: lower max
dimensions reduce token usage, higher dimensions preserve detail.
Implementation:
- `sanitizeSessionMessagesImages` in
`src/agents/embedded-agent-helpers/images.ts`
- `sanitizeContentBlocksImages` in `src/agents/tool-images.ts`
- Max image side is configurable via `agents.defaults.imageMaxDimensionPx`
(default: `1200`)
- Blank text blocks are removed while this pass walks replay content.
Assistant turns that become empty are dropped from the replay copy; user
and tool-result turns that become empty receive a non-empty
omitted-content placeholder.
---
## Global rule: malformed tool calls
Assistant tool-call blocks missing both `input` and `arguments` are dropped
before model context is built. This prevents provider rejections from
partially persisted tool calls (for example, after a rate limit failure).
Implementation:
- `sanitizeToolCallInputs` in `src/agents/session-transcript-repair.ts`
- Applied in `sanitizeSessionHistory`
(`src/agents/embedded-agent-runner/replay-history.ts`)
---
## Global rule: tool result pairing
Tool results are paired to tool-call occurrences within each assistant turn before
provider-specific call IDs are rewritten. Provider-generated IDs may repeat on later
turns, so a result adjacent to a repeated call stays with that occurrence. A displaced
result is moved only when exactly one unresolved occurrence can own it; ambiguous
extras are dropped and missing occurrences receive synthetic error results.
Implementation: `sanitizeToolUseResultPairing` in
`src/agents/session-transcript-repair.ts`
---
## Global rule: incomplete or silent reasoning-only turns
Assistant turns are omitted from the in-memory replay copy when they contain
only thinking or redacted-thinking content after either of these events:
- The provider output limit ends the turn with incomplete reasoning state.
- Silent-reply cleanup removes the turn's only visible `NO_REPLY` text.
The silent-reply cleanup prevents hidden reasoning from merging into a later
assistant tool-use turn when strict providers rebuild the conversation.
Empty length turns remain unchanged, as do length turns with visible text,
tool calls, or unknown content blocks. Silent-reply turns with tool calls or
unknown content blocks also remain unchanged. Stored transcripts are not
rewritten.
Implementation: `normalizeAssistantReplayContent` in
`src/agents/embedded-agent-runner/replay-history.ts`
---
## Global rule: inter-session input provenance
When an agent sends a prompt into another session via `sessions_send`
(including agent-to-agent reply/announce steps), OpenClaw persists the
created user turn with `message.provenance.kind = "inter_session"`.
OpenClaw also prepends a same-turn `[Inter-session message] ... isUser=false`
marker before the routed prompt text so the active model call can
distinguish foreign session output from external end-user instructions. This
marker includes the source session, channel, and tool when available. The
transcript still uses `role: "user"` for provider compatibility, but the
visible text and provenance metadata both mark the turn as inter-session
data.
During context rebuild, OpenClaw applies the same marker to older persisted
inter-session user turns that only have provenance metadata.
---
## Provider matrix (current behavior)
**OpenAI / OpenAI Codex**
- Image sanitization only.
- Drop orphaned reasoning signatures (standalone reasoning items without a
following content block) for OpenAI Responses/Codex transcripts, and drop
replayable OpenAI reasoning after a model route switch.
- Preserve replayable OpenAI Responses reasoning item payloads, including
encrypted empty-summary items, so manual/WebSocket replay keeps required
`rs_*` state paired with assistant output items.
- Native ChatGPT Codex Responses follows Codex wire parity by replaying
prior Responses reasoning/message/function payloads without prior item
IDs while preserving session `prompt_cache_key`.
- OpenAI Responses-family replay preserves canonical `call_*|fc_*`
same-model reasoning pairs, but deterministically normalizes malformed or
overlong `call_id`/function-call item ids before pi-ai payload conversion.
- Tool result pairing repair may move real matched outputs and synthesize
Codex-style `aborted` outputs for missing tool calls.
- No turn validation or reordering; no thought signature stripping.
**OpenAI-compatible Chat Completions**
- Historical assistant thinking/reasoning blocks are stripped before replay
so local and proxy-style OpenAI-compatible servers do not receive
prior-turn reasoning fields such as `reasoning` or `reasoning_content`.
- Current same-turn tool-call continuations keep the assistant reasoning
block attached to the tool call until the tool result has been replayed.
- Custom/self-hosted model entries with `reasoning: true` preserve replayed
reasoning metadata.
- Provider-owned exceptions can opt out when their wire protocol requires
replayed reasoning metadata.
**Google (Generative AI / Gemini CLI / Antigravity)**
- Tool call id sanitization: strict alphanumeric.
- Tool result pairing repair and synthetic tool results.
- Turn validation (Gemini-style turn alternation).
- Google turn ordering fixup (prepend a tiny user bootstrap if history
starts with assistant).
- Antigravity Claude: normalize thinking signatures; drop unsigned thinking
blocks.
**Anthropic / Minimax (Anthropic-compatible)**
- Tool result pairing repair and synthetic tool results.
- Turn validation (merge consecutive user turns to satisfy strict
alternation).
- Trailing assistant prefill turns are stripped from outgoing Anthropic
Messages payloads when thinking is enabled, including Cloudflare AI
Gateway routes.
- Pre-compaction assistant thinking signatures are stripped before provider
replay when a session has been compacted. Thinking signatures are
cryptographically bound to the conversation prefix at generation time;
after compaction the prefix changes (summarized content replaces the
original), so replaying the original signatures causes Anthropic to
reject the request with "Invalid signature in thinking block". The
thinking text is preserved as an unsigned block and then handled by the
rule below.
- Thinking blocks with missing, empty, or blank replay signatures are
stripped before provider conversion. If that empties an assistant turn,
OpenClaw keeps turn shape with non-empty omitted-reasoning text.
- Older thinking-only assistant turns that must be stripped are replaced
with non-empty omitted-reasoning text so provider adapters do not drop
the replay turn.
**Amazon Bedrock (Converse API)**
- Empty assistant stream-error turns are repaired to a non-empty fallback
text block before replay. Bedrock Converse rejects assistant messages
with `content: []`, so persisted assistant turns with `stopReason:
"error"` and empty content are also repaired on disk before load.
- Assistant stream-error turns with only blank text blocks are dropped from
the in-memory replay copy instead of replaying an invalid blank block.
- Pre-compaction assistant thinking signatures are stripped before Converse
replay when a session has been compacted, for the same reason as
Anthropic above.
- Claude thinking blocks with missing, empty, or blank replay signatures
are stripped before Converse replay. If that empties an assistant turn,
OpenClaw keeps turn shape with non-empty omitted-reasoning text.
- Older thinking-only assistant turns that must be stripped are replaced
with non-empty omitted-reasoning text so the Converse replay keeps
strict turn shape.
- Replay filters OpenClaw delivery-mirror and gateway-injected assistant
turns.
- Image sanitization applies through the global rule.
**Mistral (including model-id based detection)**
- Tool call id sanitization: strict9 (alphanumeric, length 9).
**OpenRouter Gemini**
- Thought signature cleanup: strip non-base64 `thought_signature` values
(keep base64).
**OpenRouter Anthropic**
- Trailing assistant prefill turns are stripped from verified OpenRouter
OpenAI-compatible Anthropic model payloads when reasoning is enabled,
matching direct Anthropic and Cloudflare Anthropic replay behavior.
**Everything else**
- Image sanitization only.
---
## Historical behavior (pre-2026.1.22)
Before the 2026.1.22 release, OpenClaw applied multiple layers of transcript
hygiene:
- A **transcript-sanitize extension** ran on every context build and could:
- Repair tool use/result pairing.
- Sanitize tool call ids (including a non-strict mode that preserved
`_`/`-`).
- The runner also performed provider-specific sanitization, which
duplicated work.
- Additional mutations occurred outside the provider policy, including
stripping `<final>` tags from assistant text before persistence, dropping
empty assistant error turns, and trimming assistant content after tool
calls.
This complexity caused cross-provider regressions (notably
`openai-responses` `call_id|fc_id` pairing). The 2026.1.22 cleanup removed
the extension, centralized logic in the runner, and made OpenAI **no-touch**
beyond image sanitization.
## Related
- [Session management](/concepts/session)
- [Session pruning](/concepts/session-pruning)