openclaw/docs/pi.md

summary, title, read_when

summary	title	read_when
Architecture of OpenClaw's embedded agent runtime and SQLite-backed session lifecycle	Embedded agent runtime architecture	Understanding OpenClaw embedded agent runtime design Modifying agent session lifecycle, tooling, provider wiring, or transcript storage Auditing the internal pi-coding-agent dependency boundary

OpenClaw owns the embedded agent runtime. It still imports selected pi-coding-agent packages for agent-loop, provider, and TUI primitives, but runtime identity, prompts, tools, auth selection, session state, transcripts, diagnostics, and persistence are OpenClaw-owned.

Overview

OpenClaw embeds the agent loop in-process instead of spawning an external CLI or using RPC mode. The current implementation constructs the upstream AgentSession through a narrow contract module, then supplies OpenClaw-owned runtime surfaces around it:

• SQLite-backed session and transcript persistence
• OpenClaw tool injection for messaging, sandboxing, VFS, browser, cron, gateway, and channel actions
• OpenClaw system prompt construction per channel, workspace, and context
• Multi-account auth profile rotation with failover
• Provider-agnostic model switching
• Event subscription, streaming, diagnostics, and compaction policy

Legacy JSON, JSONL, and transcript files are doctor migration inputs only. The runtime never chooses a transcript file, derives a transcript locator, or writes session JSONL.

External package boundary

{
  "@earendil-works/pi-agent-core": "0.75.1",
  "@earendil-works/pi-ai": "0.75.1",
  "@earendil-works/pi-coding-agent": "0.75.1",
  "@earendil-works/pi-tui": "0.75.1"
}

OpenClaw treats these as implementation dependencies, not as owners of OpenClaw runtime state.

Package	OpenClaw use
pi-ai	LLM abstractions: Model, streamSimple, message types, provider APIs
pi-agent-core	Agent loop, tool execution, AgentMessage types
pi-coding-agent	Narrow SDK entry: createAgentSession, AuthStorage, ModelRegistry, tool shapes
pi-tui	Terminal UI primitives for OpenClaw's local TUI mode

File structure

Several file names still include pi because they started as the integration layer. Treat them as OpenClaw runtime modules unless the code explicitly imports an upstream package boundary.

src/agents/
├── pi-embedded-runner.ts          # Re-exports from pi-embedded-runner/
├── pi-embedded-runner/
│   ├── run.ts                     # Main entry: runEmbeddedPiAgent()
│   ├── run/
│   │   ├── attempt.ts             # Single attempt logic with session setup
│   │   ├── params.ts              # RunEmbeddedPiAgentParams type
│   │   ├── payloads.ts            # Build response payloads from run results
│   │   ├── images.ts              # Vision model image injection
│   │   └── types.ts               # EmbeddedRunAttemptResult
│   ├── abort.ts                   # Abort error detection
│   ├── cache-ttl.ts               # Cache TTL tracking for context pruning
│   ├── compact.ts                 # Manual/auto compaction logic
│   ├── extensions.ts              # Load pi extensions for embedded runs
│   ├── extra-params.ts            # Provider-specific stream params
│   ├── google.ts                  # Google/Gemini turn ordering fixes
│   ├── history.ts                 # History limiting (DM vs group)
│   ├── lanes.ts                   # Session/global command lanes
│   ├── logger.ts                  # Subsystem logger
│   ├── model.ts                   # Model resolution via ModelRegistry
│   ├── runs.ts                    # Active run tracking, abort, queue
│   ├── sandbox-info.ts            # Sandbox info for system prompt
│   ├── system-prompt.ts           # System prompt builder
│   ├── tool-split.ts              # Split tools into builtIn vs custom
│   ├── types.ts                   # EmbeddedPiAgentMeta, EmbeddedPiRunResult
│   └── utils.ts                   # ThinkLevel mapping, error description
├── transcript/
│   ├── session-transcript-contract.ts # OpenClaw-owned transcript/session types
│   ├── session-manager.ts         # OpenClaw-owned SQLite transcript writer
│   └── transcript-state.ts        # SQLite-backed transcript state adapter
├── pi-embedded-subscribe.ts       # Session event subscription/dispatch
├── pi-embedded-subscribe.types.ts # SubscribeEmbeddedPiSessionParams
├── pi-embedded-subscribe.handlers.ts # Event handler factory
├── pi-embedded-subscribe.handlers.lifecycle.ts
├── pi-embedded-subscribe.handlers.types.ts
├── pi-embedded-block-chunker.ts   # Streaming block reply chunking
├── pi-embedded-messaging.ts       # Messaging tool sent tracking
├── pi-embedded-helpers.ts         # Error classification, turn validation
├── pi-embedded-helpers/           # Helper modules
├── pi-embedded-utils.ts           # Formatting utilities
├── pi-tools.ts                    # createOpenClawCodingTools()
├── pi-tools.abort.ts              # AbortSignal wrapping for tools
├── pi-tools.policy.ts             # Tool allowlist/denylist policy
├── pi-tools.read.ts               # Read tool customizations
├── pi-tools.schema.ts             # Tool schema normalization
├── pi-tools.types.ts              # AnyAgentTool type alias
├── pi-tool-definition-adapter.ts  # AgentTool -> ToolDefinition adapter
├── pi-settings.ts                 # Settings overrides
├── pi-hooks/                      # Custom pi hooks
│   ├── compaction-safeguard.ts    # Safeguard extension
│   ├── compaction-safeguard-runtime.ts
│   ├── context-pruning.ts         # Cache-TTL context pruning extension
│   └── context-pruning/
├── model-auth.ts                  # Auth profile resolution
├── auth-profiles.ts               # Profile store, cooldown, failover
├── model-selection.ts             # Default model resolution
├── models-config.ts               # SQLite model catalog materialization
├── model-catalog.ts               # Model catalog cache
├── context-window-guard.ts        # Context window validation
├── failover-error.ts              # FailoverError class
├── defaults.ts                    # DEFAULT_PROVIDER, DEFAULT_MODEL
├── system-prompt.ts               # buildAgentSystemPrompt()
├── system-prompt-params.ts        # System prompt parameter resolution
├── system-prompt-report.ts        # Debug report generation
├── tool-summaries.ts              # Tool description summaries
├── tool-policy.ts                 # Tool policy resolution
├── transcript-policy.ts           # Transcript validation policy
├── skills.ts                      # Skill snapshot/prompt building
├── skills/                        # Skill subsystem
├── sandbox.ts                     # Sandbox context resolution
├── sandbox/                       # Sandbox subsystem
├── channel-tools.ts               # Channel-specific tool injection
├── openclaw-tools.ts              # OpenClaw-specific tools
├── bash-tools.ts                  # exec/process tools
├── apply-patch.ts                 # apply_patch tool (OpenAI)
├── tools/                         # Individual tool implementations
│   ├── browser-tool.ts
│   ├── canvas-tool.ts
│   ├── cron-tool.ts
│   ├── gateway-tool.ts
│   ├── image-tool.ts
│   ├── message-tool.ts
│   ├── nodes-tool.ts
│   ├── session*.ts
│   ├── web-*.ts
│   └── ...
└── ...

Channel-specific message action runtimes now live in the plugin-owned extension directories instead of under src/agents/tools, for example:

• the Discord plugin action runtime files
• the Slack plugin action runtime file
• the Telegram plugin action runtime file
• the WhatsApp plugin action runtime file

Core integration flow

1. Running an Embedded Agent

The main entry point is still named runEmbeddedPiAgent() in pi-embedded-runner/run.ts. It runs an OpenClaw-owned embedded session:

import { runEmbeddedPiAgent } from "./agents/pi-embedded-runner.js";

const result = await runEmbeddedPiAgent({
  agentId: "main",
  sessionId: "user-123",
  sessionKey: "main:whatsapp:+1234567890",
  workspaceDir: "/path/to/workspace",
  config: openclawConfig,
  prompt: "Hello, how are you?",
  provider: "anthropic",
  model: "claude-sonnet-4-6",
  timeoutMs: 120_000,
  runId: "run-abc",
  onBlockReply: async (payload) => {
    await sendToChannel(payload.text, payload.mediaUrls);
  },
});

2. Session creation

Inside runEmbeddedAttempt() (called by runEmbeddedPiAgent()), OpenClaw creates the upstream session with OpenClaw-owned managers, tools, prompts, auth, and persistence:

import {
  createAgentSession,
  DefaultResourceLoader,
  SettingsManager,
} from "@earendil-works/pi-coding-agent";

const resourceLoader = new DefaultResourceLoader({
  cwd: resolvedWorkspace,
  agentDir,
  settingsManager,
  additionalExtensionPaths,
});
await resourceLoader.reload();

const sessionManager = openTranscriptSessionManagerForSession({
  agentId: params.agentId,
  sessionId: params.sessionId,
});

const { session } = await createAgentSession({
  cwd: resolvedWorkspace,
  agentDir,
  authStorage: params.authStorage,
  modelRegistry: params.modelRegistry,
  model: params.model,
  thinkingLevel: mapThinkingLevel(params.thinkLevel),
  tools: builtInTools,
  customTools: allCustomTools,
  sessionManager,
  settingsManager,
  resourceLoader,
});

applySystemPromptOverrideToSession(session, systemPromptOverride);

3. Event subscription

subscribeEmbeddedPiSession() subscribes to upstream AgentSession events and translates them into OpenClaw callbacks, transcript writes, and streaming reply blocks:

const subscription = subscribeEmbeddedPiSession({
  session: activeSession,
  runId: params.runId,
  verboseLevel: params.verboseLevel,
  reasoningMode: params.reasoningLevel,
  toolResultFormat: params.toolResultFormat,
  onToolResult: params.onToolResult,
  onReasoningStream: params.onReasoningStream,
  onBlockReply: params.onBlockReply,
  onPartialReply: params.onPartialReply,
  onAgentEvent: params.onAgentEvent,
});

Events handled include:

• message_start / message_end / message_update (streaming text/thinking)
• tool_execution_start / tool_execution_update / tool_execution_end
• turn_start / turn_end
• agent_start / agent_end
• compaction_start / compaction_end

4. Prompting

After setup, the session is prompted:

await session.prompt(effectivePrompt, { images: imageResult.images });

The SDK handles the full agent loop: sending to LLM, executing tool calls, streaming responses.

Image injection is prompt-local: OpenClaw loads image refs from the current prompt and passes them via images for that turn only. It does not re-scan older history turns to re-inject image payloads.

Tool architecture

Tool pipeline

1. Upstream shapes: OpenClaw adapts upstream tool definitions where needed
2. Custom replacements: OpenClaw replaces bash with exec/process and customizes read/edit/write for sandbox and VFS behavior
3. OpenClaw tools: messaging, browser, canvas, sessions, cron, gateway, and other runtime tools
4. Channel tools: Discord/Telegram/Slack/WhatsApp-specific action tools
5. Policy filtering: tools filtered by profile, provider, agent, group, and sandbox policy
6. Schema normalization: schemas cleaned for Gemini/OpenAI quirks
7. AbortSignal wrapping: tools wrapped to respect abort signals

Tool definition adapter

pi-agent-core's AgentTool has a different execute signature than pi-coding-agent's ToolDefinition. The adapter in pi-tool-definition-adapter.ts keeps that nullable/signature detail at one boundary:

export function toToolDefinitions(tools: AnyAgentTool[]): ToolDefinition[] {
  return tools.map((tool) => ({
    name: tool.name,
    label: tool.label ?? name,
    description: tool.description ?? "",
    parameters: tool.parameters,
    execute: async (toolCallId, params, onUpdate, _ctx, signal) => {
      // Upstream pi-coding-agent signature differs from pi-agent-core.
      return await tool.execute(toolCallId, params, signal, onUpdate);
    },
  }));
}

Tool split strategy

splitSdkTools() passes all tools via customTools:

export function splitSdkTools(options: { tools: AnyAgentTool[]; sandboxEnabled: boolean }) {
  return {
    builtInTools: [], // Empty. We override everything
    customTools: toToolDefinitions(options.tools),
  };
}

This ensures OpenClaw's policy filtering, sandbox integration, and extended toolset remain consistent across providers.

System prompt construction

The system prompt is built in buildAgentSystemPrompt() (system-prompt.ts). It assembles a full prompt with sections including Tooling, Tool Call Style, Safety guardrails, OpenClaw Control, Skills, Docs, Workspace, Sandbox, Messaging, Assistant Output Directives, Voice, Silent Replies, Heartbeats, Runtime metadata, plus Memory and Reactions when enabled, and optional context files and extra system prompt content. Sections are trimmed for minimal prompt mode used by subagents.

The prompt is applied after session creation via applySystemPromptOverrideToSession():

const systemPromptOverride = createSystemPromptOverride(appendPrompt);
applySystemPromptOverrideToSession(session, systemPromptOverride);

Session management

Session transcripts

Sessions are SQLite-backed event streams with tree structure (id/parentId linking). JSONL is legacy doctor-import input only; OpenClaw runtime code does not create, select, or bridge through transcript files or locators. OpenClaw owns the transcript writer behind src/agents/transcript/session-transcript-contract.ts:

const sessionManager = openTranscriptSessionManagerForSession({
  agentId: params.agentId,
  sessionId: params.sessionId,
});

OpenClaw wraps this with guardSessionManager() for tool result safety.

History limiting

limitHistoryTurns() trims conversation history based on channel type (DM vs group).

Compaction

Auto-compaction triggers on context overflow. Common overflow signatures include request_too_large, context length exceeded, input exceeds the maximum number of tokens, input token count exceeds the maximum number of input tokens, input is too long for the model, and ollama error: context length exceeded. compactEmbeddedPiSessionDirect() handles manual compaction:

const compactResult = await compactEmbeddedPiSessionDirect({
  agentId, sessionId, provider, model, ...
});

Authentication and model resolution

Auth profiles

OpenClaw maintains an auth profile store with multiple API keys per provider:

const authStore = ensureAuthProfileStore(agentDir, { allowKeychainPrompt: false });
const profileOrder = resolveAuthProfileOrder({ cfg, store: authStore, provider, preferredProfile });

Profiles rotate on failures with cooldown tracking:

await markAuthProfileFailure({ store, profileId, reason, cfg, agentDir });
const rotated = await advanceAuthProfile();

Model resolution

import { resolveModel } from "./pi-embedded-runner/model.js";

const { model, error, authStorage, modelRegistry } = resolveModel(
  provider,
  modelId,
  agentDir,
  config,
);

// Uses pi's ModelRegistry and AuthStorage
authStorage.setRuntimeApiKey(model.provider, apiKeyInfo.apiKey);

Failover

FailoverError triggers model fallback when configured:

if (fallbackConfigured && isFailoverErrorMessage(errorText)) {
  throw new FailoverError(errorText, {
    reason: promptFailoverReason ?? "unknown",
    provider,
    model: modelId,
    profileId,
    status: resolveFailoverStatus(promptFailoverReason),
  });
}

Runtime extensions

OpenClaw loads custom runtime extensions for specialized behavior. These extensions use the upstream extension mechanism, but their policy and state are OpenClaw-owned.

Compaction safeguard

src/agents/pi-hooks/compaction-safeguard.ts adds guardrails to compaction, including adaptive token budgeting plus tool failure and file operation summaries:

if (resolveCompactionMode(params.cfg) === "safeguard") {
  setCompactionSafeguardRuntime(params.sessionManager, { maxHistoryShare });
  paths.push(resolvePiExtensionPath("compaction-safeguard"));
}

Context pruning

src/agents/pi-hooks/context-pruning.ts implements cache-TTL based context pruning:

if (cfg?.agents?.defaults?.contextPruning?.mode === "cache-ttl") {
  setContextPruningRuntime(params.sessionManager, {
    settings,
    contextWindowTokens,
    isToolPrunable,
    lastCacheTouchAt,
  });
  paths.push(resolvePiExtensionPath("context-pruning"));
}

Streaming and block replies

Block chunking

EmbeddedBlockChunker manages streaming text into discrete reply blocks:

const blockChunker = blockChunking ? new EmbeddedBlockChunker(blockChunking) : null;

Thinking/Final Tag Stripping

Streaming output is processed to strip <think>/<thinking> blocks and extract <final> content:

const stripBlockTags = (text: string, state: { thinking: boolean; final: boolean }) => {
  // Strip <think>...</think> content
  // If enforceFinalTag, only return <final>...</final> content
};

Reply directives

Reply directives like [[media:url]], [[voice]], [[reply:id]] are parsed and extracted:

const { text: cleanedText, mediaUrls, audioAsVoice, replyToId } = consumeReplyDirectives(chunk);

Error handling

Error classification

pi-embedded-helpers.ts classifies errors for appropriate handling:

isContextOverflowError(errorText)     // Context too large
isCompactionFailureError(errorText)   // Compaction failed
isAuthAssistantError(lastAssistant)   // Auth failure
isRateLimitAssistantError(...)        // Rate limited
isFailoverAssistantError(...)         // Should failover
classifyFailoverReason(errorText)     // "auth" | "rate_limit" | "quota" | "timeout" | ...

Thinking level fallback

If a thinking level is unsupported, it falls back:

const fallbackThinking = pickFallbackThinkingLevel({
  message: errorText,
  attempted: attemptedThinking,
});
if (fallbackThinking) {
  thinkLevel = fallbackThinking;
  continue;
}

Sandbox integration

When sandbox mode is enabled, tools and paths are constrained:

const sandbox = await resolveSandboxContext({
  config: params.config,
  sessionKey: sandboxSessionKey,
  workspaceDir: resolvedWorkspace,
});

if (sandboxRoot) {
  // Use sandboxed read/edit/write tools
  // Exec runs in container
  // Browser uses bridge URL
}

Provider-Specific Handling

Anthropic

• Refusal magic string scrubbing
• Turn validation for consecutive roles
• Strict upstream Pi tool parameter validation

Google/Gemini

• Plugin-owned tool schema sanitization

OpenAI

• apply_patch tool for Codex models
• Thinking level downgrade handling

TUI Integration

OpenClaw also has a local TUI mode that uses pi-tui components directly:

// src/tui/tui.ts
import { ... } from "@earendil-works/pi-tui";

This provides OpenClaw's interactive terminal experience without moving session state back to upstream files.

Key differences from the upstream CLI

Aspect	Upstream CLI	OpenClaw embedded
Invocation	External command / RPC	In-process session via createAgentSession()
Tools	Default coding tools	Custom OpenClaw tool suite
System prompt	Upstream prompt stack	Dynamic OpenClaw prompt per channel, workspace, and context
Session storage	~/.pi/agent/sessions/	$OPENCLAW_STATE_DIR/state/openclaw.sqlite plus $OPENCLAW_STATE_DIR/agents/<agentId>/agent/openclaw-agent.sqlite
Auth	Single credential	Multi-profile with rotation
Extensions	Loaded from disk	OpenClaw policy with programmatic and disk paths
Event handling	TUI rendering	Callback-based (onBlockReply, etc.)

Future considerations

Areas for potential rework:

1. Naming cleanup: Historical pi-* file names can move toward OpenClaw runtime names once imports are fully quarantined.
2. Tool signature alignment: Upstream tool signature adapters should stay at one boundary.
3. Transcript writer wrapping: guardSessionManager adds tool-result safety around the SQLite writer but increases complexity.
4. Extension loading: OpenClaw should keep policy ownership while shrinking the integration surface.
5. Streaming handler complexity: subscribeEmbeddedPiSession has grown large.
6. Provider quirks: Provider-specific codepaths should keep moving toward owner modules or typed runtime helpers.

Tests

Embedded runtime coverage spans these suites:

• src/agents/pi-*.test.ts
• src/agents/pi-embedded-*.test.ts
• src/agents/pi-embedded-helpers*.test.ts
• src/agents/pi-embedded-runner*.test.ts
• src/agents/pi-embedded-runner/**/*.test.ts
• src/agents/pi-embedded-subscribe*.test.ts
• src/agents/pi-tools*.test.ts
• src/agents/pi-tool-definition-adapter*.test.ts
• src/agents/pi-settings.test.ts
• src/agents/pi-hooks/**/*.test.ts

Live/opt-in:

• src/agents/pi-embedded-runner-extraparams.live.test.ts (enable OPENCLAW_LIVE_TEST=1)

For current run commands, see Pi Development Workflow.

Related

• Pi development workflow
• Install overview