Gateway: fix post-compaction amnesia for injected messages (#12283)

* Gateway: preserve Pi transcript parentId for injected messages

Thread: unknown
When: 2026-02-08 20:08 CST
Repo: https://github.com/openclaw/openclaw.git
Branch: codex/wip/2026-02-09/compact-post-compaction-parentid-fix

Problem
- Post-compaction turns sometimes lost the compaction summary + kept suffix in the *next* provider request.
- Root cause was session graph corruption: gateway appended "stopReason: injected" transcript lines via raw JSONL writes without `parentId`.
- Pi's `SessionManager.buildSessionContext()` walks the `parentId` chain from the current leaf; missing `parentId` can sever the active branch and hide the compaction entry.

Fix
- Use `SessionManager.appendMessage(...)` for injected assistant transcript writes so `parentId` is set to the current leaf.
- Route `chat.inject` through the same helper to avoid duplicating the broken raw JSONL append logic.

Why This Matters
- The compaction algorithm may be correct, but if the leaf chain is broken right after compaction, the provider payload cannot include the summary/suffix "shape" Pi expects.

Testing
- pnpm test src/agents/pi-embedded-helpers.post-compaction-shape.test.ts src/agents/pi-embedded-runner/run.overflow-compaction.post-context.test.ts
- pnpm build

Notes
- This is provider-shape agnostic: it fixes transcript structure so Anthropic/Gemini/etc all see the same post-compaction context.

Resume
- If post-compaction looks wrong again, inspect the session transcript for entries missing `parentId` immediately after `type: compaction`.

* Gateway: guardrail test for transcript parentId (chat.inject)

* Gateway: guardrail against raw transcript appends (chat.ts)

* Gateway: add local AGENTS.md note to preserve Pi transcript parentId chain

* Changelog: note gateway post-compaction amnesia fix

* Gateway: store injected transcript messages with valid stopReason

* Gateway: use valid stopReason in injected fallback

CHANGELOG.md

@@ -27,6 +27,7 @@ Docs: https://docs.openclaw.ai
 - Cron: route text-only isolated agent announces through the shared subagent announce flow; add exponential backoff for repeated errors; preserve future `nextRunAtMs` on restart; include current-boundary schedule matches; prevent stale threadId reuse across targets; and add per-job execution timeout. (#11641) Thanks @tyler6204.
 - Subagents: stabilize announce timing, preserve compaction metrics across retries, clamp overflow-prone long timeouts, and cap impossible context usage token totals. (#11551) Thanks @tyler6204.
 - Agents: recover from context overflow caused by oversized tool results (pre-emptive capping + fallback truncation). (#11579) Thanks @tyler6204.
+- Gateway: no more post-compaction amnesia; injected transcript writes now preserve Pi session `parentId` chain so agents can remember again. Thanks @Takhoffman 🦞.
 - Telegram: render markdown spoilers with `<tg-spoiler>` HTML tags. (#11543) Thanks @ezhikkk.
 - Telegram: recover proactive sends when stale topic thread IDs are used by retrying without `message_thread_id`, and clear explicit no-thread route updates instead of inheriting stale thread state. (#11620)
 - Gateway/CLI: when `gateway.bind=lan`, use a LAN IP for probe URLs and Control UI links. (#11448) Thanks @AnonO6.

src/gateway/server-methods/AGENTS.md

@@ -0,0 +1,3 @@
+# Gateway Server Methods Notes
+
+- Pi session transcripts are a `parentId` chain/DAG; never append Pi `type: "message"` entries via raw JSONL writes (missing `parentId` can sever the leaf path and break compaction/history). Always write transcript messages via `SessionManager.appendMessage(...)` (or a wrapper that uses it).

src/gateway/server-methods/chat.inject.parentid.test.ts

@@ -0,0 +1,72 @@
+import { CURRENT_SESSION_VERSION } from "@mariozechner/pi-coding-agent";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { describe, expect, it, vi } from "vitest";
+import type { GatewayRequestContext } from "./types.js";
+
+// Guardrail: Ensure gateway "injected" assistant transcript messages are appended via SessionManager,
+// so they are attached to the current leaf with a `parentId` and do not sever compaction history.
+describe("gateway chat.inject transcript writes", () => {
+  it("appends a Pi session entry that includes parentId", async () => {
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-chat-inject-"));
+    const transcriptPath = path.join(dir, "sess.jsonl");
+
+    // Minimal Pi session header so SessionManager can open/append safely.
+    fs.writeFileSync(
+      transcriptPath,
+      `${JSON.stringify({
+        type: "session",
+        version: CURRENT_SESSION_VERSION,
+        id: "sess-1",
+        timestamp: new Date(0).toISOString(),
+        cwd: "/tmp",
+      })}\n`,
+      "utf-8",
+    );
+
+    vi.doMock("../session-utils.js", async (importOriginal) => {
+      const original = await importOriginal();
+      return {
+        ...original,
+        loadSessionEntry: () => ({
+          storePath: "/tmp/store.json",
+          entry: {
+            sessionId: "sess-1",
+            sessionFile: transcriptPath,
+          },
+        }),
+      };
+    });
+
+    const { chatHandlers } = await import("./chat.js");
+
+    const respond = vi.fn();
+    type InjectCtx = Pick<GatewayRequestContext, "broadcast" | "nodeSendToSession">;
+    const context: InjectCtx = {
+      broadcast: vi.fn() as unknown as InjectCtx["broadcast"],
+      nodeSendToSession: vi.fn() as unknown as InjectCtx["nodeSendToSession"],
+    };
+    await chatHandlers["chat.inject"]({
+      params: { sessionKey: "k1", message: "hello" },
+      respond,
+      context,
+    });
+
+    expect(respond).toHaveBeenCalled();
+    const [, payload, error] = respond.mock.calls.at(-1) ?? [];
+    expect(error).toBeUndefined();
+    expect(payload).toMatchObject({ ok: true });
+
+    const lines = fs.readFileSync(transcriptPath, "utf-8").split(/\r?\n/).filter(Boolean);
+    expect(lines.length).toBeGreaterThanOrEqual(2);
+
+    const last = JSON.parse(lines.at(-1) as string) as Record<string, unknown>;
+    expect(last.type).toBe("message");
+
+    // The regression we saw: raw jsonl appends omitted this field entirely.
+    expect(Object.prototype.hasOwnProperty.call(last, "parentId")).toBe(true);
+    expect(last).toHaveProperty("id");
+    expect(last).toHaveProperty("message");
+  });
+});

src/gateway/server-methods/chat.transcript-writes.guardrail.test.ts

@@ -0,0 +1,23 @@
+import fs from "node:fs";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+
+// Guardrail: the "empty post-compaction context" regression came from gateway code appending
+// Pi transcript message entries as raw JSONL without `parentId`.
+//
+// This test is intentionally simple and file-local: if someone reintroduces direct JSONL appends
+// against `transcriptPath`, Pi's SessionManager parent chain can break again.
+describe("gateway chat transcript writes (guardrail)", () => {
+  it("does not append transcript messages via raw fs.appendFileSync(transcriptPath, ...)", () => {
+    const chatTs = fileURLToPath(new URL("./chat.ts", import.meta.url));
+    const src = fs.readFileSync(chatTs, "utf-8");
+
+    // Disallow raw appends against the resolved transcript path variable.
+    // (The transcript header creation via writeFileSync is OK; the bug class is raw message appends.)
+    expect(src.includes("fs.appendFileSync(transcriptPath")).toBe(false);
+
+    // Ensure we keep using SessionManager for transcript message appends.
+    expect(src).toContain("SessionManager.open(transcriptPath)");
+    expect(src).toContain("appendMessage(");
+  });
+});

src/gateway/server-methods/chat.ts

@@ -1,5 +1,4 @@
-import { CURRENT_SESSION_VERSION } from "@mariozechner/pi-coding-agent";
-import { randomUUID } from "node:crypto";
+import { CURRENT_SESSION_VERSION, SessionManager } from "@mariozechner/pi-coding-agent";
 import fs from "node:fs";
 import path from "node:path";
 import type { MsgContext } from "../../auto-reply/templating.js";
@@ -47,6 +46,8 @@ type TranscriptAppendResult = {
   error?: string;
 };
 
+type AppendMessageArg = Parameters<SessionManager["appendMessage"]>[0];
+
 function resolveTranscriptPath(params: {
   sessionId: string;
   storePath: string | undefined;
@@ -116,29 +117,44 @@ function appendAssistantTranscriptMessage(params: {
   }
 
   const now = Date.now();
-  const messageId = randomUUID().slice(0, 8);
   const labelPrefix = params.label ? `[${params.label}]\n\n` : "";
-  const messageBody: Record<string, unknown> = {
+  const usage = {
+    input: 0,
+    output: 0,
+    cacheRead: 0,
+    cacheWrite: 0,
+    totalTokens: 0,
+    cost: {
+      input: 0,
+      output: 0,
+      cacheRead: 0,
+      cacheWrite: 0,
+      total: 0,
+    },
+  };
+  const messageBody: AppendMessageArg & Record<string, unknown> = {
     role: "assistant",
     content: [{ type: "text", text: `${labelPrefix}${params.message}` }],
     timestamp: now,
-    stopReason: "injected",
-    usage: { input: 0, output: 0, totalTokens: 0 },
-  };
-  const transcriptEntry = {
-    type: "message",
-    id: messageId,
-    timestamp: new Date(now).toISOString(),
-    message: messageBody,
+    // Pi stopReason is a strict enum; this is not model output, but we still store it as a
+    // normal assistant message so it participates in the session parentId chain.
+    stopReason: "stop",
+    usage,
+    // Make these explicit so downstream tooling never treats this as model output.
+    api: "openai-responses",
+    provider: "openclaw",
+    model: "gateway-injected",
   };
 
   try {
-    fs.appendFileSync(transcriptPath, `${JSON.stringify(transcriptEntry)}\n`, "utf-8");
+    // IMPORTANT: Use SessionManager so the entry is attached to the current leaf via parentId.
+    // Raw jsonl appends break the parent chain and can hide compaction summaries from context.
+    const sessionManager = SessionManager.open(transcriptPath);
+    const messageId = sessionManager.appendMessage(messageBody);
+    return { ok: true, messageId, message: messageBody };
   } catch (err) {
     return { ok: false, error: err instanceof Error ? err.message : String(err) };
   }
-
-  return { ok: true, messageId, message: transcriptEntry.message };
 }
 
 function nextChatSeq(context: { agentRunSeq: Map<string, number> }, runId: string) {
@@ -554,7 +570,9 @@ export const chatHandlers: GatewayRequestHandlers = {
                   role: "assistant",
                   content: [{ type: "text", text: combinedReply }],
                   timestamp: now,
-                  stopReason: "injected",
+                  // Keep this compatible with Pi stopReason enums even though this message isn't
+                  // persisted to the transcript due to the append failure.
+                  stopReason: "stop",
                   usage: { input: 0, output: 0, totalTokens: 0 },
                 };
               }
@@ -640,62 +658,37 @@ export const chatHandlers: GatewayRequestHandlers = {
       return;
     }
 
-    // Resolve transcript path
-    const transcriptPath = entry?.sessionFile
-      ? entry.sessionFile
-      : path.join(path.dirname(storePath), `${sessionId}.jsonl`);
-
-    if (!fs.existsSync(transcriptPath)) {
+    const appended = appendAssistantTranscriptMessage({
+      message: p.message,
+      label: p.label,
+      sessionId,
+      storePath,
+      sessionFile: entry?.sessionFile,
+      createIfMissing: false,
+    });
+    if (!appended.ok || !appended.messageId || !appended.message) {
       respond(
         false,
         undefined,
-        errorShape(ErrorCodes.INVALID_REQUEST, "transcript file not found"),
-      );
-      return;
-    }
-
-    // Build transcript entry
-    const now = Date.now();
-    const messageId = randomUUID().slice(0, 8);
-    const labelPrefix = p.label ? `[${p.label}]\n\n` : "";
-    const messageBody: Record<string, unknown> = {
-      role: "assistant",
-      content: [{ type: "text", text: `${labelPrefix}${p.message}` }],
-      timestamp: now,
-      stopReason: "injected",
-      usage: { input: 0, output: 0, totalTokens: 0 },
-    };
-    const transcriptEntry = {
-      type: "message",
-      id: messageId,
-      timestamp: new Date(now).toISOString(),
-      message: messageBody,
-    };
-
-    // Append to transcript file
-    try {
-      fs.appendFileSync(transcriptPath, `${JSON.stringify(transcriptEntry)}\n`, "utf-8");
-    } catch (err) {
-      const errMessage = err instanceof Error ? err.message : String(err);
-      respond(
-        false,
-        undefined,
-        errorShape(ErrorCodes.UNAVAILABLE, `failed to write transcript: ${errMessage}`),
+        errorShape(
+          ErrorCodes.UNAVAILABLE,
+          `failed to write transcript: ${appended.error ?? "unknown error"}`,
+        ),
       );
       return;
     }
 
     // Broadcast to webchat for immediate UI update
     const chatPayload = {
-      runId: `inject-${messageId}`,
+      runId: `inject-${appended.messageId}`,
       sessionKey: rawSessionKey,
       seq: 0,
       state: "final" as const,
-      message: transcriptEntry.message,
+      message: appended.message,
     };
     context.broadcast("chat", chatPayload);
     context.nodeSendToSession(rawSessionKey, "chat", chatPayload);
 
-    respond(true, { ok: true, messageId });
+    respond(true, { ok: true, messageId: appended.messageId });
   },
 };