mirror of
https://github.com/openclaw/openclaw.git
synced 2026-07-18 13:11:35 +00:00
* refactor(sessions): migrate runtime storage to sqlite * test(sessions): fix sqlite CI regressions * test(sessions): align remaining sqlite fixtures * fix(codex): require sqlite trajectory recorder * test(sessions): align orphan recovery sqlite fixture * test(sessions): align sqlite rebase fixtures * fix(sessions): finish current-main integration of the sqlite flip Resolve the whole-store SDK removal across its owner boundary: drop the loadSessionStore re-export and the registry whole-store wrappers, wire hasTrackedActiveSessionRun into gateway chat, complete the preserveLockedHarnessIds cleanup contract, flip the codex thread-history import to storePath targets, and port remaining main-side tests from file-store helpers to session accessor reads. * chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain bump gets its own review, and regenerate docs_map, the plugin SDK API baseline, and the export-surface ratchet for the merged tree. * feat(sessions): keep archived transcripts by default with zstd cold storage Codex-style retention: deleting or resetting a session archives its transcript as a zstd-compressed JSONL artifact (plain when the runtime lacks node:zlib zstd) and keeps it until the disk budget evicts oldest first. resetArchiveRetention now governs both deleted and reset archives and defaults to keep; maxDiskBytes defaults to 2gb so retention stays bounded, with archives evicted before live sessions. The cron reaper follows the same knob instead of deleting archives on its own timer. * fix(state): converge agent DB migration lineages and bound database growth Merge coherence: run both structure-gated legacy memory-schema repairs (flip-lineage drop, main-lineage identity rebuild) before the flip migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all converge, and hoist foreign_keys=OFF outside the schema transaction where the pragma was silently ignored and the v1 sessions rebuild cascade-deleted session_entries. Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL maintenance releases freed pages in bounded passes (never a blocking full VACUUM), and doctor reports state/agent DB bloat from freelist stats. * fix(codex): resolve the store path for thread-history import via the SDK The supervision catalog passed the legacy sessionFile locator to the storePath-targeted transcript mirror; resolve the agent store path with the session-store SDK helper instead of a runtime-object seam so test fakes and headless callers need no extra surface. Drop the obsolete missing-session-id preprocessing case: sessions rows are NOT NULL on session_id and upsert repairs id-less patches at write time. * fix(sessions): fail safe on malformed disk-budget config and doctor stat errors A malformed explicit maxDiskBytes disables the budget instead of falling back to the destructive 2gb default the user never chose, and the doctor bloat check skips databases whose paths stat-fail instead of aborting doctor. * fix(sessions): complete sqlite conflict translations * test(sqlite): align hardening checks with maintenance * test(sessions): inspect compressed transcript archives * fix(tests): await session seeds and drop unused helpers flagged by CI lint The five unawaited writeSessionStoreSeed calls raced their SQLite seeds against the assertions, failing compact shards; the bloat probe drops a useless initializer and the merged tests drop now-unused helpers. * test(sessions): type legacy proof events directly * test(sessions): align hardening contracts * perf(sessions): read usage transcript sizes from SQL aggregates Usage/cost scans walked every session and materialized every transcript event just to re-stringify it for a byte estimate — the #86718 stall class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes in SQLite without loading a single row. * fix(sessions): re-root foreign-root transcript paths onto the current sessions dir Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry absolute sessionFile paths from the old root; the containment fallback kept those foreign paths, so migration read (and would archive) files in the original root and reported local copies missing. Re-root the canonical agents/<id>/sessions suffix onto the current dir when the file exists there; genuine cross-root layouts still fall through unchanged. * test(agents): seed harness admission through sqlite * fix(sqlite): close agent db on pragma setup failure * fix(doctor): compact and retrofit incremental auto-vacuum after session import The migration is the sanctioned offline window: post-import compact reclaims import churn and applies auto_vacuum=INCREMENTAL to databases created before the fresh-DB pragma existed, so runtime maintenance can release pages in bounded passes on every install. --------- Co-authored-by: Peter Steinberger <steipete@gmail.com>
161 lines
5.5 KiB
TypeScript
161 lines
5.5 KiB
TypeScript
// Chat transcript injection appends gateway-authored assistant rows while
|
|
// preserving agent-session parent links and transcript update notifications.
|
|
import type { SessionManager } from "../../agents/sessions/session-manager.js";
|
|
import { persistSessionTranscriptTurn } from "../../config/sessions/session-accessor.js";
|
|
import type { OpenClawConfig } from "../../config/types.openclaw.js";
|
|
import { formatErrorMessage } from "../../infra/errors.js";
|
|
|
|
type AppendMessageArg = Parameters<SessionManager["appendMessage"]>[0];
|
|
|
|
/** Metadata persisted on gateway-injected assistant messages that mark a stopped run. */
|
|
export type GatewayInjectedAbortMeta = {
|
|
aborted: true;
|
|
origin: "rpc" | "stop-command";
|
|
runId: string;
|
|
};
|
|
|
|
/** Result shape returned after appending an assistant row to a session transcript. */
|
|
export type GatewayInjectedTranscriptAppendResult = {
|
|
ok: boolean;
|
|
messageId?: string;
|
|
message?: Record<string, unknown>;
|
|
error?: string;
|
|
};
|
|
|
|
/** Hash marker used to dedupe companion TTS text/audio supplements. */
|
|
export type GatewayInjectedTtsSupplementMarker = {
|
|
textSha256: string;
|
|
};
|
|
|
|
function resolveInjectedAssistantContent(params: {
|
|
message: string;
|
|
label?: string;
|
|
content?: Array<Record<string, unknown>>;
|
|
}): Array<Record<string, unknown>> {
|
|
const labelPrefix = params.label ? `[${params.label}]\n\n` : "";
|
|
// Preserve rich content arrays when callers already prepared media blocks;
|
|
// only the first text block is rewritten so block ordering stays intact.
|
|
if (params.content && params.content.length > 0) {
|
|
if (!labelPrefix) {
|
|
return params.content;
|
|
}
|
|
const first = params.content[0];
|
|
if (
|
|
first &&
|
|
typeof first === "object" &&
|
|
first.type === "text" &&
|
|
typeof first.text === "string"
|
|
) {
|
|
return [{ ...first, text: `${labelPrefix}${first.text}` }, ...params.content.slice(1)];
|
|
}
|
|
return [{ type: "text", text: labelPrefix.trim() }, ...params.content];
|
|
}
|
|
return [{ type: "text", text: `${labelPrefix}${params.message}` }];
|
|
}
|
|
|
|
/** Append a gateway-authored assistant message while preserving transcript parent links. */
|
|
export async function appendInjectedAssistantMessageToTranscript(params: {
|
|
transcriptPath?: string;
|
|
storePath?: string;
|
|
sessionId?: string;
|
|
sessionKey?: string;
|
|
agentId?: string;
|
|
message: string;
|
|
label?: string;
|
|
/** When set, used as the assistant `content` array (e.g. text + embedded audio blocks). */
|
|
content?: Array<Record<string, unknown>>;
|
|
idempotencyKey?: string;
|
|
abortMeta?: GatewayInjectedAbortMeta;
|
|
ttsSupplement?: GatewayInjectedTtsSupplementMarker;
|
|
now?: number;
|
|
config?: OpenClawConfig;
|
|
}): Promise<GatewayInjectedTranscriptAppendResult> {
|
|
const now = params.now ?? Date.now();
|
|
const usage = {
|
|
input: 0,
|
|
output: 0,
|
|
cacheRead: 0,
|
|
cacheWrite: 0,
|
|
totalTokens: 0,
|
|
cost: {
|
|
input: 0,
|
|
output: 0,
|
|
cacheRead: 0,
|
|
cacheWrite: 0,
|
|
total: 0,
|
|
},
|
|
};
|
|
const resolvedContent = resolveInjectedAssistantContent({
|
|
message: params.message,
|
|
label: params.label,
|
|
content: params.content,
|
|
});
|
|
const messageBody: AppendMessageArg & Record<string, unknown> = {
|
|
role: "assistant",
|
|
// Gateway-injected assistant messages can include non-model content blocks (e.g. embedded TTS audio).
|
|
content: resolvedContent as unknown as Extract<
|
|
AppendMessageArg,
|
|
{ role: "assistant" }
|
|
>["content"],
|
|
timestamp: now,
|
|
// stopReason is a strict runner enum; this is not model output, but we still store it as a
|
|
// normal assistant message so it participates in the session parentId chain.
|
|
stopReason: "stop",
|
|
usage,
|
|
// Make these explicit so downstream tooling never treats this as model output.
|
|
api: "openai-responses",
|
|
provider: "openclaw",
|
|
model: "gateway-injected",
|
|
...(params.idempotencyKey ? { idempotencyKey: params.idempotencyKey } : {}),
|
|
...(params.ttsSupplement ? { openclawTtsSupplement: params.ttsSupplement } : {}),
|
|
...(params.abortMeta
|
|
? {
|
|
openclawAbort: {
|
|
aborted: true,
|
|
origin: params.abortMeta.origin,
|
|
runId: params.abortMeta.runId,
|
|
},
|
|
}
|
|
: {}),
|
|
};
|
|
|
|
try {
|
|
if (!params.transcriptPath && (!params.storePath || !params.sessionId || !params.sessionKey)) {
|
|
return { ok: false, error: "transcript identity not resolved" };
|
|
}
|
|
const turn = await persistSessionTranscriptTurn(
|
|
{
|
|
sessionKey: params.sessionKey ?? "",
|
|
...(params.transcriptPath ? { sessionFile: params.transcriptPath } : {}),
|
|
...(params.storePath ? { storePath: params.storePath } : {}),
|
|
...(params.sessionId ? { sessionId: params.sessionId } : {}),
|
|
...(params.agentId ? { agentId: params.agentId } : {}),
|
|
},
|
|
{
|
|
updateMode: "inline",
|
|
touchSessionEntry: Boolean(params.storePath && params.sessionId && params.sessionKey),
|
|
...(params.config ? { config: params.config } : {}),
|
|
messages: [
|
|
{
|
|
message: messageBody,
|
|
idempotencyLookup: "scan-assistant",
|
|
now,
|
|
useRawWhenLinear: true,
|
|
},
|
|
],
|
|
},
|
|
);
|
|
const appended = turn.messages[0];
|
|
if (!appended) {
|
|
return { ok: false, error: "gateway-injected assistant message was not appended" };
|
|
}
|
|
return {
|
|
ok: true,
|
|
messageId: appended.messageId,
|
|
message: appended.message as Record<string, unknown>,
|
|
};
|
|
} catch (err) {
|
|
return { ok: false, error: formatErrorMessage(err) };
|
|
}
|
|
}
|