Files
openclaw/src/gateway/session-utils.single-row-cache.test.ts
Josh Lehman 0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00

323 lines
10 KiB
TypeScript

/**
* Tests single-row session cache behavior in gateway session utilities.
*/
import { afterEach, describe, expect, test, vi } from "vitest";
import { resetConfigRuntimeState, setRuntimeConfigSnapshot } from "../config/config.js";
import type { OpenClawConfig } from "../config/config.js";
import { resolveStorePath, type SessionEntry } from "../config/sessions.js";
import { replaceSessionEntry, updateSessionEntry } from "../config/sessions/session-accessor.js";
import { resetPluginRuntimeStateForTest } from "../plugins/runtime.js";
import { withStateDirEnv } from "../test-helpers/state-dir-env.js";
const subagentRegistryReadMock = vi.hoisted(() => {
let runsByChildSessionKey = new Map<string, Record<string, unknown>>();
const buildSubagentRunReadIndex = vi.fn(() => {
const runsByControllerSessionKey = new Map<string, Record<string, unknown>[]>();
for (const entry of runsByChildSessionKey.values()) {
const controllerSessionKey =
typeof entry.controllerSessionKey === "string"
? entry.controllerSessionKey
: typeof entry.requesterSessionKey === "string"
? entry.requesterSessionKey
: undefined;
if (!controllerSessionKey) {
continue;
}
const runs = runsByControllerSessionKey.get(controllerSessionKey) ?? [];
runs.push(entry);
runsByControllerSessionKey.set(controllerSessionKey, runs);
}
return {
runsByControllerSessionKey,
getDisplaySubagentRun: vi.fn(
(childSessionKey: string) => runsByChildSessionKey.get(childSessionKey) ?? null,
),
countActiveDescendantRuns: vi.fn(() => 0),
};
});
return {
buildSubagentRunReadIndex,
countActiveDescendantRuns: vi.fn(() => 0),
getSessionDisplaySubagentRunByChildSessionKey: vi.fn(
(childSessionKey: string) => runsByChildSessionKey.get(childSessionKey) ?? null,
),
getSubagentSessionRuntimeMs: vi.fn(() => undefined),
getSubagentSessionStartedAt: vi.fn(() => undefined),
isSubagentRunLive: vi.fn(() => false),
listSubagentRunsForController: vi.fn((controllerSessionKey: string) =>
[...runsByChildSessionKey.values()].filter((entry) => {
const controller =
typeof entry.controllerSessionKey === "string"
? entry.controllerSessionKey
: typeof entry.requesterSessionKey === "string"
? entry.requesterSessionKey
: undefined;
return controller === controllerSessionKey;
}),
),
resolveSubagentSessionStatus: vi.fn(() => undefined),
setSubagentRunsForTest: (runs: Record<string, unknown>[]) => {
runsByChildSessionKey = new Map(
runs
.filter((entry) => typeof entry.childSessionKey === "string")
.map((entry) => [entry.childSessionKey as string, entry]),
);
},
};
});
vi.mock("../agents/subagent-registry-read.js", () => subagentRegistryReadMock);
import {
listSessionsFromStore,
listSessionsFromStoreAsync,
loadGatewaySessionRow,
} from "./session-utils.js";
const MAIN_AGENT_ID = "main";
const TEST_MODEL = "openai/gpt-5.4";
type SingleRowCacheContext = {
now: number;
storePath: string;
};
type MovingChildFixture = {
oldParent: string;
newParent: string;
child: string;
store: Record<string, SessionEntry>;
};
async function withSingleRowCacheStore(
statePrefix: string,
workspace: string,
run: (context: SingleRowCacheContext) => Promise<void>,
): Promise<void> {
await withStateDirEnv(statePrefix, async () => {
const cfg: OpenClawConfig = {
agents: {
list: [
{
id: MAIN_AGENT_ID,
default: true,
workspace,
},
],
defaults: { model: { primary: TEST_MODEL } },
},
} as OpenClawConfig;
setRuntimeConfigSnapshot(cfg, cfg);
await run({
now: Math.floor(Date.now() / 1_000) * 1_000 + 100,
storePath: resolveStorePath(cfg.session?.store, { agentId: MAIN_AGENT_ID }),
});
});
}
function parentSession(sessionId: string, now: number): SessionEntry {
return {
sessionId,
updatedAt: now,
};
}
function runningChildSession(
sessionId: string,
parentSessionKey: string,
now: number,
): SessionEntry {
return {
sessionId,
parentSessionKey,
updatedAt: now,
status: "running",
};
}
async function seedSessionEntries(
storePath: string,
store: Record<string, SessionEntry>,
): Promise<void> {
for (const [sessionKey, entry] of Object.entries(store)) {
await replaceSessionEntry({ sessionKey, storePath }, entry);
}
}
function setSubagentControllerRun(
childSessionKey: string,
controllerSessionKey: string,
createdAt: number,
): void {
subagentRegistryReadMock.setSubagentRunsForTest([
{
childSessionKey,
controllerSessionKey,
requesterSessionKey: controllerSessionKey,
createdAt,
},
]);
}
function createMovingChildFixture(now: number): MovingChildFixture {
const oldParent = "agent:main:subagent:parent-old";
const newParent = "agent:main:subagent:parent-new";
const child = "agent:main:subagent:child";
return {
oldParent,
newParent,
child,
store: {
[oldParent]: parentSession("parent-old", now),
[newParent]: parentSession("parent-new", now),
[child]: runningChildSession("child", oldParent, now),
},
};
}
function expectChildMovedToNewParent(fixture: MovingChildFixture, now: number): void {
expect(
loadGatewaySessionRow(fixture.oldParent, { now: now + 50 })?.childSessions,
).toBeUndefined();
expect(loadGatewaySessionRow(fixture.newParent, { now: now + 50 })?.childSessions).toEqual([
fixture.child,
]);
expect(subagentRegistryReadMock.buildSubagentRunReadIndex).not.toHaveBeenCalled();
}
describe("single gateway session row child-session cache", () => {
afterEach(() => {
resetConfigRuntimeState();
resetPluginRuntimeStateForTest();
subagentRegistryReadMock.setSubagentRunsForTest([]);
vi.clearAllMocks();
});
test("shares the child-session index across repeated single-row loads for the same store", async () => {
await withSingleRowCacheStore(
"openclaw-single-row-cache-",
"/tmp/openclaw-single-row-cache",
async ({ now, storePath }) => {
const store: Record<string, SessionEntry> = {
"agent:main:subagent:parent-a": parentSession("parent-a", now),
"agent:main:subagent:child-a": runningChildSession(
"child-a",
"agent:main:subagent:parent-a",
now,
),
"agent:main:subagent:parent-b": parentSession("parent-b", now),
"agent:main:subagent:child-b": runningChildSession(
"child-b",
"agent:main:subagent:parent-b",
now,
),
};
await seedSessionEntries(storePath, store);
const rowA = loadGatewaySessionRow("agent:main:subagent:parent-a", { now });
const rowB = loadGatewaySessionRow("agent:main:subagent:parent-b", { now: now + 50 });
const rowAAfterWindow = loadGatewaySessionRow("agent:main:subagent:parent-a", {
now: now + 1_500,
});
expect(rowA?.childSessions).toEqual(["agent:main:subagent:child-a"]);
expect(rowB?.childSessions).toEqual(["agent:main:subagent:child-b"]);
expect(rowAAfterWindow?.childSessions).toEqual(["agent:main:subagent:child-a"]);
expect(subagentRegistryReadMock.buildSubagentRunReadIndex).not.toHaveBeenCalled();
},
);
});
test("refreshes subagent registry state while reusing store child candidates", async () => {
await withSingleRowCacheStore(
"openclaw-single-row-cache-fresh-registry-",
"/tmp/openclaw-single-row-cache-fresh-registry",
async ({ now, storePath }) => {
const fixture = createMovingChildFixture(now);
await seedSessionEntries(storePath, fixture.store);
setSubagentControllerRun(fixture.child, fixture.oldParent, now);
expect(loadGatewaySessionRow(fixture.oldParent, { now })?.childSessions).toEqual([
fixture.child,
]);
setSubagentControllerRun(fixture.child, fixture.newParent, now + 25);
expectChildMovedToNewParent(fixture, now);
},
);
});
test("builds shared subagent metadata context for single-row session lists", async () => {
await withSingleRowCacheStore(
"openclaw-single-row-list-context-",
"/tmp/openclaw-single-row-list-context",
async ({ now, storePath }) => {
const store: Record<string, SessionEntry> = {
"agent:main:discord:channel:parent": parentSession("parent", now),
};
const cfg: OpenClawConfig = {
agents: {
list: [
{
id: MAIN_AGENT_ID,
default: true,
workspace: "/tmp/openclaw-single-row-list-context",
},
],
defaults: { model: { primary: TEST_MODEL } },
},
} as OpenClawConfig;
const syncListed = listSessionsFromStore({
cfg,
storePath,
store,
opts: { agentId: MAIN_AGENT_ID, limit: 1 },
});
expect(syncListed.sessions).toHaveLength(1);
expect(subagentRegistryReadMock.buildSubagentRunReadIndex).toHaveBeenCalledTimes(1);
expect(
subagentRegistryReadMock.getSessionDisplaySubagentRunByChildSessionKey,
).not.toHaveBeenCalled();
vi.clearAllMocks();
const asyncListed = await listSessionsFromStoreAsync({
cfg,
storePath,
store,
opts: { agentId: MAIN_AGENT_ID, limit: 1 },
});
expect(asyncListed.sessions).toHaveLength(1);
expect(subagentRegistryReadMock.buildSubagentRunReadIndex).toHaveBeenCalledTimes(1);
expect(
subagentRegistryReadMock.getSessionDisplaySubagentRunByChildSessionKey,
).not.toHaveBeenCalled();
},
);
});
test("rebuilds store child candidates after same-object session store writes", async () => {
await withSingleRowCacheStore(
"openclaw-single-row-cache-write-version-",
"/tmp/openclaw-single-row-cache-write-version",
async ({ now, storePath }) => {
const fixture = createMovingChildFixture(now);
await seedSessionEntries(storePath, fixture.store);
expect(loadGatewaySessionRow(fixture.oldParent, { now })?.childSessions).toEqual([
fixture.child,
]);
await updateSessionEntry({ sessionKey: fixture.child, storePath }, () => ({
parentSessionKey: fixture.newParent,
updatedAt: now + 25,
}));
expectChildMovedToNewParent(fixture, now);
},
);
});
});