Files
openclaw/src/gateway/session-groups.test.ts
Josh Lehman 0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00

122 lines
4.5 KiB
TypeScript

import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import type { SessionEntry } from "../config/sessions.js";
import { loadSessionEntry, replaceSessionEntry } from "../config/sessions/session-accessor.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { closeOpenClawAgentDatabasesForTest } from "../state/openclaw-agent-db.js";
import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js";
import {
deleteSessionGroup,
ensureSessionGroupRegistered,
listSessionGroups,
putSessionGroups,
renameSessionGroup,
} from "./session-groups.js";
describe("session groups catalog", () => {
let root: string;
let env: NodeJS.ProcessEnv;
const cfg = {} as OpenClawConfig;
beforeEach(async () => {
const tempRoot = await fs.realpath(os.tmpdir());
root = await fs.mkdtemp(path.join(tempRoot, "openclaw-session-groups-"));
env = { ...process.env, OPENCLAW_STATE_DIR: root };
});
afterEach(async () => {
closeOpenClawAgentDatabasesForTest();
closeOpenClawStateDatabaseForTest();
await fs.rm(root, { recursive: true, force: true });
});
async function seedSessionStore(entries: Record<string, SessionEntry>): Promise<string> {
const storePath = path.join(root, "agents", "main", "sessions", "sessions.json");
for (const [sessionKey, entry] of Object.entries(entries)) {
await replaceSessionEntry({ agentId: "main", storePath, sessionKey }, entry);
}
return storePath;
}
it("replaces the ordered catalog with deduped trimmed names", () => {
expect(listSessionGroups(env)).toEqual([]);
const groups = putSessionGroups(["Work", " Personal ", "Work", ""], env);
expect(groups).toEqual([
{ name: "Work", position: 0 },
{ name: "Personal", position: 1 },
]);
expect(listSessionGroups(env)).toEqual(groups);
expect(putSessionGroups(["Personal"], env)).toEqual([{ name: "Personal", position: 0 }]);
});
it("absorbs ad-hoc categories at the end of the catalog", () => {
putSessionGroups(["Work"], env);
ensureSessionGroupRegistered("Travel", env);
ensureSessionGroupRegistered("Travel", env);
expect(listSessionGroups(env)).toEqual([
{ name: "Work", position: 0 },
{ name: "Travel", position: 1 },
]);
});
it("renames a group and repoints member categories without bumping updatedAt", async () => {
putSessionGroups(["Old", "Other"], env);
// Store saves run maintenance pruning; stale timestamps would be dropped.
const updatedAtA = Date.now() - 1_000;
const updatedAtB = Date.now() - 2_000;
const storePath = await seedSessionStore({
"agent:main:dashboard:a": { sessionId: "a1", updatedAt: updatedAtA, category: "Old" },
"agent:main:dashboard:b": { sessionId: "b1", updatedAt: updatedAtB, category: "Other" },
});
const result = await renameSessionGroup({ cfg, name: "Old", to: "New", env });
expect(result.updatedSessions).toBe(1);
expect(result.groups.map((group) => group.name)).toEqual(["New", "Other"]);
const sessionA = loadSessionEntry({
agentId: "main",
storePath,
sessionKey: "agent:main:dashboard:a",
});
const sessionB = loadSessionEntry({
agentId: "main",
storePath,
sessionKey: "agent:main:dashboard:b",
});
expect(sessionA?.category).toBe("New");
expect(sessionA?.updatedAt).toBe(updatedAtA);
expect(sessionB?.category).toBe("Other");
});
it("deletes a group and clears member categories", async () => {
putSessionGroups(["Gone"], env);
const storePath = await seedSessionStore({
"agent:main:dashboard:a": { sessionId: "a1", updatedAt: Date.now(), category: "Gone" },
});
const result = await deleteSessionGroup({ cfg, name: "Gone", env });
expect(result.updatedSessions).toBe(1);
expect(result.groups).toEqual([]);
expect(
loadSessionEntry({
agentId: "main",
storePath,
sessionKey: "agent:main:dashboard:a",
})?.category,
).toBeUndefined();
});
it("merges a rename into an existing target group", async () => {
putSessionGroups(["A", "B"], env);
await seedSessionStore({
"agent:main:dashboard:a": { sessionId: "a1", updatedAt: Date.now(), category: "A" },
});
const result = await renameSessionGroup({ cfg, name: "A", to: "B", env });
expect(result.groups).toEqual([{ name: "B", position: 1 }]);
expect(result.updatedSessions).toBe(1);
});
});