mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 13:10:43 +00:00
feat(memory-wiki): add agent-facing people wiki metadata
This commit is contained in:
Peter Steinberger
@@ -182,7 +182,7 @@ Telegraph style. Root rules only. Read scoped `AGENTS.md` before subtree work.
|
||||
## Ops / Footguns
|
||||
|
||||
- Remote install docs: `docs/install/{exe-dev,fly,hetzner}.md`. Parallels smoke: `$openclaw-parallels-smoke`; Discord roundtrip: `parallels-discord-roundtrip`.
|
||||
- Memory wiki: keep prompt digest tiny. The prompt should only say the wiki exists, prefer `wiki_search` / `wiki_get`, start from `reports/maintainer-agent-directory.md` for people routing, and verify contact data before use.
|
||||
- Memory wiki: keep prompt digest tiny. The prompt should only say the wiki exists, prefer `wiki_search` / `wiki_get`, start from `reports/person-agent-directory.md` for people routing, use search modes (`find-person`, `route-question`, `source-evidence`, `raw-claim`) when useful, and verify contact data before use.
|
||||
- People wiki provenance: generated identity, social, contact, and "fun detail" notes need explicit source class/confidence (`maintainer-whois`, Discrawl sample/stat, GitHub profile, maintainer repo file). Do not promote inferred details to facts.
|
||||
- Rebrand/migration/config warnings: run `openclaw doctor`.
|
||||
- Never edit `node_modules`.
|
||||
|
||||
@@ -7,6 +7,7 @@ Docs: https://docs.openclaw.ai
|
||||
### Changes
|
||||
|
||||
- Providers/NVIDIA: add the NVIDIA provider with API-key onboarding, setup docs, static catalog metadata, and literal model-ref picker support so NVIDIA hosted models can be selected with their provider prefix intact. (#71204) Thanks @eleqtrizit.
|
||||
- Memory/wiki: add agent-facing people wiki metadata, canonical aliases, person cards, relationship graphs, privacy/provenance reports, evidence-kind drilldown, and search modes for person lookup, question routing, source evidence, and raw claims. Thanks @vincentkoc.
|
||||
- Messages: add global `messages.visibleReplies` so operators can require visible output to go through `message(action=send)` for any source chat, while `messages.groupChat.visibleReplies` stays available as the group/channel override. Thanks @scoootscooob.
|
||||
- Gateway/dev: run `pnpm gateway:watch` through a named tmux session by default, with `gateway:watch:raw` and `OPENCLAW_GATEWAY_WATCH_TMUX=0` for foreground mode, so repeated starts respawn an inspectable watcher without trapping the invoking agent shell. Thanks @vincentkoc.
|
||||
- Plugin SDK: mark remaining legacy alias exports and diffs tool/config aliases with deprecation metadata, and add a guard so future legacy alias comments require `@deprecated` tags. Thanks @vincentkoc.
|
||||
|
||||
@@ -38,6 +38,7 @@ openclaw wiki ingest ./notes/alpha.md
|
||||
openclaw wiki compile
|
||||
openclaw wiki lint
|
||||
openclaw wiki search "alpha"
|
||||
openclaw wiki search "who should I ask about Teams?" --mode route-question
|
||||
openclaw wiki get entity.alpha --from 1 --lines 80
|
||||
|
||||
openclaw wiki apply synthesis "Alpha Summary" \
|
||||
@@ -135,11 +136,34 @@ Behavior depends on config:
|
||||
|
||||
- `search.backend`: `shared` or `local`
|
||||
- `search.corpus`: `wiki`, `memory`, or `all`
|
||||
- `--mode`: `auto`, `find-person`, `route-question`, `source-evidence`, or
|
||||
`raw-claim`
|
||||
|
||||
Use `wiki search` when you want wiki-specific ranking or provenance details.
|
||||
For one broad shared recall pass, prefer `openclaw memory search` when the
|
||||
active memory plugin exposes shared search.
|
||||
|
||||
Search modes help the agent choose the right surface:
|
||||
|
||||
- `find-person`: aliases, handles, socials, canonical IDs, and person pages
|
||||
- `route-question`: ask-for/best-used-for hints and relationship context
|
||||
- `source-evidence`: source pages and structured evidence fields
|
||||
- `raw-claim`: structured claim text with claim/evidence metadata
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
openclaw wiki search "bgroux" --mode find-person
|
||||
openclaw wiki search "who knows Teams rollout?" --mode route-question
|
||||
openclaw wiki search "maintainer-whois" --mode source-evidence
|
||||
openclaw wiki search "strong route Teams" --mode raw-claim --json
|
||||
```
|
||||
|
||||
Text output includes `Claim:` and `Evidence:` lines when a result matches a
|
||||
structured claim. JSON output additionally exposes `matchedClaimId`,
|
||||
`matchedClaimStatus`, `matchedClaimConfidence`, `evidenceKinds`, and
|
||||
`evidenceSourceIds` for agent-side drilldown.
|
||||
|
||||
### `wiki get <lookup>`
|
||||
|
||||
Read a wiki page by id or relative path.
|
||||
|
||||
@@ -150,16 +150,90 @@ Each claim can include:
|
||||
|
||||
Evidence entries can include:
|
||||
|
||||
- `kind`
|
||||
- `sourceId`
|
||||
- `path`
|
||||
- `lines`
|
||||
- `weight`
|
||||
- `confidence`
|
||||
- `privacyTier`
|
||||
- `note`
|
||||
- `updatedAt`
|
||||
|
||||
This is what makes the wiki act more like a belief layer than a passive note
|
||||
dump. Claims can be tracked, scored, contested, and resolved back to sources.
|
||||
|
||||
## Agent-facing entity metadata
|
||||
|
||||
Entity pages can also carry routing metadata for agent use. This is generic
|
||||
frontmatter, so it works for people, teams, systems, projects, or any other
|
||||
entity type.
|
||||
|
||||
Common fields include:
|
||||
|
||||
- `entityType`: for example `person`, `team`, `system`, or `project`
|
||||
- `canonicalId`: stable identity key used across aliases and imports
|
||||
- `aliases`: names, handles, or labels that should resolve to the same page
|
||||
- `privacyTier`: `public`, `local-private`, `sensitive`, or `confirm-before-use`
|
||||
- `bestUsedFor` / `notEnoughFor`: compact routing hints
|
||||
- `lastRefreshedAt`: source-refresh timestamp separate from page edit time
|
||||
- `personCard`: optional person-specific routing card with handles, socials,
|
||||
emails, timezone, lane, ask-for, avoid-asking-for, confidence, and privacy
|
||||
- `relationships`: typed edges to related pages with target, kind, weight,
|
||||
confidence, evidence kind, privacy tier, and note
|
||||
|
||||
For a people wiki, the agent should usually start with
|
||||
`reports/person-agent-directory.md`, then open the person page with `wiki_get`
|
||||
before using contact details or inferred facts.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
pageType: entity
|
||||
entityType: person
|
||||
id: entity.brad-groux
|
||||
canonicalId: maintainer.brad-groux
|
||||
aliases:
|
||||
- Brad
|
||||
- bgroux
|
||||
privacyTier: local-private
|
||||
bestUsedFor:
|
||||
- Microsoft Teams and Azure routing
|
||||
notEnoughFor:
|
||||
- legal approval
|
||||
lastRefreshedAt: "2026-04-29T00:00:00.000Z"
|
||||
personCard:
|
||||
handles:
|
||||
- "@bgroux"
|
||||
socials:
|
||||
- "https://x.example/bgroux"
|
||||
emails:
|
||||
- brad@example.com
|
||||
timezone: America/Chicago
|
||||
lane: Microsoft ecosystem
|
||||
askFor:
|
||||
- Teams rollout questions
|
||||
avoidAskingFor:
|
||||
- unrelated billing decisions
|
||||
confidence: 0.8
|
||||
privacyTier: confirm-before-use
|
||||
relationships:
|
||||
- targetId: entity.alice
|
||||
targetTitle: Alice
|
||||
kind: collaborates-with
|
||||
confidence: 0.7
|
||||
evidenceKind: discrawl-stat
|
||||
claims:
|
||||
- id: claim.brad.teams
|
||||
text: Brad is useful for Microsoft Teams routing.
|
||||
status: supported
|
||||
confidence: 0.9
|
||||
evidence:
|
||||
- kind: maintainer-whois
|
||||
sourceId: source.maintainers
|
||||
privacyTier: local-private
|
||||
```
|
||||
|
||||
## Compile pipeline
|
||||
|
||||
The compile step reads wiki pages, normalizes summaries, and emits stable
|
||||
@@ -190,6 +264,10 @@ Built-in reports include:
|
||||
- `reports/low-confidence.md`
|
||||
- `reports/claim-health.md`
|
||||
- `reports/stale-pages.md`
|
||||
- `reports/person-agent-directory.md`
|
||||
- `reports/relationship-graph.md`
|
||||
- `reports/provenance-coverage.md`
|
||||
- `reports/privacy-review.md`
|
||||
|
||||
These reports track things like:
|
||||
|
||||
@@ -199,6 +277,10 @@ These reports track things like:
|
||||
- low-confidence pages and claims
|
||||
- stale or unknown freshness
|
||||
- pages with unresolved questions
|
||||
- person/entity routing cards
|
||||
- structured relationship edges
|
||||
- evidence class coverage
|
||||
- non-public privacy tiers that need review before use
|
||||
|
||||
## Search and retrieval
|
||||
|
||||
@@ -219,6 +301,8 @@ Important behavior:
|
||||
- claim ids can resolve back to the owning page
|
||||
- contested/stale/fresh claims influence ranking
|
||||
- provenance labels can survive into results
|
||||
- search mode can bias ranking for person lookup, question routing, source
|
||||
evidence, or raw claims
|
||||
|
||||
Practical rule:
|
||||
|
||||
@@ -226,6 +310,22 @@ Practical rule:
|
||||
- use `wiki_search` + `wiki_get` when you care about wiki-specific ranking,
|
||||
provenance, or page-level belief structure
|
||||
|
||||
Search modes:
|
||||
|
||||
- `auto`: balanced default
|
||||
- `find-person`: boost person-like entities, aliases, handles, socials, and
|
||||
canonical IDs
|
||||
- `route-question`: boost agent cards, ask-for hints, best-used-for hints, and
|
||||
relationship context
|
||||
- `source-evidence`: boost source pages and structured evidence metadata
|
||||
- `raw-claim`: boost matching structured claims and return claim/evidence
|
||||
metadata in results
|
||||
|
||||
When a result matches a structured claim, `wiki_search` can return
|
||||
`matchedClaimId`, `matchedClaimStatus`, `matchedClaimConfidence`,
|
||||
`evidenceKinds`, and `evidenceSourceIds` in its details payload. Text output
|
||||
also includes compact `Claim:` and `Evidence:` lines when available.
|
||||
|
||||
## Agent tools
|
||||
|
||||
The plugin registers these tools:
|
||||
@@ -239,7 +339,9 @@ The plugin registers these tools:
|
||||
What they do:
|
||||
|
||||
- `wiki_status`: current vault mode, health, Obsidian CLI availability
|
||||
- `wiki_search`: search wiki pages and, when configured, shared memory corpora
|
||||
- `wiki_search`: search wiki pages and, when configured, shared memory corpora;
|
||||
accepts `mode` for person lookup, question routing, source evidence, or raw
|
||||
claim drilldown
|
||||
- `wiki_get`: read a wiki page by id/path or fall back to shared memory corpus
|
||||
- `wiki_apply`: narrow synthesis/metadata mutations without freeform page surgery
|
||||
- `wiki_lint`: structural checks, provenance gaps, contradictions, open questions
|
||||
|
||||
@@ -26,7 +26,12 @@ import {
|
||||
runObsidianOpen,
|
||||
runObsidianSearch,
|
||||
} from "./obsidian.js";
|
||||
import { getMemoryWikiPage, searchMemoryWiki } from "./query.js";
|
||||
import {
|
||||
getMemoryWikiPage,
|
||||
searchMemoryWiki,
|
||||
WIKI_SEARCH_MODES,
|
||||
type WikiSearchMode,
|
||||
} from "./query.js";
|
||||
import { syncMemoryWikiImportedSources } from "./source-sync.js";
|
||||
import type { MemoryWikiImportedSourceSyncResult } from "./source-sync.js";
|
||||
import {
|
||||
@@ -81,6 +86,7 @@ type WikiSearchCommandOptions = {
|
||||
maxResults?: number;
|
||||
backend?: ResolvedMemoryWikiConfig["search"]["backend"];
|
||||
corpus?: ResolvedMemoryWikiConfig["search"]["corpus"];
|
||||
mode?: WikiSearchMode;
|
||||
};
|
||||
|
||||
type WikiGetCommandOptions = {
|
||||
@@ -567,9 +573,13 @@ export async function runWikiSearch(params: {
|
||||
maxResults?: number;
|
||||
searchBackend?: ResolvedMemoryWikiConfig["search"]["backend"];
|
||||
searchCorpus?: ResolvedMemoryWikiConfig["search"]["corpus"];
|
||||
mode?: WikiSearchMode;
|
||||
json?: boolean;
|
||||
stdout?: Pick<NodeJS.WriteStream, "write">;
|
||||
}) {
|
||||
if (params.mode && !(WIKI_SEARCH_MODES as readonly string[]).includes(params.mode)) {
|
||||
throw new Error(`wiki search --mode must be one of: ${WIKI_SEARCH_MODES.join(", ")}.`);
|
||||
}
|
||||
await syncMemoryWikiImportedSources({ config: params.config, appConfig: params.appConfig });
|
||||
const results = await searchMemoryWiki({
|
||||
config: params.config,
|
||||
@@ -578,6 +588,7 @@ export async function runWikiSearch(params: {
|
||||
maxResults: params.maxResults,
|
||||
searchBackend: params.searchBackend,
|
||||
searchCorpus: params.searchCorpus,
|
||||
mode: params.mode,
|
||||
});
|
||||
const summary = params.json
|
||||
? JSON.stringify(results, null, 2)
|
||||
@@ -586,7 +597,7 @@ export async function runWikiSearch(params: {
|
||||
: results
|
||||
.map(
|
||||
(result, index) =>
|
||||
`${index + 1}. ${result.title} (${result.corpus}/${result.kind})\nPath: ${result.path}${typeof result.startLine === "number" && typeof result.endLine === "number" ? `\nLines: ${result.startLine}-${result.endLine}` : ""}${result.provenanceLabel ? `\nProvenance: ${result.provenanceLabel}` : ""}\nSnippet: ${result.snippet}`,
|
||||
`${index + 1}. ${result.title} (${result.corpus}/${result.kind})\nPath: ${result.path}${typeof result.startLine === "number" && typeof result.endLine === "number" ? `\nLines: ${result.startLine}-${result.endLine}` : ""}${result.provenanceLabel ? `\nProvenance: ${result.provenanceLabel}` : ""}${result.matchedClaimId ? `\nClaim: ${result.matchedClaimId}` : ""}${result.evidenceKinds && result.evidenceKinds.length > 0 ? `\nEvidence: ${result.evidenceKinds.join(", ")}` : ""}\nSnippet: ${result.snippet}`,
|
||||
)
|
||||
.join("\n\n");
|
||||
writeOutput(summary, params.stdout);
|
||||
@@ -935,7 +946,8 @@ export function registerWikiCli(
|
||||
.command("search")
|
||||
.description("Search wiki pages and, when configured, the active memory corpus")
|
||||
.argument("<query>", "Search query")
|
||||
.option("--max-results <n>", "Maximum results", (value: string) => Number(value)),
|
||||
.option("--max-results <n>", "Maximum results", (value: string) => Number(value))
|
||||
.option("--mode <mode>", `Search mode (${WIKI_SEARCH_MODES.join(", ")})`),
|
||||
)
|
||||
.option("--json", "Print JSON")
|
||||
.action(async (query: string, opts: WikiSearchCommandOptions) => {
|
||||
@@ -946,6 +958,7 @@ export function registerWikiCli(
|
||||
maxResults: opts.maxResults,
|
||||
searchBackend: opts.backend,
|
||||
searchCorpus: opts.corpus,
|
||||
mode: opts.mode,
|
||||
json: opts.json,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -353,6 +353,101 @@ describe("compileMemoryWikiVault", () => {
|
||||
await expect(fs.access(path.join(rootDir, "reports", "open-questions.md"))).rejects.toThrow();
|
||||
});
|
||||
|
||||
it("writes agent directory, relationship, provenance, and privacy reports", async () => {
|
||||
const { rootDir, config } = await createVault({
|
||||
rootDir: nextCaseRoot(),
|
||||
initialize: true,
|
||||
});
|
||||
|
||||
await fs.writeFile(
|
||||
path.join(rootDir, "entities", "brad.md"),
|
||||
renderWikiMarkdown({
|
||||
frontmatter: {
|
||||
pageType: "entity",
|
||||
entityType: "person",
|
||||
id: "entity.brad",
|
||||
title: "Brad Groux",
|
||||
canonicalId: "maintainer.brad-groux",
|
||||
aliases: ["brad"],
|
||||
privacyTier: "local-private",
|
||||
bestUsedFor: ["Microsoft routing"],
|
||||
lastRefreshedAt: "2026-04-29T00:00:00.000Z",
|
||||
personCard: {
|
||||
handles: ["@bgroux"],
|
||||
lane: "Microsoft Teams",
|
||||
askFor: ["Teams and Azure questions"],
|
||||
privacyTier: "confirm-before-use",
|
||||
},
|
||||
relationships: [
|
||||
{
|
||||
targetId: "entity.alice",
|
||||
targetTitle: "Alice",
|
||||
kind: "collaborates-with",
|
||||
evidenceKind: "discrawl-stat",
|
||||
privacyTier: "local-private",
|
||||
},
|
||||
],
|
||||
claims: [
|
||||
{
|
||||
id: "claim.brad.teams",
|
||||
text: "Brad is useful for Microsoft Teams routing.",
|
||||
status: "supported",
|
||||
confidence: 0.9,
|
||||
evidence: [
|
||||
{
|
||||
kind: "maintainer-whois",
|
||||
sourceId: "source.maintainers",
|
||||
privacyTier: "local-private",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
body: "# Brad Groux\n",
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
await compileMemoryWikiVault(config);
|
||||
|
||||
await expect(
|
||||
fs.readFile(path.join(rootDir, "reports", "person-agent-directory.md"), "utf8"),
|
||||
).resolves.toContain("Microsoft Teams");
|
||||
await expect(
|
||||
fs.readFile(path.join(rootDir, "reports", "relationship-graph.md"), "utf8"),
|
||||
).resolves.toContain("collaborates-with");
|
||||
await expect(
|
||||
fs.readFile(path.join(rootDir, "reports", "provenance-coverage.md"), "utf8"),
|
||||
).resolves.toContain("maintainer-whois: 1");
|
||||
await expect(
|
||||
fs.readFile(path.join(rootDir, "reports", "privacy-review.md"), "utf8"),
|
||||
).resolves.toContain("confirm-before-use");
|
||||
|
||||
const agentDigest = JSON.parse(
|
||||
await fs.readFile(path.join(rootDir, ".openclaw-wiki", "cache", "agent-digest.json"), "utf8"),
|
||||
) as {
|
||||
pages: Array<{
|
||||
path: string;
|
||||
canonicalId?: string;
|
||||
aliases?: string[];
|
||||
personCard?: { lane?: string };
|
||||
relationshipCount?: number;
|
||||
}>;
|
||||
};
|
||||
expect(agentDigest.pages).toContainEqual(
|
||||
expect.objectContaining({
|
||||
path: "entities/brad.md",
|
||||
canonicalId: "maintainer.brad-groux",
|
||||
aliases: ["brad"],
|
||||
personCard: expect.objectContaining({ lane: "Microsoft Teams" }),
|
||||
relationshipCount: 1,
|
||||
}),
|
||||
);
|
||||
await expect(
|
||||
fs.readFile(path.join(rootDir, ".openclaw-wiki", "cache", "claims.jsonl"), "utf8"),
|
||||
).resolves.toContain('"evidenceKinds":["maintainer-whois"]');
|
||||
});
|
||||
|
||||
it("ignores generated related links when computing backlinks on repeated compile", async () => {
|
||||
const { rootDir, config } = await createVault({
|
||||
rootDir: nextCaseRoot(),
|
||||
|
||||
@@ -28,8 +28,10 @@ import {
|
||||
renderWikiMarkdown,
|
||||
toWikiPageSummary,
|
||||
type WikiClaim,
|
||||
type WikiClaimEvidence,
|
||||
type WikiPageKind,
|
||||
type WikiPageSummary,
|
||||
type WikiRelationship,
|
||||
WIKI_RELATED_END_MARKER,
|
||||
WIKI_RELATED_START_MARKER,
|
||||
} from "./markdown.js";
|
||||
@@ -218,6 +220,106 @@ const DASHBOARD_PAGES: DashboardPageDefinition[] = [
|
||||
].join("\n");
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "report.person-agent-directory",
|
||||
title: "Person Agent Directory",
|
||||
relativePath: "reports/person-agent-directory.md",
|
||||
buildBody: ({ config, pages, now }) => {
|
||||
const matches = pages
|
||||
.filter((page) => page.kind !== "report" && isPersonLikePage(page))
|
||||
.toSorted((left, right) => left.title.localeCompare(right.title));
|
||||
if (matches.length === 0) {
|
||||
return "- No person-like entity pages with agent cards yet.";
|
||||
}
|
||||
const lines = [`- People with routing metadata: ${matches.length}`];
|
||||
for (const page of matches) {
|
||||
const freshness = assessPageFreshness(page, now);
|
||||
lines.push(`- ${formatPersonDirectoryLine(config, page, freshness)}`);
|
||||
}
|
||||
return lines.join("\n");
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "report.relationship-graph",
|
||||
title: "Relationship Graph",
|
||||
relativePath: "reports/relationship-graph.md",
|
||||
buildBody: ({ config, pages }) => {
|
||||
const relationships = pages
|
||||
.flatMap((page) => page.relationships.map((relationship) => ({ page, relationship })))
|
||||
.toSorted((left, right) => {
|
||||
const leftTitle = left.relationship.targetTitle ?? left.relationship.targetId ?? "";
|
||||
const rightTitle = right.relationship.targetTitle ?? right.relationship.targetId ?? "";
|
||||
return `${left.page.title} ${leftTitle}`.localeCompare(
|
||||
`${right.page.title} ${rightTitle}`,
|
||||
);
|
||||
});
|
||||
if (relationships.length === 0) {
|
||||
return "- No structured relationships yet.";
|
||||
}
|
||||
return [
|
||||
`- Structured relationships: ${relationships.length}`,
|
||||
"",
|
||||
...relationships.map(
|
||||
({ page, relationship }) => `- ${formatRelationshipLine(config, page, relationship)}`,
|
||||
),
|
||||
].join("\n");
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "report.provenance-coverage",
|
||||
title: "Provenance Coverage",
|
||||
relativePath: "reports/provenance-coverage.md",
|
||||
buildBody: ({ config, pages }) => {
|
||||
const evidenceEntries = pages.flatMap((page) =>
|
||||
page.claims.flatMap((claim) =>
|
||||
claim.evidence.map((evidence) => ({ page, claim, evidence })),
|
||||
),
|
||||
);
|
||||
const missingEvidence = pages.flatMap((page) =>
|
||||
page.claims
|
||||
.filter((claim) => claim.evidence.length === 0)
|
||||
.map((claim) => ({ page, claim })),
|
||||
);
|
||||
if (evidenceEntries.length === 0 && missingEvidence.length === 0) {
|
||||
return "- No structured claims with provenance coverage yet.";
|
||||
}
|
||||
const kindCounts = countBy(
|
||||
evidenceEntries.map(({ evidence }) => evidence.kind ?? "unspecified"),
|
||||
);
|
||||
const sourceCounts = countBy(
|
||||
evidenceEntries.map(({ evidence }) => evidence.sourceId ?? evidence.path ?? "inline"),
|
||||
);
|
||||
const lines = [
|
||||
`- Evidence entries: ${evidenceEntries.length}`,
|
||||
`- Claims missing evidence: ${missingEvidence.length}`,
|
||||
"",
|
||||
"### Evidence Classes",
|
||||
...formatCountLines(kindCounts),
|
||||
"",
|
||||
"### Top Evidence Sources",
|
||||
...formatCountLines(sourceCounts).slice(0, 20),
|
||||
];
|
||||
if (missingEvidence.length > 0) {
|
||||
lines.push("", "### Missing Evidence");
|
||||
for (const { page, claim } of missingEvidence) {
|
||||
lines.push(`- ${formatPageLink(config, page)}: ${formatClaimIdentityForPage(claim)}`);
|
||||
}
|
||||
}
|
||||
return lines.join("\n");
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "report.privacy-review",
|
||||
title: "Privacy Review",
|
||||
relativePath: "reports/privacy-review.md",
|
||||
buildBody: ({ config, pages }) => {
|
||||
const entries = collectPrivacyReviewEntries(config, pages);
|
||||
if (entries.length === 0) {
|
||||
return "- No non-public privacy tiers flagged right now.";
|
||||
}
|
||||
return [`- Privacy review entries: ${entries.length}`, "", ...entries].join("\n");
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
export type CompileMemoryWikiResult = {
|
||||
@@ -294,6 +396,165 @@ function formatFreshnessLabel(freshness: WikiFreshness): string {
|
||||
throw new Error("Unsupported wiki freshness level");
|
||||
}
|
||||
|
||||
function formatListPreview(values: readonly string[], maxItems = 3): string | null {
|
||||
if (values.length === 0) {
|
||||
return null;
|
||||
}
|
||||
const shown = values.slice(0, maxItems).join(", ");
|
||||
return values.length > maxItems ? `${shown}, +${values.length - maxItems}` : shown;
|
||||
}
|
||||
|
||||
function formatMaybeDetail(label: string, value: string | null | undefined): string | null {
|
||||
return value ? `${label} ${value}` : null;
|
||||
}
|
||||
|
||||
function isPersonLikePage(page: WikiPageSummary): boolean {
|
||||
const entityType = normalizeLowercaseStringOrEmpty(page.entityType);
|
||||
const pageType = normalizeLowercaseStringOrEmpty(page.pageType);
|
||||
return (
|
||||
Boolean(page.personCard) ||
|
||||
entityType === "person" ||
|
||||
entityType === "maintainer" ||
|
||||
pageType === "person" ||
|
||||
pageType === "maintainer"
|
||||
);
|
||||
}
|
||||
|
||||
function formatPersonDirectoryLine(
|
||||
config: ResolvedMemoryWikiConfig,
|
||||
page: WikiPageSummary,
|
||||
freshness: WikiFreshness,
|
||||
): string {
|
||||
const card = page.personCard;
|
||||
const details = [
|
||||
formatMaybeDetail("id", page.canonicalId ?? card?.canonicalId ?? page.id),
|
||||
formatMaybeDetail("aliases", formatListPreview(page.aliases)),
|
||||
formatMaybeDetail("handles", formatListPreview(card?.handles ?? [])),
|
||||
formatMaybeDetail("lane", card?.lane),
|
||||
formatMaybeDetail("ask", formatListPreview(card?.askFor ?? [])),
|
||||
formatMaybeDetail(
|
||||
"best",
|
||||
formatListPreview([...page.bestUsedFor, ...(card?.bestUsedFor ?? [])]),
|
||||
),
|
||||
formatMaybeDetail("privacy", page.privacyTier ?? card?.privacyTier),
|
||||
formatMaybeDetail("refreshed", page.lastRefreshedAt ?? card?.lastRefreshedAt),
|
||||
formatMaybeDetail("freshness", formatFreshnessLabel(freshness)),
|
||||
].filter(Boolean);
|
||||
return `${formatPageLink(config, page)}${details.length > 0 ? `: ${details.join("; ")}` : ""}`;
|
||||
}
|
||||
|
||||
function formatRelationshipTarget(
|
||||
config: ResolvedMemoryWikiConfig,
|
||||
relationship: WikiRelationship,
|
||||
) {
|
||||
if (relationship.targetPath && relationship.targetTitle) {
|
||||
return formatWikiLink({
|
||||
renderMode: config.vault.renderMode,
|
||||
relativePath: relationship.targetPath,
|
||||
title: relationship.targetTitle,
|
||||
});
|
||||
}
|
||||
return relationship.targetTitle ?? relationship.targetId ?? relationship.targetPath ?? "unknown";
|
||||
}
|
||||
|
||||
function formatRelationshipLine(
|
||||
config: ResolvedMemoryWikiConfig,
|
||||
page: WikiPageSummary,
|
||||
relationship: WikiRelationship,
|
||||
): string {
|
||||
const details = [
|
||||
relationship.kind ?? "related",
|
||||
typeof relationship.weight === "number" ? `weight ${relationship.weight.toFixed(2)}` : null,
|
||||
typeof relationship.confidence === "number"
|
||||
? `confidence ${relationship.confidence.toFixed(2)}`
|
||||
: null,
|
||||
relationship.evidenceKind ? `evidence ${relationship.evidenceKind}` : null,
|
||||
relationship.privacyTier ? `privacy ${relationship.privacyTier}` : null,
|
||||
relationship.note,
|
||||
].filter(Boolean);
|
||||
return `${formatPageLink(config, page)} -> ${formatRelationshipTarget(config, relationship)}${
|
||||
details.length > 0 ? ` (${details.join(", ")})` : ""
|
||||
}`;
|
||||
}
|
||||
|
||||
function countBy(values: readonly string[]): Map<string, number> {
|
||||
const counts = new Map<string, number>();
|
||||
for (const value of values) {
|
||||
counts.set(value, (counts.get(value) ?? 0) + 1);
|
||||
}
|
||||
return counts;
|
||||
}
|
||||
|
||||
function formatCountLines(counts: Map<string, number>): string[] {
|
||||
const lines = [...counts]
|
||||
.toSorted((left, right) => {
|
||||
if (left[1] !== right[1]) {
|
||||
return right[1] - left[1];
|
||||
}
|
||||
return left[0].localeCompare(right[0]);
|
||||
})
|
||||
.map(([label, count]) => `- ${label}: ${count}`);
|
||||
return lines.length > 0 ? lines : ["- None"];
|
||||
}
|
||||
|
||||
function formatClaimIdentityForPage(claim: Pick<WikiClaim, "id" | "text">): string {
|
||||
return claim.id ? `\`${claim.id}\`: ${claim.text}` : claim.text;
|
||||
}
|
||||
|
||||
function isReviewablePrivacyTier(value: string | undefined): boolean {
|
||||
const tier = normalizeLowercaseStringOrEmpty(value);
|
||||
return tier !== "" && tier !== "public";
|
||||
}
|
||||
|
||||
function formatEvidencePrivacyDetails(evidence: WikiClaimEvidence): string {
|
||||
return [
|
||||
evidence.kind ? `kind ${evidence.kind}` : null,
|
||||
evidence.sourceId ? `source ${evidence.sourceId}` : null,
|
||||
evidence.path ? `path ${evidence.path}` : null,
|
||||
evidence.lines ? `lines ${evidence.lines}` : null,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join(", ");
|
||||
}
|
||||
|
||||
function collectPrivacyReviewEntries(
|
||||
config: ResolvedMemoryWikiConfig,
|
||||
pages: WikiPageSummary[],
|
||||
): string[] {
|
||||
const entries: string[] = [];
|
||||
for (const page of pages) {
|
||||
if (isReviewablePrivacyTier(page.privacyTier)) {
|
||||
entries.push(`- ${formatPageLink(config, page)}: page privacy ${page.privacyTier}`);
|
||||
}
|
||||
if (isReviewablePrivacyTier(page.personCard?.privacyTier)) {
|
||||
entries.push(
|
||||
`- ${formatPageLink(config, page)}: person card privacy ${page.personCard?.privacyTier}`,
|
||||
);
|
||||
}
|
||||
for (const relationship of page.relationships) {
|
||||
if (isReviewablePrivacyTier(relationship.privacyTier)) {
|
||||
entries.push(
|
||||
`- ${formatPageLink(config, page)}: relationship privacy ${
|
||||
relationship.privacyTier
|
||||
} -> ${formatRelationshipTarget(config, relationship)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const claim of page.claims) {
|
||||
for (const evidence of claim.evidence) {
|
||||
if (!isReviewablePrivacyTier(evidence.privacyTier)) {
|
||||
continue;
|
||||
}
|
||||
const detail = formatEvidencePrivacyDetails(evidence);
|
||||
entries.push(
|
||||
`- ${formatPageLink(config, page)}: evidence privacy ${evidence.privacyTier} on ${formatClaimIdentityForPage(claim)}${detail ? ` (${detail})` : ""}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
function formatClaimIdentity(claim: WikiClaimHealth): string {
|
||||
return claim.claimId ? `\`${claim.claimId}\`: ${claim.text}` : claim.text;
|
||||
}
|
||||
@@ -740,12 +1001,23 @@ type AgentDigestPage = {
|
||||
title: string;
|
||||
kind: WikiPageKind;
|
||||
path: string;
|
||||
pageType?: string;
|
||||
entityType?: string;
|
||||
canonicalId?: string;
|
||||
aliases: string[];
|
||||
sourceIds: string[];
|
||||
questions: string[];
|
||||
contradictions: string[];
|
||||
confidence?: number;
|
||||
privacyTier?: string;
|
||||
personCard?: WikiPageSummary["personCard"];
|
||||
bestUsedFor: string[];
|
||||
notEnoughFor: string[];
|
||||
relationshipCount: number;
|
||||
topRelationships: WikiRelationship[];
|
||||
freshnessLevel: WikiFreshnessLevel;
|
||||
lastTouchedAt?: string;
|
||||
lastRefreshedAt?: string;
|
||||
claimCount: number;
|
||||
topClaims: AgentDigestClaim[];
|
||||
};
|
||||
@@ -878,13 +1150,24 @@ function buildAgentDigest(params: {
|
||||
title: page.title,
|
||||
kind: page.kind,
|
||||
path: page.relativePath,
|
||||
aliases: [...page.aliases],
|
||||
sourceIds: [...page.sourceIds],
|
||||
questions: [...page.questions],
|
||||
contradictions: [...page.contradictions],
|
||||
bestUsedFor: [...page.bestUsedFor],
|
||||
notEnoughFor: [...page.notEnoughFor],
|
||||
relationshipCount: page.relationships.length,
|
||||
topRelationships: page.relationships.slice(0, 5),
|
||||
},
|
||||
page.pageType ? { pageType: page.pageType } : {},
|
||||
page.entityType ? { entityType: page.entityType } : {},
|
||||
page.canonicalId ? { canonicalId: page.canonicalId } : {},
|
||||
typeof page.confidence === "number" ? { confidence: page.confidence } : {},
|
||||
page.privacyTier ? { privacyTier: page.privacyTier } : {},
|
||||
page.personCard ? { personCard: page.personCard } : {},
|
||||
{ freshnessLevel: pageFreshness.level },
|
||||
pageFreshness.lastTouchedAt ? { lastTouchedAt: pageFreshness.lastTouchedAt } : {},
|
||||
page.lastRefreshedAt ? { lastRefreshedAt: page.lastRefreshedAt } : {},
|
||||
{
|
||||
claimCount: page.claims.length,
|
||||
topClaims: sortClaims(page)
|
||||
@@ -931,10 +1214,24 @@ function buildClaimsDigestLines(params: { pages: WikiPageSummary[] }): string[]
|
||||
pageTitle: page.title,
|
||||
pageKind: page.kind,
|
||||
pagePath: page.relativePath,
|
||||
pageType: page.pageType,
|
||||
entityType: page.entityType,
|
||||
canonicalId: page.canonicalId,
|
||||
aliases: page.aliases,
|
||||
text: claim.text,
|
||||
status: normalizeClaimStatus(claim.status),
|
||||
confidence: claim.confidence,
|
||||
sourceIds: page.sourceIds,
|
||||
evidenceKinds: [...new Set(claim.evidence.flatMap((entry) => entry.kind ?? []))],
|
||||
privacyTiers: [
|
||||
...new Set(
|
||||
[
|
||||
page.privacyTier,
|
||||
page.personCard?.privacyTier,
|
||||
...claim.evidence.map((entry) => entry.privacyTier),
|
||||
].flatMap((entry) => entry ?? []),
|
||||
),
|
||||
],
|
||||
evidenceCount: claim.evidence.length,
|
||||
missingEvidence: claim.evidence.length === 0,
|
||||
evidence: claim.evidence,
|
||||
|
||||
@@ -54,6 +54,7 @@ vi.mock("./obsidian.js", () => ({
|
||||
vi.mock("./query.js", () => ({
|
||||
getMemoryWikiPage: vi.fn(),
|
||||
searchMemoryWiki: vi.fn(),
|
||||
WIKI_SEARCH_MODES: ["auto", "find-person", "route-question", "source-evidence", "raw-claim"],
|
||||
}));
|
||||
|
||||
vi.mock("./source-sync.js", () => ({
|
||||
@@ -356,6 +357,42 @@ describe("memory-wiki gateway methods", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("forwards wiki.search mode and corpus options over the gateway", async () => {
|
||||
const { config } = await createVault({ prefix: "memory-wiki-gateway-" });
|
||||
const { api, registerGatewayMethod } = createPluginApi();
|
||||
|
||||
registerMemoryWikiGatewayMethods({ api, config });
|
||||
const handler = findGatewayHandler(registerGatewayMethod, "wiki.search");
|
||||
if (!handler) {
|
||||
throw new Error("wiki.search handler missing");
|
||||
}
|
||||
const respond = vi.fn();
|
||||
|
||||
await handler({
|
||||
params: {
|
||||
query: "Teams Azure",
|
||||
maxResults: 3,
|
||||
corpus: "wiki",
|
||||
backend: "local",
|
||||
mode: "route-question",
|
||||
},
|
||||
respond,
|
||||
});
|
||||
|
||||
expect(searchMemoryWiki).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
config,
|
||||
appConfig: undefined,
|
||||
query: "Teams Azure",
|
||||
maxResults: 3,
|
||||
searchBackend: "local",
|
||||
searchCorpus: "wiki",
|
||||
mode: "route-question",
|
||||
}),
|
||||
);
|
||||
expect(respond).toHaveBeenCalledWith(true, expect.anything());
|
||||
});
|
||||
|
||||
it("forwards ingest requests over the gateway", async () => {
|
||||
const { config } = await createVault({ prefix: "memory-wiki-gateway-" });
|
||||
const { api, registerGatewayMethod } = createPluginApi();
|
||||
|
||||
@@ -19,7 +19,7 @@ import {
|
||||
runObsidianOpen,
|
||||
runObsidianSearch,
|
||||
} from "./obsidian.js";
|
||||
import { getMemoryWikiPage, searchMemoryWiki } from "./query.js";
|
||||
import { getMemoryWikiPage, searchMemoryWiki, WIKI_SEARCH_MODES } from "./query.js";
|
||||
import { syncMemoryWikiImportedSources } from "./source-sync.js";
|
||||
import { buildMemoryWikiDoctorReport, resolveMemoryWikiStatus } from "./status.js";
|
||||
import { initializeMemoryWikiVault } from "./vault.js";
|
||||
@@ -277,6 +277,7 @@ export function registerMemoryWikiGatewayMethods(params: {
|
||||
const maxResults = readNumberParam(requestParams, "maxResults");
|
||||
const searchBackend = readEnumParam(requestParams, "backend", WIKI_SEARCH_BACKENDS);
|
||||
const searchCorpus = readEnumParam(requestParams, "corpus", WIKI_SEARCH_CORPORA);
|
||||
const mode = readEnumParam(requestParams, "mode", WIKI_SEARCH_MODES);
|
||||
respond(
|
||||
true,
|
||||
await searchMemoryWiki({
|
||||
@@ -286,6 +287,7 @@ export function registerMemoryWikiGatewayMethods(params: {
|
||||
maxResults,
|
||||
searchBackend,
|
||||
searchCorpus,
|
||||
mode,
|
||||
}),
|
||||
);
|
||||
} catch (error) {
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { createWikiPageFilename, slugifyWikiSegment } from "./markdown.js";
|
||||
import {
|
||||
createWikiPageFilename,
|
||||
renderWikiMarkdown,
|
||||
slugifyWikiSegment,
|
||||
toWikiPageSummary,
|
||||
} from "./markdown.js";
|
||||
|
||||
describe("slugifyWikiSegment", () => {
|
||||
it("preserves Unicode letters and numbers in wiki slugs", () => {
|
||||
@@ -40,3 +45,103 @@ describe("slugifyWikiSegment", () => {
|
||||
expect(createWikiPageFilename(stem)).toBe(fileName);
|
||||
});
|
||||
});
|
||||
|
||||
describe("toWikiPageSummary", () => {
|
||||
it("normalizes agent-facing people wiki metadata", () => {
|
||||
const raw = renderWikiMarkdown({
|
||||
frontmatter: {
|
||||
pageType: "entity",
|
||||
entityType: "person",
|
||||
id: "entity.brad",
|
||||
title: "Brad Groux",
|
||||
canonicalId: "maintainer.brad-groux",
|
||||
aliases: ["brad", "bgroux"],
|
||||
privacyTier: "local-private",
|
||||
bestUsedFor: ["Microsoft ecosystem routing"],
|
||||
notEnoughFor: ["legal approval"],
|
||||
lastRefreshedAt: "2026-04-29T00:00:00.000Z",
|
||||
personCard: {
|
||||
handles: ["@bgroux"],
|
||||
socials: ["https://x.example/bgroux"],
|
||||
email: "brad@example.com",
|
||||
timezone: "America/Chicago",
|
||||
lane: "Microsoft Teams",
|
||||
askFor: ["Teams and Azure questions"],
|
||||
avoidAskingFor: ["unrelated billing"],
|
||||
confidence: 0.8,
|
||||
privacyTier: "confirm-before-use",
|
||||
lastRefreshedAt: "2026-04-28T00:00:00.000Z",
|
||||
},
|
||||
relationships: [
|
||||
{
|
||||
targetId: "entity.alice",
|
||||
targetTitle: "Alice",
|
||||
kind: "collaborates-with",
|
||||
weight: 0.7,
|
||||
confidence: 0.6,
|
||||
evidenceKind: "discrawl-stat",
|
||||
privacyTier: "local-private",
|
||||
},
|
||||
],
|
||||
claims: [
|
||||
{
|
||||
id: "claim.brad.teams",
|
||||
text: "Brad is useful for Microsoft Teams routing.",
|
||||
confidence: 0.9,
|
||||
evidence: [
|
||||
{
|
||||
kind: "maintainer-whois",
|
||||
sourceId: "source.maintainers",
|
||||
confidence: 0.8,
|
||||
privacyTier: "local-private",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
body: "# Brad Groux\n",
|
||||
});
|
||||
|
||||
const summary = toWikiPageSummary({
|
||||
absolutePath: "/tmp/wiki/entities/brad.md",
|
||||
relativePath: "entities/brad.md",
|
||||
raw,
|
||||
});
|
||||
|
||||
expect(summary).toEqual(
|
||||
expect.objectContaining({
|
||||
entityType: "person",
|
||||
canonicalId: "maintainer.brad-groux",
|
||||
aliases: ["brad", "bgroux"],
|
||||
privacyTier: "local-private",
|
||||
bestUsedFor: ["Microsoft ecosystem routing"],
|
||||
notEnoughFor: ["legal approval"],
|
||||
lastRefreshedAt: "2026-04-29T00:00:00.000Z",
|
||||
personCard: expect.objectContaining({
|
||||
handles: ["@bgroux"],
|
||||
emails: ["brad@example.com"],
|
||||
lane: "Microsoft Teams",
|
||||
privacyTier: "confirm-before-use",
|
||||
}),
|
||||
relationships: [
|
||||
expect.objectContaining({
|
||||
targetId: "entity.alice",
|
||||
kind: "collaborates-with",
|
||||
evidenceKind: "discrawl-stat",
|
||||
}),
|
||||
],
|
||||
claims: [
|
||||
expect.objectContaining({
|
||||
id: "claim.brad.teams",
|
||||
evidence: [
|
||||
expect.objectContaining({
|
||||
kind: "maintainer-whois",
|
||||
privacyTier: "local-private",
|
||||
}),
|
||||
],
|
||||
}),
|
||||
],
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -19,10 +19,13 @@ export type ParsedWikiMarkdown = {
|
||||
};
|
||||
|
||||
export type WikiClaimEvidence = {
|
||||
kind?: string;
|
||||
sourceId?: string;
|
||||
path?: string;
|
||||
lines?: string;
|
||||
weight?: number;
|
||||
confidence?: number;
|
||||
privacyTier?: string;
|
||||
note?: string;
|
||||
updatedAt?: string;
|
||||
};
|
||||
@@ -36,6 +39,35 @@ export type WikiClaim = {
|
||||
updatedAt?: string;
|
||||
};
|
||||
|
||||
export type WikiPersonCard = {
|
||||
canonicalId?: string;
|
||||
handles: string[];
|
||||
socials: string[];
|
||||
emails: string[];
|
||||
timezone?: string;
|
||||
lane?: string;
|
||||
askFor: string[];
|
||||
avoidAskingFor: string[];
|
||||
bestUsedFor: string[];
|
||||
notEnoughFor: string[];
|
||||
confidence?: number;
|
||||
privacyTier?: string;
|
||||
lastRefreshedAt?: string;
|
||||
};
|
||||
|
||||
export type WikiRelationship = {
|
||||
targetId?: string;
|
||||
targetPath?: string;
|
||||
targetTitle?: string;
|
||||
kind?: string;
|
||||
weight?: number;
|
||||
confidence?: number;
|
||||
evidenceKind?: string;
|
||||
privacyTier?: string;
|
||||
note?: string;
|
||||
updatedAt?: string;
|
||||
};
|
||||
|
||||
export type WikiPageSummary = {
|
||||
absolutePath: string;
|
||||
relativePath: string;
|
||||
@@ -43,12 +75,20 @@ export type WikiPageSummary = {
|
||||
title: string;
|
||||
id?: string;
|
||||
pageType?: string;
|
||||
entityType?: string;
|
||||
canonicalId?: string;
|
||||
aliases: string[];
|
||||
sourceIds: string[];
|
||||
linkTargets: string[];
|
||||
claims: WikiClaim[];
|
||||
contradictions: string[];
|
||||
questions: string[];
|
||||
confidence?: number;
|
||||
privacyTier?: string;
|
||||
personCard?: WikiPersonCard;
|
||||
relationships: WikiRelationship[];
|
||||
bestUsedFor: string[];
|
||||
notEnoughFor: string[];
|
||||
sourceType?: string;
|
||||
provenanceMode?: string;
|
||||
sourcePath?: string;
|
||||
@@ -56,6 +96,7 @@ export type WikiPageSummary = {
|
||||
bridgeWorkspaceDir?: string;
|
||||
unsafeLocalConfiguredPath?: string;
|
||||
unsafeLocalRelativePath?: string;
|
||||
lastRefreshedAt?: string;
|
||||
updatedAt?: string;
|
||||
};
|
||||
|
||||
@@ -153,21 +194,37 @@ function normalizeWikiClaimEvidence(value: unknown): WikiClaimEvidence | null {
|
||||
return null;
|
||||
}
|
||||
const record = value as Record<string, unknown>;
|
||||
const kind = normalizeOptionalString(record.kind);
|
||||
const sourceId = normalizeOptionalString(record.sourceId);
|
||||
const evidencePath = normalizeOptionalString(record.path);
|
||||
const lines = normalizeOptionalString(record.lines);
|
||||
const note = normalizeOptionalString(record.note);
|
||||
const updatedAt = normalizeOptionalString(record.updatedAt);
|
||||
const privacyTier = normalizeOptionalString(record.privacyTier);
|
||||
const weight =
|
||||
typeof record.weight === "number" && Number.isFinite(record.weight) ? record.weight : undefined;
|
||||
if (!sourceId && !evidencePath && !lines && !note && weight === undefined && !updatedAt) {
|
||||
const confidence = normalizeOptionalNumber(record.confidence);
|
||||
if (
|
||||
!kind &&
|
||||
!sourceId &&
|
||||
!evidencePath &&
|
||||
!lines &&
|
||||
!note &&
|
||||
weight === undefined &&
|
||||
confidence === undefined &&
|
||||
!privacyTier &&
|
||||
!updatedAt
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
...(kind ? { kind } : {}),
|
||||
...(sourceId ? { sourceId } : {}),
|
||||
...(evidencePath ? { path: evidencePath } : {}),
|
||||
...(lines ? { lines } : {}),
|
||||
...(weight !== undefined ? { weight } : {}),
|
||||
...(confidence !== undefined ? { confidence } : {}),
|
||||
...(privacyTier ? { privacyTier } : {}),
|
||||
...(note ? { note } : {}),
|
||||
...(updatedAt ? { updatedAt } : {}),
|
||||
};
|
||||
@@ -213,6 +270,101 @@ export function normalizeWikiClaims(value: unknown): WikiClaim[] {
|
||||
});
|
||||
}
|
||||
|
||||
function normalizeOptionalNumber(value: unknown): number | undefined {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
|
||||
}
|
||||
|
||||
function normalizeWikiPersonCard(value: unknown): WikiPersonCard | undefined {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return undefined;
|
||||
}
|
||||
const record = value as Record<string, unknown>;
|
||||
const card: WikiPersonCard = {
|
||||
...(normalizeOptionalString(record.canonicalId)
|
||||
? { canonicalId: normalizeOptionalString(record.canonicalId) }
|
||||
: {}),
|
||||
handles: normalizeSingleOrTrimmedStringList(record.handles),
|
||||
socials: normalizeSingleOrTrimmedStringList(record.socials),
|
||||
emails: normalizeSingleOrTrimmedStringList(record.emails ?? record.email),
|
||||
...(normalizeOptionalString(record.timezone)
|
||||
? { timezone: normalizeOptionalString(record.timezone) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.lane) ? { lane: normalizeOptionalString(record.lane) } : {}),
|
||||
askFor: normalizeSingleOrTrimmedStringList(record.askFor),
|
||||
avoidAskingFor: normalizeSingleOrTrimmedStringList(record.avoidAskingFor),
|
||||
bestUsedFor: normalizeSingleOrTrimmedStringList(record.bestUsedFor),
|
||||
notEnoughFor: normalizeSingleOrTrimmedStringList(record.notEnoughFor),
|
||||
...(normalizeOptionalNumber(record.confidence) !== undefined
|
||||
? { confidence: normalizeOptionalNumber(record.confidence) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.privacyTier)
|
||||
? { privacyTier: normalizeOptionalString(record.privacyTier) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.lastRefreshedAt)
|
||||
? { lastRefreshedAt: normalizeOptionalString(record.lastRefreshedAt) }
|
||||
: {}),
|
||||
};
|
||||
const hasAnyValue =
|
||||
Boolean(
|
||||
card.canonicalId || card.timezone || card.lane || card.privacyTier || card.lastRefreshedAt,
|
||||
) ||
|
||||
typeof card.confidence === "number" ||
|
||||
card.handles.length > 0 ||
|
||||
card.socials.length > 0 ||
|
||||
card.emails.length > 0 ||
|
||||
card.askFor.length > 0 ||
|
||||
card.avoidAskingFor.length > 0 ||
|
||||
card.bestUsedFor.length > 0 ||
|
||||
card.notEnoughFor.length > 0;
|
||||
return hasAnyValue ? card : undefined;
|
||||
}
|
||||
|
||||
function normalizeWikiRelationships(value: unknown): WikiRelationship[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
return value.flatMap((entry) => {
|
||||
if (!entry || typeof entry !== "object" || Array.isArray(entry)) {
|
||||
return [];
|
||||
}
|
||||
const record = entry as Record<string, unknown>;
|
||||
const relationship: WikiRelationship = {
|
||||
...(normalizeOptionalString(record.targetId)
|
||||
? { targetId: normalizeOptionalString(record.targetId) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.targetPath)
|
||||
? { targetPath: normalizeOptionalString(record.targetPath) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.targetTitle)
|
||||
? { targetTitle: normalizeOptionalString(record.targetTitle) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.kind)
|
||||
? { kind: normalizeOptionalString(record.kind) }
|
||||
: {}),
|
||||
...(normalizeOptionalNumber(record.weight) !== undefined
|
||||
? { weight: normalizeOptionalNumber(record.weight) }
|
||||
: {}),
|
||||
...(normalizeOptionalNumber(record.confidence) !== undefined
|
||||
? { confidence: normalizeOptionalNumber(record.confidence) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.evidenceKind)
|
||||
? { evidenceKind: normalizeOptionalString(record.evidenceKind) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.privacyTier)
|
||||
? { privacyTier: normalizeOptionalString(record.privacyTier) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.note)
|
||||
? { note: normalizeOptionalString(record.note) }
|
||||
: {}),
|
||||
...(normalizeOptionalString(record.updatedAt)
|
||||
? { updatedAt: normalizeOptionalString(record.updatedAt) }
|
||||
: {}),
|
||||
};
|
||||
const hasAnyValue = Object.keys(relationship).length > 0;
|
||||
return hasAnyValue ? [relationship] : [];
|
||||
});
|
||||
}
|
||||
|
||||
export function extractWikiLinks(markdown: string): string[] {
|
||||
const searchable = markdown.replace(RELATED_BLOCK_PATTERN, "");
|
||||
const links: string[] = [];
|
||||
@@ -297,6 +449,9 @@ export function toWikiPageSummary(params: {
|
||||
title,
|
||||
id: normalizeOptionalString(parsed.frontmatter.id),
|
||||
pageType: normalizeOptionalString(parsed.frontmatter.pageType),
|
||||
entityType: normalizeOptionalString(parsed.frontmatter.entityType),
|
||||
canonicalId: normalizeOptionalString(parsed.frontmatter.canonicalId),
|
||||
aliases: normalizeSingleOrTrimmedStringList(parsed.frontmatter.aliases),
|
||||
sourceIds: normalizeSourceIds(parsed.frontmatter.sourceIds),
|
||||
linkTargets: extractWikiLinks(params.raw),
|
||||
claims: normalizeWikiClaims(parsed.frontmatter.claims),
|
||||
@@ -307,6 +462,11 @@ export function toWikiPageSummary(params: {
|
||||
Number.isFinite(parsed.frontmatter.confidence)
|
||||
? parsed.frontmatter.confidence
|
||||
: undefined,
|
||||
privacyTier: normalizeOptionalString(parsed.frontmatter.privacyTier),
|
||||
personCard: normalizeWikiPersonCard(parsed.frontmatter.personCard),
|
||||
relationships: normalizeWikiRelationships(parsed.frontmatter.relationships),
|
||||
bestUsedFor: normalizeSingleOrTrimmedStringList(parsed.frontmatter.bestUsedFor),
|
||||
notEnoughFor: normalizeSingleOrTrimmedStringList(parsed.frontmatter.notEnoughFor),
|
||||
sourceType: normalizeOptionalString(parsed.frontmatter.sourceType),
|
||||
provenanceMode: normalizeOptionalString(parsed.frontmatter.provenanceMode),
|
||||
sourcePath: normalizeOptionalString(parsed.frontmatter.sourcePath),
|
||||
@@ -316,6 +476,7 @@ export function toWikiPageSummary(params: {
|
||||
parsed.frontmatter.unsafeLocalConfiguredPath,
|
||||
),
|
||||
unsafeLocalRelativePath: normalizeOptionalString(parsed.frontmatter.unsafeLocalRelativePath),
|
||||
lastRefreshedAt: normalizeOptionalString(parsed.frontmatter.lastRefreshedAt),
|
||||
updatedAt: normalizeOptionalString(parsed.frontmatter.updatedAt),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -205,6 +205,114 @@ describe("searchMemoryWiki", () => {
|
||||
expect(results[0]?.snippet).toContain("Teams");
|
||||
});
|
||||
|
||||
it("supports people-routing search modes and claim evidence drilldown metadata", async () => {
|
||||
const { rootDir, config } = await createQueryVault({
|
||||
initialize: true,
|
||||
});
|
||||
await fs.writeFile(
|
||||
path.join(rootDir, "entities", "brad.md"),
|
||||
renderWikiMarkdown({
|
||||
frontmatter: {
|
||||
pageType: "entity",
|
||||
entityType: "person",
|
||||
id: "entity.brad",
|
||||
title: "Brad Groux",
|
||||
canonicalId: "maintainer.brad-groux",
|
||||
aliases: ["bgroux"],
|
||||
privacyTier: "local-private",
|
||||
personCard: {
|
||||
handles: ["@bgroux"],
|
||||
lane: "Microsoft Teams",
|
||||
askFor: ["Teams and Azure rollout questions"],
|
||||
},
|
||||
bestUsedFor: ["Microsoft ecosystem routing"],
|
||||
relationships: [
|
||||
{
|
||||
targetId: "entity.alice",
|
||||
targetTitle: "Alice",
|
||||
kind: "works-with",
|
||||
note: "Teams escalation buddy",
|
||||
},
|
||||
],
|
||||
claims: [
|
||||
{
|
||||
id: "claim.brad.teams",
|
||||
text: "Brad is a strong route for Microsoft Teams questions.",
|
||||
status: "supported",
|
||||
confidence: 0.88,
|
||||
evidence: [
|
||||
{
|
||||
kind: "maintainer-whois",
|
||||
sourceId: "source.maintainers",
|
||||
privacyTier: "local-private",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
body: "# Brad Groux\n\nAgent card summary.\n",
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
await fs.writeFile(
|
||||
path.join(rootDir, "sources", "maintainers.md"),
|
||||
renderWikiMarkdown({
|
||||
frontmatter: {
|
||||
pageType: "source",
|
||||
id: "source.maintainers",
|
||||
title: "Maintainers Source",
|
||||
},
|
||||
body: "# Maintainers Source\n\nmaintainer-whois Teams sample.\n",
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
const personResults = await searchMemoryWiki({
|
||||
config,
|
||||
query: "bgroux",
|
||||
mode: "find-person",
|
||||
});
|
||||
expect(personResults[0]).toEqual(
|
||||
expect.objectContaining({
|
||||
path: "entities/brad.md",
|
||||
canonicalId: "maintainer.brad-groux",
|
||||
aliases: ["bgroux"],
|
||||
privacyTier: "local-private",
|
||||
searchMode: "find-person",
|
||||
}),
|
||||
);
|
||||
|
||||
const routeResults = await searchMemoryWiki({
|
||||
config,
|
||||
query: "Teams Azure",
|
||||
mode: "route-question",
|
||||
});
|
||||
expect(routeResults[0]?.path).toBe("entities/brad.md");
|
||||
|
||||
const claimResults = await searchMemoryWiki({
|
||||
config,
|
||||
query: "strong route Teams",
|
||||
mode: "raw-claim",
|
||||
});
|
||||
expect(claimResults[0]).toEqual(
|
||||
expect.objectContaining({
|
||||
path: "entities/brad.md",
|
||||
matchedClaimId: "claim.brad.teams",
|
||||
matchedClaimConfidence: 0.88,
|
||||
evidenceKinds: ["maintainer-whois"],
|
||||
evidenceSourceIds: ["source.maintainers"],
|
||||
}),
|
||||
);
|
||||
|
||||
const evidenceResults = await searchMemoryWiki({
|
||||
config,
|
||||
query: "maintainer-whois",
|
||||
mode: "source-evidence",
|
||||
maxResults: 2,
|
||||
});
|
||||
expect(evidenceResults.map((result) => result.path)).toContain("sources/maintainers.md");
|
||||
});
|
||||
|
||||
it("uses body text instead of frontmatter for fallback snippets", async () => {
|
||||
const { rootDir, config } = await createQueryVault({
|
||||
initialize: true,
|
||||
|
||||
@@ -22,14 +22,33 @@ const RELATED_BLOCK_PATTERN =
|
||||
/<!-- openclaw:wiki:related:start -->[\s\S]*?<!-- openclaw:wiki:related:end -->/g;
|
||||
const MARKDOWN_FRONTMATTER_PATTERN = /^\s*---\r?\n[\s\S]*?\r?\n---\r?\n?/;
|
||||
|
||||
export const WIKI_SEARCH_MODES = [
|
||||
"auto",
|
||||
"find-person",
|
||||
"route-question",
|
||||
"source-evidence",
|
||||
"raw-claim",
|
||||
] as const;
|
||||
|
||||
export type WikiSearchMode = (typeof WIKI_SEARCH_MODES)[number];
|
||||
|
||||
type QueryDigestPage = {
|
||||
id?: string;
|
||||
title: string;
|
||||
kind: WikiPageSummary["kind"];
|
||||
path: string;
|
||||
pageType?: string;
|
||||
entityType?: string;
|
||||
canonicalId?: string;
|
||||
aliases?: string[];
|
||||
sourceIds: string[];
|
||||
questions: string[];
|
||||
contradictions: string[];
|
||||
privacyTier?: string;
|
||||
personCard?: WikiPageSummary["personCard"];
|
||||
bestUsedFor?: string[];
|
||||
notEnoughFor?: string[];
|
||||
relationshipCount?: number;
|
||||
};
|
||||
|
||||
type QueryDigestClaim = {
|
||||
@@ -38,10 +57,16 @@ type QueryDigestClaim = {
|
||||
pageTitle: string;
|
||||
pageKind: WikiPageSummary["kind"];
|
||||
pagePath: string;
|
||||
pageType?: string;
|
||||
entityType?: string;
|
||||
canonicalId?: string;
|
||||
aliases?: string[];
|
||||
text: string;
|
||||
status?: string;
|
||||
confidence?: number;
|
||||
sourceIds?: string[];
|
||||
evidenceKinds?: string[];
|
||||
privacyTiers?: string[];
|
||||
freshnessLevel?: string;
|
||||
lastTouchedAt?: string;
|
||||
};
|
||||
@@ -68,6 +93,16 @@ export type WikiSearchResult = {
|
||||
sourcePath?: string;
|
||||
provenanceLabel?: string;
|
||||
updatedAt?: string;
|
||||
searchMode?: WikiSearchMode;
|
||||
entityType?: string;
|
||||
canonicalId?: string;
|
||||
aliases?: string[];
|
||||
privacyTier?: string;
|
||||
matchedClaimId?: string;
|
||||
matchedClaimStatus?: string;
|
||||
matchedClaimConfidence?: number;
|
||||
evidenceKinds?: string[];
|
||||
evidenceSourceIds?: string[];
|
||||
};
|
||||
|
||||
export type WikiGetResult = {
|
||||
@@ -206,11 +241,50 @@ function buildPageSearchText(page: QueryableWikiPage): string {
|
||||
page.title,
|
||||
page.relativePath,
|
||||
page.id ?? "",
|
||||
page.pageType ?? "",
|
||||
page.entityType ?? "",
|
||||
page.canonicalId ?? "",
|
||||
page.aliases.join(" "),
|
||||
page.sourceIds.join(" "),
|
||||
page.questions.join(" "),
|
||||
page.contradictions.join(" "),
|
||||
page.privacyTier ?? "",
|
||||
page.bestUsedFor.join(" "),
|
||||
page.notEnoughFor.join(" "),
|
||||
page.personCard?.canonicalId ?? "",
|
||||
page.personCard?.handles.join(" ") ?? "",
|
||||
page.personCard?.socials.join(" ") ?? "",
|
||||
page.personCard?.emails.join(" ") ?? "",
|
||||
page.personCard?.timezone ?? "",
|
||||
page.personCard?.lane ?? "",
|
||||
page.personCard?.askFor.join(" ") ?? "",
|
||||
page.personCard?.avoidAskingFor.join(" ") ?? "",
|
||||
page.personCard?.bestUsedFor.join(" ") ?? "",
|
||||
page.personCard?.notEnoughFor.join(" ") ?? "",
|
||||
page.relationships
|
||||
.flatMap((relationship) => [
|
||||
relationship.targetId ?? "",
|
||||
relationship.targetPath ?? "",
|
||||
relationship.targetTitle ?? "",
|
||||
relationship.kind ?? "",
|
||||
relationship.evidenceKind ?? "",
|
||||
relationship.note ?? "",
|
||||
])
|
||||
.join(" "),
|
||||
page.claims.map((claim) => claim.text).join(" "),
|
||||
page.claims.map((claim) => claim.id ?? "").join(" "),
|
||||
page.claims
|
||||
.flatMap((claim) =>
|
||||
claim.evidence.flatMap((evidence) => [
|
||||
evidence.kind ?? "",
|
||||
evidence.sourceId ?? "",
|
||||
evidence.path ?? "",
|
||||
evidence.lines ?? "",
|
||||
evidence.note ?? "",
|
||||
evidence.privacyTier ?? "",
|
||||
]),
|
||||
)
|
||||
.join(" "),
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n");
|
||||
@@ -247,11 +321,30 @@ function buildDigestPageSearchText(page: QueryDigestPage, claims: QueryDigestCla
|
||||
page.title,
|
||||
page.path,
|
||||
page.id ?? "",
|
||||
page.pageType ?? "",
|
||||
page.entityType ?? "",
|
||||
page.canonicalId ?? "",
|
||||
page.aliases?.join(" ") ?? "",
|
||||
page.sourceIds.join(" "),
|
||||
page.questions.join(" "),
|
||||
page.contradictions.join(" "),
|
||||
page.privacyTier ?? "",
|
||||
page.bestUsedFor?.join(" ") ?? "",
|
||||
page.notEnoughFor?.join(" ") ?? "",
|
||||
page.personCard?.canonicalId ?? "",
|
||||
page.personCard?.handles.join(" ") ?? "",
|
||||
page.personCard?.socials.join(" ") ?? "",
|
||||
page.personCard?.emails.join(" ") ?? "",
|
||||
page.personCard?.timezone ?? "",
|
||||
page.personCard?.lane ?? "",
|
||||
page.personCard?.askFor.join(" ") ?? "",
|
||||
page.personCard?.avoidAskingFor.join(" ") ?? "",
|
||||
page.personCard?.bestUsedFor.join(" ") ?? "",
|
||||
page.personCard?.notEnoughFor.join(" ") ?? "",
|
||||
claims.map((claim) => claim.text).join(" "),
|
||||
claims.map((claim) => claim.id ?? "").join(" "),
|
||||
claims.map((claim) => claim.evidenceKinds?.join(" ") ?? "").join(" "),
|
||||
claims.map((claim) => claim.privacyTiers?.join(" ") ?? "").join(" "),
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n");
|
||||
@@ -260,11 +353,13 @@ function buildDigestPageSearchText(page: QueryDigestPage, claims: QueryDigestCla
|
||||
function isClaimTextOrIdMatch(
|
||||
claim: Pick<QueryDigestClaim, "id" | "text"> | Pick<WikiClaim, "id" | "text">,
|
||||
queryLower: string,
|
||||
queryTokens: readonly string[] = buildQueryTokens(queryLower),
|
||||
): boolean {
|
||||
if (normalizeLowercaseStringOrEmpty(claim.text).includes(queryLower)) {
|
||||
const textLower = normalizeLowercaseStringOrEmpty(claim.text);
|
||||
if (lineMatchesQuery(textLower, queryLower, [...queryTokens])) {
|
||||
return true;
|
||||
}
|
||||
return normalizeLowercaseStringOrEmpty(claim.id).includes(queryLower);
|
||||
return lineMatchesQuery(normalizeLowercaseStringOrEmpty(claim.id), queryLower, [...queryTokens]);
|
||||
}
|
||||
|
||||
function scoreClaimMatch(params: {
|
||||
@@ -274,10 +369,18 @@ function scoreClaimMatch(params: {
|
||||
status?: string;
|
||||
freshnessLevel?: string;
|
||||
queryLower: string;
|
||||
queryTokens?: readonly string[];
|
||||
}): number {
|
||||
let score = 0;
|
||||
if (normalizeLowercaseStringOrEmpty(params.text).includes(params.queryLower)) {
|
||||
score += 25;
|
||||
} else if (
|
||||
params.queryTokens?.length &&
|
||||
params.queryTokens.every((token) =>
|
||||
normalizeLowercaseStringOrEmpty(params.text).includes(token),
|
||||
)
|
||||
) {
|
||||
score += 18;
|
||||
}
|
||||
if (normalizeLowercaseStringOrEmpty(params.id).includes(params.queryLower)) {
|
||||
score += 10;
|
||||
@@ -313,6 +416,7 @@ function scoreDigestClaimMatch(claim: QueryDigestClaim, queryLower: string): num
|
||||
status: claim.status,
|
||||
freshnessLevel: claim.freshnessLevel,
|
||||
queryLower,
|
||||
queryTokens: buildQueryTokens(queryLower),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -348,12 +452,212 @@ function scoreWikiMetadataMatch(params: {
|
||||
return score;
|
||||
}
|
||||
|
||||
function hasQueryMatch(
|
||||
value: string | undefined,
|
||||
queryLower: string,
|
||||
queryTokens: readonly string[],
|
||||
) {
|
||||
const normalized = normalizeLowercaseStringOrEmpty(value);
|
||||
return lineMatchesQuery(normalized, queryLower, [...queryTokens]);
|
||||
}
|
||||
|
||||
function hasAnyQueryMatch(
|
||||
values: readonly (string | undefined)[],
|
||||
queryLower: string,
|
||||
queryTokens: readonly string[],
|
||||
) {
|
||||
return values.some((value) => hasQueryMatch(value, queryLower, queryTokens));
|
||||
}
|
||||
|
||||
function isPersonLikeSummary(
|
||||
page: Pick<WikiPageSummary, "entityType" | "pageType" | "personCard">,
|
||||
): boolean {
|
||||
const entityType = normalizeLowercaseStringOrEmpty(page.entityType);
|
||||
const pageType = normalizeLowercaseStringOrEmpty(page.pageType);
|
||||
return (
|
||||
Boolean(page.personCard) ||
|
||||
entityType === "person" ||
|
||||
entityType === "maintainer" ||
|
||||
pageType === "person" ||
|
||||
pageType === "maintainer"
|
||||
);
|
||||
}
|
||||
|
||||
function scorePageSearchModeBoost(params: {
|
||||
page: QueryableWikiPage;
|
||||
matchingClaims: readonly WikiClaim[];
|
||||
queryLower: string;
|
||||
queryTokens: readonly string[];
|
||||
mode: WikiSearchMode;
|
||||
}): number {
|
||||
const { page, queryLower, queryTokens } = params;
|
||||
switch (params.mode) {
|
||||
case "auto":
|
||||
return 0;
|
||||
case "find-person": {
|
||||
let score = isPersonLikeSummary(page) ? 24 : -4;
|
||||
if (
|
||||
hasAnyQueryMatch(
|
||||
[
|
||||
page.canonicalId,
|
||||
...page.aliases,
|
||||
page.personCard?.canonicalId,
|
||||
...(page.personCard?.handles ?? []),
|
||||
...(page.personCard?.emails ?? []),
|
||||
...(page.personCard?.socials ?? []),
|
||||
],
|
||||
queryLower,
|
||||
queryTokens,
|
||||
)
|
||||
) {
|
||||
score += 24;
|
||||
}
|
||||
return score;
|
||||
}
|
||||
case "route-question": {
|
||||
let score = isPersonLikeSummary(page) ? 14 : 0;
|
||||
if (
|
||||
hasAnyQueryMatch(
|
||||
[
|
||||
page.personCard?.lane,
|
||||
...(page.personCard?.askFor ?? []),
|
||||
...(page.personCard?.avoidAskingFor ?? []),
|
||||
...page.bestUsedFor,
|
||||
...page.notEnoughFor,
|
||||
...(page.personCard?.bestUsedFor ?? []),
|
||||
...(page.personCard?.notEnoughFor ?? []),
|
||||
...page.relationships.flatMap((relationship) => [
|
||||
relationship.kind,
|
||||
relationship.targetTitle,
|
||||
relationship.note,
|
||||
]),
|
||||
],
|
||||
queryLower,
|
||||
queryTokens,
|
||||
)
|
||||
) {
|
||||
score += 32;
|
||||
}
|
||||
score += Math.min(8, page.relationships.length * 2);
|
||||
return score;
|
||||
}
|
||||
case "source-evidence": {
|
||||
let score = page.kind === "source" ? 22 : 0;
|
||||
if (
|
||||
hasAnyQueryMatch(
|
||||
[
|
||||
page.sourcePath,
|
||||
...page.sourceIds,
|
||||
...page.claims.flatMap((claim) =>
|
||||
claim.evidence.flatMap((evidence) => [
|
||||
evidence.kind,
|
||||
evidence.sourceId,
|
||||
evidence.path,
|
||||
evidence.lines,
|
||||
evidence.note,
|
||||
]),
|
||||
),
|
||||
],
|
||||
queryLower,
|
||||
queryTokens,
|
||||
)
|
||||
) {
|
||||
score += 30;
|
||||
}
|
||||
return score;
|
||||
}
|
||||
case "raw-claim":
|
||||
return params.matchingClaims.length > 0 ? 42 : 0;
|
||||
}
|
||||
}
|
||||
|
||||
function scoreDigestSearchModeBoost(params: {
|
||||
page: QueryDigestPage;
|
||||
claims: readonly QueryDigestClaim[];
|
||||
matchingClaims: readonly QueryDigestClaim[];
|
||||
queryLower: string;
|
||||
queryTokens: readonly string[];
|
||||
mode: WikiSearchMode;
|
||||
}): number {
|
||||
const { page, queryLower, queryTokens } = params;
|
||||
switch (params.mode) {
|
||||
case "auto":
|
||||
return 0;
|
||||
case "find-person": {
|
||||
let score = isPersonLikeSummary(page) ? 24 : -4;
|
||||
if (
|
||||
hasAnyQueryMatch(
|
||||
[
|
||||
page.canonicalId,
|
||||
...(page.aliases ?? []),
|
||||
page.personCard?.canonicalId,
|
||||
...(page.personCard?.handles ?? []),
|
||||
...(page.personCard?.emails ?? []),
|
||||
...(page.personCard?.socials ?? []),
|
||||
],
|
||||
queryLower,
|
||||
queryTokens,
|
||||
)
|
||||
) {
|
||||
score += 24;
|
||||
}
|
||||
return score;
|
||||
}
|
||||
case "route-question": {
|
||||
let score = isPersonLikeSummary(page) ? 14 : 0;
|
||||
if (
|
||||
hasAnyQueryMatch(
|
||||
[
|
||||
page.personCard?.lane,
|
||||
...(page.personCard?.askFor ?? []),
|
||||
...(page.personCard?.avoidAskingFor ?? []),
|
||||
...(page.bestUsedFor ?? []),
|
||||
...(page.notEnoughFor ?? []),
|
||||
...(page.personCard?.bestUsedFor ?? []),
|
||||
...(page.personCard?.notEnoughFor ?? []),
|
||||
],
|
||||
queryLower,
|
||||
queryTokens,
|
||||
)
|
||||
) {
|
||||
score += 32;
|
||||
}
|
||||
score += Math.min(8, (page.relationshipCount ?? 0) * 2);
|
||||
return score;
|
||||
}
|
||||
case "source-evidence": {
|
||||
let score = page.kind === "source" ? 22 : 0;
|
||||
if (
|
||||
hasAnyQueryMatch(
|
||||
[
|
||||
...page.sourceIds,
|
||||
...params.claims.flatMap((claim) => [
|
||||
...(claim.sourceIds ?? []),
|
||||
...(claim.evidenceKinds ?? []),
|
||||
...(claim.privacyTiers ?? []),
|
||||
]),
|
||||
],
|
||||
queryLower,
|
||||
queryTokens,
|
||||
)
|
||||
) {
|
||||
score += 30;
|
||||
}
|
||||
return score;
|
||||
}
|
||||
case "raw-claim":
|
||||
return params.matchingClaims.length > 0 ? 42 : 0;
|
||||
}
|
||||
}
|
||||
|
||||
function buildDigestCandidatePaths(params: {
|
||||
digest: QueryDigestBundle;
|
||||
query: string;
|
||||
maxResults: number;
|
||||
mode: WikiSearchMode;
|
||||
}): string[] {
|
||||
const queryLower = normalizeLowercaseStringOrEmpty(params.query);
|
||||
const queryTokens = buildQueryTokens(queryLower);
|
||||
const claimsByPage = new Map<string, QueryDigestClaim[]>();
|
||||
for (const claim of params.digest.claims) {
|
||||
const current = claimsByPage.get(claim.pagePath) ?? [];
|
||||
@@ -380,7 +684,7 @@ function buildDigestCandidatePaths(params: {
|
||||
queryLower,
|
||||
});
|
||||
const matchingClaims = claims
|
||||
.filter((claim) => isClaimTextOrIdMatch(claim, queryLower))
|
||||
.filter((claim) => isClaimTextOrIdMatch(claim, queryLower, queryTokens))
|
||||
.toSorted(
|
||||
(left, right) =>
|
||||
scoreDigestClaimMatch(right, queryLower) - scoreDigestClaimMatch(left, queryLower),
|
||||
@@ -389,6 +693,14 @@ function buildDigestCandidatePaths(params: {
|
||||
score += scoreDigestClaimMatch(matchingClaims[0], queryLower);
|
||||
score += Math.min(10, (matchingClaims.length - 1) * 2);
|
||||
}
|
||||
score += scoreDigestSearchModeBoost({
|
||||
page,
|
||||
claims,
|
||||
matchingClaims,
|
||||
queryLower,
|
||||
queryTokens,
|
||||
mode: params.mode,
|
||||
});
|
||||
return { path: page.path, score };
|
||||
})
|
||||
.filter((candidate) => candidate.score > 0)
|
||||
@@ -402,11 +714,20 @@ function buildDigestCandidatePaths(params: {
|
||||
.map((candidate) => candidate.path);
|
||||
}
|
||||
|
||||
function isClaimMatch(claim: WikiClaim, queryLower: string): boolean {
|
||||
return isClaimTextOrIdMatch(claim, queryLower);
|
||||
function isClaimMatch(
|
||||
claim: WikiClaim,
|
||||
queryLower: string,
|
||||
queryTokens: readonly string[],
|
||||
): boolean {
|
||||
return isClaimTextOrIdMatch(claim, queryLower, queryTokens);
|
||||
}
|
||||
|
||||
function rankClaimMatch(page: QueryableWikiPage, claim: WikiClaim, queryLower: string): number {
|
||||
function rankClaimMatch(
|
||||
page: QueryableWikiPage,
|
||||
claim: WikiClaim,
|
||||
queryLower: string,
|
||||
queryTokens: readonly string[],
|
||||
): number {
|
||||
const freshness = assessClaimFreshness({ page, claim });
|
||||
return scoreClaimMatch({
|
||||
text: claim.text,
|
||||
@@ -415,15 +736,18 @@ function rankClaimMatch(page: QueryableWikiPage, claim: WikiClaim, queryLower: s
|
||||
status: claim.status,
|
||||
freshnessLevel: freshness.level,
|
||||
queryLower,
|
||||
queryTokens,
|
||||
});
|
||||
}
|
||||
|
||||
function getMatchingClaims(page: QueryableWikiPage, queryLower: string): WikiClaim[] {
|
||||
const queryTokens = buildQueryTokens(queryLower);
|
||||
return page.claims
|
||||
.filter((claim) => isClaimMatch(claim, queryLower))
|
||||
.filter((claim) => isClaimMatch(claim, queryLower, queryTokens))
|
||||
.toSorted(
|
||||
(left, right) =>
|
||||
rankClaimMatch(page, right, queryLower) - rankClaimMatch(page, left, queryLower),
|
||||
rankClaimMatch(page, right, queryLower, queryTokens) -
|
||||
rankClaimMatch(page, left, queryLower, queryTokens),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -436,7 +760,7 @@ function buildPageSnippet(page: QueryableWikiPage, query: string): string {
|
||||
return buildSnippet(page.raw, query);
|
||||
}
|
||||
|
||||
function scorePage(page: QueryableWikiPage, query: string): number {
|
||||
function scorePage(page: QueryableWikiPage, query: string, mode: WikiSearchMode): number {
|
||||
const queryLower = normalizeLowercaseStringOrEmpty(query);
|
||||
const queryTokens = buildQueryTokens(queryLower);
|
||||
const titleLower = normalizeLowercaseStringOrEmpty(page.title);
|
||||
@@ -468,9 +792,16 @@ function scorePage(page: QueryableWikiPage, query: string): number {
|
||||
});
|
||||
const matchingClaims = getMatchingClaims(page, queryLower);
|
||||
if (matchingClaims.length > 0) {
|
||||
score += rankClaimMatch(page, matchingClaims[0], queryLower);
|
||||
score += rankClaimMatch(page, matchingClaims[0], queryLower, queryTokens);
|
||||
score += Math.min(10, (matchingClaims.length - 1) * 2);
|
||||
}
|
||||
score += scorePageSearchModeBoost({
|
||||
page,
|
||||
matchingClaims,
|
||||
queryLower,
|
||||
queryTokens,
|
||||
mode,
|
||||
});
|
||||
const bodyOccurrences = rawLower.split(queryLower).length - 1;
|
||||
score += Math.min(10, bodyOccurrences);
|
||||
for (const token of queryTokens) {
|
||||
@@ -585,6 +916,10 @@ function buildWikiProvenanceLabel(
|
||||
| "bridgeRelativePath"
|
||||
| "unsafeLocalRelativePath"
|
||||
| "relativePath"
|
||||
| "entityType"
|
||||
| "canonicalId"
|
||||
| "aliases"
|
||||
| "privacyTier"
|
||||
>,
|
||||
): string | undefined {
|
||||
if (page.sourceType === "memory-bridge-events") {
|
||||
@@ -610,11 +945,24 @@ function buildWikiResultMetadata(
|
||||
| "bridgeRelativePath"
|
||||
| "unsafeLocalRelativePath"
|
||||
| "relativePath"
|
||||
| "entityType"
|
||||
| "canonicalId"
|
||||
| "aliases"
|
||||
| "privacyTier"
|
||||
>,
|
||||
): Partial<
|
||||
Pick<
|
||||
WikiSearchResult,
|
||||
"id" | "sourceType" | "provenanceMode" | "sourcePath" | "provenanceLabel" | "updatedAt"
|
||||
| "id"
|
||||
| "sourceType"
|
||||
| "provenanceMode"
|
||||
| "sourcePath"
|
||||
| "provenanceLabel"
|
||||
| "updatedAt"
|
||||
| "entityType"
|
||||
| "canonicalId"
|
||||
| "aliases"
|
||||
| "privacyTier"
|
||||
>
|
||||
> {
|
||||
const provenanceLabel = buildWikiProvenanceLabel(page);
|
||||
@@ -625,22 +973,50 @@ function buildWikiResultMetadata(
|
||||
...(page.sourcePath ? { sourcePath: page.sourcePath } : {}),
|
||||
...(provenanceLabel ? { provenanceLabel } : {}),
|
||||
...(page.updatedAt ? { updatedAt: page.updatedAt } : {}),
|
||||
...("entityType" in page && page.entityType ? { entityType: page.entityType } : {}),
|
||||
...("canonicalId" in page && page.canonicalId ? { canonicalId: page.canonicalId } : {}),
|
||||
...("aliases" in page && page.aliases.length > 0 ? { aliases: [...page.aliases] } : {}),
|
||||
...("privacyTier" in page && page.privacyTier ? { privacyTier: page.privacyTier } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function toWikiSearchResult(page: QueryableWikiPage, query: string): WikiSearchResult {
|
||||
function buildClaimResultMetadata(claim: WikiClaim | undefined): Partial<WikiSearchResult> {
|
||||
if (!claim) {
|
||||
return {};
|
||||
}
|
||||
return {
|
||||
...(claim.id ? { matchedClaimId: claim.id } : {}),
|
||||
...(claim.status ? { matchedClaimStatus: claim.status } : {}),
|
||||
...(typeof claim.confidence === "number" ? { matchedClaimConfidence: claim.confidence } : {}),
|
||||
evidenceKinds: [...new Set(claim.evidence.flatMap((evidence) => evidence.kind ?? []))],
|
||||
evidenceSourceIds: [...new Set(claim.evidence.flatMap((evidence) => evidence.sourceId ?? []))],
|
||||
};
|
||||
}
|
||||
|
||||
function toWikiSearchResult(
|
||||
page: QueryableWikiPage,
|
||||
query: string,
|
||||
mode: WikiSearchMode,
|
||||
): WikiSearchResult {
|
||||
const queryLower = normalizeLowercaseStringOrEmpty(query);
|
||||
const matchingClaim = getMatchingClaims(page, queryLower)[0];
|
||||
return {
|
||||
corpus: "wiki",
|
||||
path: page.relativePath,
|
||||
title: page.title,
|
||||
kind: page.kind,
|
||||
score: scorePage(page, query),
|
||||
score: scorePage(page, query, mode),
|
||||
snippet: buildPageSnippet(page, query),
|
||||
searchMode: mode,
|
||||
...buildWikiResultMetadata(page),
|
||||
...buildClaimResultMetadata(matchingClaim),
|
||||
};
|
||||
}
|
||||
|
||||
function toMemoryWikiSearchResult(result: MemorySearchResult): WikiSearchResult {
|
||||
function toMemoryWikiSearchResult(
|
||||
result: MemorySearchResult,
|
||||
mode: WikiSearchMode,
|
||||
): WikiSearchResult {
|
||||
return {
|
||||
corpus: "memory",
|
||||
path: result.path,
|
||||
@@ -651,6 +1027,7 @@ function toMemoryWikiSearchResult(result: MemorySearchResult): WikiSearchResult
|
||||
startLine: result.startLine,
|
||||
endLine: result.endLine,
|
||||
memorySource: result.source,
|
||||
searchMode: mode,
|
||||
...(result.citation ? { citation: result.citation } : {}),
|
||||
};
|
||||
}
|
||||
@@ -659,6 +1036,7 @@ async function searchWikiCorpus(params: {
|
||||
rootDir: string;
|
||||
query: string;
|
||||
maxResults: number;
|
||||
mode: WikiSearchMode;
|
||||
}): Promise<WikiSearchResult[]> {
|
||||
const digest = await readQueryDigestBundle(params.rootDir);
|
||||
const candidatePaths = digest
|
||||
@@ -666,6 +1044,7 @@ async function searchWikiCorpus(params: {
|
||||
digest,
|
||||
query: params.query,
|
||||
maxResults: params.maxResults,
|
||||
mode: params.mode,
|
||||
})
|
||||
: [];
|
||||
const seenPaths = new Set<string>();
|
||||
@@ -678,7 +1057,7 @@ async function searchWikiCorpus(params: {
|
||||
}
|
||||
|
||||
const results = candidatePages
|
||||
.map((page) => toWikiSearchResult(page, params.query))
|
||||
.map((page) => toWikiSearchResult(page, params.query, params.mode))
|
||||
.filter((page) => page.score > 0);
|
||||
if (candidatePaths.length === 0 || results.length >= params.maxResults) {
|
||||
return results;
|
||||
@@ -691,7 +1070,7 @@ async function searchWikiCorpus(params: {
|
||||
return [
|
||||
...results,
|
||||
...remainingPages
|
||||
.map((page) => toWikiSearchResult(page, params.query))
|
||||
.map((page) => toWikiSearchResult(page, params.query, params.mode))
|
||||
.filter((page) => page.score > 0),
|
||||
];
|
||||
}
|
||||
@@ -728,16 +1107,19 @@ export async function searchMemoryWiki(params: {
|
||||
maxResults?: number;
|
||||
searchBackend?: WikiSearchBackend;
|
||||
searchCorpus?: WikiSearchCorpus;
|
||||
mode?: WikiSearchMode;
|
||||
}): Promise<WikiSearchResult[]> {
|
||||
const effectiveConfig = applySearchOverrides(params.config, params);
|
||||
await initializeMemoryWikiVault(effectiveConfig);
|
||||
const maxResults = Math.max(1, params.maxResults ?? 10);
|
||||
const mode = params.mode ?? "auto";
|
||||
|
||||
const wikiResults = shouldSearchWiki(effectiveConfig)
|
||||
? await searchWikiCorpus({
|
||||
rootDir: effectiveConfig.vault.path,
|
||||
query: params.query,
|
||||
maxResults,
|
||||
mode,
|
||||
})
|
||||
: [];
|
||||
|
||||
@@ -750,7 +1132,7 @@ export async function searchMemoryWiki(params: {
|
||||
: null;
|
||||
const memoryResults = sharedMemoryManager
|
||||
? (await sharedMemoryManager.search(params.query, { maxResults })).map((result) =>
|
||||
toMemoryWikiSearchResult(result),
|
||||
toMemoryWikiSearchResult(result, mode),
|
||||
)
|
||||
: [];
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ import {
|
||||
type ResolvedMemoryWikiConfig,
|
||||
} from "./config.js";
|
||||
import { lintMemoryWikiVault } from "./lint.js";
|
||||
import { getMemoryWikiPage, searchMemoryWiki } from "./query.js";
|
||||
import { getMemoryWikiPage, searchMemoryWiki, WIKI_SEARCH_MODES } from "./query.js";
|
||||
import { syncMemoryWikiImportedSources } from "./source-sync.js";
|
||||
import { renderMemoryWikiStatus, resolveMemoryWikiStatus } from "./status.js";
|
||||
|
||||
@@ -17,12 +17,14 @@ const WikiSearchBackendSchema = Type.Union(
|
||||
WIKI_SEARCH_BACKENDS.map((value) => Type.Literal(value)),
|
||||
);
|
||||
const WikiSearchCorpusSchema = Type.Union(WIKI_SEARCH_CORPORA.map((value) => Type.Literal(value)));
|
||||
const WikiSearchModeSchema = Type.Union(WIKI_SEARCH_MODES.map((value) => Type.Literal(value)));
|
||||
const WikiSearchSchema = Type.Object(
|
||||
{
|
||||
query: Type.String({ minLength: 1 }),
|
||||
maxResults: Type.Optional(Type.Number({ minimum: 1 })),
|
||||
backend: Type.Optional(WikiSearchBackendSchema),
|
||||
corpus: Type.Optional(WikiSearchCorpusSchema),
|
||||
mode: Type.Optional(WikiSearchModeSchema),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
);
|
||||
@@ -126,6 +128,7 @@ export function createWikiSearchTool(
|
||||
maxResults?: number;
|
||||
backend?: ResolvedMemoryWikiConfig["search"]["backend"];
|
||||
corpus?: ResolvedMemoryWikiConfig["search"]["corpus"];
|
||||
mode?: (typeof WIKI_SEARCH_MODES)[number];
|
||||
};
|
||||
await syncImportedSourcesIfNeeded(config, appConfig);
|
||||
const results = await searchMemoryWiki({
|
||||
@@ -137,6 +140,7 @@ export function createWikiSearchTool(
|
||||
maxResults: params.maxResults,
|
||||
...(params.backend ? { searchBackend: params.backend } : {}),
|
||||
...(params.corpus ? { searchCorpus: params.corpus } : {}),
|
||||
...(params.mode ? { mode: params.mode } : {}),
|
||||
});
|
||||
const text =
|
||||
results.length === 0
|
||||
@@ -144,7 +148,7 @@ export function createWikiSearchTool(
|
||||
: results
|
||||
.map(
|
||||
(result, index) =>
|
||||
`${index + 1}. ${result.title} (${result.corpus}/${result.kind})\nPath: ${result.path}${typeof result.startLine === "number" && typeof result.endLine === "number" ? `\nLines: ${result.startLine}-${result.endLine}` : ""}${result.provenanceLabel ? `\nProvenance: ${result.provenanceLabel}` : ""}\nSnippet: ${result.snippet}`,
|
||||
`${index + 1}. ${result.title} (${result.corpus}/${result.kind})\nPath: ${result.path}${typeof result.startLine === "number" && typeof result.endLine === "number" ? `\nLines: ${result.startLine}-${result.endLine}` : ""}${result.provenanceLabel ? `\nProvenance: ${result.provenanceLabel}` : ""}${result.matchedClaimId ? `\nClaim: ${result.matchedClaimId}` : ""}${result.evidenceKinds && result.evidenceKinds.length > 0 ? `\nEvidence: ${result.evidenceKinds.join(", ")}` : ""}\nSnippet: ${result.snippet}`,
|
||||
)
|
||||
.join("\n\n");
|
||||
return {
|
||||
|
||||
Reference in New Issue
Block a user