mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-03 10:21:40 +00:00
* feat(memory): add provenance and recall metadata to the memory index * feat(memory): provenance-gated promotion and capture hygiene * feat(dreaming): LLM consolidation with deterministic gates, on by default * feat(active-memory): deterministic recall lane with escalation default * feat(memory): user model file and standing intents * docs(memory): document the memory architecture * fix(memory): live-QA fixes — metadata writers, provenance classes, intent scope, claim accumulation
166 lines
6.1 KiB
Markdown
166 lines
6.1 KiB
Markdown
---
|
|
summary: "The default SQLite-based memory backend with keyword, vector, and hybrid search"
|
|
title: "Builtin memory engine"
|
|
read_when:
|
|
- You want to understand the default memory backend
|
|
- You want to configure embedding providers or hybrid search
|
|
---
|
|
|
|
The builtin engine is the default memory backend. It stores your memory index
|
|
in a per-agent SQLite database and needs no extra dependencies to get
|
|
started.
|
|
|
|
## What it provides
|
|
|
|
- **Keyword search** via FTS5 full-text indexing (BM25 scoring).
|
|
- **Vector search** via embeddings from any supported provider.
|
|
- **Hybrid search** that combines both for best results.
|
|
- **Deterministic ranking** by relevance, recency, and write-time importance.
|
|
- **Trusted trigger recall** for bounded pre-reply context without a recall model.
|
|
- **CJK support** via trigram tokenization for Chinese, Japanese, and Korean.
|
|
- **sqlite-vec acceleration** for in-database vector queries (optional).
|
|
|
|
## Getting started
|
|
|
|
By default, the builtin engine uses OpenAI embeddings. If `OPENAI_API_KEY` or
|
|
`models.providers.openai.apiKey` is already configured, vector search works
|
|
with no extra memory config.
|
|
|
|
To set a provider explicitly:
|
|
|
|
```json5
|
|
{
|
|
memory: {
|
|
search: {
|
|
provider: "openai",
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
Without an embedding provider, only keyword search is available.
|
|
|
|
To force local GGUF embeddings, install the official llama.cpp provider
|
|
plugin, then point `local.modelPath` at a GGUF file:
|
|
|
|
```bash
|
|
openclaw plugins install @openclaw/llama-cpp-provider
|
|
```
|
|
|
|
```json5
|
|
{
|
|
memory: {
|
|
search: {
|
|
provider: "local",
|
|
fallback: "none",
|
|
local: {
|
|
modelPath: "~/.node-llama-cpp/models/embeddinggemma-300m-qat-Q8_0.gguf",
|
|
},
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
## Supported embedding providers
|
|
|
|
| Provider | ID | Notes |
|
|
| ----------------- | ------------------- | ----------------------------------- |
|
|
| Bedrock | `bedrock` | Uses the AWS credential chain |
|
|
| DeepInfra | `deepinfra` | Default: `BAAI/bge-m3` |
|
|
| Gemini | `gemini` | Supports multimodal (image + audio) |
|
|
| GitHub Copilot | `github-copilot` | Uses your Copilot subscription |
|
|
| LM Studio | `lmstudio` | Local/self-hosted |
|
|
| Local | `local` | `@openclaw/llama-cpp-provider` |
|
|
| Mistral | `mistral` | |
|
|
| Ollama | `ollama` | Local/self-hosted |
|
|
| OpenAI | `openai` | Default: `text-embedding-3-small` |
|
|
| OpenAI-compatible | `openai-compatible` | Generic `/v1/embeddings` endpoint |
|
|
| Voyage | `voyage` | |
|
|
|
|
Set `memory.search.provider` to switch away from OpenAI.
|
|
|
|
## How indexing works
|
|
|
|
OpenClaw indexes `MEMORY.md`, an existing root `USER.md`, and `memory/*.md` into
|
|
chunks (400 tokens with 80-token overlap by default) and stores them in a
|
|
per-agent SQLite database. OpenClaw does not create `USER.md` automatically.
|
|
|
|
Each chunk can carry nullable importance and trigger metadata. Null values are
|
|
neutral, so older indexes remain usable. Search combines hybrid relevance,
|
|
recency decay, and importance; trigger recall only injects curated or
|
|
promoted-trusted entries.
|
|
|
|
Each indexed chunk also has SQLite-owned provenance: origin class (`owner`,
|
|
`agent`, `untrusted`, or `system`), session kind, observation time, and an
|
|
optional supersession key. This metadata is stored separately from Markdown
|
|
so recalled prose cannot rewrite its own trust classification.
|
|
|
|
- **Index location:** the owning agent database at
|
|
`~/.openclaw/agents/<agentId>/agent/openclaw-agent.sqlite`
|
|
- **Storage maintenance:** SQLite WAL sidecars are bounded with periodic and
|
|
shutdown checkpoints.
|
|
- **File watching:** changes to memory files trigger a debounced reindex
|
|
(1.5s default).
|
|
- **Auto-reindex:** the index rebuilds automatically when the embedding
|
|
provider, model, chunking config, configured sources, or scope change.
|
|
- **Reindex on demand:** `openclaw memory index --force`
|
|
|
|
<Info>
|
|
You can also index Markdown files outside the workspace with
|
|
`memory.search.extraPaths`. See the
|
|
[configuration reference](/reference/memory-config#additional-memory-paths).
|
|
</Info>
|
|
|
|
## When to use
|
|
|
|
The builtin engine is the right choice for most users:
|
|
|
|
- Works out of the box with no extra dependencies.
|
|
- Handles keyword and vector search well.
|
|
- Supports all embedding providers.
|
|
- Hybrid search combines the best of both retrieval approaches.
|
|
|
|
Consider switching to [QMD](/concepts/memory-qmd) if you need reranking, query
|
|
expansion, or want to index directories outside the workspace.
|
|
|
|
Consider [Honcho](/concepts/memory-honcho) if you want cross-session memory
|
|
with automatic user modeling.
|
|
|
|
## Troubleshooting
|
|
|
|
**Memory search disabled?** Check `openclaw memory status`. If no provider is
|
|
detected, set one explicitly or add an API key.
|
|
|
|
**Local provider not detected?** Confirm the local path exists and run:
|
|
|
|
```bash
|
|
openclaw memory status --deep --agent main
|
|
openclaw memory index --force --agent main
|
|
```
|
|
|
|
Both standalone CLI commands and the Gateway use the same `local` provider id.
|
|
Set `memory.search.provider: "local"` when you want local embeddings.
|
|
|
|
**Stale results?** Run `openclaw memory index --force` to rebuild. The watcher
|
|
may miss changes in rare edge cases.
|
|
|
|
**sqlite-vec not loading?** OpenClaw falls back to in-process cosine
|
|
similarity automatically. `openclaw memory status --deep` reports the local
|
|
vector store separately from the embedding provider, so `Vector store:
|
|
unavailable` points at sqlite-vec loading while `Embeddings: unavailable`
|
|
points at provider/auth or model readiness. Check logs for the specific load
|
|
error.
|
|
|
|
## Configuration
|
|
|
|
For embedding provider setup, hybrid search tuning (weights, MMR, temporal
|
|
decay), batch indexing, multimodal memory, sqlite-vec, extra paths, and all
|
|
other config knobs, see the
|
|
[Memory configuration reference](/reference/memory-config).
|
|
|
|
## Related
|
|
|
|
- [Memory overview](/concepts/memory)
|
|
- [Memory search](/concepts/memory-search)
|
|
- [Active memory](/concepts/active-memory)
|