Files
openclaw/docs/concepts/memory-builtin.md
Peter Steinberger 28630a9a65 feat(memory): provenance-gated memory with dreaming on by default (#114819)
* feat(memory): add provenance and recall metadata to the memory index

* feat(memory): provenance-gated promotion and capture hygiene

* feat(dreaming): LLM consolidation with deterministic gates, on by default

* feat(active-memory): deterministic recall lane with escalation default

* feat(memory): user model file and standing intents

* docs(memory): document the memory architecture

* fix(memory): live-QA fixes — metadata writers, provenance classes, intent scope, claim accumulation
2026-07-28 06:04:25 -04:00

6.1 KiB

summary, title, read_when
summary title read_when
The default SQLite-based memory backend with keyword, vector, and hybrid search Builtin memory engine
You want to understand the default memory backend
You want to configure embedding providers or hybrid search

The builtin engine is the default memory backend. It stores your memory index in a per-agent SQLite database and needs no extra dependencies to get started.

What it provides

  • Keyword search via FTS5 full-text indexing (BM25 scoring).
  • Vector search via embeddings from any supported provider.
  • Hybrid search that combines both for best results.
  • Deterministic ranking by relevance, recency, and write-time importance.
  • Trusted trigger recall for bounded pre-reply context without a recall model.
  • CJK support via trigram tokenization for Chinese, Japanese, and Korean.
  • sqlite-vec acceleration for in-database vector queries (optional).

Getting started

By default, the builtin engine uses OpenAI embeddings. If OPENAI_API_KEY or models.providers.openai.apiKey is already configured, vector search works with no extra memory config.

To set a provider explicitly:

{
  memory: {
    search: {
      provider: "openai",
    },
  },
}

Without an embedding provider, only keyword search is available.

To force local GGUF embeddings, install the official llama.cpp provider plugin, then point local.modelPath at a GGUF file:

openclaw plugins install @openclaw/llama-cpp-provider
{
  memory: {
    search: {
      provider: "local",
      fallback: "none",
      local: {
        modelPath: "~/.node-llama-cpp/models/embeddinggemma-300m-qat-Q8_0.gguf",
      },
    },
  },
}

Supported embedding providers

Provider ID Notes
Bedrock bedrock Uses the AWS credential chain
DeepInfra deepinfra Default: BAAI/bge-m3
Gemini gemini Supports multimodal (image + audio)
GitHub Copilot github-copilot Uses your Copilot subscription
LM Studio lmstudio Local/self-hosted
Local local @openclaw/llama-cpp-provider
Mistral mistral
Ollama ollama Local/self-hosted
OpenAI openai Default: text-embedding-3-small
OpenAI-compatible openai-compatible Generic /v1/embeddings endpoint
Voyage voyage

Set memory.search.provider to switch away from OpenAI.

How indexing works

OpenClaw indexes MEMORY.md, an existing root USER.md, and memory/*.md into chunks (400 tokens with 80-token overlap by default) and stores them in a per-agent SQLite database. OpenClaw does not create USER.md automatically.

Each chunk can carry nullable importance and trigger metadata. Null values are neutral, so older indexes remain usable. Search combines hybrid relevance, recency decay, and importance; trigger recall only injects curated or promoted-trusted entries.

Each indexed chunk also has SQLite-owned provenance: origin class (owner, agent, untrusted, or system), session kind, observation time, and an optional supersession key. This metadata is stored separately from Markdown so recalled prose cannot rewrite its own trust classification.

  • Index location: the owning agent database at ~/.openclaw/agents/<agentId>/agent/openclaw-agent.sqlite
  • Storage maintenance: SQLite WAL sidecars are bounded with periodic and shutdown checkpoints.
  • File watching: changes to memory files trigger a debounced reindex (1.5s default).
  • Auto-reindex: the index rebuilds automatically when the embedding provider, model, chunking config, configured sources, or scope change.
  • Reindex on demand: openclaw memory index --force
You can also index Markdown files outside the workspace with `memory.search.extraPaths`. See the [configuration reference](/reference/memory-config#additional-memory-paths).

When to use

The builtin engine is the right choice for most users:

  • Works out of the box with no extra dependencies.
  • Handles keyword and vector search well.
  • Supports all embedding providers.
  • Hybrid search combines the best of both retrieval approaches.

Consider switching to QMD if you need reranking, query expansion, or want to index directories outside the workspace.

Consider Honcho if you want cross-session memory with automatic user modeling.

Troubleshooting

Memory search disabled? Check openclaw memory status. If no provider is detected, set one explicitly or add an API key.

Local provider not detected? Confirm the local path exists and run:

openclaw memory status --deep --agent main
openclaw memory index --force --agent main

Both standalone CLI commands and the Gateway use the same local provider id. Set memory.search.provider: "local" when you want local embeddings.

Stale results? Run openclaw memory index --force to rebuild. The watcher may miss changes in rare edge cases.

sqlite-vec not loading? OpenClaw falls back to in-process cosine similarity automatically. openclaw memory status --deep reports the local vector store separately from the embedding provider, so Vector store: unavailable points at sqlite-vec loading while Embeddings: unavailable points at provider/auth or model readiness. Check logs for the specific load error.

Configuration

For embedding provider setup, hybrid search tuning (weights, MMR, temporal decay), batch indexing, multimodal memory, sqlite-vec, extra paths, and all other config knobs, see the Memory configuration reference.