mirror of https://github.com/openclaw/openclaw.git synced 2026-07-22 00:01:37 +00:00

Files

Peter Steinberger 658b601ee5 feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

* feat(llama-cpp): add in-process text inference

* test(llama-cpp): narrow setup provider fixture

* fix(llama-cpp): trim public surface and refresh docs map

* fix(llama-cpp): import Context type in inference test

2026-07-16 18:53:55 -07:00

src

feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

2026-07-16 18:53:55 -07:00

index.test.ts

feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

2026-07-16 18:53:55 -07:00

index.ts

feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

2026-07-16 18:53:55 -07:00

npm-shrinkwrap.json

chore(release): set version to 2026.7.2

2026-07-11 04:00:49 +01:00

openclaw.plugin.json

feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

2026-07-16 18:53:55 -07:00

package.json

feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

2026-07-16 18:53:55 -07:00

README.md

feat(llama-cpp): in-process local GGUF text inference provider (#109444 )

2026-07-16 18:53:55 -07:00

README.md

@openclaw/llama-cpp-provider

Official llama.cpp text-inference and embedding provider for OpenClaw.

This plugin runs local GGUF chat and embedding models in-process through node-llama-cpp.

Install

openclaw plugins install @openclaw/llama-cpp-provider

Restart the Gateway after installing or updating the plugin. Use Node 24 for native installs and updates.

Configure text inference

Choose Local model (llama.cpp) during onboarding. After explicit consent, OpenClaw downloads the approximately 2.5 GB Qwen3 4B Instruct 2507 Q4_K_M default. Discovery never downloads a model.

See the llama.cpp provider guide for custom GGUF model configuration and hardware guidance.

Configure embeddings

Set agents.defaults.memorySearch.provider to local. By default, the plugin downloads and uses the EmbeddingGemma GGUF model. Configure agents.defaults.memorySearch.local.modelPath to use another local path, Hugging Face model URI, or HTTPS model URL.

Package

Plugin id: llama-cpp
Package: @openclaw/llama-cpp-provider
Minimum OpenClaw host: 2026.6.2