mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-03 10:50:20 +00:00
feat(memory): add Bedrock embedding provider for memory search (#61547)
* feat(memory): add Bedrock embedding provider for memory search Add Amazon Bedrock as a native embedding provider for memory search. Supports Titan Embed Text v1/v2 and Cohere Embed models via AWS SDK. - New embeddings-bedrock.ts: BedrockRuntimeClient + InvokeModel - Auth via AWS default credential chain (same as Bedrock inference) - Auto-selected in 'auto' mode when AWS credentials are detected - Titan V2: configurable dimensions (256/512/1024), normalization - Cohere: native batch support with search_query/search_document types - 16 new tests covering all model types, auth detection, edge cases Closes #26289 * fix(memory): harden bedrock embedding selection --------- Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
This commit is contained in:
wirjo
@@ -1,4 +1,4 @@
|
||||
57a3b1cc7d573c3788a670d927eac947fb1685384804f5c3c926f702a27fe00b config-baseline.json
|
||||
82163136ff466db3caa61290fd65a8b8dd9487fc61f3871c177f96fcecf9e29b config-baseline.core.json
|
||||
ae67508350baf891b902348d55fada6c17e9c053adf53aaf3a8b92cd364ef3f1 config-baseline.channel.json
|
||||
d972a11d0f86080a722bddfe48990dd1b8fa16eb8e157e83f49bd46a5941c512 config-baseline.plugin.json
|
||||
73fbcd00d17685b462dfb11aff74baae99265ae5671db28893d8608456daa44e config-baseline.json
|
||||
effaf240920c16fce2c78af52dec15aa9ceb049e34f703c568669cb6beef3f91 config-baseline.core.json
|
||||
3c999707b167138de34f6255e3488b99e404c5132d3fc5879a1fa12d815c31f5 config-baseline.channel.json
|
||||
031b237717ca108ea2cd314413db4c91edfdfea55f808179e3066331f41af134 config-baseline.plugin.json
|
||||
|
||||
@@ -35,14 +35,15 @@ node-llama-cpp).
|
||||
|
||||
## Supported providers
|
||||
|
||||
| Provider | ID | Needs API key | Notes |
|
||||
| -------- | --------- | ------------- | ----------------------------- |
|
||||
| OpenAI | `openai` | Yes | Auto-detected, fast |
|
||||
| Gemini | `gemini` | Yes | Supports image/audio indexing |
|
||||
| Voyage | `voyage` | Yes | Auto-detected |
|
||||
| Mistral | `mistral` | Yes | Auto-detected |
|
||||
| Ollama | `ollama` | No | Local, must set explicitly |
|
||||
| Local | `local` | No | GGUF model, ~0.6 GB download |
|
||||
| Provider | ID | Needs API key | Notes |
|
||||
| -------- | --------- | ------------- | ---------------------------------------------------- |
|
||||
| OpenAI | `openai` | Yes | Auto-detected, fast |
|
||||
| Gemini | `gemini` | Yes | Supports image/audio indexing |
|
||||
| Voyage | `voyage` | Yes | Auto-detected |
|
||||
| Mistral | `mistral` | Yes | Auto-detected |
|
||||
| Bedrock | `bedrock` | No | Auto-detected when the AWS credential chain resolves |
|
||||
| Ollama | `ollama` | No | Local, must set explicitly |
|
||||
| Local | `local` | No | GGUF model, ~0.6 GB download |
|
||||
|
||||
## How search works
|
||||
|
||||
|
||||
@@ -271,3 +271,32 @@ grounding checks.
|
||||
|
||||
The IAM principal used by the gateway must have the `bedrock:ApplyGuardrail`
|
||||
permission in addition to the standard invoke permissions.
|
||||
|
||||
## Embeddings for memory search
|
||||
|
||||
Bedrock can also serve as the embedding provider for
|
||||
[memory search](/concepts/memory-search). This is configured separately from the
|
||||
inference provider — set `agents.defaults.memorySearch.provider` to `"bedrock"`:
|
||||
|
||||
```json5
|
||||
{
|
||||
agents: {
|
||||
defaults: {
|
||||
memorySearch: {
|
||||
provider: "bedrock",
|
||||
model: "amazon.titan-embed-text-v2:0", // default
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Bedrock embeddings use the same AWS SDK credential chain as inference (instance
|
||||
roles, SSO, access keys, shared config, and web identity). No API key is
|
||||
needed. When `provider` is `"auto"`, Bedrock is auto-detected if that
|
||||
credential chain resolves successfully.
|
||||
|
||||
Supported embedding models include Amazon Titan Embed (v1, v2), Amazon Nova
|
||||
Embed, Cohere Embed (v3, v4), and TwelveLabs Marengo. See
|
||||
[Memory configuration reference — Bedrock](/reference/memory-config#bedrock-embedding-config)
|
||||
for the full model list and dimension options.
|
||||
|
||||
@@ -25,12 +25,12 @@ All memory search settings live under `agents.defaults.memorySearch` in
|
||||
|
||||
## Provider selection
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
| ---------- | --------- | ---------------- | -------------------------------------------------------------------------------- |
|
||||
| `provider` | `string` | auto-detected | Embedding adapter ID: `openai`, `gemini`, `voyage`, `mistral`, `ollama`, `local` |
|
||||
| `model` | `string` | provider default | Embedding model name |
|
||||
| `fallback` | `string` | `"none"` | Fallback adapter ID when the primary fails |
|
||||
| `enabled` | `boolean` | `true` | Enable or disable memory search |
|
||||
| Key | Type | Default | Description |
|
||||
| ---------- | --------- | ---------------- | ------------------------------------------------------------------------------------------- |
|
||||
| `provider` | `string` | auto-detected | Embedding adapter ID: `openai`, `gemini`, `voyage`, `mistral`, `bedrock`, `ollama`, `local` |
|
||||
| `model` | `string` | provider default | Embedding model name |
|
||||
| `fallback` | `string` | `"none"` | Fallback adapter ID when the primary fails |
|
||||
| `enabled` | `boolean` | `true` | Enable or disable memory search |
|
||||
|
||||
### Auto-detection order
|
||||
|
||||
@@ -41,13 +41,14 @@ When `provider` is not set, OpenClaw selects the first available:
|
||||
3. `gemini` -- if a Gemini key can be resolved.
|
||||
4. `voyage` -- if a Voyage key can be resolved.
|
||||
5. `mistral` -- if a Mistral key can be resolved.
|
||||
6. `bedrock` -- if the AWS SDK credential chain resolves (instance role, access keys, profile, SSO, web identity, or shared config).
|
||||
|
||||
`ollama` is supported but not auto-detected (set it explicitly).
|
||||
|
||||
### API key resolution
|
||||
|
||||
Remote embeddings require an API key. OpenClaw resolves from:
|
||||
auth profiles, `models.providers.*.apiKey`, or environment variables.
|
||||
Remote embeddings require an API key. Bedrock uses the AWS SDK default
|
||||
credential chain instead (instance roles, SSO, access keys).
|
||||
|
||||
| Provider | Env var | Config key |
|
||||
| -------- | ------------------------------ | --------------------------------- |
|
||||
@@ -55,6 +56,7 @@ auth profiles, `models.providers.*.apiKey`, or environment variables.
|
||||
| Gemini | `GEMINI_API_KEY` | `models.providers.google.apiKey` |
|
||||
| Voyage | `VOYAGE_API_KEY` | `models.providers.voyage.apiKey` |
|
||||
| Mistral | `MISTRAL_API_KEY` | `models.providers.mistral.apiKey` |
|
||||
| Bedrock | AWS credential chain | No API key needed |
|
||||
| Ollama | `OLLAMA_API_KEY` (placeholder) | -- |
|
||||
|
||||
Codex OAuth covers chat/completions only and does not satisfy embedding
|
||||
@@ -104,6 +106,84 @@ Changing model or `outputDimensionality` triggers an automatic full reindex.
|
||||
|
||||
---
|
||||
|
||||
## Bedrock embedding config
|
||||
|
||||
Bedrock uses the AWS SDK default credential chain -- no API keys needed.
|
||||
If OpenClaw runs on EC2 with a Bedrock-enabled instance role, just set the
|
||||
provider and model:
|
||||
|
||||
```json5
|
||||
{
|
||||
agents: {
|
||||
defaults: {
|
||||
memorySearch: {
|
||||
provider: "bedrock",
|
||||
model: "amazon.titan-embed-text-v2:0",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
| ---------------------- | -------- | ------------------------------ | ------------------------------- |
|
||||
| `model` | `string` | `amazon.titan-embed-text-v2:0` | Any Bedrock embedding model ID |
|
||||
| `outputDimensionality` | `number` | model default | For Titan V2: 256, 512, or 1024 |
|
||||
|
||||
### Supported models
|
||||
|
||||
The following models are supported (with family detection and dimension
|
||||
defaults):
|
||||
|
||||
| Model ID | Provider | Default Dims | Configurable Dims |
|
||||
| ------------------------------------------ | ---------- | ------------ | -------------------- |
|
||||
| `amazon.titan-embed-text-v2:0` | Amazon | 1024 | 256, 512, 1024 |
|
||||
| `amazon.titan-embed-text-v1` | Amazon | 1536 | -- |
|
||||
| `amazon.titan-embed-g1-text-02` | Amazon | 1536 | -- |
|
||||
| `amazon.titan-embed-image-v1` | Amazon | 1024 | -- |
|
||||
| `amazon.nova-2-multimodal-embeddings-v1:0` | Amazon | 1024 | 256, 384, 1024, 3072 |
|
||||
| `cohere.embed-english-v3` | Cohere | 1024 | -- |
|
||||
| `cohere.embed-multilingual-v3` | Cohere | 1024 | -- |
|
||||
| `cohere.embed-v4:0` | Cohere | 1536 | 256-1536 |
|
||||
| `twelvelabs.marengo-embed-3-0-v1:0` | TwelveLabs | 512 | -- |
|
||||
| `twelvelabs.marengo-embed-2-7-v1:0` | TwelveLabs | 1024 | -- |
|
||||
|
||||
Throughput-suffixed variants (e.g., `amazon.titan-embed-text-v1:2:8k`) inherit
|
||||
the base model's configuration.
|
||||
|
||||
### Authentication
|
||||
|
||||
Bedrock auth uses the standard AWS SDK credential resolution order:
|
||||
|
||||
1. Environment variables (`AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`)
|
||||
2. SSO token cache
|
||||
3. Web identity token credentials
|
||||
4. Shared credentials and config files
|
||||
5. ECS or EC2 metadata credentials
|
||||
|
||||
Region is resolved from `AWS_REGION`, `AWS_DEFAULT_REGION`, the
|
||||
`amazon-bedrock` provider `baseUrl`, or defaults to `us-east-1`.
|
||||
|
||||
### IAM permissions
|
||||
|
||||
The IAM role or user needs:
|
||||
|
||||
```json
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": "bedrock:InvokeModel",
|
||||
"Resource": "*"
|
||||
}
|
||||
```
|
||||
|
||||
For least-privilege, scope `InvokeModel` to the specific model:
|
||||
|
||||
```
|
||||
arn:aws:bedrock:*::foundation-model/amazon.titan-embed-text-v2:0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local embedding config
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|
||||
Reference in New Issue
Block a user