mirror of
https://github.com/openclaw/openclaw.git
synced 2026-03-14 03:20:49 +00:00
* feat(secrets): expand secret target coverage and gateway tooling * docs(secrets): align gateway and CLI secret docs * chore(protocol): regenerate swift gateway models for secrets methods * fix(config): restore talk apiKey fallback and stabilize runner test * ci(windows): reduce test worker count for shard stability * ci(windows): raise node heap for test shard stability * test(feishu): make proxy env precedence assertion windows-safe * fix(gateway): resolve auth password SecretInput refs for clients * fix(gateway): resolve remote SecretInput credentials for clients * fix(secrets): skip inactive refs in command snapshot assignments * fix(secrets): scope gateway.remote refs to effective auth surfaces * fix(secrets): ignore memory defaults when enabled agents disable search * fix(secrets): honor Google Chat serviceAccountRef inheritance * fix(secrets): address tsgo errors in command and gateway collectors * fix(secrets): avoid auth-store load in providers-only configure * fix(gateway): defer local password ref resolution by precedence * fix(secrets): gate telegram webhook secret refs by webhook mode * fix(secrets): gate slack signing secret refs to http mode * fix(secrets): skip telegram botToken refs when tokenFile is set * fix(secrets): gate discord pluralkit refs by enabled flag * fix(secrets): gate discord voice tts refs by voice enabled * test(secrets): make runtime fixture modes explicit * fix(cli): resolve local qr password secret refs * fix(cli): fail when gateway leaves command refs unresolved * fix(gateway): fail when local password SecretRef is unresolved * fix(gateway): fail when required remote SecretRefs are unresolved * fix(gateway): resolve local password refs only when password can win * fix(cli): skip local password SecretRef resolution on qr token override * test(gateway): cast SecretRef fixtures to OpenClawConfig * test(secrets): activate mode-gated targets in runtime coverage fixture * fix(cron): support SecretInput webhook tokens safely * fix(bluebubbles): support SecretInput passwords across config paths * fix(msteams): make appPassword SecretInput-safe in onboarding/token paths * fix(bluebubbles): align SecretInput schema helper typing * fix(cli): clarify secrets.resolve version-skew errors * refactor(secrets): return structured inactive paths from secrets.resolve * refactor(gateway): type onboarding secret writes as SecretInput * chore(protocol): regenerate swift models for secrets.resolve * feat(secrets): expand extension credential secretref support * fix(secrets): gate web-search refs by active provider * fix(onboarding): detect SecretRef credentials in extension status * fix(onboarding): allow keeping existing ref in secret prompt * fix(onboarding): resolve gateway password SecretRefs for probe and tui * fix(onboarding): honor secret-input-mode for local gateway auth * fix(acp): resolve gateway SecretInput credentials * fix(secrets): gate gateway.remote refs to remote surfaces * test(secrets): cover pattern matching and inactive array refs * docs(secrets): clarify secrets.resolve and remote active surfaces * fix(bluebubbles): keep existing SecretRef during onboarding * fix(tests): resolve CI type errors in new SecretRef coverage * fix(extensions): replace raw fetch with SSRF-guarded fetch * test(secrets): mark gateway remote targets active in runtime coverage * test(infra): normalize home-prefix expectation across platforms * fix(cli): only resolve local qr password refs in password mode * test(cli): cover local qr token mode with unresolved password ref * docs(cli): clarify local qr password ref resolution behavior * refactor(extensions): reuse sdk SecretInput helpers * fix(wizard): resolve onboarding env-template secrets before plaintext * fix(cli): surface secrets.resolve diagnostics in memory and qr * test(secrets): repair post-rebase runtime and fixtures * fix(gateway): skip remote password ref resolution when token wins * fix(secrets): treat tailscale remote gateway refs as active * fix(gateway): allow remote password fallback when token ref is unresolved * fix(gateway): ignore stale local password refs for none and trusted-proxy * fix(gateway): skip remote secret ref resolution on local call paths * test(cli): cover qr remote tailscale secret ref resolution * fix(secrets): align gateway password active-surface with auth inference * fix(cli): resolve inferred local gateway password refs in qr * fix(gateway): prefer resolvable remote password over token ref pre-resolution * test(gateway): cover none and trusted-proxy stale password refs * docs(secrets): sync qr and gateway active-surface behavior * fix: restore stability blockers from pre-release audit * Secrets: fix collector/runtime precedence contradictions * docs: align secrets and web credential docs * fix(rebase): resolve integration regressions after main rebase * fix(node-host): resolve gateway secret refs for auth * fix(secrets): harden secretinput runtime readers * gateway: skip inactive auth secretref resolution * cli: avoid gateway preflight for inactive secret refs * extensions: allow unresolved refs in onboarding status * tests: fix qr-cli module mock hoist ordering * Security: align audit checks with SecretInput resolution * Gateway: resolve local-mode remote fallback secret refs * Node host: avoid resolving inactive password secret refs * Secrets runtime: mark Slack appToken inactive for HTTP mode * secrets: keep inactive gateway remote refs non-blocking * cli: include agent memory secret targets in runtime resolution * docs(secrets): sync docs with active-surface and web search behavior * fix(secrets): keep telegram top-level token refs active for blank account tokens * fix(daemon): resolve gateway password secret refs for probe auth * fix(secrets): skip IRC NickServ ref resolution when NickServ is disabled * fix(secrets): align token inheritance and exec timeout defaults * docs(secrets): clarify active-surface notes in cli docs * cli: require secrets.resolve gateway capability * gateway: log auth secret surface diagnostics * secrets: remove dead provider resolver module * fix(secrets): restore gateway auth precedence and fallback resolution * fix(tests): align plugin runtime mock typings --------- Co-authored-by: Peter Steinberger <steipete@gmail.com>
331 lines
12 KiB
Markdown
331 lines
12 KiB
Markdown
---
|
||
summary: "Web search + fetch tools (Brave, Perplexity, Gemini, Grok, and Kimi providers)"
|
||
read_when:
|
||
- You want to enable web_search or web_fetch
|
||
- You need Brave Search API key setup
|
||
- You want to use Perplexity Sonar for web search
|
||
- You want to use Gemini with Google Search grounding
|
||
title: "Web Tools"
|
||
---
|
||
|
||
# Web tools
|
||
|
||
OpenClaw ships two lightweight web tools:
|
||
|
||
- `web_search` — Search the web via Brave Search API (default), Perplexity Sonar, Gemini with Google Search grounding, Grok, or Kimi.
|
||
- `web_fetch` — HTTP fetch + readable extraction (HTML → markdown/text).
|
||
|
||
These are **not** browser automation. For JS-heavy sites or logins, use the
|
||
[Browser tool](/tools/browser).
|
||
|
||
## How it works
|
||
|
||
- `web_search` calls your configured provider and returns results.
|
||
- **Brave** (default): returns structured results (title, URL, snippet).
|
||
- **Perplexity**: returns AI-synthesized answers with citations from real-time web search.
|
||
- **Gemini**: returns AI-synthesized answers grounded in Google Search with citations.
|
||
- Results are cached by query for 15 minutes (configurable).
|
||
- `web_fetch` does a plain HTTP GET and extracts readable content
|
||
(HTML → markdown/text). It does **not** execute JavaScript.
|
||
- `web_fetch` is enabled by default (unless explicitly disabled).
|
||
|
||
## Choosing a search provider
|
||
|
||
| Provider | Pros | Cons | API Key |
|
||
| ------------------- | -------------------------------------------- | ---------------------------------------- | -------------------------------------------- |
|
||
| **Brave** (default) | Fast, structured results, free tier | Traditional search results | `BRAVE_API_KEY` |
|
||
| **Perplexity** | AI-synthesized answers, citations, real-time | Requires Perplexity or OpenRouter access | `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` |
|
||
| **Gemini** | Google Search grounding, AI-synthesized | Requires Gemini API key | `GEMINI_API_KEY` |
|
||
| **Grok** | xAI web-grounded responses | Requires xAI API key | `XAI_API_KEY` |
|
||
| **Kimi** | Moonshot web search capability | Requires Moonshot API key | `KIMI_API_KEY` / `MOONSHOT_API_KEY` |
|
||
|
||
See [Brave Search setup](/brave-search) and [Perplexity Sonar](/perplexity) for provider-specific details.
|
||
|
||
### Auto-detection
|
||
|
||
If no `provider` is explicitly set, OpenClaw auto-detects which provider to use based on available API keys, checking in this order:
|
||
|
||
1. **Brave** — `BRAVE_API_KEY` env var or `tools.web.search.apiKey` config
|
||
2. **Gemini** — `GEMINI_API_KEY` env var or `tools.web.search.gemini.apiKey` config
|
||
3. **Kimi** — `KIMI_API_KEY` / `MOONSHOT_API_KEY` env var or `tools.web.search.kimi.apiKey` config
|
||
4. **Perplexity** — `PERPLEXITY_API_KEY` / `OPENROUTER_API_KEY` env var or `tools.web.search.perplexity.apiKey` config
|
||
5. **Grok** — `XAI_API_KEY` env var or `tools.web.search.grok.apiKey` config
|
||
|
||
If no keys are found, it falls back to Brave (you'll get a missing-key error prompting you to configure one).
|
||
|
||
### Explicit provider
|
||
|
||
Set the provider in config:
|
||
|
||
```json5
|
||
{
|
||
tools: {
|
||
web: {
|
||
search: {
|
||
provider: "brave", // or "perplexity" or "gemini" or "grok" or "kimi"
|
||
},
|
||
},
|
||
},
|
||
}
|
||
```
|
||
|
||
Example: switch to Perplexity Sonar (direct API):
|
||
|
||
```json5
|
||
{
|
||
tools: {
|
||
web: {
|
||
search: {
|
||
provider: "perplexity",
|
||
perplexity: {
|
||
apiKey: "pplx-...",
|
||
baseUrl: "https://api.perplexity.ai",
|
||
model: "perplexity/sonar-pro",
|
||
},
|
||
},
|
||
},
|
||
},
|
||
}
|
||
```
|
||
|
||
## Getting a Brave API key
|
||
|
||
1. Create a Brave Search API account at [https://brave.com/search/api/](https://brave.com/search/api/)
|
||
2. In the dashboard, choose the **Data for Search** plan (not “Data for AI”) and generate an API key.
|
||
3. Run `openclaw configure --section web` to store the key in config (recommended), or set `BRAVE_API_KEY` in your environment.
|
||
|
||
Brave provides a free tier plus paid plans; check the Brave API portal for the
|
||
current limits and pricing.
|
||
|
||
### Where to set the key (recommended)
|
||
|
||
**Recommended:** run `openclaw configure --section web`. It stores the key in
|
||
`~/.openclaw/openclaw.json` under `tools.web.search.apiKey`.
|
||
|
||
**Environment alternative:** set `BRAVE_API_KEY` in the Gateway process
|
||
environment. For a gateway install, put it in `~/.openclaw/.env` (or your
|
||
service environment). See [Env vars](/help/faq#how-does-openclaw-load-environment-variables).
|
||
|
||
## Using Perplexity (direct or via OpenRouter)
|
||
|
||
Perplexity Sonar models have built-in web search capabilities and return AI-synthesized
|
||
answers with citations. You can use them via OpenRouter (no credit card required - supports
|
||
crypto/prepaid).
|
||
|
||
### Getting an OpenRouter API key
|
||
|
||
1. Create an account at [https://openrouter.ai/](https://openrouter.ai/)
|
||
2. Add credits (supports crypto, prepaid, or credit card)
|
||
3. Generate an API key in your account settings
|
||
|
||
### Setting up Perplexity search
|
||
|
||
```json5
|
||
{
|
||
tools: {
|
||
web: {
|
||
search: {
|
||
enabled: true,
|
||
provider: "perplexity",
|
||
perplexity: {
|
||
// API key (optional if OPENROUTER_API_KEY or PERPLEXITY_API_KEY is set)
|
||
apiKey: "sk-or-v1-...",
|
||
// Base URL (key-aware default if omitted)
|
||
baseUrl: "https://openrouter.ai/api/v1",
|
||
// Model (defaults to perplexity/sonar-pro)
|
||
model: "perplexity/sonar-pro",
|
||
},
|
||
},
|
||
},
|
||
},
|
||
}
|
||
```
|
||
|
||
**Environment alternative:** set `OPENROUTER_API_KEY` or `PERPLEXITY_API_KEY` in the Gateway
|
||
environment. For a gateway install, put it in `~/.openclaw/.env`.
|
||
|
||
If no base URL is set, OpenClaw chooses a default based on the API key source:
|
||
|
||
- `PERPLEXITY_API_KEY` or `pplx-...` → `https://api.perplexity.ai`
|
||
- `OPENROUTER_API_KEY` or `sk-or-...` → `https://openrouter.ai/api/v1`
|
||
- Unknown key formats → OpenRouter (safe fallback)
|
||
|
||
### Available Perplexity models
|
||
|
||
| Model | Description | Best for |
|
||
| -------------------------------- | ------------------------------------ | ----------------- |
|
||
| `perplexity/sonar` | Fast Q&A with web search | Quick lookups |
|
||
| `perplexity/sonar-pro` (default) | Multi-step reasoning with web search | Complex questions |
|
||
| `perplexity/sonar-reasoning-pro` | Chain-of-thought analysis | Deep research |
|
||
|
||
## Using Gemini (Google Search grounding)
|
||
|
||
Gemini models support built-in [Google Search grounding](https://ai.google.dev/gemini-api/docs/grounding),
|
||
which returns AI-synthesized answers backed by live Google Search results with citations.
|
||
|
||
### Getting a Gemini API key
|
||
|
||
1. Go to [Google AI Studio](https://aistudio.google.com/apikey)
|
||
2. Create an API key
|
||
3. Set `GEMINI_API_KEY` in the Gateway environment, or configure `tools.web.search.gemini.apiKey`
|
||
|
||
### Setting up Gemini search
|
||
|
||
```json5
|
||
{
|
||
tools: {
|
||
web: {
|
||
search: {
|
||
provider: "gemini",
|
||
gemini: {
|
||
// API key (optional if GEMINI_API_KEY is set)
|
||
apiKey: "AIza...",
|
||
// Model (defaults to "gemini-2.5-flash")
|
||
model: "gemini-2.5-flash",
|
||
},
|
||
},
|
||
},
|
||
},
|
||
}
|
||
```
|
||
|
||
**Environment alternative:** set `GEMINI_API_KEY` in the Gateway environment.
|
||
For a gateway install, put it in `~/.openclaw/.env`.
|
||
|
||
### Notes
|
||
|
||
- Citation URLs from Gemini grounding are automatically resolved from Google's
|
||
redirect URLs to direct URLs.
|
||
- Redirect resolution uses the SSRF guard path (HEAD + redirect checks + http/https validation) before returning the final citation URL.
|
||
- Redirect resolution uses strict SSRF defaults, so redirects to private/internal targets are blocked.
|
||
- The default model (`gemini-2.5-flash`) is fast and cost-effective.
|
||
Any Gemini model that supports grounding can be used.
|
||
|
||
## web_search
|
||
|
||
Search the web using your configured provider.
|
||
|
||
### Requirements
|
||
|
||
- `tools.web.search.enabled` must not be `false` (default: enabled)
|
||
- API key for your chosen provider:
|
||
- **Brave**: `BRAVE_API_KEY` or `tools.web.search.apiKey`
|
||
- **Perplexity**: `OPENROUTER_API_KEY`, `PERPLEXITY_API_KEY`, or `tools.web.search.perplexity.apiKey`
|
||
- **Gemini**: `GEMINI_API_KEY` or `tools.web.search.gemini.apiKey`
|
||
- **Grok**: `XAI_API_KEY` or `tools.web.search.grok.apiKey`
|
||
- **Kimi**: `KIMI_API_KEY`, `MOONSHOT_API_KEY`, or `tools.web.search.kimi.apiKey`
|
||
|
||
### Config
|
||
|
||
```json5
|
||
{
|
||
tools: {
|
||
web: {
|
||
search: {
|
||
enabled: true,
|
||
apiKey: "BRAVE_API_KEY_HERE", // optional if BRAVE_API_KEY is set
|
||
maxResults: 5,
|
||
timeoutSeconds: 30,
|
||
cacheTtlMinutes: 15,
|
||
},
|
||
},
|
||
},
|
||
}
|
||
```
|
||
|
||
### Tool parameters
|
||
|
||
- `query` (required)
|
||
- `count` (1–10; default from config)
|
||
- `country` (optional): 2-letter country code for region-specific results (e.g., "DE", "US", "ALL"). If omitted, Brave chooses its default region.
|
||
- `search_lang` (optional): ISO language code for search results (e.g., "de", "en", "fr")
|
||
- `ui_lang` (optional): ISO language code for UI elements
|
||
- `freshness` (optional): filter by discovery time
|
||
- Brave: `pd`, `pw`, `pm`, `py`, or `YYYY-MM-DDtoYYYY-MM-DD`
|
||
- Perplexity: `pd`, `pw`, `pm`, `py`
|
||
|
||
**Examples:**
|
||
|
||
```javascript
|
||
// German-specific search
|
||
await web_search({
|
||
query: "TV online schauen",
|
||
count: 10,
|
||
country: "DE",
|
||
search_lang: "de",
|
||
});
|
||
|
||
// French search with French UI
|
||
await web_search({
|
||
query: "actualités",
|
||
country: "FR",
|
||
search_lang: "fr",
|
||
ui_lang: "fr",
|
||
});
|
||
|
||
// Recent results (past week)
|
||
await web_search({
|
||
query: "TMBG interview",
|
||
freshness: "pw",
|
||
});
|
||
```
|
||
|
||
## web_fetch
|
||
|
||
Fetch a URL and extract readable content.
|
||
|
||
### web_fetch requirements
|
||
|
||
- `tools.web.fetch.enabled` must not be `false` (default: enabled)
|
||
- Optional Firecrawl fallback: set `tools.web.fetch.firecrawl.apiKey` or `FIRECRAWL_API_KEY`.
|
||
|
||
### web_fetch config
|
||
|
||
```json5
|
||
{
|
||
tools: {
|
||
web: {
|
||
fetch: {
|
||
enabled: true,
|
||
maxChars: 50000,
|
||
maxCharsCap: 50000,
|
||
maxResponseBytes: 2000000,
|
||
timeoutSeconds: 30,
|
||
cacheTtlMinutes: 15,
|
||
maxRedirects: 3,
|
||
userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
|
||
readability: true,
|
||
firecrawl: {
|
||
enabled: true,
|
||
apiKey: "FIRECRAWL_API_KEY_HERE", // optional if FIRECRAWL_API_KEY is set
|
||
baseUrl: "https://api.firecrawl.dev",
|
||
onlyMainContent: true,
|
||
maxAgeMs: 86400000, // ms (1 day)
|
||
timeoutSeconds: 60,
|
||
},
|
||
},
|
||
},
|
||
},
|
||
}
|
||
```
|
||
|
||
### web_fetch tool parameters
|
||
|
||
- `url` (required, http/https only)
|
||
- `extractMode` (`markdown` | `text`)
|
||
- `maxChars` (truncate long pages)
|
||
|
||
Notes:
|
||
|
||
- `web_fetch` uses Readability (main-content extraction) first, then Firecrawl (if configured). If both fail, the tool returns an error.
|
||
- Firecrawl requests use bot-circumvention mode and cache results by default.
|
||
- `web_fetch` sends a Chrome-like User-Agent and `Accept-Language` by default; override `userAgent` if needed.
|
||
- `web_fetch` blocks private/internal hostnames and re-checks redirects (limit with `maxRedirects`).
|
||
- `maxChars` is clamped to `tools.web.fetch.maxCharsCap`.
|
||
- `web_fetch` caps the downloaded response body size to `tools.web.fetch.maxResponseBytes` before parsing; oversized responses are truncated and include a warning.
|
||
- `web_fetch` is best-effort extraction; some sites will need the browser tool.
|
||
- See [Firecrawl](/tools/firecrawl) for key setup and service details.
|
||
- Responses are cached (default 15 minutes) to reduce repeated fetches.
|
||
- If you use tool profiles/allowlists, add `web_search`/`web_fetch` or `group:web`.
|
||
- If the Brave key is missing, `web_search` returns a short setup hint with a docs link.
|