Files
openclaw/docs/tools/tool-search.md
Peter Steinberger d413fbd8ec fix: Tool Search finds tools by how agents actually phrase requests (#114285)
* feat(agents): rank Tool Search with BM25 over names, descriptions, and parameters

Ranking was case-insensitive substring matching with hand-tuned weights, which
failed in ways that made tools unreachable rather than merely mis-ordered:

- "scheduling" found nothing against a tool described "Schedule a recurring
  task" — no stemming.
- "read" ranked spreadsheet_open, because "sp-read-sheet" contains it.
- A non-English query tokenized to zero terms, and the scorer returned 1 for
  every entry, so the model received an arbitrary alphabetical slice of the
  catalog presented as a ranked result.

Replace it with Okapi BM25 over a tokenizer that splits on Unicode word
boundaries, drops stopwords, and collapses light English inflection. Index
parameter names and descriptions too, which Codex (bm25 crate) and the Claude
API tool-search tools both do; a query like "repository" now reaches a tool
whose description never says it. Underscore-joined names index as both the whole
name and its parts.

A small query-expansion table bridges intent to description vocabulary ("look up
the price" -> search/web), which pure lexical overlap cannot do. It holds only
generic capability words, never plugin or vendor names.

Empty queries now score nothing instead of everything, and both `tool_search`
and the code-mode bridge tell the model to query in English, so the degenerate
case is steered away from rather than silently mishandled.

Untrusted schemas are still never traversed: parameters are indexed only for
`openclaw` entries, matching the boundary compactToolSearchCatalogEntry already
enforces by reporting MCP and client inputs as "unknown".

* fix(agents): undouble inflected consonants and state the real English contract

Autoreview caught two defects in the new stemmer and its documentation.

"running" stripped to "runn", which can never meet "run", so a tool named
task_runner described "Running tasks" became unreachable for the query "run" —
a regression the substring scorer did not have. English doubles the final
consonant before -ing/-ed/-er, so undo that, while keeping doubles that belong
to the root (call, process, off, buzz).

The English-only claim was also stronger than the code: the tokenizer keeps
Unicode letters, so a non-English query yields terms and can match. Rejecting
them would make a catalog that legitimately names a tool in another script
permanently unreachable, so the behavior stays and the wording now matches it —
catalogs are written in English, so other languages usually match nothing, and
the model is asked to query in English for that reason rather than because the
input is filtered. The previous test only proved an unrelated document scored
zero, so it is replaced by one that asserts each half directly.

* fix(agents): split camelCase, discount expansions, and stop trigger collisions

Three defects in the new tokenizer, all found by autoreview and all reproduced
before fixing.

`splitWords` lowercased before looking for boundaries, so `readFile` produced
only `readfil` and the natural query `read file` could not meet it. MCP catalogs
commonly use camelCase, and the old substring scorer matched those. Split case
transitions before lowercasing.

Expansion terms carried the same BM25 weight as words the caller typed, so
`weather` — which expands to search/web — could rank a general web tool above
the exact weather tool by matching two terms instead of one. Expansions are a
guess about how the catalog words a capability, so they now score at 0.35, and a
term the caller actually wrote keeps full weight even when an expansion repeats
it.

Trigger matching ran the document stemmer, which collapsed unrelated vocabulary:
`news` became `new`, so "open a new issue" silently acquired a web-search intent.
Triggers now normalize by singularization only, which leaves `news` intact, and
the table lists `reminder` explicitly rather than relying on the stemmer to
reach it.

* fix(agents): normalize -ies plurals and tier literal matches above expansions

`repositories` stemmed to `repositori` while `repository` stayed put, so the two
never met; `-ies` now normalizes back to `-y` in both the document stemmer and
the expansion triggers, which also lets `directories` and `memories` reach their
intended groups instead of stalling at `directorie`.

The 0.35 expansion discount is not sufficient on its own to keep a literal match
ranked first: BM25 sums per term, so a common literal like `weather` carries
little IDF while a short document collecting two rare expansions can outscore
it, and a small result limit then drops every tool matching the typed word.
Literal overlap is now reported per hit and ranked as a tier ahead of score, so
the discount orders within a tier rather than trying to carry the invariant.

Verified by running the ranker over the failing inputs rather than reasoning
about them; the first attempt at the trigger fix silently did not apply because
the formatter had reflowed the function, which the matrix run caught.

* fix(agents): keep non-plural -s words out of the plural stem rule

"news" stemmed to "new", which literal-matched every "Create a new ..." tool;
because literal overlap now outranks expansion-only hits, a search for news
returned creation tools instead of the web tool the query meant.

The collision is not unique to news — "status", "canvas", and "alias" are all
ordinary tool vocabulary here and all lose their meaning under the same rule, so
the exemption covers that class rather than the single reported word.

* fix(agents): restore the exact-name tier and keep `get` searchable

Two signals the old substring scorer had and BM25 alone does not.

It gave an exact name/id match +20, which flattening everything into one
document removed: querying a known tool name could rank a shorter entry that
merely mentions the word above the tool itself, and the result limit would then
drop the tool asked for. Exact name/id is now a sort tier ahead of literal
overlap.

`get` was in the stopword list, but it names real operations in a tool catalog.
Discarding it made `search("get")` empty and reduced "get issue" to "issue",
where a shorter delete_issue or update_issue entry can win on length
normalization. Capability verbs stay indexed.

* fix(agents): keep acronyms whole, both -ies readings, and stopword-named tools

* refactor(agents): keep the ranking types and entry-text helper module-local

Knip's hard-zero unused-export gate flagged WeightedTerm, RankedDocument,
LexicalIndex, and toolSearchEntryText: nothing outside their own modules
imported them. Narrowing the surface is what the repo asks for anyway.

The untrusted-schema test now drives ToolSearchRuntime.search instead of
calling toolSearchEntryText directly, which proves the boundary through the
real entry point. Verified it still bites: removing the source gate makes it
fail with "client properties must remain deferred".
2026-07-27 01:10:53 -04:00

14 KiB

summary, title, read_when
summary title read_when
Tool Search: compact large OpenClaw tool catalogs behind search, describe, and call Tool Search
You want OpenClaw agents to use a large tool catalog without adding every tool schema to the prompt
You want OpenClaw tools, MCP tools, and client tools exposed through one compact runtime surface
You are implementing or debugging tool discovery for OpenClaw runs

Tool Search is an experimental OpenClaw agent runtime feature. It gives agents one compact way to discover and call large tool catalogs. It is useful when the run has many available tools but the model is likely to need only a few of them.

This page documents OpenClaw Tool Search. It is not the Codex-native tool search or dynamic-tools surface. Codex-native code mode, tool search, deferred dynamic tools, and nested tool calls are stable Codex harness surfaces and do not depend on tools.toolSearch.

For the generic OpenClaw runtime that exposes a QuickJS-WASI exec/wait surface instead of Tool Search controls, see Code Mode.

When enabled for OpenClaw runs, the model receives one tool_search_code tool by default, plus any direct-only tools whose structured results cannot cross the compact bridge. The code tool runs a short JavaScript body in an isolated Node subprocess with an openclaw.tools bridge:

const hits = await openclaw.tools.search("create a GitHub issue");
const tool = await openclaw.tools.describe(hits[0].id);
return await openclaw.tools.call(tool.id, {
  title: "Crash on startup",
  body: "Steps to reproduce...",
});

The catalog can include catalog-eligible OpenClaw tools, plugin tools, MCP tools, and client-provided tools. The model does not see every cataloged schema up front. Instead, it searches compact descriptors, describes one selected tool when it needs the exact schema, and calls that tool through OpenClaw. Direct-only tools remain model-visible and are not added to the catalog.

Codex harness runs do not receive these experimental OpenClaw Tool Search controls. OpenClaw passes product capabilities to Codex as dynamic tools, and Codex owns the stable native code mode, native tool search, deferred dynamic tools, and nested tool calls.

How a turn runs

At planning time the OpenClaw embedded runner builds the effective catalog for the run:

  1. Resolve the active tool policy for the agent, profile, sandbox, and session.
  2. List eligible OpenClaw and plugin tools.
  3. List eligible MCP tools through the session MCP runtime.
  4. Add eligible client tools supplied for the current run.
  5. Keep direct-only tools model-visible and index compact descriptors for the remaining catalog-eligible tools.
  6. Expose the OpenClaw code bridge, the structured fallback tools, or the compact directory surface alongside those direct-only tools.

At execution time every real tool call returns to OpenClaw. The isolated Node runtime does not hold plugin implementations, MCP client objects, or secrets. openclaw.tools.call(...) crosses the bridge back into the Gateway, where the normal policy, approval, hook, logging, and result handling still apply.

Modes

tools.toolSearch has three model-facing modes:

  • code: exposes tool_search_code, the default compact JavaScript bridge, alongside direct-only tools.
  • tools: exposes tool_search, tool_describe, and tool_call as plain structured tools for providers that should not receive code, alongside direct-only tools.
  • directory: exposes tool_search, tool_describe, and tool_call plus a bounded prompt directory of available tool names and descriptions for providers that should see tool names without every full schema. OpenClaw can also expose a small bounded set of likely or required tool schemas directly for the current turn. Direct-only tools remain visible in this mode too.

All modes use the same policy-filtered catalog and normal OpenClaw execution path. Tools marked catalogMode: "direct-only" stay outside that catalog and remain model-visible. If the current runtime cannot launch the isolated Node code-mode child process, the default code mode falls back to tools before catalog compaction. In directory mode, client-provided tools stay directly visible for the current run while OpenClaw tools, plugin tools, and MCP tools can be compacted behind the directory catalog. A direct call to an exact hidden directory name is hydrated from that same authorized catalog before execution.

All modes are experimental. Prefer direct tool exposure for small OpenClaw tool catalogs, and prefer the Codex-native stable surfaces for Codex harness runs.

There is no separate source-selection config. When Tool Search is enabled, the catalog includes catalog-eligible OpenClaw, MCP, and client tools after normal policy filtering; direct-only tools are retained separately.

Why this exists

Large catalogs are useful but expensive. Sending every tool schema to the model makes the request larger, slows planning, and increases accidental tool selection.

Tool Search changes the shape:

  • direct tools: the model sees every selected schema before the first token
  • Tool Search code mode: the model sees one compact code tool, a short API contract, and any direct-only tools
  • Tool Search tools mode: the model sees three compact structured fallback tools plus any direct-only tools
  • Tool Search directory mode: the model sees a bounded directory plus search/describe/call controls and a small bounded set of likely or required schemas, plus any direct-only tools
  • during the turn: the model can load remaining schemas as needed

Direct tool exposure is still the right default for small catalogs. Tool Search is best when one run can see many tools, especially from MCP servers or client-provided app tools.

API

openclaw.tools.search(query, options?)

Searches the effective catalog for the current run.

Queries must be written in English. Ranking is lexical (Okapi BM25 over tool names, descriptions, and first-party parameter names and descriptions), with light English stemming so scheduling reaches a tool described as Schedule a recurring task, and a small intent expansion so look up the price reaches one described as Search the web. Tool names and descriptions are written in English, so a query in another language will usually match nothing. It is not rejected — a catalog may legitimately describe a tool in another script — but it is also no longer answered with an arbitrary slice of the catalog presented as if it were ranked, which is what the previous scorer did whenever a query produced no usable terms. Both tool_search and the code-mode bridge state this requirement in their model-facing descriptions.

Untrusted parameter schemas are never indexed. MCP and client tools are matched on name and description only, which is the same boundary that defers their input signatures as input: "unknown".

Results are compact and safe to put back into prompt context. Each hit includes a bounded TypeScript-style input signature, such as { id: string; mode?: "drip" | "flood" }, so the model can skip describe when that signature is sufficient. A trusted OpenClaw core or plugin tool may also include a compact output hint, such as Array<{ id: string; paid: boolean }>. MCP and client output-schema claims are not promoted into this trusted hint. Their untrusted input schemas are also deferred as input: "unknown"; use describe before calling them. Open, oversized, or otherwise partial output schemas omit the hint and remain available through describe instead.

const hits = await openclaw.tools.search("calendar event", { limit: 5 });

openclaw.tools.describe(id)

Loads full metadata for one search result, including the exact input schema and the trusted full outputSchema when the tool declares one.

const calendarCreate = await openclaw.tools.describe("mcp:calendar:create_event");

openclaw.tools.call(id, args)

Calls a selected tool through OpenClaw and returns the raw { tool, result } envelope. JSON-returning tools normally place their value in result.details. If a trusted tool declares outputSchema, OpenClaw compiles the schema before execution and validates final details after normal tool hooks before returning the catalog call.

await openclaw.tools.call(calendarCreate.id, {
  summary: "Planning",
  start: "2026-05-09T14:00:00Z",
});

Tool authors declare output contracts on the tool's outputSchema property. It describes AgentToolResult.details, not rendered content blocks. Include all non-throwing variants or omit it for unstable results. See Code Mode output contracts and Tool plugins.

The structured fallback mode exposes the same operations as tools:

  • tool_search
  • tool_describe
  • tool_call

Directory mode exposes:

  • tool_search
  • tool_describe
  • tool_call

It also keeps client-provided tools and all direct-only tools directly visible, and may expose a small bounded set of likely or required catalog tool schemas directly for the current turn. If the bounded directory omits entries, use tool_search to find them. If the model requests an exact hidden directory tool name directly, OpenClaw hydrates it from the authorized catalog before normal execution. Directory-mode client tool names must not collide with OpenClaw, plugin, or MCP tool names because exact deferred dispatch uses those names.

Runtime boundary

The code bridge runs in a short-lived Node subprocess. The subprocess starts with Node permission mode enabled, an empty environment, no filesystem or network grants, and no child-process or worker grants. OpenClaw enforces a parent-process wall-clock timeout and kills the subprocess on timeout, including after async continuations.

The runtime exposes only:

  • console.log, console.warn, and console.error
  • openclaw.tools.search
  • openclaw.tools.describe
  • openclaw.tools.call

Normal OpenClaw behavior still applies to final calls:

  • tool allow and deny policies
  • per-agent and per-sandbox tool restrictions
  • channel/runtime tool policy
  • approval hooks
  • plugin before_tool_call hooks
  • session identity, logs, and telemetry

Config

Enable Tool Search for OpenClaw runs with the default code bridge:

openclaw config set tools.toolSearch true

Equivalent JSON:

{
  tools: {
    toolSearch: true,
  },
}

Use the structured fallback tools instead for OpenClaw runs:

{
  tools: {
    toolSearch: {
      mode: "tools",
    },
  },
}

Use the compact directory surface instead for OpenClaw runs:

{
  tools: {
    toolSearch: {
      mode: "directory",
    },
  },
}

Tune code-mode timeout and search result limits (values shown are the defaults):

{
  tools: {
    toolSearch: {
      mode: "code",
      codeTimeoutMs: 10000,
      searchDefaultLimit: 8,
      maxSearchLimit: 20,
    },
  },
}

The runtime clamps codeTimeoutMs to 1000-60000, maxSearchLimit to 1-50, and searchDefaultLimit to 1..maxSearchLimit.

Disable it:

{
  tools: {
    toolSearch: false,
  },
}

Prompt and telemetry

Code mode attaches a telemetry object to every tool_search_code result:

  • catalogSize: number of catalog entries the runtime resolved
  • sources: catalog entry counts split into openclaw, mcp, and client
  • searchCount, describeCount, callCount: running totals for the catalog session, carried across calls rather than reset per call

tools and directory mode emit no telemetry object; their tool_search, tool_describe, and tool_call results carry only the catalog data for that operation. OpenClaw does not record serialized tool or prompt byte counts. The E2E scenario measures provider payload bytes separately from the mock provider lane, not from the runtime.

Regardless of mode, target tool calls are projected into the session transcript as normal tool call and tool result pairs, and search, describe, and call results carry each tool's id and source. Session logs therefore still answer:

  • how many tool schemas the model saw up front
  • how many search and describe operations it performed
  • which final tool was called
  • whether the result came from OpenClaw, MCP, or a client tool

E2E validation

The QA Lab gateway scenario proves both paths with the OpenClaw runtime:

pnpm openclaw qa suite --provider-mode mock-openai --scenario tool-search-gateway-e2e

It creates a temporary fake plugin with a large tool catalog, starts the mock OpenAI provider, starts a Gateway once in direct mode and once with Tool Search enabled, then compares provider request payloads and session logs.

The regression proves:

  1. Direct mode can call the fake plugin tool.
  2. Tool Search can call the same fake plugin tool.
  3. Direct mode exposes the fake plugin tool schemas directly to the provider.
  4. Tool Search exposes only the compact bridge plus any direct-only tools.
  5. The Tool Search request payload is smaller for the large fake catalog.
  6. Session logs show the expected tool-call counts and bridged call telemetry.

Failure behavior

Tool Search should fail closed:

  • if a tool is not in the effective policy, search should not return it
  • if a selected tool becomes unavailable, tool_call should fail
  • if policy or approval blocks execution, the call result should report that block instead of bypassing it
  • if the code bridge cannot create an isolated runtime, use mode: "tools" or disable Tool Search for that deployment