mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-02 12:01:35 +00:00
d413fbd8ec
* feat(agents): rank Tool Search with BM25 over names, descriptions, and parameters
Ranking was case-insensitive substring matching with hand-tuned weights, which
failed in ways that made tools unreachable rather than merely mis-ordered:
- "scheduling" found nothing against a tool described "Schedule a recurring
task" — no stemming.
- "read" ranked spreadsheet_open, because "sp-read-sheet" contains it.
- A non-English query tokenized to zero terms, and the scorer returned 1 for
every entry, so the model received an arbitrary alphabetical slice of the
catalog presented as a ranked result.
Replace it with Okapi BM25 over a tokenizer that splits on Unicode word
boundaries, drops stopwords, and collapses light English inflection. Index
parameter names and descriptions too, which Codex (bm25 crate) and the Claude
API tool-search tools both do; a query like "repository" now reaches a tool
whose description never says it. Underscore-joined names index as both the whole
name and its parts.
A small query-expansion table bridges intent to description vocabulary ("look up
the price" -> search/web), which pure lexical overlap cannot do. It holds only
generic capability words, never plugin or vendor names.
Empty queries now score nothing instead of everything, and both `tool_search`
and the code-mode bridge tell the model to query in English, so the degenerate
case is steered away from rather than silently mishandled.
Untrusted schemas are still never traversed: parameters are indexed only for
`openclaw` entries, matching the boundary compactToolSearchCatalogEntry already
enforces by reporting MCP and client inputs as "unknown".
* fix(agents): undouble inflected consonants and state the real English contract
Autoreview caught two defects in the new stemmer and its documentation.
"running" stripped to "runn", which can never meet "run", so a tool named
task_runner described "Running tasks" became unreachable for the query "run" —
a regression the substring scorer did not have. English doubles the final
consonant before -ing/-ed/-er, so undo that, while keeping doubles that belong
to the root (call, process, off, buzz).
The English-only claim was also stronger than the code: the tokenizer keeps
Unicode letters, so a non-English query yields terms and can match. Rejecting
them would make a catalog that legitimately names a tool in another script
permanently unreachable, so the behavior stays and the wording now matches it —
catalogs are written in English, so other languages usually match nothing, and
the model is asked to query in English for that reason rather than because the
input is filtered. The previous test only proved an unrelated document scored
zero, so it is replaced by one that asserts each half directly.
* fix(agents): split camelCase, discount expansions, and stop trigger collisions
Three defects in the new tokenizer, all found by autoreview and all reproduced
before fixing.
`splitWords` lowercased before looking for boundaries, so `readFile` produced
only `readfil` and the natural query `read file` could not meet it. MCP catalogs
commonly use camelCase, and the old substring scorer matched those. Split case
transitions before lowercasing.
Expansion terms carried the same BM25 weight as words the caller typed, so
`weather` — which expands to search/web — could rank a general web tool above
the exact weather tool by matching two terms instead of one. Expansions are a
guess about how the catalog words a capability, so they now score at 0.35, and a
term the caller actually wrote keeps full weight even when an expansion repeats
it.
Trigger matching ran the document stemmer, which collapsed unrelated vocabulary:
`news` became `new`, so "open a new issue" silently acquired a web-search intent.
Triggers now normalize by singularization only, which leaves `news` intact, and
the table lists `reminder` explicitly rather than relying on the stemmer to
reach it.
* fix(agents): normalize -ies plurals and tier literal matches above expansions
`repositories` stemmed to `repositori` while `repository` stayed put, so the two
never met; `-ies` now normalizes back to `-y` in both the document stemmer and
the expansion triggers, which also lets `directories` and `memories` reach their
intended groups instead of stalling at `directorie`.
The 0.35 expansion discount is not sufficient on its own to keep a literal match
ranked first: BM25 sums per term, so a common literal like `weather` carries
little IDF while a short document collecting two rare expansions can outscore
it, and a small result limit then drops every tool matching the typed word.
Literal overlap is now reported per hit and ranked as a tier ahead of score, so
the discount orders within a tier rather than trying to carry the invariant.
Verified by running the ranker over the failing inputs rather than reasoning
about them; the first attempt at the trigger fix silently did not apply because
the formatter had reflowed the function, which the matrix run caught.
* fix(agents): keep non-plural -s words out of the plural stem rule
"news" stemmed to "new", which literal-matched every "Create a new ..." tool;
because literal overlap now outranks expansion-only hits, a search for news
returned creation tools instead of the web tool the query meant.
The collision is not unique to news — "status", "canvas", and "alias" are all
ordinary tool vocabulary here and all lose their meaning under the same rule, so
the exemption covers that class rather than the single reported word.
* fix(agents): restore the exact-name tier and keep `get` searchable
Two signals the old substring scorer had and BM25 alone does not.
It gave an exact name/id match +20, which flattening everything into one
document removed: querying a known tool name could rank a shorter entry that
merely mentions the word above the tool itself, and the result limit would then
drop the tool asked for. Exact name/id is now a sort tier ahead of literal
overlap.
`get` was in the stopword list, but it names real operations in a tool catalog.
Discarding it made `search("get")` empty and reduced "get issue" to "issue",
where a shorter delete_issue or update_issue entry can win on length
normalization. Capability verbs stay indexed.
* fix(agents): keep acronyms whole, both -ies readings, and stopword-named tools
* refactor(agents): keep the ranking types and entry-text helper module-local
Knip's hard-zero unused-export gate flagged WeightedTerm, RankedDocument,
LexicalIndex, and toolSearchEntryText: nothing outside their own modules
imported them. Narrowing the surface is what the repo asks for anyway.
The untrusted-schema test now drives ToolSearchRuntime.search instead of
calling toolSearchEntryText directly, which proves the boundary through the
real entry point. Verified it still bites: removing the source gate makes it
fail with "client properties must remain deferred".