vultr/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-05-06 05:10:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(kimi): reject ungrounded web search answers

Browse Source
This commit is contained in:
Peter Steinberger
2026-05-02 07:03:16 +01:00
parent 2cc79ff184
commit 9008fa445d
5 changed files with 199 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@@ -30,6 +30,7 @@ Docs: https://docs.openclaw.ai
- CLI/directory: report unsupported directory operations for installed channel plugins instead of prompting to reinstall the plugin when it lacks a directory adapter. Fixes #75770. Thanks @lawong888.
- Web search/SearXNG: show the JSON API `search.formats` prerequisite during SearXNG setup before prompting for the base URL. Supersedes #65592. Thanks @evanpaul14.
- Web search/SearXNG: pass through `img_src` image URLs from SearXNG image-category results. Supersedes #61416. Thanks @sghael.
- Web search/Kimi: fail explicitly when Moonshot returns an ungrounded chat answer instead of native web-search evidence, so Kimi no longer reports generic fallback text as a successful search. Fixes #52573. Thanks @wangwllu.
- Web search: keep public provider requests on the strict SSRF guard and reserve private-network access for explicit self-hosted SearXNG/Firecrawl endpoints. Fixes #74357 and supersedes #74360. Thanks @fede-kamel.
- Web search/Firecrawl: allow self-hosted private/internal Firecrawl `baseUrl` endpoints, including HTTP for private targets, while keeping hosted Firecrawl on the strict official endpoint. Fixes #63877 and supersedes #59666, #63941, and #74013. Thanks @jhthompson12, @jzakirov, @Mlightsnow, and @shad0wca7.
- Providers/OpenRouter: strip trailing assistant prefill turns from verified OpenRouter Anthropic model requests when reasoning is enabled, so Claude 4.6 routes no longer fail with Anthropic's prefill rejection through the OpenAI-compatible adapter. Fixes #75395. Thanks @sbmilburn.

9
docs/tools/kimi-search.md
View File

@@ -79,6 +79,15 @@ If you omit `model`, OpenClaw defaults to `kimi-k2.6`.
Kimi uses Moonshot web search to synthesize answers with inline citations,
similar to Gemini and Grok's grounded response approach.
OpenClaw treats Kimi `web_search` as successful only after Moonshot returns
native web-search grounding evidence, such as a replayable `$web_search` tool
payload, `search_results`, or citation URLs. If Kimi stops immediately with a
plain chat answer like "I cannot browse the internet" and no grounding evidence,
OpenClaw returns a structured `kimi_web_search_ungrounded` error instead of
wrapping that text as a search result. Retry the query, switch to a structured
provider such as Brave, or use `web_fetch` / the browser tool when you already
have a target URL.
## Supported parameters
Kimi search supports `query`.

30
docs/tools/web.md
View File

@@ -76,7 +76,7 @@ local while `web_search` and `x_search` can use xAI Responses under the hood.
AI-synthesized answers with citations via xAI web grounding.
</Card>
<Card title="Kimi" icon="moon" href="/tools/kimi-search">
AI-synthesized answers with citations via Moonshot web search.
AI-synthesized answers with citations via Moonshot web search; ungrounded chat fallbacks fail explicitly.
</Card>
<Card title="MiniMax Search" icon="globe" href="/tools/minimax-search">
Structured results via the MiniMax Token Plan search API.
@@ -97,20 +97,20 @@ local while `web_search` and `x_search` can use xAI Responses under the hood.
### Provider comparison
| Provider | Result style | Filters | API key |
| ----------------------------------------- | -------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------- |
| [Brave](/tools/brave-search) | Structured snippets | Country, language, time, `llm-context` mode | `BRAVE_API_KEY` |
| [DuckDuckGo](/tools/duckduckgo-search) | Structured snippets | -- | None (key-free) |
| [Exa](/tools/exa-search) | Structured + extracted | Neural/keyword mode, date, content extraction | `EXA_API_KEY` |
| [Firecrawl](/tools/firecrawl) | Structured snippets | Via `firecrawl_search` tool | `FIRECRAWL_API_KEY` |
| [Gemini](/tools/gemini-search) | AI-synthesized + citations | -- | `GEMINI_API_KEY` |
| [Grok](/tools/grok-search) | AI-synthesized + citations | -- | `XAI_API_KEY` |
| [Kimi](/tools/kimi-search) | AI-synthesized + citations | -- | `KIMI_API_KEY` / `MOONSHOT_API_KEY` |
| [MiniMax Search](/tools/minimax-search) | Structured snippets | Region (`global` / `cn`) | `MINIMAX_CODE_PLAN_KEY` / `MINIMAX_CODING_API_KEY` / `MINIMAX_OAUTH_TOKEN` |
| [Ollama Web Search](/tools/ollama-search) | Structured snippets | -- | None for signed-in local hosts; `OLLAMA_API_KEY` for direct `https://ollama.com` search |
| [Perplexity](/tools/perplexity-search) | Structured snippets | Country, language, time, domains, content limits | `PERPLEXITY_API_KEY` / `OPENROUTER_API_KEY` |
| [SearXNG](/tools/searxng-search) | Structured snippets | Categories, language | None (self-hosted) |
| [Tavily](/tools/tavily) | Structured snippets | Via `tavily_search` tool | `TAVILY_API_KEY` |
| Provider | Result style | Filters | API key |
| ----------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------- |
| [Brave](/tools/brave-search) | Structured snippets | Country, language, time, `llm-context` mode | `BRAVE_API_KEY` |
| [DuckDuckGo](/tools/duckduckgo-search) | Structured snippets | -- | None (key-free) |
| [Exa](/tools/exa-search) | Structured + extracted | Neural/keyword mode, date, content extraction | `EXA_API_KEY` |
| [Firecrawl](/tools/firecrawl) | Structured snippets | Via `firecrawl_search` tool | `FIRECRAWL_API_KEY` |
| [Gemini](/tools/gemini-search) | AI-synthesized + citations | -- | `GEMINI_API_KEY` |
| [Grok](/tools/grok-search) | AI-synthesized + citations | -- | `XAI_API_KEY` |
| [Kimi](/tools/kimi-search) | AI-synthesized + citations; fails on ungrounded chat fallbacks | -- | `KIMI_API_KEY` / `MOONSHOT_API_KEY` |
| [MiniMax Search](/tools/minimax-search) | Structured snippets | Region (`global` / `cn`) | `MINIMAX_CODE_PLAN_KEY` / `MINIMAX_CODING_API_KEY` / `MINIMAX_OAUTH_TOKEN` |
| [Ollama Web Search](/tools/ollama-search) | Structured snippets | -- | None for signed-in local hosts; `OLLAMA_API_KEY` for direct `https://ollama.com` search |
| [Perplexity](/tools/perplexity-search) | Structured snippets | Country, language, time, domains, content limits | `PERPLEXITY_API_KEY` / `OPENROUTER_API_KEY` |
| [SearXNG](/tools/searxng-search) | Structured snippets | Categories, language | None (self-hosted) |
| [Tavily](/tools/tavily) | Structured snippets | Via `tavily_search` tool | `TAVILY_API_KEY` |
## Auto-detection

55
extensions/moonshot/src/kimi-web-search-provider.runtime.ts
View File

@@ -75,6 +75,12 @@ type KimiSearchResponse = {
}>;
};
type KimiSearchResult = {
content: string;
citations: string[];
grounded: boolean;
};
function resolveKimiConfig(searchConfig?: SearchConfigRecord): KimiConfig {
const kimi = searchConfig?.kimi;
return kimi && typeof kimi === "object" && !Array.isArray(kimi) ? (kimi as KimiConfig) : {};
@@ -155,6 +161,15 @@ function extractKimiCitations(data: KimiSearchResponse): string[] {
return [...new Set(citations)];
}
function hasKimiSearchResults(data: KimiSearchResponse): boolean {
return (data.search_results ?? []).some(
(entry) =>
Boolean(normalizeOptionalString(entry.url)) ||
Boolean(normalizeOptionalString(entry.title)) ||
Boolean(normalizeOptionalString(entry.content)),
);
}
function extractKimiToolResultContent(toolCall: KimiToolCall): string | undefined {
const rawArguments = toolCall.function?.arguments;
if (typeof rawArguments !== "string" || rawArguments.trim().length === 0) {
@@ -169,10 +184,11 @@ async function runKimiSearch(params: {
baseUrl: string;
model: string;
timeoutSeconds: number;
}): Promise<{ content: string; citations: string[] }> {
}): Promise<KimiSearchResult> {
const endpoint = `${params.baseUrl.trim().replace(/\/$/, "")}/chat/completions`;
const messages: Array<Record<string, unknown>> = [{ role: "user", content: params.query }];
const collectedCitations = new Set<string>();
let hasGroundingEvidence = false;
for (let round = 0; round < 3; round += 1) {
const next = await withTrustedWebSearchEndpoint(
@@ -201,16 +217,26 @@ async function runKimiSearch(params: {
}
const data = (await res.json()) as KimiSearchResponse;
if (hasKimiSearchResults(data)) {
hasGroundingEvidence = true;
}
for (const citation of extractKimiCitations(data)) {
collectedCitations.add(citation);
}
if (collectedCitations.size > 0) {
hasGroundingEvidence = true;
}
const choice = data.choices?.[0];
const message = choice?.message;
const text = extractKimiMessageText(message);
const toolCalls = message?.tool_calls ?? [];
if (choice?.finish_reason !== "tool_calls" || toolCalls.length === 0) {
return { done: true, content: text ?? "No response", citations: [...collectedCitations] };
return {
done: true,
content: text ?? "No response",
citations: [...collectedCitations],
};
}
messages.push({
@@ -228,6 +254,9 @@ async function runKimiSearch(params: {
if (!toolCallId || !toolCallName || !toolContent) {
continue;
}
if (toolCallName === KIMI_WEB_SEARCH_TOOL.function.name) {
hasGroundingEvidence = true;
}
pushed = true;
messages.push({
role: "tool",
@@ -237,20 +266,25 @@ async function runKimiSearch(params: {
});
}
if (!pushed) {
return { done: true, content: text ?? "No response", citations: [...collectedCitations] };
return {
done: true,
content: text ?? "No response",
citations: [...collectedCitations],
};
}
return { done: false };
},
);
if (next.done) {
return { content: next.content, citations: next.citations };
return { content: next.content, citations: next.citations, grounded: hasGroundingEvidence };
}
}
return {
content: "Search completed but no final answer was produced.",
citations: [...collectedCitations],
grounded: hasGroundingEvidence,
};
}
@@ -304,6 +338,18 @@ export async function executeKimiWebSearchProviderTool(
model,
timeoutSeconds: resolveSearchTimeoutSeconds(searchConfig),
});
if (!result.grounded) {
return {
error: "kimi_web_search_ungrounded",
message:
"Kimi returned a chat completion without native web-search grounding. Retry the query, switch to a structured provider such as Brave, or use web_fetch/browser for a specific URL.",
query,
provider: "kimi",
model,
docs: "https://docs.openclaw.ai/tools/kimi-search",
tookMs: Date.now() - start,
};
}
const payload = {
query,
provider: "kimi",
@@ -410,5 +456,6 @@ export const __testing = {
resolveKimiModel,
resolveKimiBaseUrl,
extractKimiCitations,
hasKimiSearchResults,
extractKimiToolResultContent,
} as const;

124
extensions/moonshot/src/kimi-web-search-provider.test.ts
View File

@@ -1,6 +1,6 @@
import type { OpenClawConfig } from "openclaw/plugin-sdk/provider-onboard";
import { withEnvAsync } from "openclaw/plugin-sdk/test-env";
import { describe, expect, it } from "vitest";
import { afterEach, describe, expect, it, vi } from "vitest";
import { __testing } from "../test-api.js";
import { createKimiWebSearchProvider } from "./kimi-web-search-provider.js";
@@ -25,7 +25,27 @@ function withEnv(overrides: Record<string, string>, run: () => void): void {
}
}
function jsonResponse(body: unknown): Response {
return new Response(JSON.stringify(body), {
status: 200,
headers: { "Content-Type": "application/json" },
});
}
async function executeKimiSearch(query: string): Promise<Record<string, unknown>> {
const provider = createKimiWebSearchProvider();
const tool = provider.createTool({ config: {}, searchConfig: {} });
if (!tool) {
throw new Error("Expected tool definition");
}
return await tool.execute({ query });
}
describe("kimi web search provider", () => {
afterEach(() => {
vi.unstubAllGlobals();
});
it("points missing-key users to fetch/browser alternatives", async () => {
await withEnvAsync({ KIMI_API_KEY: undefined, MOONSHOT_API_KEY: undefined }, async () => {
const provider = createKimiWebSearchProvider();
@@ -108,6 +128,108 @@ describe("kimi web search provider", () => {
).toEqual(["https://a.test", "https://b.test", "https://c.test"]);
});
it("returns a structured failure for ungrounded chat-only responses", async () => {
const fetchMock = vi.fn().mockResolvedValue(
jsonResponse({
choices: [
{
finish_reason: "stop",
message: { content: "I cannot browse the internet." },
},
],
}),
);
vi.stubGlobal("fetch", fetchMock);
await withEnvAsync({ KIMI_API_KEY: "kimi-test-key" }, async () => {
const result = await executeKimiSearch("kimi ungrounded chat fallback");
expect(result).toMatchObject({
error: "kimi_web_search_ungrounded",
provider: "kimi",
message: expect.stringContaining("without native web-search grounding"),
});
});
});
it("accepts final responses backed by Kimi web search tool replay", async () => {
const toolArguments = JSON.stringify({
query: "OpenClaw GitHub repository",
usage: { total_tokens: 1200 },
});
const fetchMock = vi
.fn()
.mockResolvedValueOnce(
jsonResponse({
choices: [
{
finish_reason: "tool_calls",
message: {
content: "",
tool_calls: [
{
id: "call-1",
function: {
name: "$web_search",
arguments: toolArguments,
},
},
],
},
},
],
}),
)
.mockResolvedValueOnce(
jsonResponse({
choices: [
{
finish_reason: "stop",
message: { content: "OpenClaw is available on GitHub." },
},
],
}),
);
vi.stubGlobal("fetch", fetchMock);
await withEnvAsync({ KIMI_API_KEY: "kimi-test-key" }, async () => {
const result = await executeKimiSearch("kimi grounded tool replay");
expect(result).toMatchObject({
provider: "kimi",
content: expect.stringContaining("OpenClaw is available on GitHub."),
citations: [],
});
expect(result).not.toHaveProperty("error");
});
});
it("accepts final responses with search result citations", async () => {
const fetchMock = vi.fn().mockResolvedValue(
jsonResponse({
search_results: [{ title: "OpenClaw", url: "https://github.com/openclaw/openclaw" }],
choices: [
{
finish_reason: "stop",
message: { content: "OpenClaw is on GitHub." },
},
],
}),
);
vi.stubGlobal("fetch", fetchMock);
await withEnvAsync({ KIMI_API_KEY: "kimi-test-key" }, async () => {
const result = await executeKimiSearch("kimi grounded citation");
expect(result).toMatchObject({
provider: "kimi",
content: expect.stringContaining("OpenClaw is on GitHub."),
citations: ["https://github.com/openclaw/openclaw"],
});
expect(result).not.toHaveProperty("error");
});
});
it("returns original tool arguments as tool content", () => {
const rawArguments = ' {"query":"MacBook Neo","usage":{"total_tokens":123}} ';