* fix(gateway): keep chat history display text truncation surrogate-safe
truncateChatHistoryText uses slice(0, N) to truncate chat history text
for the Control UI. When the truncation boundary falls inside a UTF-16
surrogate pair (emoji, CJK extended), the resulting string contains a
dangling surrogate that browsers render as U+FFFD (�).
Replace slice(0, maxChars) with truncateUtf16Safe(text, maxChars) so
the truncation point always falls on a complete code-point boundary.
* test(gateway): cover chat display UTF-16 boundary
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(tool-policy-audit): use truncateUtf16Safe for audit field truncation
Replace naive .slice(0, MAX) with truncateUtf16Safe() to prevent
surrogate pair splitting in tool policy audit log output.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(agents): cover audit UTF-16 boundary
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(agents): keep tool-result context guard truncation UTF-16 safe
Replace the raw text.slice(0, cutPoint) in truncateTextToBudget with
truncateUtf16Safe so oversized tool results do not emit lone surrogates
when an emoji falls on the truncation boundary.
Adds a regression test that places an emoji exactly at the legacy cut
point and asserts the truncated output contains no lone surrogates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(agents): report surrogate-safe omitted count
---------
Co-authored-by: chengzhichao-xydt <chengzhichao-xydt@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(matrix): truncate inbound preview on UTF-16 code-point boundary
The matrix inbound verbose preview used bodyText.slice(0, 200) before logging.
When a supplementary-plane character (emoji, extended CJK) straddles the 200th
code unit, the slice splits its surrogate pair and emits a lone surrogate into
the verbose log line. Lone surrogates corrupt JSON log serialization, break
terminal rendering, and crash UTF-8-validating log shippers.
Use the shared truncateUtf16Safe helper (already used by the mattermost sibling
monitor after #101630) to keep complete surrogate pairs intact. No behavior
change for ASCII input; the preview is still capped at 200 code units.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(matrix): fix inbound-preview test for lint and lib target
- Avoid String.prototype.isWellFormed (ES2024 lib): use the encodeURIComponent
well-formedness probe instead, which works on the project's TS lib target.
- Replace literal string concatenation with template strings to satisfy the
no-useless-concat lint rule. No behavior change in the assertions.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(channels): keep inbound log previews UTF-16 safe
* test(line): type verbose preview fixtures
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat(stepfun): add step-3.7-flash model and make it default
* fix(stepfun): set step-3.7-flash cacheRead cost to 0.04
* fix(stepfun): raise step-3.7-flash maxTokens to context limit
Output was artificially capped at 65536; StepFun docs set max_tokens
default to INF, bounded only by the 256K context window. Raise
step-3.7-flash maxTokens to 262144 on both stepfun and stepfun-plan,
matching contextWindow. Per models.dev anomalyco/models.dev#1903.
step-3.5-flash entries unchanged.
* fix(stepfun): zero step-plan step-3.7-flash cost for plan billing
Step Plan endpoint bills per subscription plan, not per token, matching
the sibling stepfun-plan/step-3.5-flash and step-3.5-flash-2603 entries
which already carry zero cost. Standard stepfun/step-3.7-flash keeps its
per-token rate. Per models.dev anomalyco/models.dev#1903, whose step-plan
providers omit cost entirely.
* docs(changelog): note StepFun 3.7 support
* style(stepfun): format provider table
* fix(stepfun): forward reasoning effort
* fix(stepfun): scope reasoning compat to 3.7
* test(stepfun): keep 3.5 reasoning compat unchanged
* fix(stepfun): carry off fallback through agent turns
* fix(stepfun): use documented token limit field
* fix(stepfun): match chat completions contract
* chore(stepfun): keep changelog release-owned
---------
Co-authored-by: Tianning Li <litianning@stepfun.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(security): keep channel-metadata and install-policy truncation surrogate-safe
channel-metadata.ts and install-policy.ts both had private truncateText
functions using String.prototype.slice(0, N) on user-visible text.
channel-metadata is explicitly user-controlled untrusted content that gets
injected into LLM prompt context -- a dangling surrogate from unsafe
truncation would corrupt the prompt with invalid Unicode, affecting token
counting and potentially LLM response quality.
Replace .slice(0, ...) with truncateUtf16Safe(...) in both files so the
truncation point always falls on a complete code-point boundary.
Add test coverage for buildUntrustedChannelMetadata with emoji at both
the entry-level (400-char) and outer (800-char) truncation boundaries.
* test(security): prove UTF-16 truncation boundaries
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* [AI] fix(memory): use truncateUtf16Safe for dreaming snippet truncation
Replace .slice(0, N) with truncateUtf16Safe() at 5 call sites in
dreaming-phases.ts so complex emoji and surrogate pairs near the
truncation boundary are not split into lone surrogates.
truncateUtf16Safe is the standard SDK helper, already used in
session-cost-usage, cron, exec-approval, and node-host modules
for the same purpose.
* [AI] fix(memory): use SDK facade import for truncateUtf16Safe, add surrogate-proof test
Replace direct @openclaw/normalization-core/utf16-slice import with
openclaw/plugin-sdk/memory-core-host-engine-foundation SDK facade,
matching manager-search.ts in the same extension.
Add surrogate-proof.test.ts that runs a full dreaming ingestion sweep
with emoji at the 280-char boundary and verifies zero lone surrogates.
* [AI] fix(memory-proof): use relative imports in surrogate-proof test
* [AI] fix(memory-proof): place emoji at post-prefix 280-char boundary, assert session corpus file
* [AI] fix(memory-proof): avoid unnecessary String callback in oxlint
* [AI] fix(memory-proof): add curly braces per eslint curly rule
* [AI] feat(proof): add standalone dreaming surrogate truncation proof script
* [AI] chore(proof): remove unused SQLite-dependent proof script
* [AI] fix(proof): correct emoji boundary placement for all 5 call sites
* [AI] fix(proof): add curly braces per eslint curly rule in proof script
* [AI] fix(proof): session emoji at actual corpus 280-char boundary
* test: streamline memory dreaming UTF-16 coverage
---------
Co-authored-by: hailory <hailory@xydigit.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(acp): keep session update text truncation surrogate-safe
acp-projector's truncateText function used String.prototype.slice(0, N)
on session update text that is sent to AI providers in the prompt context.
When the truncation boundary falls inside a UTF-16 surrogate pair (emoji,
CJK extended), the resulting string contains a dangling surrogate that
can corrupt the prompt.
Replace .slice(0, maxChars-1) with truncateUtf16Safe(input, maxChars-1)
so the truncation point always falls on a complete code-point boundary.
* fix: make ACP projection truncation UTF-16 safe
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
The bounded task label and progress summary shown for child ACP background runs used raw String.slice, which can split a surrogate pair at the 160/240-char boundaries and surface U+FFFD in requester-facing task status. Part of the UTF-16 safety sweep.
* fix(gateway): keep session title and preview text truncation UTF-16 safe
* test(gateway): cover session UTF-16 boundaries in place
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(ui): keep clampText/truncateText surrogate-safe at emoji boundaries
clampText and truncateText in format.ts used String.prototype.slice(0, N)
to truncate user-visible text. When the truncation boundary falls inside
a UTF-16 surrogate pair (emoji, CJK extended, etc.), the resulting
string contains a dangling surrogate that browsers render as U+FFFD (�).
Replace .slice(0, ...) with truncateUtf16Safe(...) which skips past
incomplete surrogate pairs so the truncation point always falls on a
code-point boundary.
Affected UI surfaces: markdown rendering, command descriptions, skill
summaries, exec-approval rules, tool-stream output previews.
Add surrogate-safe truncation test cases for both functions using a
real emoji at the exact truncation boundary.
* test(ui): tighten UTF-16 truncation coverage
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
