* fix(matrix): truncate inbound preview on UTF-16 code-point boundary
The matrix inbound verbose preview used bodyText.slice(0, 200) before logging.
When a supplementary-plane character (emoji, extended CJK) straddles the 200th
code unit, the slice splits its surrogate pair and emits a lone surrogate into
the verbose log line. Lone surrogates corrupt JSON log serialization, break
terminal rendering, and crash UTF-8-validating log shippers.
Use the shared truncateUtf16Safe helper (already used by the mattermost sibling
monitor after #101630) to keep complete surrogate pairs intact. No behavior
change for ASCII input; the preview is still capped at 200 code units.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(matrix): fix inbound-preview test for lint and lib target
- Avoid String.prototype.isWellFormed (ES2024 lib): use the encodeURIComponent
well-formedness probe instead, which works on the project's TS lib target.
- Replace literal string concatenation with template strings to satisfy the
no-useless-concat lint rule. No behavior change in the assertions.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(channels): keep inbound log previews UTF-16 safe
* test(line): type verbose preview fixtures
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* feat(stepfun): add step-3.7-flash model and make it default
* fix(stepfun): set step-3.7-flash cacheRead cost to 0.04
* fix(stepfun): raise step-3.7-flash maxTokens to context limit
Output was artificially capped at 65536; StepFun docs set max_tokens
default to INF, bounded only by the 256K context window. Raise
step-3.7-flash maxTokens to 262144 on both stepfun and stepfun-plan,
matching contextWindow. Per models.dev anomalyco/models.dev#1903.
step-3.5-flash entries unchanged.
* fix(stepfun): zero step-plan step-3.7-flash cost for plan billing
Step Plan endpoint bills per subscription plan, not per token, matching
the sibling stepfun-plan/step-3.5-flash and step-3.5-flash-2603 entries
which already carry zero cost. Standard stepfun/step-3.7-flash keeps its
per-token rate. Per models.dev anomalyco/models.dev#1903, whose step-plan
providers omit cost entirely.
* docs(changelog): note StepFun 3.7 support
* style(stepfun): format provider table
* fix(stepfun): forward reasoning effort
* fix(stepfun): scope reasoning compat to 3.7
* test(stepfun): keep 3.5 reasoning compat unchanged
* fix(stepfun): carry off fallback through agent turns
* fix(stepfun): use documented token limit field
* fix(stepfun): match chat completions contract
* chore(stepfun): keep changelog release-owned
---------
Co-authored-by: Tianning Li <litianning@stepfun.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(security): keep channel-metadata and install-policy truncation surrogate-safe
channel-metadata.ts and install-policy.ts both had private truncateText
functions using String.prototype.slice(0, N) on user-visible text.
channel-metadata is explicitly user-controlled untrusted content that gets
injected into LLM prompt context -- a dangling surrogate from unsafe
truncation would corrupt the prompt with invalid Unicode, affecting token
counting and potentially LLM response quality.
Replace .slice(0, ...) with truncateUtf16Safe(...) in both files so the
truncation point always falls on a complete code-point boundary.
Add test coverage for buildUntrustedChannelMetadata with emoji at both
the entry-level (400-char) and outer (800-char) truncation boundaries.
* test(security): prove UTF-16 truncation boundaries
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* [AI] fix(memory): use truncateUtf16Safe for dreaming snippet truncation
Replace .slice(0, N) with truncateUtf16Safe() at 5 call sites in
dreaming-phases.ts so complex emoji and surrogate pairs near the
truncation boundary are not split into lone surrogates.
truncateUtf16Safe is the standard SDK helper, already used in
session-cost-usage, cron, exec-approval, and node-host modules
for the same purpose.
* [AI] fix(memory): use SDK facade import for truncateUtf16Safe, add surrogate-proof test
Replace direct @openclaw/normalization-core/utf16-slice import with
openclaw/plugin-sdk/memory-core-host-engine-foundation SDK facade,
matching manager-search.ts in the same extension.
Add surrogate-proof.test.ts that runs a full dreaming ingestion sweep
with emoji at the 280-char boundary and verifies zero lone surrogates.
* [AI] fix(memory-proof): use relative imports in surrogate-proof test
* [AI] fix(memory-proof): place emoji at post-prefix 280-char boundary, assert session corpus file
* [AI] fix(memory-proof): avoid unnecessary String callback in oxlint
* [AI] fix(memory-proof): add curly braces per eslint curly rule
* [AI] feat(proof): add standalone dreaming surrogate truncation proof script
* [AI] chore(proof): remove unused SQLite-dependent proof script
* [AI] fix(proof): correct emoji boundary placement for all 5 call sites
* [AI] fix(proof): add curly braces per eslint curly rule in proof script
* [AI] fix(proof): session emoji at actual corpus 280-char boundary
* test: streamline memory dreaming UTF-16 coverage
---------
Co-authored-by: hailory <hailory@xydigit.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(acp): keep session update text truncation surrogate-safe
acp-projector's truncateText function used String.prototype.slice(0, N)
on session update text that is sent to AI providers in the prompt context.
When the truncation boundary falls inside a UTF-16 surrogate pair (emoji,
CJK extended), the resulting string contains a dangling surrogate that
can corrupt the prompt.
Replace .slice(0, maxChars-1) with truncateUtf16Safe(input, maxChars-1)
so the truncation point always falls on a complete code-point boundary.
* fix: make ACP projection truncation UTF-16 safe
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
The bounded task label and progress summary shown for child ACP background runs used raw String.slice, which can split a surrogate pair at the 160/240-char boundaries and surface U+FFFD in requester-facing task status. Part of the UTF-16 safety sweep.
* fix(gateway): keep session title and preview text truncation UTF-16 safe
* test(gateway): cover session UTF-16 boundaries in place
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(ui): keep clampText/truncateText surrogate-safe at emoji boundaries
clampText and truncateText in format.ts used String.prototype.slice(0, N)
to truncate user-visible text. When the truncation boundary falls inside
a UTF-16 surrogate pair (emoji, CJK extended, etc.), the resulting
string contains a dangling surrogate that browsers render as U+FFFD (�).
Replace .slice(0, ...) with truncateUtf16Safe(...) which skips past
incomplete surrogate pairs so the truncation point always falls on a
code-point boundary.
Affected UI surfaces: markdown rendering, command descriptions, skill
summaries, exec-approval rules, tool-stream output previews.
Add surrogate-safe truncation test cases for both functions using a
real emoji at the exact truncation boundary.
* test(ui): tighten UTF-16 truncation coverage
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(agents): keep steering metadata truncation UTF-16 safe
Replace raw string slicing with truncateUtf16Safe in promptLiteral so
emoji and CJK surrogate pairs in steering queue metadata are not split
mid-pair at the 500-char MAX_METADATA_CHARS cap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(agents): add emoji surrogate boundary regression for steering metadata
Drive buildMergedAgentSteeringPrompt with a task label placing an emoji
at the 500-char MAX_METADATA_CHARS boundary to prove truncateUtf16Safe
preserves the surrogate pair instead of splitting it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(agents): assert steering truncation output
* test(agents): remove useless string concat in emoji boundary test
---------
Co-authored-by: Alix-007 <li.long15@xydigit.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: chengzhichao-xydt <chengzhichao-xydt@users.noreply.github.com>
* fix: keep emoji intact at remaining text truncation boundaries
Replace .slice(0, N) with truncateUtf16Safe() in five remaining
user-visible text truncation sites that were missed by earlier
UTF-16 safety sweeps:
- restart.ts: restart reason text (2 sites)
- chat-composer.ts: reply target text preview
- chat-thread.ts: thread text preview + message text extraction
All sites truncate user-generated content where emoji or other
supplementary-plane characters could produce lone surrogates.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: complete UTF-16-safe restart and chat boundaries
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Adds GitHub Enterprise data-residency support to the existing bundled GitHub Copilot provider.
Maintainer proof:
- GitHub CI green on head 54010a6538
- `check-lint`, `check-additional-extension-bundled`, and `check-shrinkwrap` passed in CI
- local `pnpm lint:extensions:bundled`, `pnpm lint`, and focused GitHub Copilot Vitest passed
- AWS Crabbox proof passed
- live microsoft.ghe.com device-flow/token-exchange/model-catalog proof passed
Co-authored-by: Tobias Oort <tobias.oort@ict.nl>
Co-authored-by: Gio Della-Libera <235387111+giodl73-repo@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
