mirror of
https://github.com/openclaw/openclaw.git
synced 2026-03-30 19:32:27 +00:00
- Use code-point length instead of UTF-16 length in estimateStringChars() so that CJK Extension B+ surrogate pairs (U+20000+) are counted as 1 character, not 2 (fixes ~25% overestimate for rare characters). - Change long-line split step from maxChars to chunking.tokens so that CJK lines are sliced into token-budget-sized segments instead of char-budget-sized segments that produce ~4x oversized chunks. - Add tests for both fixes: surrogate-pair handling and long CJK line splitting. Addresses review feedback from Greptile and Codex bots.