## Summary
- Problem: `normalizeDirectiveWhitespace` applied whitespace-collapsing regexes globally, including inside fenced code blocks (` ``` ` / `~~~`) and indent-code-blocks (4-space / tab), corrupting indentation in assistant replies that contain code snippets
- Why it matters: Any language where indentation is significant (Python, Go, YAML, etc.) or visually meaningful would render incorrectly after stripping inline directive tags
- What changed: Stash code blocks under a Unicode private-use sentinel (`\uE000`) before normalization, run the existing prose regexes on the masked text, then restore the original blocks verbatim
- What did NOT change: All prose normalization rules are retained as-is (`\r\n`, multi-space collapse, leading blank-line strip, trailing whitespace, 3+ newline fold)
## Change Type
- [x] Bug fix
## Scope
- [ ] Gateway / orchestration
## Root Cause
- Root cause: Prose whitespace regexes were applied to the full text string with no awareness of Markdown code block boundaries
- Missing detection / guardrail: No tests covered indented content inside fenced blocks
- Contributing context: Directive tag stripping (`[[reply_to_current]]`, `[[audio_as_voice]]`) is applied before delivery, making the normalization step a silent corruption point for code-heavy replies
## Regression Test Plan
- Coverage level that should have caught this:
- [x] Unit test
- Target test or file: `src/utils/directive-tags.test.ts`
- Scenario the test should lock in: `parseInlineDirectives` with fenced/indent code blocks must preserve all leading whitespace inside those blocks
- Why this is the smallest reliable guardrail: Pure function with deterministic string in/out; no mocks needed
- If no new test is added, why not: 7 new unit tests added
## User-visible / Behavior Changes
Code blocks in assistant replies containing `[[reply_to_current]]` or `[[audio_as_voice]]` directives now retain correct indentation after the directive is stripped.
## Security Impact
- New permissions/capabilities? No
- Secrets/tokens handling changed? No
- New/changed network calls? No
- Command/tool execution surface changed? No
- Data access scope changed? No
## Compatibility / Migration
- Backward compatible? Yes
- Config/env changes? No
- Migration needed? No
Co-Authored-By: Codemax <codemax@binance.com>
- Two-pass line splitting: first slice at maxChars (unchanged for Latin),
then re-split only CJK-heavy segments at chunking.tokens. This preserves
the original ~800-char segments for ASCII lines while keeping CJK chunks
within the token budget.
- Narrow surrogate-pair adjustment to CJK Extension B+ range (D840–D87E)
only, so emoji surrogate pairs are not affected. Mixed CJK+emoji text
is now handled consistently regardless of composition.
- Add tests: emoji handling (2), Latin backward-compat long-line (1).
Addresses Codex P1 (oversized CJK segments) and P2s (Latin over-splitting,
emoji surrogate inconsistency).
- Use code-point length instead of UTF-16 length in estimateStringChars()
so that CJK Extension B+ surrogate pairs (U+20000+) are counted as 1
character, not 2 (fixes ~25% overestimate for rare characters).
- Change long-line split step from maxChars to chunking.tokens so that
CJK lines are sliced into token-budget-sized segments instead of
char-budget-sized segments that produce ~4x oversized chunks.
- Add tests for both fixes: surrogate-pair handling and long CJK line
splitting.
Addresses review feedback from Greptile and Codex bots.
The QMD memory system uses a fixed 4:1 chars-to-tokens ratio for chunk
sizing, which severely underestimates CJK (Chinese/Japanese/Korean) text
where each character is roughly 1 token. This causes oversized chunks for
CJK users, degrading vector search quality and wasting context window space.
Changes:
- Add shared src/utils/cjk-chars.ts module with CJK-aware character
counting (estimateStringChars) and token estimation helpers
- Update chunkMarkdown() in src/memory/internal.ts to use weighted
character lengths for chunk boundary decisions and overlap calculation
- Replace hardcoded estimateTokensFromChars in the context report
command with the shared utility
- Add 13 unit tests for the CJK estimation module and 5 new tests for
CJK-aware memory chunking behavior
Backward compatible: pure ASCII/Latin text behavior is unchanged.
Closes#39965
Related: #40216
* fix: preserve indentation when stripping reply directives
* fix: preserve word boundaries when stripping reply directives
* fix: drop separator space after leading reply directives
* fix: preserve indentation when stripping reply directives (#55960) (thanks @Nanako0129)
---------
Co-authored-by: Ayaan Zaidi <hi@obviy.us>
* fix: make cleanup "keep" persist subagent sessions indefinitely
* feat: expose subagent session metadata in sessions list
* fix: include status and timing in sessions_list tool
* fix: hide injected timestamp prefixes in chat ui
* feat: push session list updates over websocket
* feat: expose child subagent sessions in subagents list
* feat: add admin http endpoint to kill sessions
* Emit session.message websocket events for transcript updates
* Estimate session costs in sessions list
* Add direct session history HTTP and SSE endpoints
* Harden dashboard session events and history APIs
* Add session lifecycle gateway methods
* Add dashboard session API improvements
* Add dashboard session model and parent linkage support
* fix: tighten dashboard session API metadata
* Fix dashboard session cost metadata
* Persist accumulated session cost
* fix: stop followup queue drain cfg crash
* Fix dashboard session create and model metadata
* fix: stop guessing session model costs
* Gateway: cache OpenRouter pricing for configured models
* Gateway: add timeout session status
* Fix subagent spawn test config loading
* Gateway: preserve operator scopes without device identity
* Emit user message transcript events and deduplicate plugin warnings
* feat: emit sessions.changed lifecycle event on subagent spawn
Adds a session-lifecycle-events module (similar to transcript-events)
that emits create events when subagents are spawned. The gateway
server.impl.ts listens for these events and broadcasts sessions.changed
with reason=create to SSE subscribers, so dashboards can pick up new
subagent sessions without polling.
* Gateway: allow persistent dashboard orchestrator sessions
* fix: preserve operator scopes for token-authenticated backend clients
Backend clients (like agent-dashboard) that authenticate with a valid gateway
token but don't present a device identity were getting their scopes stripped.
The scope-clearing logic ran before checking the device identity decision,
so even when evaluateMissingDeviceIdentity returned 'allow' (because
roleCanSkipDeviceIdentity passed for token-authed operators), scopes were
already cleared.
Fix: also check decision.kind before clearing scopes, so token-authenticated
operators keep their requested scopes.
* Gateway: allow operator-token session kills
* Fix stale active subagent status after follow-up runs
* Fix dashboard image attachments in sessions send
* Fix completed session follow-up status updates
* feat: stream session tool events to operator UIs
* Add sessions.steer gateway coverage
* Persist subagent timing in session store
* Fix subagent session transcript event keys
* Fix active subagent session status in gateway
* bump session label max to 512
* Fix gateway send session reactivation
* fix: publish terminal session lifecycle state
* feat: change default session reset to effectively never
- Change DEFAULT_RESET_MODE from "daily" to "idle"
- Change DEFAULT_IDLE_MINUTES from 60 to 0 (0 = disabled/never)
- Allow idleMinutes=0 through normalization (don't clamp to 1)
- Treat idleMinutes=0 as "no idle expiry" in evaluateSessionFreshness
- Default behavior: mode "idle" + idleMinutes 0 = sessions never auto-reset
- Update test assertion for new default mode
* fix: prep session management followups (#50101) (thanks @clay-datacurve)
---------
Co-authored-by: Tyler Yust <TYTYYUST@YAHOO.COM>
