* fix: enforce embedding model token limit to prevent 8192 overflow
- Replace EMBEDDING_APPROX_CHARS_PER_TOKEN=1 with UTF-8 byte length
estimation (safe upper bound for tokenizer output)
- Add EMBEDDING_MODEL_MAX_TOKENS=8192 hard cap
- Add splitChunkToTokenLimit() that binary-searches for the largest
safe split point, with surrogate pair handling
- Add enforceChunkTokenLimit() wrapper called in indexFile() after
chunkMarkdown(), before any embedding API call
- Fixes: session files with large JSONL entries could produce chunks
exceeding text-embedding-3-small's 8192 token limit
Tests: 2 new colocated tests in manager.embedding-token-limit.test.ts
- Verifies oversized ASCII chunks are split to <=8192 bytes each
- Verifies multibyte (emoji) content batching respects byte limits
* fix: make embedding token limit provider-aware
- Add optional maxInputTokens to EmbeddingProvider interface
- Each provider (openai, gemini, voyage) reports its own limit
- Known-limits map as fallback: openai 8192, gemini 2048, voyage 32K
- Resolution: provider field > known map > default 8192
- Backward compatible: local/llama uses fallback
* fix: enforce embedding input size limits (#13455) (thanks @rodrigouroz)
---------
Co-authored-by: Tak Hoffman <781889+Takhoffman@users.noreply.github.com>
7f1712c1ba