Commit Graph

3 Commits

Author SHA1 Message Date
ZOOWH
fb702e5cae fix(memory): harden embedding batch workflows (#103472)
* fix(memory): harden embedding batch workflows

Co-authored-by: 徐闻涵0668001344 <xu.wenhan1@xydigit.com>
Co-authored-by: ben.li <li.yang6@xydigit.com>

* test(memory): align batch redaction expectation

* test(memory): tighten batch dependency types

* refactor(google): clarify batch state control flow

* test(google): make batch stage fallback lint-safe

* build(plugin-sdk): refresh memory batch surface budget

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: ben.li <li.yang6@xydigit.com>
2026-07-11 16:43:58 -07:00
solodmd
1ec42c9acc fix(voyage): close response body stream when batch output JSONL parsing throws (#98840)
The batch output file download path creates a readline interface over a
Readable.fromWeb() response body stream. If JSON.parse throws on a malformed
JSONL line, the for-await loop exits via exception but the readline interface
and underlying Readable stream were never explicitly closed or destroyed,
leaving the HTTP response body stream dangling.

Extract the stream reading into , a testable helper
that wraps the iteration in a try-finally so both reader.close() and
inputStream.destroy() are always called, matching the pattern established in
#98493 for the same class of leak.
2026-07-06 06:32:50 +00:00
Alix-007
d3620da3e0 fix(voyage): bound embedding-batch status, error, and non-OK responses (#96608)
The batch status read (fetchVoyageBatchStatus) parsed its response with an
unbounded await res.json(), and the batch error-file read (readVoyageBatchError)
buffered the whole body via await res.text(). On top of that, the non-OK
(4xx/5xx) diagnostic body was still read unbounded: assertVoyageResponseOk did
await res.text() before throwing, and the non-OK output-file branch in
runVoyageEmbeddingBatches did the same. Voyage base URLs are user-supplied and
reachable via SSRF, so a misbehaving or hostile endpoint could stream an
unbounded body into memory on any of these paths before parsing.

Route the status JSON through the shared readProviderJsonResponse, the error
file through readResponseWithLimit, and now the non-OK diagnostic body through
readResponseWithLimit as well, all under a single 16 MiB cap, cancelling the
stream on overflow before decode/parse. assertVoyageResponseOk preserves its
original "${context}: ${status} ${text}" diagnostic shape for under-cap bodies
and throws a bounded "(error body exceeds <N> bytes)" on overflow; the non-OK
output-file branch now reuses it instead of a duplicate unbounded read. The
existing error-file fail-soft handling (formatUnavailableBatchError) is
preserved, so a capped endpoint degrades gracefully. The submit path already
bounds its body via postJsonWithRetry/maxResponseBytes and is left untouched.

Symmetric counterpart to the #96027/#96038 response-limit campaign.
2026-06-25 13:52:36 -04:00