* fix: sendPolicy deny suppresses delivery, not inbound processing (#53328)
Previously, sendPolicy "deny" returned early before the agent dispatch,
preventing the agent from ever seeing the message. This broke the use
case of an agent listening on WhatsApp groups with sendPolicy: deny to
read messages without replying — the agent couldn't read them at all.
Move the deny gate from before the agent dispatch to after it. The agent
now processes inbound messages normally (context, memory, tool calls),
but all outbound delivery paths are suppressed: final replies, tool
results, block replies, working status, plan updates, typing indicators,
and TTS payloads.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: propagate sendPolicy to ACP tail dispatch instead of hardcoded allow
The ACP tail dispatch path (ctx.AcpDispatchTailAfterReset) was passing
sendPolicy: "allow" unconditionally, which would bypass delivery
suppression in a /reset <tail> turn when the session has sendPolicy deny.
Pass through the resolved sendPolicy so the tail dispatch respects it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard before_dispatch hook and ACP tail dispatch under sendPolicy deny
before_dispatch handled replies were leaking through sendFinalPayload
before the suppressDelivery guard was checked. ACP tail dispatch (from
/new <tail>) was being rejected by acp-runtime.ts deny checks instead
of proceeding with delivery suppression handled downstream.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* auto-reply: propagate deny suppression to reply_dispatch
* fix(acp): suppress onReplyStart when user delivery is denied
When sendPolicy resolves to "deny", ACP tail dispatch still invoked
onReplyStart via startReplyLifecycle before the suppressUserDelivery
check. Channels wire onReplyStart to typing indicators, so deny-scoped
sessions could still emit outbound typing events on /reset <tail>
flows and command bypass paths.
Gate startReplyLifecycleOnce on suppressUserDelivery so the lifecycle
is marked started but the callback is skipped. Payload delivery was
already suppressed; this closes the typing-indicator leak flagged by
Codex review (PR #65461 P1/P2).
* fix(acp): route non-tail deny turns through ACP when suppression is wired
tryDispatchAcpReplyHook was returning early for non-tail, non-command ACP
turns under sendPolicy: "deny", causing ACP-bound sessions to fall back
to the embedded reply path instead of flowing through acpManager.runTurn.
That diverged ACP session state, tool calls, and memory whenever
delivery suppression was active.
Now the early-return only fires when sendPolicy is "deny" AND the event
lacks suppressUserDelivery — i.e., when downstream delivery suppression
is not wired up. When suppressUserDelivery is set, dispatch-acp-delivery
already drops outbound sends (see onReplyStart / deliver guards), so ACP
can safely run the turn with state consistency preserved.
Existing behavior preserved:
- Command bypass still overrides deny
- Tail dispatch still overrides deny
- Plain-text deny turns without suppression still short-circuit
Addresses Codex bot P1 feedback on #65461.
* fix: gate empty-body typing indicator behind suppressTyping (#53328)
* fix: guard plugin-binding + fast-abort outbound paths under sendPolicy deny
The original PR computed suppressDelivery inside the try block, which was
after two outbound paths:
1. The plugin-owned binding block (sendBindingNotice calls for
unavailable/declined/error outcomes, plus the plugin's own "handled"
outcome) ran before the suppressDelivery flag existed, so plugin
notices still leaked under deny.
2. The fast-abort path dispatched "Agent was aborted." via
routeReplyToOriginating / sendFinalReply before the flag existed.
Move resolveSendPolicy() above the plugin-binding block so suppressDelivery
covers every outbound path downstream, matching the PR description's claim
that "all outbound paths are guarded by the flag."
Plugin-bound inbound handling under deny: plugin handlers can emit
outbound replies we cannot rewind, so skip the claim hook entirely under
deny and fall through to normal (suppressed) agent processing.
touchConversationBindingRecord still runs so binding activity stays
tracked.
Fast-abort under deny: still run the abort and record the completed
state, just don't emit the abort reply.
Tests:
- suppresses the fast-abort reply under sendPolicy deny
- delivers the fast-abort reply normally when sendPolicy is allow
(regression guard)
- skips plugin-bound claim hook under deny and falls through to
suppressed agent dispatch
Addresses Codex review findings on PR #65461.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Lobster <lobster@shahine.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Feat: LM Studio Integration
* Format
* Support usage in streaming true
Fix token count
* Add custom window check
* Drop max tokens fallback
* tweak docs
Update generated
* Avoid error if stale header does not resolve
* Fix test
* Fix test
* Fix rebase issues
Trim code
* Fix tests
Drop keyless
Fixes
* Fix linter issues in tests
* Update generated artifacts
* Do not have fatal header resoltuion for discovery
* Do the same for API key as well
* fix: honor lmstudio preload runtime auth
* fix: clear stale lmstudio header auth
* fix: lazy-load lmstudio runtime facade
* fix: preserve lmstudio shared synthetic auth
* fix: clear stale lmstudio header auth in discovery
* fix: prefer lmstudio header auth for discovery
* fix: honor lmstudio header auth in warmup paths
* fix: clear stale lmstudio profile auth
* fix: ignore lmstudio env auth on header migration
* fix: use local lmstudio setup seam
* fix: resolve lmstudio rebase fallout
---------
Co-authored-by: Frank Yang <frank.ekn@gmail.com>
* video_generate: add providerOptions, inputAudios, and imageRoles
- VideoGenerationSourceAsset gains an optional `role` field (e.g.
"first_frame", "last_frame"); core treats it as opaque and forwards it
to the provider unchanged.
- VideoGenerationRequest gains `inputAudios` (reference audio assets,
e.g. background music) and `providerOptions` (arbitrary
provider-specific key/value pairs forwarded as-is).
- VideoGenerationProviderCapabilities gains `maxInputAudios`.
- video_generate tool schema adds:
- `imageRoles` array (parallel to `images`, sets role per asset)
- `audioRef` / `audioRefs` (single/multi reference audio inputs)
- `providerOptions` (JSON object passed through to the provider)
- `MAX_INPUT_IMAGES` bumped 5 → 9; `MAX_INPUT_AUDIOS` = 3
- Capability validation extended to gate on `maxInputAudios`.
- runtime.ts threads `inputAudios` and `providerOptions` through to
`provider.generateVideo`.
- Docs and runtime tests updated.
Made-with: Cursor
* docs: fix BytePlus Seedance capability table — split 1.5 and 2.0 rows
1.5 Pro supports at most 2 input images (first_frame + last_frame);
2.0 supports up to 9 reference images, 3 videos, and 3 audios.
Provider notes section updated accordingly.
Made-with: Cursor
* docs: list all Seedance 1.0 models in video-generation provider table
- Default model updated to seedance-1-0-pro-250528 (was the T2V lite)
- Provider notes now enumerate all five 1.0 model IDs with T2V/I2V capability notes
Made-with: Cursor
* video_generate: address review feedback (P1/P2)
P1: Add "adaptive" to SUPPORTED_ASPECT_RATIOS so provider-specific ratio
passthrough (used by Seedance 1.5/2.0) is accepted instead of throwing.
Update error message to include "adaptive" in the allowed list.
P1: Fix audio input capability default — when a provider does not declare
maxInputAudios, default to 0 (no audio support) instead of MAX_INPUT_AUDIOS.
Providers must explicitly opt in via maxInputAudios to accept audio inputs.
P2: Remove unnecessary type cast in imageRoles assignment; VideoGenerationSourceAsset
already declares role?: string so a non-null assertion suffices.
P2: Add videoRoles and audioRoles tool parameters, parallel to imageRoles,
so callers can assign semantic role hints to reference video and audio assets
(e.g. "reference_video", "reference_audio" for Seedance 2.0).
Made-with: Cursor
* video_generate: fix check-docs formatting and snake_case param reading
Made-with: Cursor
* video_generate: clarify *Roles are parallel to combined input list (P2)
Made-with: Cursor
* video_generate: add missing duration import; fix corrupted docs section
Made-with: Cursor
* video_generate: pass mode inputs to duration resolver; note plugin requirement (P2)
Made-with: Cursor
* plugin-sdk: sync new video-gen fields — role, inputAudios, providerOptions, maxInputAudios
Add fields introduced by core in the PR1 batch to the public plugin-sdk
mirror so TypeScript provider plugins can declare and consume them
without type assertions:
- VideoGenerationSourceAsset.role?: string
- VideoGenerationRequest.inputAudios and .providerOptions
- VideoGenerationModeCapabilities.maxInputAudios
The AssertAssignable bidirectional checks still pass because all new
fields are optional; this change makes the SDK surface complete.
Made-with: Cursor
* video-gen runtime: skip failover candidates lacking audio capability
Made-with: Cursor
* video-gen: fall back to flat capabilities.maxInputAudios in failover and tool validation
Made-with: Cursor
* video-gen: defer audio-count check to runtime, enabling fallback for audio-capable candidates
Made-with: Cursor
* video-gen: defer maxDurationSeconds check to runtime, enabling fallback for higher-cap candidates
Made-with: Cursor
* video-gen: add VideoGenerationAssetRole union and typed providerOptions capability
Introduces a canonical VideoGenerationAssetRole union (first_frame,
last_frame, reference_image, reference_video, reference_audio) for the
source-asset role hint, and a VideoGenerationProviderOptionType tag
('number' | 'boolean' | 'string') plus a new capabilities.providerOptions
schema that providers use to declare which opaque providerOptions keys
they accept and with what primitive type.
Types are additive and backwards compatible. The role field accepts both
canonical union values and arbitrary provider-specific strings via a
`VideoGenerationAssetRole | (string & {})` union, so autocomplete works
for the common case without blocking provider-specific extensions.
Runtime enforcement of providerOptions (skip-in-fallback, unknown key
and type mismatch) lands in a follow-up commit.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: enforce typed providerOptions schema via skip-in-fallback
Adds `validateProviderOptionsAgainstDeclaration` in the video-generation
runtime and wires it into the `generateVideo` candidate loop alongside
the existing audio-count and duration-cap skip guards.
Behavior:
- Candidates with no declared `capabilities.providerOptions` skip any
non-empty providerOptions payload with a clear skip reason, so a
provider that would ignore `{seed: 42}` and succeed without the
caller's intent never gets reached.
- Candidates that declare a schema reject unknown keys with the list
of accepted keys in the error.
- Candidates that declare a schema reject type mismatches (expected
number/boolean/string) with the declared type in the error.
- All skip reasons push into `attempts` so the aggregated failure
message at the end of the fallback chain explains exactly why each
candidate was rejected.
Also hardens the tool boundary: `providerOptions` that is not a plain
JSON object (including bogus arrays like `["seed", 42]`) now throws a
`ToolInputError` up front instead of being cast to `Record` and
forwarded with numeric-string keys.
Consistent with the audio/duration skip-in-fallback pattern introduced
by yongliang.xie in earlier commits on this branch.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: harden *Roles parity + document canonical role values
Replaces the inline `parseRolesArg` lambda with a dedicated
`parseRoleArray` helper that throws a ToolInputError when the caller
supplies more roles than assets. Off-by-one alignment mistakes in
`imageRoles` / `videoRoles` / `audioRoles` now fail loudly at the tool
boundary instead of silently dropping trailing roles.
Also tightens the schema descriptions to document the canonical
VideoGenerationAssetRole values (first_frame, last_frame, reference_*)
and the skip-in-fallback contract on providerOptions, and rejects
non-array inputs to any `*Roles` field early rather than coercing them
to an empty list.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: surface dropped aspectRatio sentinels in ignoredOverrides
"adaptive" and other provider-specific sentinel aspect ratios are
unparseable as numeric ratios, so when the active provider does not
declare the sentinel in caps.aspectRatios, `resolveClosestAspectRatio`
returns undefined and the previous code silently nulled out
`aspectRatio` without surfacing a warning.
Push the dropped value into `ignoredOverrides` so the tool result
warning path ("Ignored unsupported overrides for …") picks it up, and
the caller gets visible feedback that the request was dropped instead
of a silent no-op. Also corrects the tool-side comment on
SUPPORTED_ASPECT_RATIOS to describe actual behavior.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: surface declared providerOptions + maxInputAudios in action=list
`video_generate action=list` now includes the declared providerOptions
schema (key:type) per provider, so agents can discover which opaque
keys each provider accepts without trial and error. Both mode-level and
flat-provider providerOptions declarations are merged, matching the
runtime lookup order in `generateVideo`.
Also surfaces `maxInputAudios` alongside the other max-input counts for
completeness — previously the list output did not expose the audio cap
at all, even though the tool validates against it.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: warn once per request when runtime skips a fallback candidate
The skip-in-fallback guards (audio cap, duration cap, providerOptions)
all logged at debug level, which meant operators had no visible signal
when the primary provider was silently passed over in favor of a
fallback. Add a first-skip log.warn in the runtime loop so the reason
for the first rejection is surfaced once per request, and leave the
rest of the skip events at debug to avoid flooding on long chains.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: cover new tool-level behavior with regression tests
Adds regression tests for:
- providerOptions shape rejection (arrays, strings)
- providerOptions happy-path forwarding to runtime
- imageRoles length-parity guard
- *Roles non-array rejection
- positional role attachment to loaded reference images
- audio data: URL templated rejection branch
- aspectRatio='adaptive' acceptance and forwarding
- unsupported aspectRatio rejection (mentions 'adaptive' in the error)
All eight new cases run in the existing video-generate-tool suite and
use the same provider-mock pattern already established in the file.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* video-gen: cover runtime providerOptions skip-in-fallback branches
Adds runtime regression tests for the new typed-providerOptions guard:
- candidates without a declared providerOptions schema are skipped
when any providerOptions is supplied (prevents silent drop)
- candidates that declare a schema skip on unknown keys with the
accepted-key list surfaced in the error
- candidates that declare a schema skip on type mismatches with the
declared type surfaced in the error
- end-to-end fallback: openai (no providerOptions) is skipped and
byteplus (declared schema) accepts the same request, with an
attempt entry recording the first skip reason
Also updates the existing 'forwards providerOptions to the provider
unchanged' case so the destination provider declares the matching
typed schema, and wires a `warn` stub into the hoisted logger mock
so the new first-skip log.warn call path does not blow up.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* changelog: note video_generate providerOptions / inputAudios / role hints
Adds an Unreleased Changes entry describing the user-visible surface
expansion for video_generate: typed providerOptions capability,
inputAudios reference audio, per-asset role hints via the canonical
VideoGenerationAssetRole union, the 'adaptive' aspect-ratio sentinel,
maxInputAudios capability, and the relaxed 9-image cap.
Credits the original PR author.
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
* byteplus: declare providerOptions schema (seed, draft, camerafixed) and forward to API
Made-with: Cursor
* byteplus: fix camera_fixed body field (API uses underscore, not camerafixed)
Made-with: Cursor
* fix(byteplus): normalize resolution to lowercase before API call
The Seedance API rejects resolution values with uppercase letters —
"480P", "720P" etc return InvalidParameter, while "480p", "720p"
are accepted. This was breaking the video generation live test
(resolveLiveVideoResolution returns "480P").
Normalize req.resolution to lowercase at the provider layer before
setting body.resolution, so any caller-supplied casing is corrected
without requiring changes to the VideoGenerationResolution type or
live-test helpers.
Verified via direct API call:
body.resolution = "480P" → HTTP 400 InvalidParameter
body.resolution = "480p" → task created successfully
body.resolution = "720p" → task created successfully (t2v, i2v, 1.5-pro)
body.resolution = "1080p" → task created successfully
Made-with: Cursor
* video-gen/byteplus: auto-select i2v model when input images provided with t2v model
Seedance 1.0 uses separate model IDs for T2V (seedance-1-0-lite-t2v-250428)
and I2V (seedance-1-0-lite-i2v-250428). When the caller requests a T2V model
but also provides inputImages, the API rejects with task_type i2v not supported
on t2v model.
Fix: when inputImages are present and the requested model contains "-t2v-",
auto-substitute "-i2v-" so the API receives the correct model. Seedance 1.5 Pro
uses a single model ID for both modes and is unaffected by this substitution.
Verified via live test: both mode=generate and mode=imageToVideo pass for
byteplus/seedance-1-0-lite-t2v-250428 with no failures.
Co-authored-by: odysseus0 <odysseus0@example.com>
Made-with: Cursor
* video-gen: fix duration rounding + align BytePlus (1.0) docs (P2)
Made-with: Cursor
* video-gen: relax providerOptions gate for undeclared-schema providers (P1)
Distinguish undefined (not declared = backward-compat pass-through) from
{} (explicitly declared empty = no options accepted) in
validateProviderOptionsAgainstDeclaration. Providers without a declared
schema receive providerOptions as-is; providers with an explicit empty
schema still skip. Typed schemas continue to validate key names and types.
Also: restore camera_fixed (underscore) in BytePlus provider schema and
body key (regression from earlier rebase), remove duplicate local
readBooleanToolParam definition now imported from media-tool-shared,
update tests and docs accordingly.
Made-with: Cursor
* video_generate: add landing follow-up coverage
* video_generate: finalize plugin-sdk baseline (#61987) (thanks @xieyongliang)
---------
Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
Co-authored-by: George Zhang <georgezhangtj97@gmail.com>
Co-authored-by: odysseus0 <odysseus0@example.com>
