feat: add video generation core infrastructure and extend image generation parameters (#53681)

* feat: add video generation core infrastructure and extend image generation parameters

Add full video generation capability to OpenClaw core:

- New `video_generate` agent tool with support for prompt, duration, aspect ratio,
  resolution, seed, watermark, I2V (first/last frame), camerafixed, and draft mode
- New `VideoGenerationProvider` plugin SDK type and `registerVideoGenerationProvider` API
- New `src/video-generation/` module (types, runtime with fallback, provider registry)
- New `openclaw/plugin-sdk/video-generation` export for external plugins
- 200MB max file size for generated videos (vs default 5MB for images)

Extend image generation with additional parameters:
- `seed`, `watermark`, `guidanceScale`, `optimizePrompt`, `providerOptions`
- New `readBooleanParam()` helper in tool common utilities

Update plugin registry, contracts, and all test mocks to include
`videoGenerationProviders` and `videoGenerationProviderIds`.

Made-with: Cursor

* fix: validate aspect ratio against target provider when model override is set

* cleanup: remove redundant ?? undefined from video/image generate tools

* chore: regenerate plugin SDK API baseline after video generation additions

---------

Co-authored-by: yongliang.xie <yongliang.xie@bytedance.com>
This commit is contained in:
xieyongliang
2026-03-26 09:45:06 +08:00
committed by GitHub
parent 39fbfd9b28
commit 4cb8dde894
37 changed files with 1380 additions and 241 deletions

View File

@@ -30,6 +30,11 @@ export type GenerateImageParams = {
aspectRatio?: string;
resolution?: ImageGenerationResolution;
inputImages?: ImageGenerationSourceImage[];
seed?: number;
watermark?: boolean;
guidanceScale?: number;
optimizePrompt?: boolean;
providerOptions?: Record<string, unknown>;
};
export type GenerateImageRuntimeResult = {
@@ -153,6 +158,11 @@ export async function generateImage(
aspectRatio: params.aspectRatio,
resolution: params.resolution,
inputImages: params.inputImages,
seed: params.seed,
watermark: params.watermark,
guidanceScale: params.guidanceScale,
optimizePrompt: params.optimizePrompt,
providerOptions: params.providerOptions,
});
if (!Array.isArray(result.images) || result.images.length === 0) {
throw new Error("Image generation provider returned no images.");

View File

@@ -31,6 +31,12 @@ export type ImageGenerationRequest = {
aspectRatio?: string;
resolution?: ImageGenerationResolution;
inputImages?: ImageGenerationSourceImage[];
seed?: number;
watermark?: boolean;
guidanceScale?: number;
optimizePrompt?: boolean;
/** Provider-specific options (e.g. sequential generation). */
providerOptions?: Record<string, unknown>;
};
export type ImageGenerationResult = {