Video Generation with AI Gateway

AI Gateway has added video generation, accessible through AI SDK 6 alongside the text and image interfaces already in use. The capability is in beta and open to Pro and Enterprise plans and paid AI Gateway users.

Access paths

Two entry points are available. Programmatic generation runs through AI SDK 6, which keeps the video path on the same interface, authentication flow and observability dashboard used for text and images.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'xai/grok-imagine-video',
  prompt: 'A golden retriever catching a frisbee mid-air at the beach',
});

For experimentation without code, each model page embeds a configurable AI Gateway Playground where providers can be compared, prompts adjusted, and results downloaded. Playground access starts from any video generation model in the model list.

Models and provider capabilities

Four model families ship at launch, across 17 variations:

• Grok Imagine (xAI): fast, strong instruction following, with style transfer for creating and editing video in seconds.
• Wan (Alibaba): reference-based generation and multi-shot storytelling, preserving identity between scenes.
• Kling: image-to-video and native audio; the 3.0 models handle multishot video with automatic scene transitions.
• Veo (Google): high visual fidelity and physics realism, with native audio generation, cinematic lighting and physics.

Four generation types

Video prompting differs from image prompting. Alongside a description of the content, prompts can carry motion cues — camera movement, object actions, timing — and optionally audio direction. Providers expose their distinct capabilities through providerOptions, which select between fundamentally different generation modes; model-specific options are covered in the documentation.

The capabilities currently available across the model creators on AI Gateway are listed below.

Text-to-video

A text prompt produces the video, with the model handling visuals, motion and optionally audio. Typical uses include hyperrealistic, production-quality footage from a simple prompt.

For programmatic generation at scale — on-demand video for an app, platform or content pipeline, without licensing fees or production — the example uses klingai/kling-v2.6-t2v to build a video from a text prompt with a defined aspect ratio and duration.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'klingai/kling-v2.6-t2v',
  prompt: `Wide shot of a rocket lifting off from launch pad at dawn.
    Massive plume of orange fire and white smoke billows outward
    from the base. The rocket rises slowly at first, engines blazing,
    then accelerates upward. Pink and orange sunrise sky
    in the background. Ocean visible in the distance.`,
  aspectRatio: '16:9',
  duration: 5,
  providerOptions: {
    klingai: {
      mode: 'pro',
      sound: 'on',
    },
  },
});

For creative content, a single prompt can yield polished clips with natural motion and cinematic quality for social media, ads or storytelling. A highly specific, descriptive prompt drives google/veo-3.1-generate-001 to produce heavy detail and the exact motion requested.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'google/veo-3.1-generate-001',
  prompt: `Close-up of a great horned owl turning its head slowly.
    Enormous yellow eyes with intricate iris detail.
    Every feather texture visible, from soft facial disc to ear tufts.
    The owl blinks once, deliberately.`
  aspectRatio: '16:9',
});

Image-to-video

Starting from a supplied image, the model controls the initial composition and generates motion from there.

Product photography is one target: the klingai/kling-v2.6-i2v model animates a product image once an image URL and a motion description are passed in the prompt.

iconst { videos } = await generateVideo({
  model: 'klingai/kling-v2.6-i2v',
  prompt: {
    image: blackHoodie,
    text: `The orange tabby cat walks slowly across the black hoodie.
      Warm natural light. Cozy lifestyle scene. Smooth, cinematic.`,
  },
  duration: 5,
  providerOptions: {
    klingai: { mode: 'pro' },
  },
});

The same mode covers animated illustrations, where static artwork gains subtle motion for thematic content or marketing at scale, and lifestyle and product shots — food, beverage and similar subjects — that need light motion for social content.

In the example, a coffee image is rendered into a more interactive video with controlled lighting direction and minute detail.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'alibaba/wan-v2.6-i2v',
  prompt: {
    image: 'https://your-storage.com/coffee-pour.png',
    text: `Coffee swirls gently in the cup, steam rises slowly,
     warm morning light shifts subtly`,
  },
  resolution: '1280x720',
  duration: 3,
});

First and last frame

Supplying both a start and an end state lets the model generate a seamless transition between them — useful for before/after reveals, outfit swaps, product comparisons and changes over time, driven by two uploaded images.

The two states are defined through images used in the prompt and provider options.

With klingai/kling-v3.0-i2v, the start frame goes in image and the end frame in lastFrameImage; the model fills in the transition.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'klingai/kling-v3.0-i2v',
  prompt: {
    image: startFrameDataUrl, // Empty room
    text: `Smooth cinematic transition: The empty loft fills with furniture.
      A green velvet sofa fades into view, followed by a wooden coffee table.
      Potted plants rise from the floor. A patterned rug materializes.
      Framed artwork appears on the walls. Bookshelves on the back wall.
      Gentle, seamless transformation.`,
  },
  duration: 5,
  providerOptions: {
    klingai: {
      lastFrameImage: endFrameDataUrl, // Furnished room
      mode: 'std',
    },
  },
});

Reference-to-video

Given reference videos or images of a person or character, the model extracts appearance and voice to generate new scenes starring the same identity. In the example, two reference images of dogs feed the final video.

alibaba/wan-v2.6-r2v-flash consumes people and characters named in the prompt. For best multi-reference results, Wan recommends addressing them as character1, character2 and so on.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'alibaba/wan-v2.6-r2v-flash',
  prompt:
   `character1 and character2 are playing together on the beach in San Francisco
    with the Golden Gate Bridge in the background, sunny day, waves crashing`,
  resolution: '1280x720',
  duration: 5,
  providerOptions: {
    alibaba: {
      referenceUrls: [shibaImage, yorkieImage],
    },
  },
});

Video editing

Existing video can be transformed by style transfer: pass a video URL and describe the change, and the model applies the new style while retaining the original motion. In the example, xai/grok-imagine-video takes a source video from an earlier generation and reworks it into a watercolor style.

import { experimental_generateVideo as generateVideo } from 'ai';
const { videos } = await generateVideo({
  model: 'xai/grok-imagine-video',
  prompt: `Transform into watercolor painting style, soft flowing brushstrokes,
   paint bleeding at edges, delicate washes of color, artistic and dreamlike`,
  providerOptions:
    xai: {
      videoUrl: dogVideo,
    },
  },
});

Documentation and examples

The Video Generation Documentation covers further examples and configuration options for video models, and the Video Generation Quick Start provides simple getting-started scripts. Model changelogs carry additional examples and prompts for Grok Imagine, Alibaba Wan, Veo and Kling.