If you've been following the AI video space lately, you've probably noticed that it's exploding. New models are coming out every week with better outputs, higher resolution, and faster generation speeds.
Wan2.1 is the newest and most capable open-source video model. It was released last week, and it's topping the leaderboards.
There's a lot to like about Wan2.1:
In this post we'll cover the new models and how to run them with an API.
The model is available on Replicate in a number of different flavors:
The 480p models are great for experimentation because they run faster.
Use 720p if you need a higher resolution.
The 1.3b models are smaller, and designed to run on consumer GPUs.
The 14b model excels at real-world physics, and you can push it to do things most other models struggle with:
Every model on Replicate has a scalable cloud API, and Wan2.1 is no exception.
Here's a code snippet for running the Wan2.1 text-to-video model using the Replicate JavaScript client:
import Replicate from "replicate";
const replicate = new Replicate()
const model = "wavespeedai/wan-2.1-i2v-480p"
const input = {
image: "https://replicate.delivery/pbxt/MZZyui7brAbh1d2AsyPtgPIByUwzSv6Uou8objC7zXEjLySc/1a8nt7yw5drm80cn05r89mjce0.png",
prompt: "A woman is talking",
}
const output = await replicate.run(model, { input })
console.log(output)
The code for the image-to-video model is nearly identical. Just omit the image
input when calling the model:
import Replicate from "replicate"
const replicate = new Replicate()
const model = "wavespeedai/wan-2.1-t2v-480p";
const input = {
prompt: "A woman is talking"
}
const output = await replicate.run(model, { input })
console.log(output.url())
The Wavespeed Wan2.1 models also expose a number of different settings for you to experiment with.
Try experimenting with guide_scale
, shift
and steps
. We've found that a lower guide_scale
and shift
(about 4 and 2) can give lovely realistic videos.
This model wouldn't exist without the work of numerous open-source contributors. We're using WavespeedAI's optimizations to bring you the fastest generations in the world.
Big shout-outs to Alibaba for open sourcing the model, and to @chengzeyi and @wavespeed_ai for working with us to bring you these speeds. ⚡️