New AI Text-to-Video Model powered by Stable Diffusion + ControlNet

Text2Video-Zero, a first zero-shot text-to-video synthesis diffusion framework, that enables low cost yet high-quality and consistent video generation with only pre-trained text-to-image diffusion models without any training on videos or optimization! Text2Video-Zero also naturally supports cool extension works of pre-trained text-to-image models such as Instruct Pix2Pix, ControlNet and DreamBooth, and based on which we present Video Instruct Pix2Pix, Pose Conditional, Edge Conditional and, Edge Conditional and DreamBooth Specialized applications.

[ysy]

thanks for keeping us on the cutting edge. we hope someone is going to make this available for A1111

[Henryfroio]

It’s been only a week since t2v was release, and look how much it has already evolved

[technoprincess]

now i can't see the demo interface like you, when i get access to it, i just see all codes and some words "runtime error..."

[schlingel]

can you please make a deep inside into the method section from that paper?