TUTORIAL - Runway Multi-Speaker Lip Sync

Runway allows you to lip sync up to four characters in an image or video. There are some limitations, but this is a huge step forward for cinematic storytelling using AI. This in-depth tutorial will show you how to use multi-speaker lip sync and cover some tips and tricks. Hopefully this demo saves you time and Runway credits!

Subscribe to my podcast where I interview AI creators:

00:00 Introduction
00:53 How to use Runway's multi-speaker lip sync
05:02 Trying faces in profile
07:25 Images with more than 2 people speaking (4 total)
09:58 Trying inhuman face (non-human facial features)
10:40 Making characters in a video talk
13:07 Making animated characters talk
14:37 Have you heard about my AI podcast "How I Generate It"?
15:10 Here's the movie we made: "Runway and Chill"

#runway #runwayml #lipsync #aivideocreation #aitutorial #futureoffilm #aivideo #aicharacters

[wholeness]

That animation was a winner l, been waiting for someone to nail that. Thanks

[Smirk_Station]

Just got the unlimited! “Stay up til 3 generating videos”. You got me!!

[massimilianosessantini]

I use Runway and Hedra for lip sync. Runway team MUST take a look at Hedra because is awesome. Runway lip sync works better with an animated video of a person it is too static with a sigle pic (you must have a pay account), hedra uses a single pic of a person and gives you back all body and facial animation with lip sync and AI voices (512x512 resolution, free account).

[QueenOneStudios]

I love that Luma + Runway walking/talking scene! I totally get how that's an aha moment for you. Appreciate all your videos and enthusiasm 🙏

[DirectorDanielMason]

Since people are facing forward, you could probably split screen and intercut the different jury members, so that more than four talk, by recording it twice (for now cropping out the first 4 so the next 4 are the “new” first 4).

Then splice the shots together in the order you want, blending the portions of the shots together in the video editor or any program with compositing (perhaps something free like the basic Da Vinci Resolve:).

Maybe when blending together you can take reaction shots from different takes at different moments, so everyone stays lively (feathering the edges between characters to blend the whole scene together and keep it lively/to sell it as real).

Very good tutorial!:)

[mistertiggy]

This was a really well developed video and to use Luma to tie it all into that one scene was next level. Luma uses mid journey which is the best image generator in my opinion

[CharlotteNguyen-rj]

hi, do you know hơ to make the characters speak at the same time? im struggling

[TashaCaufield]

Great epsiode. Always nice to see a tutorial on some of the lesser-talked about but massively important features on these apps 🤗

[CineFlavor]

I just uploaded a short test of me morphing into Tony Soprano and then using LivePortrait to animate the face exactly how I wanted it to. And it works 10x better and looks more realistic. But the only issue is that you have to track it to your face in After Effects or another software

[thenintendohub]

really enjoyed this, and learned a lot too. The link to your podcast doesn't seem to be working or is that just me? thanks

[taoprompts]

oh wow, I didn't realize it could do multiple people, that's pretty cool

[justshredit]

Is there a way to lip sync to voice tracks you’ve recorded already? Thinking of using it to make an avatar for a speaker in a film. Any suggestions?

[FilmSpook]

Very Awesome, thanks Brother!! Just subscribed!!

[planetofshireen]

i didn't know about this feature . thanks for making this video . a question- can we upload our own voice in it like the voice cloning feature . can we do that?

[BabylonBaller]

If it cant handle a slightly angled face like at 14:15 then its not ready for filmmakers just yet as over the shoulder shot is the core of storytelling and this may not get that right however were headed in a positive direction for sure. Another year or two and wamn

[richardginocchio]

Hi, I really appreciate your videos. I had a question about lip sync if you can help me.
I until a year ago used to use an app (reface app) that had lipsync, you could talk over a video for a few seconds making the people in the video say what you wanted. Some time ago they removed lip sync and I am not finding a viable alternative. Can you tell me if runway gen 3 has the ability to insert its own video and give the characters in the video their own phrases to say?
If runway doesn't do this can you tell me if there are any alternatives? thanks for everything.

[Bartetmedia]

It's a great tool to start with but they need to make it better cause it's a little glitchy.

[NEURAMos]

Lip Sync is a cool tool, but it's a shame it only works with human faces.

[Nidaqaiser]

Can anyone provide me with runway umlimited for lip syn?

[PTISUPPORT-xi]

It's free or paid
If it's paid how much subscription?