Multimodal AI Video Relight with IC-Light (ComfyUI, non-AnimateDiff)

Try RunComfy and run this workflow on the Cloud without any installation needed, with lightning fast GPUs!

REDEMPTION INSTRUCTIONS: Sign in to RunComfy → Click your profile at the top right → Select Redeem a coupon.

COUPON CODE: RCABA10 (Expires August 31)

Can't stop / won't stop: video relighting for your product videos, with detail and color transfer, using IC-Light in a non-AnimateDiff environment!

This workflow allows the users to get:
- moving video + moving light mask
- moving video + still light mask
- still frame + moving light mask
- still frame + still light mask (if you really think that's a good idea)

Thanks to LinkedIn's Mauricio Tonon for the RIFE final pass suggestion!

Drag and drop the json workflow file from openart into ComfyUI and install the missing custom nodes, and grab the following models in case you're missing them:

Timestamps:
00:00 - Intro
00:30 - Sponsor: RunComfy
00:46 - Workflow (Why Video Relighting)
01:37 - What it can do
02:38 - How it works, in depth (Inputs)
06:37 - How it works, in depth (Results)
07:13 - How it works, in depth (Generation)
11:06 - Cloud demo via RunComfy
16:25 - Frame Interpolation issues
17:27 - Disabling RAVE
18:18 - Final Considerations
18:59 - Outro

#runcomfy #stablediffusion #stablediffusiontutorial #videorelight #comfyui #comfyuitutorial #relight #iclight

[risunobushi_ai]

Try RunComfy and run this workflow on the Cloud without any installation needed, with lightning fast GPUs!
Visit, and get 10% off for GPU time or subscriptions with the Coupon below.

REDEMPTION INSTRUCTIONS: Sign in to RunComfy → Click your profile at the top right → Select Redeem a coupon.

COUPON CODE: RCABA10 (Expires August 31)

[ВикторияВасилькова-ди]

Oh my god, this morning's discussion on reddit was so heated that you repeated and improved that controversial workflow and provided it for free. Bravo!!!

[yotraxx]

Andrea, you are a genius. Thank you so much for sharing your acknowledgement.

[Zanroff]

I have green screen footage of a person I'd like to key out and subtly relight using this. Perhaps it would be better to use the still photo relighting workflow and use the incremental batch image loaders for the footage, luma matte and lighting footage? I'll split everything up into still frame images. Any suggestions beyond that?

[tonon_AI]

Hey man no problem. Thanks for sharing such nice content.

[DanielPartzsch]

I think you should always use lanczos algorithm in the resizing nodes. Nearest exact gives very pixilated results.

[xandervera]

Love it, can't wait to fiddle with this!!
ALSO what do you have against set/get nodes 😆I hate spaghetti!! Just set and get so everything looks clean and color code them xD

[Scrapemist]

The previews of the result are a bit small. To hide the inconsistency/artefacts?

[kennedysworks]

Dear Andrea Baioni,

The workflow you created is absolutely brilliant and has had a significant impact on my video work. I believe its versatility is truly limitless.

I have one question: Is there a way to adjust the flickering of the lighting in the final video output? Currently, the flickering is consistent regardless of the lighting I use. I wonder if it’s possible to either reduce the flickering or increase it.

If it’s not too much trouble, I would appreciate your advice.

[jonathanperreault]

@risunobushi_ai can you make a video on how to relight flat images like a picture of a wall or floor ? im making textures for video games and id like to remove all lighting from a picture i took or change the lighting bakes in a texture

also would it be possible to isolate different aspects of the lighting a bit like segment anything? im trying to find a way to make diferent maps for my materials

[PaganDeus]

Hi, your user interface is unique, but it causes me an error. I tried to find a solution, but it didn't work out. Maybe you, as the father of this interface, will decide)

Error occurred when executing LoadAndApplyICLightUnet:

IC-Light: Could not patch calculate_weight - IC-Light: The 'calculate_weight' function does not exist in 'lora'