Kasucast #21 - Improving Stable Diffusion images with FreeU (SDXL, LCM, Turbo) and ComfyUI API

#sdxl #ComfyUI #comfyui #api #LoRA #LCM #sdxlturbo #freeu #stablediffusion

This is a video about tuning FreeU hyperparameters for Stable Diffusion. I show how to use both the ComfyRoll custom nodes and the ComfyUI API to grid search for the ideal b1, b2, s1, s2 parameters.

Models:

Final hyperparameters:
Normal SDXL kasukanra style finetune:
b1: 1.5
b2: 1.3
s1: 1.1
s2: 0.2

SDXL kasukanra style finetune + LCM LoRA:
b1: 1.6
b2: 1.1
s1: 1.3
s2: 1.1

SDXL kasukanra style finetune + Turbo LoRA:
b1: 1.8
b2: 1.3
s1: 1.6
s2: 1.4

Resources:

Timestamps:
00:00 Introduction
00:56 FreeU paper
03:43 How do b and s value interact
04:52 ComfyRoll nodes
05:47 CivitAI workflows downloads
06:30 ComfyRoll workflows
08:41 ComfyRoll Grid error
09:06 Debugging the grid error
11:07 Trying out ComfyRoll XY list
14:16 Finding a fix for grid error
15:54 ComfyUI API
16:36 ComfyUI API JSON
18:45 How to request ComfyUI API
20:13 Re-evaluating the API JSON file
20:54 Code overview
22:59 Selecting baseline freeU s values
23:46 Handling asynchronous calls as ComfyUI API doesn't have callbacks
24:52 Setting up code environment
26:44 Executing the code
28:01 Analyzing the dynamic b values for normal FreeU
32:23 Narrowing the search window
36:04 Why choose static s values of 1.1 and 0.2? Why not use 0, 0 or 1, 1?
38:14 Dynamic s values
41:52 Getting original seed value
43:22 Pointing out my mistake with my seed value wasn't working
44:57 Comparing with original generations as a sanity check
45:32 Comparing results so far
46:30 Workflow for kohya deep shrink
47:00 LCM workflow
47:31 Dynamic b values for LCM
48:48 Dynamic s values for LCM
50:51 Updated comparison of all results so far
51:22 Extracting SDXL turbo LoRA
52:58 SDXL Turbo LoRA workflow
54:30 Dynamic b values for SDXL Turbo
55:37 Comparing different b1, b2 values for SDXL Turbo
56:11 Dynamic s values for SDXL Turbo
57:00 Final comparison of all results so far

🎉 Social Media:

Images/processes may be fabricated and therefore not real. I am unaware of any illegal activities. Documentation will not be taken as admission of guilt.

[SakuraSuzie]

I'm the creator of Comfyroll Nodes. There is a workaround for the missing last image on the CivitAI post for these nodes. The problem is caused by the processing order. The last image is generating after the image grid is generated. I made a temporary fix for this, but plan a full rework.

[raneahester]

kasukanra: *holds our hand*

US: *smiling like the lost child we are*

[Latentnaut]

Massive video, thank you. In my context, I would love have seen an example with photorealistic portrait, as FreeU seems quite intense in the outcome.

[Soul-rrus]

This was great. Thank you for making it!

[DOntTouCHmYPaNDa]

Awesome, I’m stoked you released a new video!!

[isaacjohn]

Loved the short paper review, in-depth analysis as well as you problem solving journey, I have Subbed to your channel so keep it up.

[morphiussys]

Awesome video, also woohoo for being #1000th view!

[mrti]

good stuff. thanks for sharing! Question: What image explorer are you using?

[kasukanra]

These are the final hyperparameters that worked for me:

Normal SDXL kasukanra style finetune:
b1: 1.5
b2: 1.3
s1: 1.1
s2: 0.2

SDXL kasukanra style finetune + LCM LoRA:
b1: 1.6
b2: 1.1
s1: 1.3
s2: 1.1

SDXL kasukanra style finetune + Turbo LoRA:
b1: 1.8
b2: 1.3
s1: 1.6
s2: 1.4

[daniel]

u should try basic shapes, artworks are very subjective, i would say try, vector shapes or things like grid lines where u can easily spot the flaws

[AnotherComment-rlfv]

you should try on realistic images, artistic comparisons are always subjective.