ComfyUI - Generate Images with Flux & ControlNet SDXL

With this method, you can use SDXL ControlNets like Openpose, Depth, Canny, etc., for image generation and transfer them to Flux.
Since the specific ControlNet model for FLUX has not been released yet, we can use a trick to utilize the SDXL ControlNet models in FLUX, which will help you achieve almost what you want.

FLUX is an advanced image generation model, available in three variants: 
- FLUX.1 [pro] for top-tier performance, 
- FLUX.1 [dev] for efficient non-commercial use,
- FLUX.1 [schnell] for fast local development. These models excel in prompt adherence, visual quality, and output diversity. 

Download the needed files from this address: 

Unet Folder: 

Clip Folder: 

VAE folder: 

***********************************

Comfyui tutorial, Учебное пособие по Comfyui, Comfyui ट्यूटोरियल, Tutoriel Comfyui, Tutorial Comfyui, Comfyui 튜토리얼
Comfyui stable diffusion, Install comfyui, comfyui video, controlnet comfyui, comfyui animateddiff, comfyui sdxl, comfyui upscale, comfyui video to video, comfyui manager, comfyui inpainting, comfyui ipadapter, comfyui faceswap

***********************************

------------------------------------
🌍 SOCIAL 

------------------------------------

#ComfyUI
#Flux

[NeonXXP]

I like he new Flux model it made me learn Comfy UI over the weekend. In a month it will be awesome when the degenerates give it the pony treatment and we get more control nets for things like colour matching and LoRa for consistent characters.

[Huang-ujrt]

I think what you said is great. I am using mimicpc which can also achieve such effect. You can try it for free. In comparison, I think the use process of mimicpc is more streamlined and friendly.

[GamerConnection]

Unless I use the basic model any other model I get RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x2048 and 768x320) (NM I fixed it by downloading the models on the manager)

[AIFuzz]

Isn’t that just an img2img workflow using Flux as the base model?

[eveekiviblog]

Could you please make a video about triggers. Alwys, never, on event.... Like if there is preprocessed image, then no need to use preprocessor again in next generations

[krasen]

is there a way to control this with the inpainting?

[MilesBellas]

All the comments from the Channel owner are hidden ?

[messlav]

I don’t get how theoretically that works. You need to have special controlnet variant for each model, e.g. “pose controlnet flux dev”, you can’t just take controlnet from other model and add it to flux. So for such pipeline we need to wait specific trained controlnet for flux. Am I wrong?

[TailspinMedia]

does controlnet only work with the Schnell model?? not the Dev?

[telecaster]

Hi and thanks but the address not work