ComfyUI Tutorial Series: Ep09 - How to Use SDXL ControlNet Union

In Episode 9 of our ComfyUI tutorial series, we explore ControlNet, focusing on the new Union ControlNet for SDXL. This powerful tool unifies multiple models, such as pose or Canny, into one, making it easier to guide your AI in creating images with specific compositions. Learn how to download, install, and integrate the ControlNet Union ProMax model into your workflow, and discover how to stack multiple ControlNet models for more detailed and controlled results.

In this tutorial, you’ll learn:

- How to download and install the SDXL ControlNet Union ProMax model.
- The basics of ControlNet and how it guides AI to create images with specific structures.
- How to update ComfyUI and explore the new interface options.

Download Control Net Union
Place it in ..ComfyUI\models\controlnet folder

Get the custom nodes

Go to Manager - Custom Nodes Manager
Search for venture
Install the node: comfyui-art-venture

Search for Preprocessor
Preprocessors
Install the node: ComfyUI's ControlNet Auxiliary Preprocessorsc

Search for comfyroll
Install the node: Comfyroll Studio
Restart comfyui

Download all the workflows from Discord
look for the channel pixaroma-workflows

Unlock exclusive perks by joining our channel:

Chapters:
00:00 Introduction
00:17 Download the ControlNet Model
00:53 Understanding ControlNet
01:38 ComfyUI and New Interface Overview
02:55 Setting Up Custom Nodes for ControlNet
03:40 Exploring Preprocessors: Canny, Depth, and More
08:52 Integrating ControlNet into Your Workflow
15:13 Stacking Multiple ControlNet Models
18:15 Testing and Fine-Tuning Your Workflow

[pixaroma]

Just found a way to use it without Art-Venture node (for some caused some errors), you can use instead the AIO Aux Preprocessor - it comes with
Update 2: Added also one workflow for canny with no extra custom nodes need it. only the comfyui defaults.

You can now support the channel and unlock exclusive perks by becoming a member:

[DandelionStrawart]

This is a great explanation. The great thing is you always start with the basic model and start adding nodes to that, and it's easy to follow how to do this. Other guys start with a super complicated workflow to start with and you're supposed to follow what they did, it just makes it very complicated. But yours is simple and direct, in the end, you can use Controlnet with comfyui, and you have the control to expand it or keep it simple. Thanks very much for the video.

[Fenkreg]

Thank you for the tutors and ESPECIALLY that you hadn't add background music so I can listen to my own while learning cool stuff :3

[aml]

Once again a not over wordy or hard to follow tutorial. Concise and to the point. Thank you.

[aml]

Just a little something for your help today. It was very kind of you. Thank's once again.

[ivo_tm]

Short, clear and informative. Thank you!

[Fayrus_Fuma]

Again a very useful video, but unfortunately not on my models. It will be very useful for others. Thank you for your work. Your work is very detailed and informative indeed.

[KimikimiYKim]

Hi are you running comfyui now experiencing frequent bugs? I've come to the point where I'm experiencing frequent disconects for no reason after generating too large an image in the 3rd image since comfyui update on August 15th yesterday. Do you think this is a bug or something?

[indigoood]

That's is an amazing skill to operate ComfyUI show by you btw.

[pedrogorilla]

Very good man! And the voice is very natural, how are you doing it?

[danteshume]

Sir, thank you very much! I have finally waited for your update.

[pavelponcar]

Thanks man, lot of things to experiment with

[AndyTune]

What was the file that caused an error with "venture", problem with downloading a file or a file was a problem?

[jennifertsang]

Great video!! Always learning a lot from them.

[robrever]

Cant wait to try! But my hopes are low. I have tried to implement similar control net workflows recently on different machines with the same reaults: a bunch of red errors.

[cncpts]

followed till you mentioned "Art-Venture" I always had an issue with their custom nodes, and getting updated, instead I will subscribe and wait until something similar comes up for flux!

[Raphy_Afk]

Amazing video ! Could you please make a video on how to use a controlnet to inpaint a face from a reference image ?

[MrRaycaster]

Have to say first your videos are top notch and as others have noted CONCISE! I’m very interested in controlnet as I can’t seem to find a way to implement Canny with xsarchirectual_v11.ckpt checkpoint. The checkpoint is perfect for interior design but I hope to add basic “sketchup” designs to guide it. Of course Im getting errors because the control does not match as you noted. Is there a workable controlnet or another ckpt suitable for architecture images? Thank you.

[CsokaErno]

Does ControlNet work with ForgeUI properly or is it better to use with Comfy? Another one. You made me love ForgeUI, I think it is the best for me. But ComfyUI seems more flexible. Do you think it is better to use Comfy or I can stay with ForgeUI? Besides it seems the most improvement go with Comfy. Do you think it is a better choice?

[Boiller]

How would you use pixel perfect in Comfy? There is a node for it, but I'm not exactly sure where to plug it into in the comfy graph.