Using SDXL Controlnet with Automatic1111

A quick tutorial about installing and using the new Controlnets for Stable Diffusion SDXL in Automatic1111 (yes, it's finally here!).
Giving you a short introduction about the new SDXL models for Automatic1111, as well as the new Controlnets.

The combination of SDXL and Controlnet allows you to produce stunning, high quality images of people in different poses, while keeping the original image of a person stable to a high extent.

00:00 Introduction to Stable Diffusion SDXL
01:26 Updating Automatic1111 and Controlnet
02:01 Download and install SDXL base + refiner models
03:32 Download and install SDXL Controlnet models into Automatic1111
04:33 Install SDXL Style Selector
05:02 Using SDXL in Automatic1111
08:10 Using SDXL Controlnets in Automatic1111


======================================
Download the SDXL base + refiner models here:
======================================

Download the SDXL Controlnet models here:

or get them from Huggingface:

Install Automatic1111:

Get Controlnet here (or install it via the Autmoatic1111 Extensions tab):

#automatic1111 #stablediffusion #controlnet #sdxl

[XraynPR]

Hi, Ive tried a few Controlnets - the larger ones in filesize seem to really tank the performance (e.g. taking an hour for the image, where I need a minute or so without the CN) and the smaller ones are so-so.

Is that first issue a one time thing, does CN need to load some files in the background?

E.g. Im always often getting this message for CN modules: "RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x2816 and 2560x1536)"

Edit: A restart seems to have helped with the runtime error, maybe it was related to HiRes upscale ...

[ukrolelo]

Mat1 and ma2 shapes error fixed with your model ❤ thx for video

[dkamhaji]

hello! can you tell me the difference between the SDXL/REFINER from Civit with the _FIX - compared to what is in hugging face as sd_xl_base_1.0.safetensors

[milovanribic]

Is there any option here for AI video creation, is there some extension's for that?

[aletergut]

Does anyone know why my SDXL in process image changes completely around 73%? It's like starting from scratch again. Thanks!

[alexpolymath]

Have you changed anything inside settings > control net ->Config file for Control Net models. I have models/cldm_v15.yaml, not sure if I should change it

[hogstoothairsoft]

sadly doesn't work for me, always getting: "ControlNet - WARNING - ControlNet does not support SDXL -- disabling" (using your models and everything up to date)

[thanksfernuthin]

There's no tile resample yet? That's a shame. That's really helpful.

[seancannon]

I assume you meant "1024 x 1024" not "124 x 124"?

[faddlewaddle]

Thank you. I've now lost my Deforum tab.