ComfyUI: FLUX + ControlNet + IPAdapter (SDXL Version) Integration

Learn how to leverage Flux with Controlnet and IPAdapter (SDXL) in ComfyUI. This up-to-date tutorial covers integration tricks.

Workflow:

Resources using in this video:
Matteo channel: @latentvision

#comfyui #flux #controlnet #IPAdapter #sdxl #stablediffusion

[Kirmm]

I'm stuck on Anyline Preprocessor. Just can't import it. Says (IMPORT FAILED). Tried "Try Fix" button, tried Uninstalling and reinstalling but no go. Any tips?

EDIT: OKAY fixed it with AIO Aux Preprocessor.

But now I get an error with ClipVision.

I downloaded the IP-Adapter models into a IPAdapter folder but still no go.

Error occurred when executing IPAdapterUnifiedLoader:

ClipVision model not found.

File "C:\COMFY-UI\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)

File "C:\COMFY-UI\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)

File "C:\COMFY-UI\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))

File "C:\COMFY-UI\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus-main\IPAdapterPlus.py", line 559, in load_models
raise Exception("ClipVision model not found.")

[eveekiviblog]

you can also try with cpds controlnet and add lora for flux. You can use Florence 2 and Olama LLMs to remove style from first text bar. BTW I really expect to see comparison of GGUF (lets say Q8) and NF4 flux models comparison using Union!

[Huang-ujrt]

For a beginner also say that your videos are really very friendly, thank you very much. Because of my professional needs and the high learning threshold of flux, I've been using mimicpc to run flux before, it can load the workflow directly, I just want to download the flux model, and it handles the details wonderfully, but after watching your video, I'm using mimicpc to run flux again finally have a different experience, it's like I'm starting to get started! I feel like I'm starting to get the hang of it.

[Rachelcenter]

5:55 but why do you have positive/negative prompt in the middle of the workflow and then another prompt box on the far left hand side of the workflow?

[fabioroncal]

it just doesnt work... i can get those Krita and the checkpoint. models..it would be more helful if you put the links to all the necessary assets

[filterophilicxx]

which llm did you use to convert image to prompt? is it open source please guide

[Rachelcenter]

when i open the workflow it says "set node, . get node and anyLineProcessor" are missing, so i go to "install missing" and it prompts me to install "anyline", program restarts, i hit refresh on browser and its still missing.

[BrowneNorton-hu]

Thomas Ronald Williams Brenda Lopez Susan

[kronosiii]

Thank you for the tutorial. AnylinePreprocessor by themisto does not load even after update all comfyui and a manual installation,