Real time Stable Diffusion in TouchDesigner

Video Tutorial on TouchDiffusion: Interactive Real-Time Generation using StreamDiffusion and TensorRT in TouchDesigner

Socials:

00:00 - Intro
00:09 - TouchDiffusion features
00:42 - Beginning of installation
00:50 - Python install
01:08 - Git install
01:18 - Cuda Toolkit install
01:38 - TouchDiffusion install
01:55 - where python
02:13 - webui install
03:10 - Fix for error in pop up window
03:27 - Model downloading & engine preparation
04:45 - SD Turbo engine
05:07 - TouchDiffusion settings
05:46 - Usage example
07:13 - Parameters explanation

[bujhhhhhggyfv]

amazing job explaining the entire process...

[Remvisuals]

instnaly fast and so nice to be in 512, congrats on this one, cant wait for more features!

[alessandrosaracino]

this is great! lean and clean, thank you!

[MohamedAli-nbqk]

THIS video talks about the portable version and NOT the manual installation. If you try following this installation for the manual installation, you are going to face errors related to models and engines not being found, for example "cannot access local variable 'pipe' where it is not associated with a value", because the paths for the portable version and the manual version are slightly different. That's why I recommend using the portable version

[marlus]

Thanks for this great solution! Amazing performance on 4070 12GB! SD Turbo and 1 sample step. I will test making other engine now. 3 sample steps were slow on 4070.

[zackel-baz]

Hey VJ SCHOOL!thanks for the tutorial. I get to the point where you have to click "Load Engine" pulse button, but then it doesn't do anything. I'm not sure if this helps figure out what's going wrong, but I noticed in your video at 6:26 the "Active engine" variable updates from "none" to "sd-turbo-512-512-1" whereas for me the "active engine" variable stays on "none" no matter how many times i reload or press the pulse button!

[lematchafrappe]

Thanks for making this free, much love

[birderwang]

Thank you very much for sharing this video!

[Azizinegin]

Thanks for sharing your knowledge, i appreciate it 🎉

[andreyeduardo]

Increible trabajo bro, saludos desde Costa Rica

[famnak]

I also have errors: Failed to import because of the following error (look up to see its traceback):
module 'numpy' has no attribute '_core' and name 'StreamDiffusionWrapper' is not defined

[elf-py]

how can i fix the error in TD "name 'StreamDiffusionWrapper' is not defined"plzzzz

[julianhaidbaner]

Hi, i want to tell if there is some solucion to this, when a press load engine, nothing happens at all, thanks you very much for the sharing this video is great !

[terryyu]

The test was successful, very cool...
May I ask when ControlNet will be supported?

[fulongfu]

plz help me, i have errors: Failed to import because of the following error (look up to see its traceback):
module 'numpy' has no attribute 'dtypes'
name 'StreamDiffusionWrapper' is not defined

[zohebmamoo]

Amazing video, what are the steps to have live feedback from the camera, remove background then generate on what's left? Like reacting to someone moving live

[nicholasd]

Thanks very much Oleg: Is support for Blackwell GPU's being developed? Obviously this works well with 4080's and 4090's but I just entered the market with a 5070 ti and have tried everything to get this working. Fingers crossed.

[sjdfhcreeper]

Hi! Thnaks for the video Oleg, I just have one problem at the end of the whole porcess, in TD when I select the touchdiffusion main folder it seem like it dont read the location, I restart the program and pc, but the error keep there, :( what could it be? anyone knows? thnks!!!

[alicekebornel]

when i hit pulse to refresh the engine nothing happens, engine is missing inside TD but not inside folder "checkpoints". started the pc and TD several times, doesnt help :( what could it be?

[천재영-lf]

Nevertheless I got finished engine preparation and selected the file which webui includes, it doesnt work.. I tried to press load engine and refresh engine several times, but it dosent work.. how to resolve this :(