Real-time diffusion in TouchDesigner - StreamdiffusionTD Setup + Install + Settings

Important ! Read description for Installation Tips ! This uses TouchDesigner 2023 and will not work in TouchDesigner 2022.
Currently only working using windows + nvidia graphics card.

BEFORE Installing - make sure to have:
Python 3.10 (make sure Python is added to Path in your system's environment variables)
CUDA 11.8 or CUDA 12.1

00:00 Introduction to StreamDiffusionTD
00:48 System Requirements for StreamDiffusionTD
01:12 Setting up operator in network
01:37 Downloading StreamDiffusion repo
02:10 Installing venv (first install only)
03:04 Setting Model + parameters before Stream
03:30 Step Schedule part 1
03:46 Similar Image Filter
04:00 Starting the Stream / Cmd window
04:41 Disable / Enable to see Stream bug
05:10 Seed
05:47 Prompts
06:16 Setting up Prompt Box
07:10 Negative Prompts do NOT work
07:22 Step Schedule part 2
08:16 Guidance Scale / Delta
08:47 Experiment !
09:16 Lora loader page
09:33 Callbacks
10:03 Stopping the Stream
10:09 Setting OSC Ports / Stream name
10:18 Visible Window (ON is default)
10:32 Say hey on discord !
11:00 Thanks !!

[elekktronaut]

absolutely mindblowing!! getting 6fps on my laptop 3070, thank you so much ❤

[TouchDesignerOfficial]

Amazing work and great tutorial. A small tip for your callbacks DAT, on the Common page of parameters if you switch 'Content Language' to Python then the DAT will use python syntax highlighting.

[gleix_]

Lyell you've got a voice ready for public access. ALSO this looks amazing! For people who know TD and already have a competent workflow, but are lacking in knowledge of SD, tools like this immediately open the door and make it more accessible than it has any right to be. I think seeing SD generating images in real-time, within TD in a format I can immediately wrap my head around to a better degree, is a pull I haven't had with SD until now. Awesome work!

[jakoblacour]

Really great! Got 1.5 fps with my GTX 1070 😅

[blankensmithing]

Lyell! Amazing work and tutorial!! Would love to see you create more tutorials in the future!!

[FunctionStore]

he's done it! I know what I'll be doing tomorrow...

[nothingrhymeswithferg]

this is actually just the craziest thing i've ever seen

[nicsnutzius]

Thanks a lot for this amazing work on StreamDiffusion! Regarding the NDI source name issue: I had similiar issues with the refresh on another project and realized that using 'Bind' instead of 'Reference' fixes the issue.

[InOneArt]

And now. Imagine an AI chatbot, when communicating with which you are accompanied by a visual series that displays your adventures in real time. Impressive

[ChyaBra]

Super excited to try this! The tutorial starts out with the Stream Diffusion operator on the screen but I'm not sure how to get that operator. Do I install the repo somewhere and point to the directory in the parameters box?

[cristianpenas]

This is super inspiring, it gives a clear visual image of the fundamentals of diffusion models

[Adellovisuals]

Thanks a lot for this! Running at 8 FPS with 4070

[ChristianBrinkmannLive]

Yeees finally! 😍 You‘re the best for putting this out! And the tutorial is well made made too 🖤

[Jonny]

Guys this is frickin insane. Amazing work!

[freivonstil]

Thank you so much for sharing! I'm a big fan even after the first minute already! Sub <3
Edit:
I just became a patreon and tried it out. A full day of setup, to get git and python running - but DAMN! Running at 15fps on a laptop 4070, what the heck?

[xavierg_music]

Thank you so much for sharing this!! ❤ My dreams just came true..

[brvoigt]

Nice use of those new sequential parameters! 🔥

[RetropunkAI]

This is EXACTLY what I needed this weekend. Thank you so much for this.

[MrJgrez]

Nice job bro. Appreciate the information.

[jaysonwall]

Whoa! That was cool. Thanks for the breakdown. Downloading stream diffusion now. :)