DreamBooth Got Buffed - 22 January Update - Much Better Success Train Stable Diffusion Models Web UI

Playlist of Stable Diffusion Tutorials, #Automatic1111 and Google Colab Guides, DreamBooth, Textual Inversion / Embedding, #LoRA, AI Upscaling, Pix2Pix, Img2Img:

In this video, I have explained how to use the newest DreamBooth update of Automatic1111 Web UI extension. With new update, now it is much more successful to teach your subjects into any Stable Diffusion model.

The update has just been released today : 22 January 2023

Zero To Hero Stable Diffusion DreamBooth Tutorial By Using Automatic1111 Web UI - Ultra Detailed

#Dreambooth revision: fd51c0b2ed20566c60affa853a32ebce1b0a1139

SD-WebUI revision: d8f8bcb821fa62e943eb95ee05b8a949317326fe

How To Do Stable Diffusion Textual Inversion (TI) / Text Embeddings By Automatic1111 Web UI Tutorial

0:00 Introduction to the new buffed DreamBooth extension
0:30 How to checkout the SD and DreamBooth version used in this video by commit hash IDs
1:40 How to compose DreamBooth training model
2:13 Best configuration of settings tab of DreamBooth extension training
3:37 Lowest VRAM settings to use DreamBooth extension and do DreamBooth training
3:59 Why not use --no-half on SD 1.5 and use on SD 2.1
4:46 New setting AdamW Weight Decay
5:10 New setting Scale Prior Loss
6:14 How exactly filewords work in Stable Diffusion DreamBooth training
8:53 Sample images generated during training
9:30 Prompting difference of new DreamBooth extension than previous versions
10:25 How to test different checkpoints saved during training by X/Y plot script

Our new approach, DreamBooth, addresses the limitation of current text-to-image models by allowing for "personalization" of these models to better fit the specific needs of users. By providing just a few images of a subject as input, DreamBooth fine-tunes a pre-trained text-to-image model (such as Imagen) to learn to associate a unique identifier with that subject. This allows for the generation of novel, photorealistic images of the subject in various scenes, poses, views, and lighting conditions, even those not present in the reference images.

Our technique utilizes a new autogenous class-specific prior preservation loss which enables the preservation of the subject's key features while still allowing for diverse synthesis of the subject. This opens up possibilities for a wide range of previously unassailable tasks such as subject recontextualization, text-guided view synthesis, appearance modification, and artistic rendering.

Imagine your own dog traveling the world, your favorite bag on display in the most exclusive showrooms, or your parrot as the main character of an illustrated storybook. These are just a few examples of the type of creative and unique content that can be generated using DreamBooth. Our approach allows for the natural and seamless integration of specific subjects into new and diverse contexts, making the impossible possible.

Our goal is to use just a few casually captured images of a specific subject, without any textual description, to generate new images of the subject with high detail fidelity and variations guided by text prompts. The input images can be captured in varying settings and contexts and the output variations can include changes in the subject's location, properties such as color, shape, and species, as well as modifications to the subject's pose, expression, material, and other semantic changes. Our approach utilizes the powerful prior of text-to-image models to enable a wide range of modifications.

To accomplish this, we first implant the subject instance into the output domain of the model and assign it a unique identifier. We present a new method for fine-tuning the model to use its prior for the specific subject instance while also addressing issues of overfitting and language drift. Our approach includes an autogenous class-specific prior preservation loss which encourages the model to generate diverse instances of the same class as the subject.

Our goal is to add a new key-value pair to the text-to-image model's "dictionary" that will allow us to generate fully-novel images of a specific subject with meaningful semantic modifications guided by a text prompt. We achieve this by fine-tuning the model with a small number of images of the subject. The question then becomes how to guide this process.

[SECourses]

Please join discord, mention me and ask me any questions. Thank you for like, subscribe, share and Patreon support. I am open to private consulting with Patreon subscription.
I have used S.D. 1.5 Official version to generate the thumbnail image
Dreambooth model was only 30 epoch training by using only 12 images

Prompt: face photo of (ohwx man) in ironman suit, handsome, artstation, concept art, cinematic lighting, insanely detailed, octane, digital painting, smooth, sharp focus, illustration, photorealistic, hdr, 8k, exquisite, Trending on ArtStation HQ trending in deviantart close up

Negative prompt: ((bw)), ((black and white)), blurry, bad, worse, worst, cheap, sketch, basic, juvenile, unprofessional, failure, oil, label, signature, watermark, amateur, grotesque, misshapen, deformed, distorted, malformed, unsightly, terrible, awful, repellent, disgusting, revolting, loathsome, mangled, awkward, twisted, contorted, lopsided, asymmetrical, irregular, unnatural, botched, mutilated, disfigured, ugly, offensive, repulsive, ghastly, hideous, unappealing, frightful, odious, obnoxious, detestable, hateful, repugnant, sickening, vile, abhorrent, contemptible, execrable, distasteful, abominable, tiling


Steps: 30, Sampler: Euler a, CFG scale: 8, Seed: 3693089256, Size: 512x512, Model hash: 6f2a10a576, Model: testDream1_testDream1_720, Batch size: 8, Batch pos: 4

[crazyarda]

elinize sağlık, yine çok iyi bir anlatım olmuş sizde olmasanız yetişemeyeceğiz gelişmelerin hızına. 😀

[michaelsnow]

Thank you for including VRAM requirements!

[ChrisWiggins]

Great video and thanks for the explanation of filewords!

[MagicOfBarca]

The default for "Class Batch Size" is 4 for me (I have 3090). So I keep it at 4 instead of 1 like you?

[AP-rjls]

Спасибо за хорошее видео. Наверное ты самый основательный автор на ютуб по этой теме

[chrisb]

Great and helpful videos! Just one question.. how come my ckpt files are being generated as .safetensors? I am using the latest WebUI & Dreambooth, with Torch 2. Some screenshots show a safetensors checkbox, but I don't see this on mine.

[MagicOfBarca]

Also what is "Class Images Per Instance Image"? and why did you type 48?

[hceely]

Latest update seems to be broken according to others too, getting a RuntimeError("No executable batch size found, reached zero.")

[haelee]

Hi, I've been watching your videos for stable diffusion related videos! it was so helpful. Thank you. I have some questions about WebUI. I followed your videos and no errors occurred, but the UI doesn't look the same as yours. Do you have any suggestions I could follow? dreambooth tab is in the UI, but only <load parameters>, <cancle> <train> are only tab that exists.

[galbalandroid]

Thank you for giving the exact commits for both dreambooth & automatic1111 that has been working for you! The latest dreambooth & webui doesn't allow me to create models for some reason. this has solved it!

EDIT : I'm getting CUDA OUT OF MEMORY error when trying to create a model. 3070 RTX with 8GB VRAM, so this doesn't make sense...

[EranMahalu]

Hi, I have an error in Dreambooth, thought you might have stumbled upon it
"if checkpoint_file is None or not
TypeError: 'CheckpointInfo' object is not subscriptable"
this happens when I try to create a model

[robert_magalhaes]

Excellent tutorial, thank you! it's the first time I've been able to make a model that can convincingly replicate my face. I don't know what I did wrong, but my model only generates photos similar to those used as a sample, which means that no matter what prompt I write, the result doesn't change much. I tried generating a photo of myself in the iron man suit and it didn't work, as well as several other prompts it didn't work either, unfortunately the result is always similar to the photos used as input.

[AlexAtiq]

thank you for the video, what is the min amount of v-ram you need to use dream booth locally

[izz]

Thank you for forwarding this to me. I appreciate your hard work. Memory errors were solved once I unchecked "Train UNET" . What is the disadvantage of not having this checked? I don't really understand it. But unchecking it worked for me.

[Ghostyshev]

Could be great to have a video about training style - not person. Saw a lot of requests on youtube and discord about it
Thanks a lot for you videos!

[TrueRewire]

My Dreambooth keeps trying to make THOUSANDS of class images, so training is taking hours if I just let it run?

Why does it do this?

[urasmutlu]

merhaba, oncelikle video icin cok tesekkurler, oldukca bilgilendirici olmus.

sizin class image uretme hiziniz yaklasik 6 saniyeyken benimki 10-11 saniye arasi suruyor. ben de rtx 3060 kullaniyorum, 32 gb ram ve i5-12400 islemci var. ayni commit hashleri kullandim ve ben de web UI'i windows uzerinden calistiriyorum. acaba aradaki hiz farki neden kaynaklaniyor olabilir? ekstra bir sey yaptiniz mi diye sormak istedim.

tekrardan video icin tesekkurler

[Hypersniper]

Do you get a black image when it is over trained? Seems to work fine at the start then it goes black. I'm on a 3080ti

[David-kufy]

v2.4.24 stable difusion has none of these options im very confused seeing as this vid was 9 days ago. how can i dreambooth on the new diffusion?