Transform Yourself into a Superhero with Dreambooth and Automatic1111

Welcome to this new tutorial. Today, we will see how to transform Yourself into a Superhero. We will be using the Dreambooth extension for Automatic1111. All of this can be done on your home PC.

Workflow

1. Install Dreambooth Extension
2. Restart Automatic1111
5. Create a Copy of Checkpoint starting from one you like
6. pepare ~20 images of yourself in different pose and clothes. Image size must match model training size. For SD 1.5 it's 512x512.
8. Use parameters in the video (also below) to much ~2200 steps
9. Generate class images.
10. Train
11. Have fun. You can use the trick explained in the video to obtain better results.

Parameters used in the video :

{
"adamw_weight_decay": 0.01,
"adaptation_beta1": 0,
"adaptation_beta2": 0,
"adaptation_d0": 1e-08,
"adaptation_eps": 1e-08,
"attention": "xformers",
"cache_latents": false,
"clip_skip": 1,
"concepts_list": [
{
"class_data_dir": "",
"class_guidance_scale": 7,
"class_infer_steps": 40,
"class_negative_prompt": "",
"class_prompt": "photo of a man",
"class_token": "",
"instance_data_dir": "G:\\Download\\cstndr\\Foto512",
"instance_prompt": "kcka man",
"instance_token": "",
"is_valid": true,
"n_save_sample": 1,
"num_class_images_per": 1,
"sample_seed": -1,
"save_guidance_scale": 7,
"save_infer_steps": 20,
"save_sample_negative_prompt": "",
"save_sample_prompt": "photo of kcka man",
"save_sample_template": ""
}
],
"concepts_path": "",
"custom_model_name": "",
"noise_scheduler": "DDPM",
"deterministic": false,
"ema_predict": false,
"epoch": 60,
"epoch_pause_frequency": 0,
"epoch_pause_time": 0,
"freeze_clip_normalization": false,
"gradient_accumulation_steps": 1,
"gradient_checkpointing": false,
"gradient_set_to_none": true,
"graph_smoothing": 50,
"half_model": false,
"train_unfrozen": false,
"has_ema": false,
"hflip": false,
"infer_ema": false,
"initial_revision": 0,
"learning_rate": 2e-06,
"learning_rate_min": 1e-06,
"lifetime_revision": 0,
"lora_learning_rate": 0.0001,
"lora_model_name": "",
"lora_unet_rank": 4,
"lora_txt_rank": 4,
"lora_txt_learning_rate": 5e-05,
"lora_txt_weight": 1,
"lora_weight": 1,
"lr_cycles": 1,
"lr_factor": 0.5,
"lr_power": 1,
"lr_scale_pos": 0.5,
"lr_scheduler": "constant_with_warmup",
"lr_warmup_steps": 0,
"max_token_length": 75,
"mixed_precision": "fp16",
"model_name": "AndreaCheckpoint",
"model_dir": "D:\\stable-diffusion-webui\\models\\dreambooth\\AndreaCheckpoint",
"model_path": "D:\\stable-diffusion-webui\\models\\dreambooth\\AndreaCheckpoint",
"num_train_epochs": 60,
"offset_noise": 0,
"optimizer": "8bit AdamW",
"pad_tokens": true,
"pretrained_model_name_or_path": "D:\\stable-diffusion-webui\\models\\dreambooth\\AndreaCheckpoint\\working",
"pretrained_vae_name_or_path": null,
"prior_loss_scale": false,
"prior_loss_target": 100.0,
"prior_loss_weight": 0.75,
"prior_loss_weight_min": 0.1,
"resolution": 512,
"revision": 2640,
"sample_batch_size": 1,
"sanity_prompt": "",
"sanity_seed": 420420.0,
"save_ckpt_after": true,
"save_ckpt_cancel": false,
"save_ckpt_during": false,
"save_ema": true,
"save_embedding_every": 25,
"save_lora_after": true,
"save_lora_cancel": false,
"save_lora_during": false,
"save_lora_for_extra_net": false,
"save_preview_every": 5,
"save_safetensors": true,
"save_state_after": false,
"save_state_cancel": false,
"save_state_during": false,
"scheduler": "DEISMultistep",
"shuffle_tags": true,
"snapshot": "",
"split_loss": true,
"stop_text_encoder": 0.75,
"strict_tokens": false,
"tf32_enable": false,
"train_batch_size": 2,
"train_imagic": false,
"train_unet": true,
"use_concepts": false,
"use_ema": false,
"use_lora": false,
"use_lora_extended": false,
"use_subdir": true,
"v2": false
}

[HarKanJel]

I had a lot of trouble to install properly AUTOMATIC1111, install Dreambooth and make xtensors working ... but after that, your video was perfect!! Thanks for sharing

[MohawkGCreative]

Amazing Tutorial!!! Just ordered a new MSI RS 32gb 2TB with 4070ti and will be creating incredible images just like yours!!!

[StefanKastner]

excellent tutorial, you deserve way more subs

[BasicShowTime]

Nice and clean explanation with a little bit of technical knowlegde... nice video bro, will help a lot of people.

[RobertManzanilla]

Thanks for the great tutorial.
I was able to successfully run the training on a 3070ti (8GB VRAM) by changing the following:
• Leave the "Gradient Checkpointing" checkbox selected
• Training Steps Per Image (Epochs) at 40
• Mixed Precision at bf16 [all in the Settings tab]

Although it took about 6 hours. I will be running tests and try to get that time down. But at least it worked!

[GenCodeFPV]

The version I got today 4-22-24 is different, its close but a few things changed. If you updated the video I didnt see it, thanks good video btw

[Deathexdraw]

Finally someone made a video of what I was searching! Thank you! Does anyone knows if blending different faces during the training would create a new model?

[ChillwaveMeditations]

when i create the model i can't see the input column. why?

[tvkoreantraditionalhouse]

잘보고 있습니다. 좋은영상 만들어주셔서 감사합니다. 만화캐릭터를 얼굴과 특별한 복장을 훈련시키려면 어떻게 훈련해야 합니까?

[guigui]

Hi, thank you for this tutorial. I think I have a problem filling in the tokens, samples and prompts fields. My version of dreambooth differs from yours and the fields are not the same. Can you help me to see more clearly. Thank you

[onur]

When choosing the Model Type, I don't have the option for 512x Model. The available options are v1x, v2x-512, v2x, SDXL, and ControlNet. Which one should I choose?

[HaoLe-tsfw]

Hmm, can't get it to work on my M1 Mac; Dreambooth seems to require NVidia chip…

[Kcolus]

Complimenti ottimo video, ma ora dreambooth ha un’interfaccia leggermente diversa dopo l’ultimo aggiornamento, e anche alcune spunte e valori non sembrano avere lo stesso significato… potresti aggiornare il tutorial all’ultima versione??

[Hocamatik]

Thank you very much for the video, but I encountered a problem. Dreambooth button does not appear. I wonder what I can do, I couldn't find a solution anywhere. Could you please help me?

[Theinnersearcher]

Why doesnt my dreambooth look anything like this ?=??? i dont havew the wizard or anything!!!

[ozanyaman]

hello there, got this error : Exception training model: ''NoneType' object has no attribute 'dtype''. There is no article for the solution :(

[zakhiltn]

I have a 3070, with the same settings in the video its taking about 10 hours to train. Not sure what is wrong, can see GPU usage at aprox 100% so its being used. Also when i click train it takes about 2 to 3 minutes before it even starts.

[lithium]

The tip about using one model for a reference img and the use img2img is really good, Ill try that. Did you use an inpaint model for the learning?
What was the GPU usage?

[holo]

Exception training model: 'local variable 'instance_loss' referenced before assignment'.
how to solve this ?

[tienlaihsia]

@DigitalArtGuide-bz3gn Dear, I was following your steps to train the model on the Runpod instance, but I alway got the issue:
Exception training model: ' Some tensors share memory, this will lead to duplicate memory on disk and potential differences when loading them again: [{'encoder.down_blocks.2.resnets.1.norm2.weight', 'decoder.up_blocks.0.upsamplers.0.conv.weight', 'decoder.up_blocks.2.resnets.2.conv1.weight', 'decoder.up_blocks.3.resnets.1.norm2.weight', 'decoder.up_blocks.3.resnets.2.conv2.weight', 'decoder.mid_block.resnets.1.conv1.weight', 'post_quant_conv.bias', 'encoder.down_blocks.2.resnets.1.conv1.weight', 'decoder.up_blocks.3.resnets.0.norm1.bias', 'encoder.conv_out.weight', 'encoder.mid_block.resnets.0.conv1.bias', 'decoder.up_blocks.0.resnets.0.conv2.weight',

Do you know the reason?