Creating a LoRA weight using kohya_ss GUI, part 2: Training the model and creating outputs

This is the part two of creating a LoRA weight. It will explain how you go about creating your model and generating realistic outputs from prompts.

[abhinavmohan]

Thanks a lot, your tutorial is the first one that has worked for me, after trying 4-5 different ones. Kudos to you and the details you provide :)

[scarpusgaming]

0:00 Intro
1:40 Config - Source Model
3:09 Config - Folders
6:12 Config - Training Parameters
13:22 Config - Json file
15:11 Start training
19:11 Model finished
20:48 Setting up model testing
26:11 Running test renders

[Vigilence]

Would love you make a similar tutorial for Lora, but focusing on art styles.

[a.........._...__..._.....__]

Thanks for the video, was a good watch while I compiled my first model. Will try your settings next

[tonychong]

Thank you so much for all your work! Learned a lot from this video. Subscribed!

[DianneStudio]

Amazing tutorial. Thank you so much for the great tutorial. Is it possible to make a update video as the Kohya_SS got a an update and now has more options. It will be much appreciated if you can make a catchup video of that. Thank you

[CharlotteGrossin]

Really great tutorials, very enjoyable and formative!

[leeren_]

amazing video, you deserve millions more subscribers

[MonkeChillVibes]

Golden content, been looking for this for a while

[AeonPrime]

There seems to be a bug in Auto1111 currently where it doesn't use the small LoRA files from kohya when they are in the default Auto1111\models\lora\ folder. I had to install the extension, add the file to the folder of the extensions (extensions\sd-webui-additional-networks\models\lora), restart the entire thing (including the terminal window), in Auto1111 I had to open the new tab at the bottom of the image generation page, enable additional networks and select my LoRa model there and reference it by it's name in the prompt.

[vaneaph]

@ 3:43
i just run into this at reddit
The program expect to find a folder like " [classname] "
so " 100_jennifer aniston" wouldn't be ideal in this case, aniston as a class name.

could anyone shed some light please ?

[endymion]

Thank you for the tutorial, i'll try it soon. Any recommendations for training art styles using this method?

[YyJester]

Thanks for taking us along for this this step by step. The 29:10 results are incredible considering the sample size and training time. Simply amazing.
Going through the parameters, I have a wide range of questions that could most easily be answered with the following:
Do you have any helpful reading material bookmarked where we can read about the different training variables used during the LoRA training?

Personal anecdote: I attempted to train with a very large sample size (1000+ images, handpicked, cropped, checked for quality), but I went through the kohya pop-up script which is much more user-unfriendly than the web-ui. The core problem is that I just blindly followed some recommended values and ended up with a training time of 4 hours and subpar results. Well, I expected my first attempt to be a failure but it's still frustrating.

Essentially, I have no clue what a regularization image can do and how it can be helpful. Pretty sure I can't run 2 batches either. I assume that would require 12-16 GB of VRAM? Why change the learning rate, what impact does it have? What's the math and reasoning behind it all? LR Schedulers, what do they do? What's warm-up and why does no one seem to want to do it? What's rank and network alpha? Just sooo many question marks. It's nice if it works so well with 10-20 images, but what if our use case differs? What if we want to train a style or concept instead of a person? (I tried a person, but just in case). I read through some guides on rentry, watched some lengthy videos, read the official LoRA github. Everyone seems to just say "here are some numbers, good luck, lol". /rant over. Apologies.

[voEovove]

Very simple and easy to follow along. Thank you.

[pastuh]

Currently training model with over 2.4k images.. and 1024, 1024 size
I'm not sure if I need to wait for all 10 epochs to finish (120950 steps in total..)
I hope training will be without any problems..

My question..
Can you suggest the best way to continue training/extending of Dreambooth model (used kohya_ss for training).
I see there is tab called 'Finetune', so maybe I need to choose trained model instead SD1.5?

My plan to add more different objects and backgrounds.

Would be nice to see video tutorial :)

[NeilHodgson]

If you are training on something that is already in the model and you are getting cross contamination you can give it a unique token name in the captioning like JnfAni instead of the full jennifer aniston. Then when using the lora you just use the new unique token you created. This way you can avoid contamination with what is already is in the model

[lemur]

I have trained the lora, and I have used it for stable diffusion, but I keep on getting black images, doesnt mater what i type in

[PasMTV]

Hello Bernard,
I followed your excellent presentation step by step, but I cannot obtain models under the "safetensors" extension! In the "Models" folder there is only one .json file...
Do you have a solution to offer me?
Thanking you.

[panzerswineflu]

I use a 1660ti and for some reason we have to use an older version and turn mixed precision off.

That said i had to set the Lora weight to 0.0? Something to not just have a random mess of colors with these settings. Not sure which thing is the culprit.

I've tuned it down some but my airplane photos still come out cartoon illustration look to them

[チャンネル-ds]

Can you do tutoral for saved training state?