Curated by our fantastic community member Adam. Thank you! The below tutorial is for Dreambooth in Automatic1111.
Adam created an updated Tutorial using Kohya here: https://learn.rundiffusion.com/kohya-training/
Preprocess Images
- Step 1 on a Small or Medium Box.
a. Create a Folder/mnt/private/name_your_project
b. Put your images in that folder your named can be jpeg or png.
c. Create a second folder/mnt/private/name_your_project/train
Be aware train/dreambooth are case sensitive when it comes to your folder names
d. Train → Preprocess Images → Source Directory box:/mnt/private/name_your_project
e. Train → Preprocess Images → Destination directory box:
/mnt/private/name_your_project/train
f. Select Create flipped copies if you are doing a person or object. Also click USE BLIP for Caption if you are doing non anime. For Anime use Deepbooru.
g. Click Preprocess. This will put all the images in that/mnt/private/name_your_project/train
folder you created with the appropriate .txt files.
h. You can edit the text files as needed.
Lora Settings
This assumes that you already have a dataset with your images and .txt files ready.
- Start a Medium or Large Box
- Click on the Dreambooth Tab
a. Click The button that says Create. Select the source checkpoint this is the model you are basing your Lora on. Select create model.
3.** Settings** Tab
a. Check Use Lora.
b. Training steps Per Image: 100-128
c. Amount of Time to Pause between epochs: .3 seconds to 60 seconds.
d. Learning rate if you use lion 0.00005, 8bitAdam 0.0001. 8Bit Adam is Standard
e. Xformer: will speed up the Process but may cause more mutations like deformed fingers/hands.
f. Mixed precision: bf16
g. At the bottom under Advanced Options: Sanity Samples. Go ahead and fill that out as if
you were prompting for the image so that you can see examples. - Concepts Tab
a. Enter your dataset director:/mnt/private/name_of_folder_/train
Your dataset can be stored in the train folder.
b. Instance Token: Is the unique token you are creating that would activate your Lora. Like
“John48”
c. Training Prompt: A photo of John48
d. Class prompt: Not needed, don't use class images until dreambooth is fixed.
e. Negative Prompt: blurry, bad quality, blahblah
f. Number of samples: 2
g. Seed: Don’t use random use the same seed so you can see the changes over time.
h. If you get a "check datasets path" and everything seems to be right, try clicking any dataset tabs, even blank ones, and pressing delete a few times to get rid of spaces. - Saving Tab
a. Save EMA Weights should be checked.
b. Custom Model Name: Enter a name example: “John48”
c. Uncheck All the checkpoints otherwise it’ll end up like 4gbs and you don’t need that with a Lora.
d. Lora Text Encoder Rank: 128
e. Check: Generate Lora weights during training
f. Check: Generate Long Lora weights for extra networks (this will save a model in your Lora folder to use for testing when saving)
g. Check: Generate weights when canceled.
h. At the Top click Save Settings
i. Click Train - Notes
If you get a "batch size 0" error, this means your model has failed. Create a new model! Use same settings and try again. Sometimes when you train and get an error, it will ruin the model. You can delete it in the dreambooth folder, create a new one, and try again!
Class Images are currently causing errors - it's recommended not to use them.