Lora Training Walkthrough

Curated by our fantastic community member Adam. Thank you! The below tutorial is for Dreambooth in Automatic1111.

Adam created an updated Tutorial using Kohya here: https://learn.rundiffusion.com/kohya-training/

Preprocess Images

  1. Step 1 on a Small or Medium Box.
    a. Create a Folder /mnt/private/name_your_project
    b. Put your images in that folder your named can be jpeg or png.
    c. Create a second folder /mnt/private/name_your_project/train
    Be aware train/dreambooth are case sensitive when it comes to your folder names
    d. Train → Preprocess Images → Source Directory box: /mnt/private/name_your_project
    e. Train → Preprocess Images → Destination directory box:
    /mnt/private/name_your_project/train
    f. Select Create flipped copies if you are doing a person or object. Also click USE BLIP for Caption if you are doing non anime. For Anime use Deepbooru.
    g. Click Preprocess. This will put all the images in that /mnt/private/name_your_project/train
    folder you created with the appropriate .txt files.
    h. You can edit the text files as needed.

Lora Settings

This assumes that you already have a dataset with your images and .txt files ready.

  1. Start a Medium or Large Box
  2. Click on the Dreambooth Tab
    a. Click The button that says Create. Select the source checkpoint this is the model you are basing your Lora on. Select create model.
    3.** Settings** Tab
    a. Check Use Lora.
    b. Training steps Per Image: 100-128
    c. Amount of Time to Pause between epochs: .3 seconds to 60 seconds.
    d. Learning rate if you use lion 0.00005, 8bitAdam 0.0001. 8Bit Adam is Standard
    e. Xformer: will speed up the Process but may cause more mutations like deformed fingers/hands.
    f. Mixed precision: bf16
    g. At the bottom under Advanced Options: Sanity Samples. Go ahead and fill that out as if
    you were prompting for the image so that you can see examples.
  3. Concepts Tab
    a. Enter your dataset director: /mnt/private/name_of_folder_/train
    Your dataset can be stored in the train folder.
    b. Instance Token: Is the unique token you are creating that would activate your Lora. Like
    “John48”
    c. Training Prompt: A photo of John48
    d. Class prompt: Not needed, don't use class images until dreambooth is fixed.
    e. Negative Prompt: blurry, bad quality, blahblah
    f. Number of samples: 2
    g. Seed: Don’t use random use the same seed so you can see the changes over time.
    h. If you get a "check datasets path" and everything seems to be right, try clicking any dataset tabs, even blank ones, and pressing delete a few times to get rid of spaces.
  4. Saving Tab
    a. Save EMA Weights should be checked.
    b. Custom Model Name: Enter a name example: “John48”
    c. Uncheck All the checkpoints otherwise it’ll end up like 4gbs and you don’t need that with a Lora.
    d. Lora Text Encoder Rank: 128
    e. Check: Generate Lora weights during training
    f. Check: Generate Long Lora weights for extra networks (this will save a model in your Lora folder to use for testing when saving)
    g. Check: Generate weights when canceled.
    h. At the Top click Save Settings
    i. Click Train
  5. Notes
    If you get a "batch size 0" error, this means your model has failed. Create a new model! Use same settings and try again. Sometimes when you train and get an error, it will ruin the model. You can delete it in the dreambooth folder, create a new one, and try again!
    Class Images are currently causing errors - it's recommended not to use them.
About the author
Ed

Ed

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to .

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.