What are LoRA models?

LoRA Models: A Beginner’s Guide

In this tutorial, we will explore LoRA (Low-Rank Adaptation) models for fine-tuning Stable Diffusion models. LoRA models offer a balance between file size and training power, making them a popular choice for users with extensive model collections.

LoRA models are typically 10 to 100 times smaller than standard checkpoint models. This makes them attractive for users who face storage limitations on their personal computers. With manageable file sizes ranging from 2 to 200 MBs, these models offer an excellent solution to the storage problem.

Unlike textual inversion or Dreambooth, LoRA models cannot be used alone. They require a model checkpoint file to function. These models modify styles by applying small changes to the accompanying model file, specifically targeting the cross-attention layers.

The key advantage of LoRA models lies in their ability to break down matrices into low-rank matrices. By doing this, they can store significantly fewer numbers, resulting in smaller file sizes. However, this reduction in file size does not significantly impact the fine-tuning power of the models.

To find these models, you can visit Civitai, a platform that hosts a large collection of LoRA models. By applying the LoRA filter, you can browse through various styles such as female portraits, anime, and realistic illustrations. It is worth mentioning that Civitai may contain NSFW (Not Safe For Work) content, so make sure to apply the appropriate filter if necessary. Alternatively, Hugging Face also offers a smaller collection of these models.

To use these models in the AUTOMATIC1111 Stable Diffusion GUI, follow these steps:

• 1.
  Install the LoRA model: Place the model files in the folder stable-diffusion-webui/models/Lora.
• 2.
  Use the LoRA model in prompts: Include the phrase '<lora:filename:multiplier>' in your prompt, where 'filename' is the file name of the LoRA model (without the extension) and 'multiplier' is the weight applied to the LoRA model. The default multiplier is 1, and setting it to 0 disables the model.

To ensure the correct filename is used, you can click on the model button within the GUI. Under the Lora tab, you will find a list of installed LoRA models to choose from. Clicking on a specific model will automatically insert the correct LoRA phrase into the prompt.

Some additional notes on using LoRA models:

By following these guidelines, you can access and utilize LoRA models in AUTOMATIC1111, enhancing your ability to fine-tune and experiment with different styles for image generation.