AI Art

What Are LoRAs? A Beginner's Guide to LoRA Models

Generative Labs/
Screenshot of LoRA models on civitai.com
Screenshot of LoRAs on civitai.com

LoRA (Low-Rank Adaptation) models let you fine-tune Stable Diffusion for specific characters, styles, or concepts that base models don't handle well. The files are small, the community libraries are free, and the setup takes a few minutes. This guide covers what LoRAs are, where to find them, and how to use them in Automatic1111.

What Are LoRA Models?

A LoRA (Low-Rank Adaptation) is a small file that fine-tunes Stable Diffusion to generate a specific character, style, or concept the base model can't produce well on its own. Instead of retraining the entire model, a LoRA trains a small set of additional weights and layers them on top of an existing model. That's why the files are so small (often just a few megabytes) and easy to collect by the dozen.

What Types of LoRA Models Exist?

  • LoRA models come in a variety of flavors, each suited to different tasks.
  • Character LoRA specializes in creating specific characters like popular video game or anime personas.
  • Style LoRA is all about mimicking artistic styles, letting you create pieces that mirror the signature style of a certain artist.
  • Concept LoRA is the Picasso of the bunch, great at conceptualizing specific ideas that are tough to achieve with just prompt engineering.
  • Pose LoRA lets your characters strike a pose, perfect for dynamic scenes.
  • Clothing LoRA is your AI fashion designer, adept at changing your characters' attire, while ...
  • Object LoRA is a whizz at generating objects like furniture or vehicles.

Where Can You Find LoRA Models?

AI-generated character using LoRA model

Open-source repositories like Civitai and Hugging Face host a plethora of LoRA models. They are small, easy to download, and absolutely free.

These repositories offer a user-friendly interface and comprehensive documentation, making it straightforward for both beginners and experienced users to navigate and understand the available models.

Additionally, due to their open-source nature, they foster an active community of developers and users, allowing for continuous model improvements, feedback, and troubleshooting help.

How Do You Install and Use LoRA Models?

First, you'll need to install the LoRA extension into your Automatic1111 web UI. Next, you install your chosen LoRA models into the correct folder. Assuming your A1111 is installed in /workspace/sd, you'd place the LoRA model in:

/workspace/sd/stable-diffusion/webui/models/Lora

To use the LoRA, you need to reference it in your prompt in the following way:

[TRIGGER_WORDS] [PROMPT] <lora:MODEL_NAME:WEIGHT>

Here, you'll replace:

  • TRIGGER_WORDS with the trigger words associated with the LoRA
  • PROMPT with the details of your prompt
  • MODEL_NAME with the filename of your LoRA without the extension
  • WEIGHT with the weight you want the LoRA to have with the output

Here's an example:

3dmm style,(masterpiece, top quality, best quality, official art,
beautiful and aesthetic:1.2), (fractal art:1.3), 1girl, beautiful,
high detailed, purple hair with a hint of pink, pink eyes, dark
lighting, serious face, looking the sky, sky, medium shot, black
sweater, jewelry <lora:revAnimated_v122:1.2>

In this example the trigger word is "3dmm" and the model name is "revAnimated_v122".

The (masterpiece:1.2) and (fractal art:1.3) bits are prompt weights, a separate syntax from the LoRA tag. For the full rundown on parentheses, brackets, weights, and the BREAK keyword, see our guide to Stable Diffusion prompt syntax.

That's it, your image is LoRA-enhanced.

Using LoRAs with A1111 API

If you're looking to set up a Stable Diffusion API using A1111 that leverages LoRAs, check out this YouTube tutorial:

Setting Up a Stable Diffusion API with LoRAs

In Conclusion

LoRA models add a whole new level of detail and accuracy to AI-generated images. With just a few steps, they can supercharge your Automatic1111 workflow, opening a world of possibilities for your projects.

Frequently asked

What are LoRAs?
›LoRAs (Low-Rank Adaptation models) are small files that fine-tune Stable Diffusion to generate a specific character, style, or concept the base model can't produce well on its own.
⌄LoRAs (Low-Rank Adaptation models) are small files that fine-tune Stable Diffusion to generate a specific character, style, or concept the base model can't produce well on its own. They load on top of an existing model without retraining it, are only a few megabytes in size, and are free to download from community libraries.
What does LoRA stand for?
›LoRA stands for Low-Rank Adaptation. It is a technique for fine-tuning large image models efficiently by training a small set of additional weights instead of the whole model, which is why the resulting files are so small.
⌄LoRA stands for Low-Rank Adaptation. It is a technique for fine-tuning large image models efficiently by training a small set of additional weights instead of the whole model, which is why the resulting files are so small.
What is a character LoRA?
›A character LoRA is trained to reproduce one specific character, such as a video game or anime persona, consistently across generations.
⌄A character LoRA is trained to reproduce one specific character, such as a video game or anime persona, consistently across generations. You activate it with the character's trigger word plus a `<lora:MODEL_NAME:WEIGHT>` tag in your prompt.
What is a style LoRA?
›A style LoRA mimics a particular artistic style rather than a subject.
⌄A style LoRA mimics a particular artistic style rather than a subject. It lets you render images in the look of a specific artist or aesthetic while you keep full control over the subject through the rest of your prompt.
What is a LoRA in Stable Diffusion?
›In Stable Diffusion, a LoRA is an add-on model you place in your `models/Lora` folder and reference in the prompt as `<lora:MODEL_NAME:WEIGHT>`.
⌄In Stable Diffusion, a LoRA is an add-on model you place in your `models/Lora` folder and reference in the prompt as `<lora:MODEL_NAME:WEIGHT>`. It nudges the base model toward the character, style, or concept it was trained on, scaled by the weight value.
Where do you download LoRA models?
›The main open-source libraries are Civitai and Hugging Face.
⌄The main open-source libraries are Civitai and Hugging Face. Both host thousands of free LoRA models with previews and documentation, and the files are small enough to download in seconds.
Subscribe

Considered takes, in your inbox.

We write when we learn something worth sharing. No schedule, no marketing digests. Built for engineers and product owners shipping with agents.

~1 email/wk · Unsubscribe anytime