Skip to main content

​
Overview

train_network.py is the core LoRA training script for Stable Diffusion v1.x and v2.x models. It supports LoRA-LierLa (the default), LoRA-C3Lier (with 3×3 convolution layers), and a range of optimizers and learning rate schedulers.
For SDXL, FLUX.1, or SD3 models, use the dedicated scripts described in their respective pages. This page covers SD 1.x and 2.x only.

​
Prerequisites

  • The sd-scripts repository cloned and the Python environment set up.
  • A prepared dataset and a TOML dataset config file.
  • A Stable Diffusion v1.x or v2.x .safetensors or .ckpt base model.

​
Training command

Below is a complete example that you can adapt to your setup. Use \ for line continuation on Linux/macOS or ^ on Windows. 
accelerate launch --num_cpu_threads_per_process 1 train_network.py \
  --pretrained_model_name_or_path="/path/to/model.safetensors" \
  --dataset_config="my_dataset_config.toml" \
  --output_dir="./output" \
  --output_name="my_lora" \
  --save_model_as=safetensors \
  --network_module=networks.lora \
  --network_dim=16 \
  --network_alpha=8 \
  --learning_rate=1e-4 \
  --unet_lr=1e-4 \
  --text_encoder_lr=1e-5 \
  --optimizer_type="AdamW8bit" \
  --lr_scheduler="cosine_with_restarts" \
  --lr_warmup_steps=100 \
  --sdpa \
  --max_train_epochs=10 \
  --save_every_n_epochs=1 \
  --mixed_precision="fp16" \
  --gradient_checkpointing
Write the command on a single line or use the appropriate line-continuation character for your shell.

​
Key arguments

​
Required arguments

​
--pretrained_model_name_or_path
string
required
Path to the base Stable Diffusion model. Accepts a local .ckpt or .safetensors file, a Diffusers-format directory, or a Hugging Face Hub model ID such as "stabilityai/stable-diffusion-2-1-base".
​
--dataset_config
string
required
Path to the .toml file that describes the training dataset. See the dataset configuration guide for the full format.
​
--output_dir
string
required
Directory where trained LoRA models, sample images, and logs are saved.
​
--output_name
string
required
Base filename for the saved LoRA (without extension). The final file will be <output_name>.safetensors.
​
--network_module
string
required
Network type to train. Use networks.lora for standard LoRA. Use networks.dylora for DyLoRA.
​
--network_dim
integer
required
LoRA rank (dimension). Higher values increase expressiveness and file size. Common values: 4, 8, 16, 32, 64, 128.

​
Network parameters

​
--network_alpha
number
default:"1"
Alpha value that scales the LoRA output. Setting this equal to network_dim matches older behavior. A value of half network_dim is a common default.
​
--network_dropout
number
Dropout rate (0.0–1.0) applied inside LoRA modules. Can help reduce overfitting. Omit to disable.
​
--network_args
string[]
Additional key=value arguments passed to the network module. For LoRA-C3Lier (3×3 convolution layers), add "conv_dim=4" "conv_alpha=1".

​
Saving

​
--save_model_as
string
default:"safetensors"
File format for the saved model. Options: safetensors (recommended), ckpt, pt.
​
--save_every_n_epochs
integer
Save a checkpoint every N epochs. If omitted, only the final model is saved.
​
--save_every_n_steps
integer
Save a checkpoint every N steps. Can be used alongside --save_every_n_epochs.

​
Learning rate

​
--learning_rate
number
default:"1e-4"
Global learning rate used when --unet_lr and --text_encoder_lr are not specified.
​
--unet_lr
number
Learning rate for LoRA modules inside the U-Net. Falls back to --learning_rate when omitted.
​
--text_encoder_lr
number
Learning rate for LoRA modules inside the text encoder. Recommended to be lower than the U-Net rate (e.g., 1e-5).
​
--lr_scheduler
string
default:"constant"
Learning rate schedule. Options: constant, cosine, linear, constant_with_warmup, cosine_with_restarts, polynomial.
​
--lr_warmup_steps
integer
Number of steps over which the learning rate ramps up from zero to the target value at the start of training.

​
Training duration

​
--max_train_epochs
integer
Number of epochs to train. When set, --max_train_steps is ignored.
​
--max_train_steps
integer
Total number of training steps. Used only when --max_train_epochs is not set.

​
Optimizer

​
--optimizer_type
string
default:"AdamW8bit"
Optimizer to use. Options include AdamW8bit (requires bitsandbytes), AdamW, Lion (requires lion-pytorch), Adafactor, DAdaptation, Prodigy, schedulefree.

​
Memory and speed

​
--mixed_precision
string
default:"no"
Mixed precision training mode. Use fp16 or bf16 to reduce VRAM usage and speed up training. Requires GPU support.
​
--gradient_checkpointing
boolean
Enables gradient checkpointing to reduce VRAM at the cost of slightly slower training.
​
--sdpa
boolean
Uses PyTorch’s Scaled Dot-Product Attention. Reduces memory and can improve speed.
​
--gradient_accumulation_steps
integer
default:"1"
Accumulate gradients over N steps before updating weights. Effectively multiplies the batch size without increasing VRAM.

​
SD 2.x specific flags

When training on Stable Diffusion v2.x models, add the following flags: 
--v2
Use --v2 for v2.x models that use epsilon (noise) parameterization, such as stable-diffusion-2-base.
Omitting --v2 when training on a v2.x model will cause incorrect behavior because the text encoder architecture differs from v1.x.

​
Conv2d LoRA (LoRA-C3Lier)

By default, LoRA targets only Linear layers and 1×1 Conv2d layers. To also apply LoRA to 3×3 Conv2d layers (LoRA-C3Lier), pass conv_dim in --network_args: 
--network_args "conv_dim=4" "conv_alpha=1"
This produces slightly larger LoRA files and may improve quality for style training, at the cost of overfitting risk with small datasets.

​
Monitoring training

If you specify --logging_dir, you can visualize the training loss and learning rate with TensorBoard: 
tensorboard --logdir ./logs

​
Using the trained LoRA

When training completes, a .safetensors file is saved to your --output_dir. You can load it in:
  • AUTOMATIC1111 stable-diffusion-webui — place in models/Lora/ and reference with <lora:my_lora:1> in your prompt.
  • ComfyUI — use a LoraLoader node pointing to the file.

Build docs developers (and LLMs) love

Get started for freeTalk to us