> ## Documentation Index
> Fetch the complete documentation index at: https://docs.comfy.org/llms.txt
> Use this file to discover all available pages before exploring further.
# TrainLoraNode - ComfyUI Built-in Node Documentation
> Complete documentation for the TrainLoraNode node in ComfyUI. Learn its inputs, outputs, parameters and usage.
> This documentation was AI-generated. If you find any errors or have suggestions for improvement, please feel free to contribute! [Edit on GitHub](https://github.com/Comfy-Org/embedded-docs/blob/main/comfyui_embedded_docs/docs/TrainLoraNode/en.md)
The TrainLoraNode creates and trains a LoRA (Low-Rank Adaptation) model on a diffusion model using provided latents and conditioning data. It allows you to fine-tune a model with custom training parameters, optimizers, and loss functions. The node outputs the trained LoRA weights, a loss history map, and the total training steps completed.
## Inputs
| Parameter | Data Type | Required | Range | Description |
| ------------------------- | ------------ | -------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model` | MODEL | Yes | - | The model to train the LoRA on. |
| `latents` | LATENT | Yes | - | The Latents to use for training, serve as dataset/input of the model. |
| `positive` | CONDITIONING | Yes | - | The positive conditioning to use for training. |
| `batch_size` | INT | Yes | 1-10000 | The batch size to use for training (default: 1). |
| `grad_accumulation_steps` | INT | Yes | 1-1024 | The number of gradient accumulation steps to use for training (default: 1). |
| `steps` | INT | Yes | 1-100000 | The number of steps to train the LoRA for (default: 16). |
| `learning_rate` | FLOAT | Yes | 0.0000001-1.0 | The learning rate to use for training (default: 0.0005). |
| `rank` | INT | Yes | 1-128 | The rank of the LoRA layers (default: 8). |
| `optimizer` | COMBO | Yes | "AdamW"
"Adam"
"SGD"
"RMSprop" | The optimizer to use for training (default: "AdamW"). |
| `loss_function` | COMBO | Yes | "MSE"
"L1"
"Huber"
"SmoothL1" | The loss function to use for training (default: "MSE"). |
| `seed` | INT | Yes | 0-18446744073709551615 | The seed to use for training (used in generator for LoRA weight initialization and noise sampling) (default: 0). |
| `training_dtype` | COMBO | Yes | "bf16"
"fp32"
"none" | The dtype to use for training. 'none' preserves the model's native compute dtype instead of overriding it. For fp16 models, GradScaler is automatically enabled (default: "bf16"). |
| `lora_dtype` | COMBO | Yes | "bf16"
"fp32" | The dtype to use for lora (default: "bf16"). |
| `quantized_backward` | BOOLEAN | Yes | - | When using training\_dtype 'none' and training on quantized model, doing backward with quantized matmul when enabled (default: False). |
| `algorithm` | COMBO | Yes | Multiple options available | The algorithm to use for training. |
| `gradient_checkpointing` | BOOLEAN | Yes | - | Use gradient checkpointing for training (default: True). |
| `checkpoint_depth` | INT | Yes | 1-5 | Depth level for gradient checkpointing (default: 1). |
| `offloading` | BOOLEAN | Yes | - | Offload model weights to CPU during training to save GPU memory (default: False). |
| `existing_lora` | COMBO | Yes | Multiple options available | The existing LoRA to append to. Set to None for new LoRA (default: "\[None]"). |
| `bucket_mode` | BOOLEAN | Yes | - | Enable resolution bucket mode. When enabled, expects pre-bucketed latents from ResolutionBucket node (default: False). |
| `bypass_mode` | BOOLEAN | Yes | - | Enable bypass mode for training. When enabled, adapters are applied via forward hooks instead of weight modification. Useful for quantized models where weights cannot be directly modified (default: False). |
**Note:** The number of positive conditioning inputs must match the number of latent images. If only one positive conditioning is provided with multiple images, it will be automatically repeated for all images.
**Note on `training_dtype`:** When set to "none", the model's native compute dtype is preserved. For fp16 models, GradScaler is automatically enabled to prevent underflow during gradient computation. If `fp16_accumulation` is also enabled (via `--fast` flags), this combination can be numerically unstable and may cause NaN values.
**Note on `quantized_backward`:** This parameter is only relevant when `training_dtype` is set to "none" and the model is a quantized model. It enables quantized matrix multiplication during the backward pass.
**Note on `bypass_mode`:** When enabled, adapters are applied via forward hooks instead of modifying the model weights directly. This is particularly useful for quantized models where weights cannot be directly modified.
## Outputs
| Output Name | Data Type | Description |
| ----------- | ----------- | ----------------------------------------------------------------------------------------------- |
| `lora` | LORA\_MODEL | The trained LoRA weights that can be saved or applied to other models. |
| `loss_map` | LOSS\_MAP | A dictionary containing the training loss values over time. |
| `steps` | INT | The total number of training steps completed (including any previous steps from existing LoRA). |
***
**Source fingerprint (SHA-256):** `e145145306330099293c8b4bb344ec0a38de690040bd796b84225f54638900c3`