This guide will introduce you to the basic concepts of ControlNet and demonstrate how to generate corresponding images in ComfyUI
Image source: ComfyUI ControlNet auxSince the current Comfy Core nodes do not include all types of preprocessors, in the actual examples in this documentation, we will provide pre-processed images. However, in practical use, you may need to use custom nodes to preprocess images to meet the requirements of different ControlNet models. Below are some relevant custom nodes:
Workflows
-> Open (ctrl+o)
.
This image already includes download links for the corresponding models, and dragging it into ComfyUI will automatically prompt for downloads.Load Checkpoint
can load dreamCreationVirtual3DECommerce_v10.safetensorsLoad VAE
can load vae-ft-mse-840000-ema-pruned.safetensorsUpload
in the Load Image
node to upload the input image provided earlierLoad ControlNet
can load control_v11p_sd15_scribble_fp16.safetensorsQueue
button or use the shortcut Ctrl(cmd) + Enter
to execute the image generationComfyUI\models\controlnet
will be detected by ComfyUI and can be loaded through this node.
load controlnet
and generates corresponding control conditions based on the input image.
Input Types
Parameter Name | Function |
---|---|
positive | Positive conditioning |
negative | Negative conditioning |
control_net | The ControlNet model to be applied |
image | Preprocessed image used as reference for ControlNet application |
vae | VAE model input |
strength | Strength of ControlNet application; higher values increase ControlNet’s influence on the generated image |
start_percent | Determines when to start applying ControlNet as a percentage; e.g., 0.2 means ControlNet guidance begins when 20% of diffusion is complete |
end_percent | Determines when to stop applying ControlNet as a percentage; e.g., 0.8 means ControlNet guidance stops when 80% of diffusion is complete |
Parameter Name | Function |
---|---|
positive | Positive conditioning data processed by ControlNet |
negative | Negative conditioning data processed by ControlNet |
Apply ControlNet(Old)
node in some early workflows, which is an early version of the ControlNet node. It is currently deprecated and not visible by default in search and node lists.
Show deprecated nodes in search
option. However, it’s recommended to use the new node.Control Strength
parameter in the Apply ControlNet node to control the influence of the ControlNet model on the generated image.