This guide demonstrates how to use Wan2.1 Fun Control in ComfyUI to generate videos with control videos
Wan2.1-Fun-14B-Control.safetensors
after downloadingLoad Image
node doesn’t currently support mp4 format. We also use Canny Edge to preprocess the original video.
Because many users encounter installation failures and environment issues when installing custom nodes, this version of the workflow uses only native nodes to ensure a smoother experience.
Thanks to our powerful ComfyUI authors who provide feature-rich nodes. If you want to directly check the related version, see Workflow Using Custom Nodes.
Load Diffusion Model
node has loaded wan2.1_fun_control_1.3B_bf16.safetensors
Load CLIP
node has loaded umt5_xxl_fp8_e4m3fn_scaled.safetensors
Load VAE
node has loaded wan_2.1_vae.safetensors
Load CLIP Vision
node has loaded clip_vision_h.safetensors
Load Image
node (renamed to Start_image
)Load Image
node. Note: This node currently doesn’t support mp4, only WebP videosWanFunControlToVideo
, avoiding overly large dimensionsRun
button or use the shortcut Ctrl(cmd) + Enter
to execute video generationWanFunControlToVideo
node, if the specified frame count exceeds the actual control video frames, the excess frames may display scenes not conforming to control conditions. We’ll address this issue in the Workflow Using Custom NodesThe model part is essentially the same. If you’ve already experienced the native-only workflow, you can directly upload the corresponding images and run it.
Load Diffusion Model
node has loaded wan2.1_fun_control_1.3B_bf16.safetensors
Load CLIP
node has loaded umt5_xxl_fp8_e4m3fn_scaled.safetensors
Load VAE
node has loaded wan_2.1_vae.safetensors
Load CLIP Vision
node has loaded clip_vision_h.safetensors
Load Image
nodeLoad Video(Upload)
custom node. Note that the workflow has adjusted the default frame_load_cap
DWPose Estimator
only uses the detect_face
optionWanFunControlToVideo
, avoiding overly large dimensionsRun
button or use the shortcut Ctrl(cmd) + Enter
to execute video generationLoad Video(Upload)
to support mp4 videosvideo_info
obtained from Load Video(Upload)
allows us to maintain the same fps
for the output videoDWPose Estimator
with other preprocessors from the ComfyUI-comfyui_controlnet_aux
node packageImage Blend
node to achieve the goal of applying multiple control methods simultaneously.
Video Combine
node from ComfyUI-VideoHelperSuite
to save videos in mp4 format
SaveAnimatedWEBP
because we currently don’t support embedding workflow into mp4 and some other custom nodes may not support embedding workflow too. To preserve the workflow in the video, we choose SaveAnimatedWEBP
node.
WanFunControlToVideo
node, control_video
is not mandatory, so sometimes you can skip using a control video, first generate a very small video size like 320x320, and then use them as control video input to achieve consistent results.