Skip to main content

โ€‹
About OmniGen2

OmniGen2 is a powerful and efficient unified multimodal generation model with approximately 7B total parameters (3B text model + 4B image generation model). Unlike OmniGen v1, OmniGen2 adopts an innovative dual-path Transformer architecture with completely independent text autoregressive model and image diffusion model, achieving parameter decoupling and specialized optimization.

โ€‹
Model Highlights

  • Visual Understanding: Inherits the powerful image content interpretation and analysis capabilities of the Qwen-VL-2.5 foundation model
  • Text-to-Image Generation: Creates high-fidelity and aesthetically pleasing images from text prompts
  • Instruction-guided Image Editing: Performs complex, instruction-based image modifications, achieving state-of-the-art performance among open-source models
  • Contextual Generation: Versatile capabilities to process and flexibly combine diverse inputs (including people, reference objects, and scenes), producing novel and coherent visual outputs

โ€‹
Technical Features

  • Dual-path Architecture: Based on Qwen 2.5 VL (3B) text encoder + independent diffusion Transformer (4B)
  • Omni-RoPE Position Encoding: Supports multi-image spatial positioning and identity distinction
  • Parameter Decoupling Design: Avoids negative impact of text generation on image quality
  • Support for complex text understanding and image understanding
  • Controllable image generation and editing
  • Excellent detail preservation capabilities
  • Unified architecture supporting multiple image generation tasks
  • Text generation capability: Can generate clear text content within images
Make sure your ComfyUI is updated.Workflows in this guide can be found in the Workflow Templates. If you canโ€™t find them in the template, your ComfyUI may be outdated.If nodes are missing when loading a workflow, possible reasons:
  1. You are not using the latest ComfyUI version (Nightly version)
  2. Some nodes failed to import at startup

โ€‹
OmniGen2 Model Download

Since this article involves different workflows, the corresponding model files and installation locations are as follows. The download information for model files is also included in the corresponding workflows: Diffusion Models VAE Text Encoders File save location:

โ€‹
ComfyUI OmniGen2 Text-to-Image Workflow

โ€‹
OmniGen2: Text to Image

Generate high-quality images from text prompts using OmniGen2โ€™s unified 7B multimodal model with dual-path architecture. OmniGen2 text-to-image workflow preview

Run on Comfy Cloud

Open and run this workflow directly in Comfy Cloud

Download Workflow

Download JSON or search โ€œOmniGen2โ€ in Template Library
Example output OmniGen2 text-to-image example output

โ€‹
1. Download Workflow File

โ€‹
2. Complete Workflow Step by Step

Workflow Step Guide Please follow the numbered steps in the image for step-by-step confirmation to ensure smooth operation of the corresponding workflow:
  1. Load Main Model: Ensure the Load Diffusion Model node loads omnigen2_fp16.safetensors
  2. Load Text Encoder: Ensure the Load CLIP node loads qwen_2.5_vl_fp16.safetensors
  3. Load VAE: Ensure the Load VAE node loads ae.safetensors
  4. Set Image Dimensions: Set the generated image dimensions in the EmptySD3LatentImage node (recommended 1024x1024)
  5. Input Prompts:
    • Input positive prompts in the first CLipTextEncode node (content you want to appear in the image)
    • Input negative prompts in the second CLipTextEncode node (content you donโ€™t want to appear in the image)
  6. Start Generation: Click the Queue Prompt button, or use the shortcut Ctrl(cmd) + Enter to execute text-to-image generation
  7. View Results: After generation is complete, the corresponding images will be automatically saved to the ComfyUI/output/ directory, and you can also preview them in the SaveImage node

โ€‹
ComfyUI OmniGen2 Image Editing Workflow

OmniGen2 has rich image editing capabilities and supports adding text to images.

โ€‹
OmniGen2 Image Edit

Edit images with natural language instructions using OmniGen2โ€™s advanced image editing capabilities and text rendering support. OmniGen2 image edit workflow preview

Run on Comfy Cloud

Open and run this workflow directly in Comfy Cloud

Download Workflow

Download JSON or search โ€œOmniGen2 Image Editโ€ in Template Library
Input materials Upload this file to the matching LoadImage node:

image_omnigen2_image_edit_input_image.png

LoadImage node 16 ยท image_omnigen2_image_edit_input_image.png
image_omnigen2_image_edit_input_image.png
Example output
Input imageOmniGen2 image edit example output

โ€‹
1. Download Workflow File

โ€‹
2. Complete Workflow Step by Step

Workflow Step Guide
  1. Load Main Model: Ensure the Load Diffusion Model node loads omnigen2_fp16.safetensors
  2. Load Text Encoder: Ensure the Load CLIP node loads qwen_2.5_vl_fp16.safetensors
  3. Load VAE: Ensure the Load VAE node loads ae.safetensors
  4. Upload Image: Upload the provided image in the Load Image node
  5. Input Prompts:
    • Input positive prompts in the first CLipTextEncode node (content you want to appear in the image)
    • Input negative prompts in the second CLipTextEncode node (content you donโ€™t want to appear in the image)
  6. Start Generation: Click the Queue Prompt button, or use the shortcut Ctrl(cmd) + Enter to execute text-to-image generation
  7. View Results: After generation is complete, the corresponding images will be automatically saved to the ComfyUI/output/ directory, and you can also preview them in the SaveImage node

โ€‹
3. Additional Workflow Instructions

  • If you want to enable the second image input, you can use the shortcut Ctrl + B to enable the corresponding node inputs for nodes that are in pink/purple state in the workflow
  • If you want to customize dimensions, you can delete the Get image size node linked to the EmptySD3LatentImage node and input custom dimensions