​
Introduction to T2I Adapter

T2I-Adapter is a lightweight adapter developed by Tencent ARC Lab designed to enhance the structural, color, and style control capabilities of text-to-image generation models (such as Stable Diffusion). It works by aligning external conditions (such as edge detection maps, depth maps, sketches, or color reference images) with the model’s internal features, achieving high-precision control without modifying the original model structure. With only about 77M parameters (approximately 300MB in size), its inference speed is about 3 times faster than ControlNet, and it supports multiple condition combinations (such as sketch + color grid). Application scenarios include line art to image conversion, color style transfer, multi-element scene generation, and more.

​
Comparison Between T2I Adapter and ControlNet

Although their functions are similar, there are notable differences in implementation and application:

  1. Lightweight Design: T2I Adapter has fewer parameters and a smaller memory footprint
  2. Inference Speed: T2I Adapter is typically about 3 times faster than ControlNet
  3. Control Precision: ControlNet offers more precise control in certain scenarios, while T2I Adapter is more suitable for lightweight control
  4. Multi-condition Combination: T2I Adapter shows more significant resource advantages when combining multiple conditions

​
Main Types of T2I Adapter

T2I Adapter provides various types to control different aspects:

  • Depth: Controls the spatial structure and depth relationships in images
  • Line Art (Canny/Sketch): Controls image edges and lines
  • Keypose: Controls character poses and actions
  • Segmentation (Seg): Controls scene layout through semantic segmentation
  • Color: Controls the overall color scheme of images

In ComfyUI, using T2I Adapter is similar to ControlNet in terms of interface and workflow. In this example, we will demonstrate how to use a depth T2I Adapter to control an interior scene.

​
Value of Depth T2I Adapter Applications

Depth maps have several important applications in image generation:

  1. Spatial Layout Control: Accurately describes three-dimensional spatial structures, suitable for interior design and architectural visualization
  2. Object Positioning: Controls the relative position and size of objects in a scene, suitable for product showcases and scene construction
  3. Perspective Relationships: Maintains reasonable perspective and proportions, suitable for landscape and urban scene generation
  4. Light and Shadow Layout: Natural light and shadow distribution based on depth information, enhancing realism

We will use interior design as an example to demonstrate how to use the depth T2I Adapter, but these techniques are applicable to other scenarios as well.

​
ComfyUI Depth T2I Adapter Workflow Example Explanation

​
1. Depth T2I Adapter Workflow Assets

Please download the workflow image below and drag it into ComfyUI to load the workflow:

Images with workflow JSON in their metadata can be directly dragged into ComfyUI or loaded using the menu Workflows -> Open (ctrl+o). This image already includes download links for the corresponding models, and dragging it into ComfyUI will automatically prompt for downloads.

Please download the image below, which we will use as input:

​
2. Model Installation

If your network cannot successfully complete the automatic download of the corresponding models, please try manually downloading the models below and placing them in the specified directories:

ComfyUI/
├── models/
│   ├── checkpoints/
│   │   └── interiordesignsuperm_v2.safetensors
│   └── controlnet/
│       └── t2iadapter_depth_sd15v2.pth

​
3. Step-by-Step Workflow Execution

  1. Ensure that Load Checkpoint can load interiordesignsuperm_v2.safetensors
  2. Ensure that Load ControlNet can load t2iadapter_depth_sd15v2.pth
  3. Click Upload in the Load Image node to upload the input image provided earlier
  4. Click the Queue button or use the shortcut Ctrl(cmd) + Enter to execute the image generation

​
General Tips for Using T2I Adapter

​
Input Image Quality Optimization

Regardless of the application scenario, high-quality input images are key to successfully using T2I Adapter:

  1. Moderate Contrast: Control images (such as depth maps, line art) should have clear contrast, but not excessively extreme
  2. Clear Boundaries: Ensure that major structures and element boundaries are clearly distinguishable in the control image
  3. Noise Control: Try to avoid excessive noise in control images, especially for depth maps and line art
  4. Reasonable Layout: Control images should have a reasonable spatial layout and element distribution

​
Characteristics of T2I Adapter Usage

One major advantage of T2I Adapter is its ability to easily combine multiple conditions for complex control effects:

  1. Depth + Edge: Control spatial layout while maintaining clear structural edges, suitable for architecture and interior design
  2. Line Art + Color: Control shapes while specifying color schemes, suitable for character design and illustrations
  3. Pose + Segmentation: Control character actions while defining scene areas, suitable for complex narrative scenes

Mixing different T2I Adapters, or combining them with other control methods (such as ControlNet, regional prompts, etc.), can further expand creative possibilities. To achieve mixing, simply chain multiple Apply ControlNet nodes together in the same way as described in Mixing ControlNet.

Was this page helpful?

Suggest edits