CLIPMergeSimple is an advanced model merging node used to combine two CLIP text encoder models based on a specified ratio.
This node specializes in merging two CLIP models based on a specified ratio, effectively blending their characteristics. It selectively applies patches from one model to another, excluding specific components like position IDs and logit scale, to create a hybrid model that combines features from both source models.
Inputs
| Parameter | Description | Data Type | Input Type | Default | Range |
|---|---|---|---|---|---|
clip1 | The first CLIP model to be merged. It serves as the base model for the merging process. | CLIP | REQUIRED | - | - |
clip2 | The second CLIP model to be merged. Its key patches, except for position IDs and logit scale, are applied to the first model based on the specified ratio. | CLIP | REQUIRED | - | - |
ratio | Determines the proportion of features from the second model to blend into the first model. A ratio of 1.0 means fully adopting the second model’s features, while 0.0 retains only the first model’s features. | FLOAT | REQUIRED | 1.0 | 0.0 - 1.0 (step: 0.01) |
Outputs
| Output Name | Description | Data Type |
|---|---|---|
clip | The resulting merged CLIP model, incorporating features from both input models according to the specified ratio. | CLIP |
Merging Mechanism Explained
Merging Algorithm
The node uses weighted averaging to merge the two models:- Clone Base Model: First clones
clip1as the base model - Get Patches: Obtains all key patches from
clip2 - Filter Special Keys: Skips keys ending with
.position_idsand.logit_scale - Apply Weighted Merge: Uses the formula
(1.0 - ratio) * clip1 + ratio * clip2
Ratio Parameter Explained
- ratio = 0.0: Fully uses clip1, ignores clip2
- ratio = 0.5: 50% contribution from each model
- ratio = 1.0: Fully uses clip2, ignores clip1
Use Cases
- Model Style Fusion: Combine characteristics of CLIP models trained on different data
- Performance Optimization: Balance strengths and weaknesses of different models
- Experimental Research: Explore combinations of different CLIP encoders
This documentation was AI-generated. If you find any errors or have suggestions for improvement, please feel free to contribute! Edit on GitHub
Source fingerprint (SHA-256):
0d3c8388dbe88675ea7fb51161ab41ce898bcf63983b3d2817b16ec5bfa613e5