CLIPMergeSimple is an advanced model merging node used to combine two CLIP text encoder models based on a specified ratio. This node specializes in merging two CLIP models based on a specified ratio, effectively blending their characteristics. It selectively applies patches from one model to another, excluding specific components like position IDs and logit scale, to create a hybrid model that combines features from both source models.

​
Inputs

ParameterData TypeDescription
clip1CLIPThe first CLIP model to be merged. It serves as the base model for the merging process.
clip2CLIPThe second CLIP model to be merged. Its key patches, except for position IDs and logit scale, are applied to the first model based on the specified ratio.
ratioFLOATRange 0.0 - 1.0, determines the proportion of features from the second model to blend into the first model. A ratio of 1.0 means fully adopting the second model’s features, while 0.0 retains only the first model’s features.

​
Outputs

ParameterData TypeDescription
clipCLIPThe resulting merged CLIP model, incorporating features from both input models according to the specified ratio.

​
Merging Mechanism Explained

​
Merging Algorithm

The node uses weighted averaging to merge the two models:
  1. Clone Base Model: First clones clip1 as the base model
  2. Get Patches: Obtains all key patches from clip2
  3. Filter Special Keys: Skips keys ending with .position_ids and .logit_scale
  4. Apply Weighted Merge: Uses the formula (1.0 - ratio) * clip1 + ratio * clip2

​
Ratio Parameter Explained

  • ratio = 0.0: Fully uses clip1, ignores clip2
  • ratio = 0.5: 50% contribution from each model
  • ratio = 1.0: Fully uses clip2, ignores clip1

​
Use Cases

  1. Model Style Fusion: Combine characteristics of CLIP models trained on different data
  2. Performance Optimization: Balance strengths and weaknesses of different models
  3. Experimental Research: Explore combinations of different CLIP encoders