BlockSparseAttention - ComfyUI Built-in Node Documentation

The Block Sparse Attention node modifies a model so its attention layers focus on only the most relevant parts of the input instead of everything at once, which reduces the computing work needed for long sequences. The savings grow with sequence length, since short sequences are usually faster with normal (dense) attention.

Inputs

Common Inputs

Parameter	Description	Data Type	Required	Range
`model`	The model to patch.	MODEL	Yes	N/A
`selection`	Method used to choose key blocks for full token-level attention (displayed as `method`). `sol-attn`: Sparsifying Online Attention uses a training-free adaptive threshold for each attention head and query block. `sla`: Sparse-Linear Attention keeps a fixed percentage of the highest-scoring key blocks; use only with model weights trained for this pattern. `vsa`: Video Sparse Attention (FastVideo) uses 3D video-cube tiling and a learned coarse attention branch; requires FastH3 model weights.	DYNAMIC_COMBO	Yes	`"sol-attn"` `"sla"` `"vsa"`
`start_percent`	Percentage point when sparse attention begins. Before this point, attention stays dense. Default: 0.2.	FLOAT	No	min: 0.0, max: 1.0, step: 0.01
`end_percent`	Percentage point when sparse attention ends. After this point, attention returns to dense. Default: 1.0.	FLOAT	No	min: 0.0, max: 1.0, step: 0.01
`dense_blocks`	Transformer blocks that always run dense, e.g. ‘0, 1, 47-49’. Default: "" (empty). Advanced input.	STRING	No	Default: ""
`min_tokens`	Sequences shorter than this stay dense. Default: 12288. Advanced input.	INT	No	min: 0, max: 1048576, step: 512
`extra_tokens`	Extra top-scoring tokens each query block attends beyond its selected blocks. Closer to dense for more attention time; 256 recommended, 0 disables. Ignored for VSA. Default: 256. Advanced input.	INT	No	min: 0, max: 256, step: 64
`sink_conditioning`	MiniMax-H3 only. `exact_kv`: every query attends the packed text/audio/reference rows exactly (about 3% cost). `exact_kv_and_rows`: additionally runs the target-audio query rows dense (keeps generated audio intact). `off` disables this behavior. Default: “exact_kv_and_rows”. Advanced input.	COMBO	No	`"exact_kv"` `"exact_kv_and_rows"` `"off"`
`verbose`	Logs whether each attention shape used sparse attention or why it stayed dense. Default: False. Advanced input.	BOOLEAN	No	Default: False

sol-attn Inputs

Parameter	Description	Data Type	Required	Range
`tau`	Threshold in score-distribution sigmas. Higher is sparser: 1.0 keeps about 16% of key blocks exact, 1.5 about 7%, 2.0 about 2.7%. Default: 1.3.	FLOAT	No	min: 0.0, max: 4.0, step: 0.05

sla Inputs

Parameter	Description	Data Type	Required	Range
`keep_percent`	Percent of key blocks each query block keeps exactly (sinks and the diagonal ride on top). The selection SLA-style LoRAs are distilled against; without such a LoRA higher is closer to dense. Default: 10.0.	FLOAT	No	min: 0.5, max: 95.0, step: 0.5

vsa Inputs

Parameter	Description	Data Type	Required	Range
`keep_percent`	Percent of video cubes each query cube keeps; FastH3-VSA checkpoints are trained at 10. Uses the model’s `to_gate_compress` layers for the coarse branch when present. Default: 10.0.	FLOAT	No	min: 0.5, max: 95.0, step: 0.5

Note: Only the parameters belonging to the currently selected method are shown in the interface.

Outputs

Output Name	Description	Data Type
`model`	The model with block-sparse attention applied.	MODEL

Constraints and Limitations

Sequences shorter than min_tokens, blocks listed in dense_blocks, and sampling steps outside the start_percent to end_percent window fall back to the dense model attention backend selected by the Model Attention Backend node.
extra_tokens is ignored when the vsa method is selected. A message is logged because VSA weights were trained against their sparse pattern.
The vsa method requires a MiniMax-H3 model; any other model raises an error. If the model lacks to_gate_compress layers, the fine stage runs without the coarse branch and a warning is logged.
dense_blocks is ignored for models that do not report block indices, which is noted in the log when verbose is enabled.
sink_conditioning applies only to MiniMax-H3 models that report a layout matching the current sequence length.

This documentation was AI-generated. If you find any errors or have suggestions for improvement, please feel free to contribute! Edit on GitHub

Source fingerprint (SHA-256): 0c34876b49a04db0ab265526e2bb5f784e150591aab631713ad2ab420a3327c4

​Inputs

​Common Inputs

​sol-attn Inputs

​sla Inputs

​vsa Inputs

​Outputs

​Constraints and Limitations