Inputs
Note: When context windows are used, each window is cached separately, so memory usage scales with the window count. The static_standard schedule should be used, because uniform schedules shift the windows on every step and the cache is never reused.
Outputs
This documentation was AI-generated. If you find any errors or have suggestions for improvement, please feel free to contribute! Edit on GitHub
Source fingerprint (SHA-256):
06305432601afd7c797ef29ef4be3f2bb1aa660e05edde270499e94ccdd54f84