ggml/llama.cpp releases: b11043
Sentiment: neutral
TL;DR
The ggml/llama.cpp project released an update that enables the HMX flash-attention mechanism to support head_dim values not divisible by 64, enhancing flexibility in model configurations. This update is significant as it broadens compatibility and potential optimizations for models like SigLIP with specific head_dim requirements.
Detailed Summary
The ggml/llama.cpp project has released an update that includes a patch allowing the HMX flash-attention mechanism to handle head_dim values not divisible by 64, specifically supporting a head_dim of 72. This update is part of pull request #26539 and enhances compatibility with models like SigLIP. The broader impact benefits developers working with attention mechanisms in large language models, potentially improving the flexibility and efficiency of their applications.
Key Points
- • HMX flash-attention now supports head_dim padding for DK=DV=72
- • Allows running HMX flash-attention with non-multiple-of-64 head_dim
- • Operates on DK/DV rounded up to 64 with zero-filled tail lanes