ggml/llama.cpp releases: b11064
Sentiment: neutral
TL;DR
The ggml/llama.cpp project updated its dsv4_hc_pre kernels to support arbitrary hardware contexts (hc), addressing a limitation that previously forced it to use CPU fallbacks. This change is significant as it enhances compatibility and performance across different hardware configurations, particularly for Kimi-K3 which uses varying hc values for banked checkpoints.
Detailed Summary
The ggml/llama.cpp project released a update that includes support for arbitrary high compute (hc) values in the dsv4_hc_pre kernels, addressing an issue where the operation was rejected by supports_op for any other hc value and forced to fall back to CPU execution. This change is particularly relevant for Kimi-K3, which uses this kernel with hc equal to the number of banked checkpoints. The broader impact includes improved flexibility and performance in handling different hardware configurations without relying on hardcoded values.
Key Points
- • metal support added for arbitrary hc in dsv4_hc_pre
- • Previous kernels hardcoded hc = 4
- • Op rejected by supports_op for other hc values, causing CPU fallback
- • Now works with any hc value used by dsv4_hc_pre