ggml/llama.cpp releases: b10545
Sentiment: neutral
TL;DR
A recent update to ggml/llama.cpp addresses an issue in the tensor API mat-mat kernel by clamping K extent to ensure proper handling of non-32 multiple values, improving performance and stability. This update is significant as it enhances the efficiency of tensor operations, particularly in scenarios where K is not a multiple of 32.
Detailed Summary
The ggml/llama.cpp project released an update that includes a change in the Tensor API mat-mat kernel to clamp K extent, addressing issues with non-32 multiples in the kernel_mul_mm function. This update was made through pull request #27450 and impacts performance optimizations for tensor operations. The broader impact is improved efficiency in handling tensor multiplications where the kernel size does not evenly divide by 32.
Key Points
- • metal updates clamp K extent in tensor API mat-mat kernel
- • Adjusted static K=32 tile for matmul2d op iterations
- • Handle partial K tile on last iteration correctly