ggml/llama.cpp releases: b10181

TL;DR

The ggml-cuda project updated its code to disable Multi-Memory Queue (MMQ) on devices with less than 48 KiB of shared memory, ensuring optimal performance across different hardware configurations. This update is crucial for maintaining compatibility and efficiency in various computational environments.

Detailed Summary

The ggml/llama.cpp project updated its ggml-cuda component to disable Multi-Memory Queue (MMQ) functionality on devices with less than 48 KiB of shared memory. This change was implemented in response to optimizing performance for hardware with lower shared memory capacity, ensuring that the software runs efficiently across a wider range of GPU architectures. The update impacts developers and users working with quantized models who need to ensure compatibility with various GPU configurations.

Key Points

• • ggml-cuda disables MMQ on devices with less than 48 KiB shared memory
• • ggml_cuda_should_use_mmq() chooses MMQ based on quantization type
• • Minimum 48 KiB per-block shared memory is required for current MMQ configurations