ggml/llama.cpp releases: b10831

TL;DR

The ggml/llama.cpp project added support for TQ1_0 in Vulkan, enhancing matrix operations and dequantization processes. This update is significant as it optimizes memory usage and performance for quantized models.

Detailed Summary

The ggml/llama.cpp project has released updates that include adding TQ1_0 support for matrix multiplication, vector-matrix operations, and dequantization in Vulkan. These changes also involve packing TQ1_0 powers of 3 into a 32-bit constant to optimize performance. The broader impact includes improved efficiency in handling specific types of computations relevant to machine learning models on GPU architectures.

Key Points

• • vulkan: add TQ1_0 support (mm, mat-vec, dequant, get_rows)
• • vulkan: pack TQ1_0 powers of 3 into a 32-bit constant