ggml/llama.cpp releases: b12031

TL;DR

The ggml/llama.cpp project released updates that enhance initial IQ3 support in OpenCL, specifically adding matrix multiplication methods for different memory layouts and optimizations for Adreno GPUs. These changes are crucial for improving the performance of machine learning models on mobile devices.

Detailed Summary

The ggml/llama.cpp project has released updates that include initial support for IQ3 (Integer Quantized) operations in OpenCL, specifically adding matrix multiplication and gemm/gemv functions optimized for Adreno GPUs. These changes were co-authored by Shaofei Qi and assisted by Codex. The broader impact includes enhanced performance of quantized models on mobile devices equipped with Adreno GPUs.

Key Points

• • opencl: add initial iq3 support
• • opencl: add iq3_s AOS mv and SOA mv with tiled mm
• • opencl: add iq3_s noshuffle gemm/gemv for adreno