ggml/llama.cpp releases: b11095

TL;DR

The ggml/llama.cpp project has released updates focusing on HMX-optimized GATED_DELTA_NET, marking progress towards faster and more efficient processing. These developments are crucial as they aim to enhance the performance of large language models by optimizing hardware support.

Detailed Summary

The ggml/llama.cpp project has released updates focusing on HMX-optimized GATED_DELTA_NET, with ongoing work on integrating Hardware Matrix Extensions (HMX) support for the Gated Delta Network. The development involves initial implementation of HMX but is currently slow and not pipelined. Efforts are also underway to re-write VTCM layout handling and prepare for pipelining, which will enhance performance in future updates.

Key Points

• • hex-gdn: start putting together HMX support for GDN
• • hex-gdn: working hmx but not-pipelined and slow for now
• • hex-gdn: re-write vtcm layout handling and prep for pipelining