ggml/llama.cpp releases: b10907

TL;DR

The ggml/llama.cpp project released updates to fix MTP context kv cache allocation issues for specific architectures like deepseek2, glm4moe, and cohere2moe. These changes also include adding inverse architecture gating and comprehensive testing for the MTP layer filter, enhancing model stability and performance.

Detailed Summary

The ggml/llama.cpp project released updates addressing issues with memory cache allocation for specific model architectures. The changes include fixes for the deepseek2, glm4moe, and cohere2moe architectures, as well as additions to inverse architecture gating and comprehensive testing for MTP layer filtering. These updates aim to enhance the performance and reliability of the models across various applications.

Key Points

• • Fix MTP context KV cache allocation for deepseek2, glm4moe, cohere2moe architectures
• • Add inverse architecture gating
• • Perform comprehensive architecture testing for MTP layer filter