Topic: cpu

The ggml/llama.cpp project updated its dsv4_hc_pre kernels to support arbitrary hardware contexts (hc), addressing a limitation that previously forced it to use CPU fallbacks. This change is significant as it enhances compatibility and performance across different hardware configurations, particularly for Kimi-K3 which uses varying hc values for banked checkpoints.

A new programming language called Bend is designed to prevent AI mistakes by using formal proof verification, which can be executed on both CPUs and GPUs. This development is significant because it enhances the reliability of AI systems by mathematically proving their correctness, potentially reducing errors in critical applications.

bend-lang.com↗

ggml/llama.cpp released an update that includes a new SYCL backend feature using radix select for top_k operations, enabling GPU-resident processing for larger k values and improving performance by parallelizing computations across devices. This update is significant as it addresses the previous limitation of the SYCL backend for large k values, previously restricted to k = 32, thereby enhancing the efficiency of top-k operations on GPUs.

The ggml/llama.cpp project released version b10901, which includes a Vulkan update allowing CPU writes in asynchronous tensor copies when the context is idle. This update aims to optimize performance by leveraging CPU resources more efficiently during idle GPU contexts.