Topic: async

vLLM released version 0.30.0 with significant contributions from many developers, introducing new models like DeepSeek-V4.1-Flash and DeepGEMM Mega-mHC, marking a substantial update in the model's capabilities.

The ggml/llama.cpp project updated to hash-cache only weights for transfers above a certain threshold, allowing the rpc-server to serve cached files from a weight-specific cache, enhancing efficiency in model loading and management. This update is significant as it optimizes the workflow by reducing redundant data transfers and improving the performance of the model deployment process.

The ggml/llama.cpp project released version b10901, which includes a Vulkan update allowing CPU writes in asynchronous tensor copies when the context is idle. This update aims to optimize performance by leveraging CPU resources more efficiently during idle GPU contexts.