Topic: workload

The paper "Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding" addresses the issue of drafters' performance degradation when faced with workload shifts in speculative decoding, by proposing a target-agnostic pre-training approach that enhances their robustness and effectiveness. This advancement is crucial for improving the reliability and efficiency of large language model inference.

arxiv.org↗

The ggml/llama.cpp project updated its codebase to convert the FILL operation into a 2D distribution of workgroups in Vulkan, addressing an issue with Intel GPUs on Qwen 3.8, ensuring compatibility and performance optimization. This update is crucial for maintaining functionality across different hardware architectures.