ggml/llama.cpp releases: b11114

TL;DR

The ggml/llama.cpp server was updated to fix router eviction race conditions by routing all model loads through a queue, ensuring that no model is evicted prematurely during loading. This update is crucial for improving the stability and reliability of model handling in the server.

Detailed Summary

The ggml/llama.cpp server was updated to fix router eviction races with the existing queue by routing every model load through a queue. This change ensures that all loads join the queue, preventing evictions from occurring prematurely. The update impacts model loading efficiency and reliability in the llm framework.

Key Points

• • fix router eviction races with the existing queue
• • route every model load through the queue
• • a model loaded by the fast path has no queue entry