ggml/llama.cpp releases: b10199

TL;DR

The ggml/llama.cpp project released version b10199, which adds support for input embedding to generate the next token and fixes issues with server_batch(). These updates enhance the model's functionality and performance.

Detailed Summary

The ggml/llama.cpp project released version b10199, which includes server-side updates to support embedding for generating the next token and sampled tokens, along with a fix for `server_batch()`. This update impacts users of the website (<https://llama.app>) and macOS/iOS devices specifically supporting Apple Silicon (arm64). The broader impact is enhanced functionality in natural language processing tasks through improved server-side token generation capabilities.

Key Points

• • server: support inp embd to generate next token
• • server: support embd for sampled token
• • fix ~server_batch()