Topic: splicing

Nexus introduces a depth-adaptive key-value cache and retrieval-decoupled tool routing to improve the efficiency of agentic large language models operating on the Model Context Protocol, addressing the issue of prefill time dominating due to growing tool registries. This innovation aims to reduce time-to-first-token (TTFT) by re-encoding fewer tool schemas each turn.

arxiv.org↗