MATRIX ยท llama.cpp https://derekwang2002.github.io/ai-intelligence/en/projects/llama-cpp/ Material changes in AI models, agents, engineering and infrastructure, with primary sources and recommendations. en Mon, 02 Nov 2026 16:56:00 GMT llama.cpp 0.7.0 adds MoE caching and changes its server API https://derekwang2002.github.io/ai-intelligence/en/events/ev-20261015-04/ ev-20261015-04-published Fri, 16 Oct 2026 15:56:07 GMT New TRIAL <p>llama.cpp 0.7.0 adds a GPU LRU cache for MoE experts kept in host memory, LiLiCorr speculative decoding and support for more model architectures. It also changes the non-Docker server port from 8080 to 9931, returns formatted chat as an array of parts, and advances the RPC protocol to v8. The reported 2.2x decode gain is a project measurement, not an independent benchmark.</p><p><strong>Why read: </strong>llama.cpp 0.7.0 may improve MoE throughput, while port, response-shape and RPC changes require upgrade regression.</p><p>Recommendation: TRIAL</p><p>Sources: <a href="https://github.com/ggml-org/llama.cpp/releases/tag/v0.7.0">llama.cpp v0.7.0</a></p>