llama.cpp 0.7.0 adds MoE caching and changes its server API
llama.cpp 0.7.0 adds MoE caching and changes its server API
2026-10-16Verified announcementTRIAL
llama.cpp 0.7.0 adds a GPU LRU cache for MoE experts kept in host memory, LiLiCorr speculative decoding and support for more model architectures. It also changes the non-Docker server port from 8080 to 9931, returns formatted chat as an array of parts, and advances the RPC protocol to v8. The reported 2.2x decode gain is a project measurement, not an independent benchmark.
What this means for you
For teams serving GGUF and MoE models, the cache may improve throughput, while the port, response-shape and RPC changes can break clients. Upgrade only with compatibility and output checks.
Evidence & related material
Technical details & scores
| Release | 0.7.0 stable |
|---|---|
| Moe Cache | GPU LRU for experts stored in host memory |
| Reported Gain | up to 2.2x decode, project-reported |
| Breaking Changes | server port 9931 outside Docker; formatted chat parts array; RPC v8 |
Technical impact 3
Engineering value 4
Adoption signal 4
Maturity 4
Verification cost 2
Risk 2