vLLM 0.34.0 removes Model Runner V1 and sizes its KV cache from free GPU memory
TL;DR
- The vLLM project published v0.34.0 on 16 November, with 10 new models, Mistral Large 4 among them.
- The extensible KV cache is on by default, gpu_memory_utilization defaults to 1.0, and the default block size is 64.
- Model Runner V1 is gone, so VLLM_USE_V2_MODEL_RUNNER=0 now fails at startup.