vLLM 0.33.0 ships CUDA 13.0-only wheels and images and drops CUDA 12.9

TL;DR

  • On 2 November the vLLM project published 0.33.0, with 927 commits from 389 contributors.
  • Wheels and Docker images are now CUDA 13.0 only, vLLM moves to PyTorch 2.14.0, and a /v1/decisions endpoint is on by default for supported generation models.
  • Medusa, the MLP speculator and the legacy Ray compiled-graph executor are gone.

Read the full story

Sign in with your email to read AI News. It’s free.

Share this story

Explain like I’m 15