vLLM Hits 500K GPUs as Open-Model Push Gains $150M Backing

vLLM, the open inference engine, says half a million GPUs are now running it in production. The milestone matters because vLLM tackles a key bottleneck in large language model inference: expensive GPU memory usage. At the core is “PagedAttention,” described as a form of virtual-memory style management for the model’s KV cache. By allocating memory more efficiently on the fly, vLLM can serve far more concurrent requests. The project now supports 500+ model architectures and 200+ accelerator types. In May 2025 it also became a PyTorch Foundation project. The push for open-weight models is being backed by funding. Simon Mo—one of vLLM’s core maintainers—co-founded Inferact in 2025. Inferact raised a $150M seed round at an $800M valuation, led by a16z and Lightspeed. Mo argues open models win for three reasons: control (less dependency risk from API pricing/model changes/outages), customization (fine-tuning and quantization flexibility), and cost (API per-token fees can become expensive at production scale versus running optimized infrastructure). The “500K GPUs” figure reflects aggregated deployment scale across cloud, on-prem clusters, and enterprise environments. Typical setups use tensor parallelism and pipeline parallelism, splitting layers across GPU groups to scale throughput. For traders, this is an AI infrastructure signal rather than a direct crypto protocol or token catalyst.