Highlights

Model Support

• Enhanced pipeline parallelism support for DeepSeek v2 (#6519), Qwen (#6974), Qwen2 (#6924), and Nemotron (#6863)
• Enhanced vision language model support for InternVL2 (#6514, #7067), BLIP-2 (#5920), MiniCPM-V (#4087, #7122).
• Added H2O Danube3-4b (#6451)
• Added Nemotron models (Nemotron-3, Nemotron-4, Minitron) (#6611)

Hardware Support

• TPU enhancements: collective communication, TP for async engine, faster compile time (#6891, #6933, #6856, #6813, #5871)
• Intel CPU: enable multiprocessing and tensor parallelism (#6125)

Performance

We are progressing along our quest to quickly improve performance. Each of the following PRs contributed some improvements, and we anticipate more enhancements in the next release.

• Separated OpenAI Server's HTTP request handling and model inference loop with zeromq. This brought 20% speedup over time to first token and 2x speedup over inter token latency. (#6883)
• Used Python's native array data structure speedup padding. This bring 15% throughput enhancement in large batch size scenarios. (#6779)
• Reduce unnecessary compute when logprobs=None. This reduced latency of get log probs from ~30ms to ~5ms in large batch size scenarios. (#6532)
• Optimize get_seqs function, bring 2% throughput enhancements. (#7051)

Production Features

• Enhancements to speculative decoding: FlashInfer in DraftModelRunner (#6926), observability (#6963), and benchmarks (#6964)
• Refactor the punica kernel based on Triton (#5036)
• Support for guided decoding for offline LLM (#6878)

Quantization

• Support W4A8 quantization for vllm (#5218)
• Tuned FP8 and INT8 Kernels for Ada Lovelace and SM75 T4 (#6677, #6996, #6848)
• Support reading bitsandbytes pre-quantized model (#5753)

What's Changed

• [Docs] Announce llama3.1 support by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6688
• [doc][distributed] fix doc argument order by @youkaichao in https://github.com/vllm-project/vllm/pull/6691
• [Bugfix] Fix a log error in chunked prefill by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6694
• [BugFix] Fix RoPE error in Llama 3.1 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6693
• Bump version to 0.5.3.post1 by @simon-mo in https://github.com/vllm-project/vllm/pull/6696
• [Misc] Add ignored layers for fp8 quantization by @mgoin in https://github.com/vllm-project/vllm/pull/6657
• [Frontend] Add Usage data in each chunk for chat_serving. #6540 by @yecohn in https://github.com/vllm-project/vllm/pull/6652
• [Model] Pipeline Parallel Support for DeepSeek v2 by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/6519
• Bump transformers version for Llama 3.1 hotfix and patch Chameleon by @ywang96 in https://github.com/vllm-project/vllm/pull/6690
• [build] relax wheel size limit by @youkaichao in https://github.com/vllm-project/vllm/pull/6704
• [CI] Add smoke test for non-uniform AutoFP8 quantization by @mgoin in https://github.com/vllm-project/vllm/pull/6702
• [Bugfix] StatLoggers: cache spec decode metrics when they get collected. by @tdoublep in https://github.com/vllm-project/vllm/pull/6645
• [bitsandbytes]: support read bnb pre-quantized model by @thesues in https://github.com/vllm-project/vllm/pull/5753
• [Bugfix] fix flashinfer cudagraph capture for PP by @SolitaryThinker in https://github.com/vllm-project/vllm/pull/6708
• [SpecDecoding] Update MLPSpeculator CI tests to use smaller model by @njhill in https://github.com/vllm-project/vllm/pull/6714
• [Bugfix] Fix token padding for chameleon by @ywang96 in https://github.com/vllm-project/vllm/pull/6724
• [Docs][ROCm] Detailed instructions to build from source by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6680
• [Build/CI] Update run-amd-test.sh. Enable Docker Hub login. by @Alexei-V-Ivanov-AMD in https://github.com/vllm-project/vllm/pull/6711
• [Bugfix]fix modelscope compatible issue by @liuyhwangyh in https://github.com/vllm-project/vllm/pull/6730
• Adding f-string to validation error which is missing by @luizanao in https://github.com/vllm-project/vllm/pull/6748
• [Bugfix] Fix speculative decode seeded test by @njhill in https://github.com/vllm-project/vllm/pull/6743
• [Bugfix] Miscalculated latency lead to time_to_first_token_seconds inaccurate. by @AllenDou in https://github.com/vllm-project/vllm/pull/6686
• [Frontend] split run_server into build_server and run_server by @dtrifiro in https://github.com/vllm-project/vllm/pull/6740
• [Kernels] Add fp8 support to reshape_and_cache_flash by @Yard1 in https://github.com/vllm-project/vllm/pull/6667
• [Core] Tweaks to model runner/input builder developer APIs by @Yard1 in https://github.com/vllm-project/vllm/pull/6712
• [Bugfix] Bump transformers to 4.43.2 by @mgoin in https://github.com/vllm-project/vllm/pull/6752
• [Doc][AMD][ROCm]Added tips to refer to mi300x tuning guide for mi300x users by @hongxiayang in https://github.com/vllm-project/vllm/pull/6754
• [core][distributed] fix zmq hang by @youkaichao in https://github.com/vllm-project/vllm/pull/6759
• [Frontend] Represent tokens with identifiable strings by @ezliu in https://github.com/vllm-project/vllm/pull/6626
• [Model] Adding support for MiniCPM-V by @HwwwwwwwH in https://github.com/vllm-project/vllm/pull/4087
• [Bugfix] Fix decode tokens w. CUDA graph by @comaniac in https://github.com/vllm-project/vllm/pull/6757
• [Bugfix] Fix awq_marlin and gptq_marlin flags by @alexm-neuralmagic in https://github.com/vllm-project/vllm/pull/6745
• [Bugfix] Fix encoding_format in examples/openai_embedding_client.py by @CatherineSue in https://github.com/vllm-project/vllm/pull/6755
• [Bugfix] Add image placeholder for OpenAI Compatible Server of MiniCPM-V by @HwwwwwwwH in https://github.com/vllm-project/vllm/pull/6787
• [ Misc ] fp8-marlin channelwise via compressed-tensors by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/6524
• [Bugfix] Fix kv_cache_dtype=fp8 without scales for FP8 checkpoints by @mgoin in https://github.com/vllm-project/vllm/pull/6761
• [Bugfix] Add synchronize to prevent possible data race by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6788
• [Doc] Add documentations for nightly benchmarks by @KuntaiDu in https://github.com/vllm-project/vllm/pull/6412
• [Bugfix] Fix empty (nullptr) channelwise scales when loading wNa16 using compressed tensors by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/6798
• [doc][distributed] improve multinode serving doc by @youkaichao in https://github.com/vllm-project/vllm/pull/6804
• [Docs] Publish 5th meetup slides by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6799
• [Core] Fix ray forward_dag error mssg by @rkooo567 in https://github.com/vllm-project/vllm/pull/6792
• [ci][distributed] fix flaky tests by @youkaichao in https://github.com/vllm-project/vllm/pull/6806
• [ci] Mark tensorizer test as soft fail and separate it from grouped test in fast check by @khluu in https://github.com/vllm-project/vllm/pull/6810
• Fix ReplicatedLinear weight loading by @qingquansong in https://github.com/vllm-project/vllm/pull/6793
• [Bugfix] [Easy] Fixed a bug in the multiprocessing GPU executor. by @eaplatanios in https://github.com/vllm-project/vllm/pull/6770
• [Core] Use array to speedup padding by @peng1999 in https://github.com/vllm-project/vllm/pull/6779
• [doc][debugging] add known issues for hangs by @youkaichao in https://github.com/vllm-project/vllm/pull/6816
• [Model] Support Nemotron models (Nemotron-3, Nemotron-4, Minitron) by @mgoin in https://github.com/vllm-project/vllm/pull/6611
• [Bugfix][Kernel] Promote another index to int64_t by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6838
• [Build/CI][ROCm] Minor simplification to Dockerfile.rocm by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6811
• [Misc][TPU] Support TPU in initialize_ray_cluster by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6812
• [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/6125
• [Doc] Add Nemotron to supported model docs by @mgoin in https://github.com/vllm-project/vllm/pull/6843
• [Doc] Update SkyPilot doc for wrong indents and instructions for update service by @Michaelvll in https://github.com/vllm-project/vllm/pull/4283
• Update README.md by @gurpreet-dhami in https://github.com/vllm-project/vllm/pull/6847
• enforce eager mode with bnb quantization temporarily by @chenqianfzh in https://github.com/vllm-project/vllm/pull/6846
• [TPU] Support collective communications in XLA devices by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6813
• [Frontend] Factor out code for running uvicorn by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/6828
• [Bug Fix] Illegal memory access, FP8 Llama 3.1 405b by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/6852
• [Bugfix]: Fix Tensorizer test failures by @sangstar in https://github.com/vllm-project/vllm/pull/6835
• [ROCm] Upgrade PyTorch nightly version by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6845
• [Doc] add VLLM_TARGET_DEVICE=neuron to documentation for neuron by @omrishiv in https://github.com/vllm-project/vllm/pull/6844
• [Bugfix][Model] Jamba assertions and no chunked prefill by default for Jamba by @tomeras91 in https://github.com/vllm-project/vllm/pull/6784
• [Model] H2O Danube3-4b by @g-eoj in https://github.com/vllm-project/vllm/pull/6451
• [Hardware][TPU] Implement tensor parallelism with Ray by @WoosukKwon in https://github.com/vllm-project/vllm/pull/5871
• [Doc] Add missing mock import to docs conf.py by @hmellor in https://github.com/vllm-project/vllm/pull/6834
• [Bugfix] Use torch.set_num_threads() to configure parallelism in multiproc_gpu_executor by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/6802
• [Misc][VLM][Doc] Consolidate offline examples for vision language models by @ywang96 in https://github.com/vllm-project/vllm/pull/6858
• [Bugfix] Fix VLM example typo by @ywang96 in https://github.com/vllm-project/vllm/pull/6859
• [bugfix] make args.stream work by @WrRan in https://github.com/vllm-project/vllm/pull/6831
• [CI/Build][Doc] Update CI and Doc for VLM example changes by @ywang96 in https://github.com/vllm-project/vllm/pull/6860
• [Model] Initial support for BLIP-2 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/5920
• [Docs] Add RunLLM chat widget by @cw75 in https://github.com/vllm-project/vllm/pull/6857
• [TPU] Reduce compilation time & Upgrade PyTorch XLA version by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6856
• [Kernel] Increase precision of GPTQ/AWQ Marlin kernel by @alexm-neuralmagic in https://github.com/vllm-project/vllm/pull/6795
• Add Nemotron to PP_SUPPORTED_MODELS by @mgoin in https://github.com/vllm-project/vllm/pull/6863
• [Misc] Pass cutlass_fp8_supported correctly in fbgemm_fp8 by @zeyugao in https://github.com/vllm-project/vllm/pull/6871
• [Model] Initialize support for InternVL2 series models by @Isotr0py in https://github.com/vllm-project/vllm/pull/6514
• [Kernel] Tuned FP8 Kernels for Ada Lovelace by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/6677
• [Core] Reduce unnecessary compute when logprobs=None by @peng1999 in https://github.com/vllm-project/vllm/pull/6532
• [Kernel] Fix deprecation function warnings squeezellm quant_cuda_kernel by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6901
• [TPU] Add TPU tensor parallelism to async engine by @etwk in https://github.com/vllm-project/vllm/pull/6891
• [Bugfix] Allow vllm to still work if triton is not installed. by @tdoublep in https://github.com/vllm-project/vllm/pull/6786
• [Frontend] New allowed_token_ids decoding request parameter by @njhill in https://github.com/vllm-project/vllm/pull/6753
• [Kernel] Remove unused variables in awq/gemm_kernels.cu by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6908
• [ci] GHA workflow to remove ready label upon "/notready" comment by @khluu in https://github.com/vllm-project/vllm/pull/6921
• [Kernel] Fix marlin divide-by-zero warnings by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6904
• [Kernel] Tuned int8 kernels for Ada Lovelace by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/6848
• [TPU] Fix greedy decoding by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6933
• [Bugfix] Fix PaliGemma MMP by @ywang96 in https://github.com/vllm-project/vllm/pull/6930
• [Doc] Super tiny fix doc typo by @fzyzcjy in https://github.com/vllm-project/vllm/pull/6949
• [BugFix] Fix use of per-request seed with pipeline parallel by @njhill in https://github.com/vllm-project/vllm/pull/6698
• [Kernel] Squash a few more warnings by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6914
• [OpenVINO] Updated OpenVINO requirements and build docs by @ilya-lavrenov in https://github.com/vllm-project/vllm/pull/6948
• [Bugfix] Fix tensorizer memory profiling bug during testing by @sangstar in https://github.com/vllm-project/vllm/pull/6881
• [Kernel] Remove scaled_fp8_quant kernel padding footgun by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/6842
• [core][misc] improve free_finished_seq_groups by @youkaichao in https://github.com/vllm-project/vllm/pull/6865
• [Build] Temporarily Disable Kernels and LoRA tests by @simon-mo in https://github.com/vllm-project/vllm/pull/6961
• [Nightly benchmarking suite] Remove pkill python from run benchmark suite by @cadedaniel in https://github.com/vllm-project/vllm/pull/6965
• [CI] [nightly benchmark] Do not re-download sharegpt dataset if exists by @cadedaniel in https://github.com/vllm-project/vllm/pull/6706
• [Speculative decoding] Add serving benchmark for llama3 70b + speculative decoding by @cadedaniel in https://github.com/vllm-project/vllm/pull/6964
• [mypy] Enable following imports for some directories by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/6681
• [Bugfix] Fix broadcasting logic for multi_modal_kwargs by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/6836
• [CI/Build] Fix mypy errors by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/6968
• [Bugfix][TPU] Set readonly=True for non-root devices by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6980
• [Bugfix] fix logit processor excceed vocab size issue by @FeiDeng in https://github.com/vllm-project/vllm/pull/6927
• Support W4A8 quantization for vllm by @HandH1998 in https://github.com/vllm-project/vllm/pull/5218
• [Bugfix] Clean up MiniCPM-V by @HwwwwwwwH in https://github.com/vllm-project/vllm/pull/6939
• [Bugfix] Fix feature size calculation for LLaVA-NeXT by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/6982
• [Model] use FusedMoE layer in Jamba by @avshalomman in https://github.com/vllm-project/vllm/pull/6935
• [MISC] Introduce pipeline parallelism partition strategies by @comaniac in https://github.com/vllm-project/vllm/pull/6920
• [Bugfix] Support cpu offloading with quant_method.process_weights_after_loading by @mgoin in https://github.com/vllm-project/vllm/pull/6960
• [Kernel] Enable FP8 Cutlass for Ada Lovelace by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/6950
• [Kernel] Tuned int8 Cutlass Kernels for SM75 (T4) by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/6996
• [Misc] Add compressed-tensors to optimized quant list by @mgoin in https://github.com/vllm-project/vllm/pull/7006
• Revert "[Frontend] Factor out code for running uvicorn" by @simon-mo in https://github.com/vllm-project/vllm/pull/7012
• [Kernel][RFC] Refactor the punica kernel based on Triton by @jeejeelee in https://github.com/vllm-project/vllm/pull/5036
• [Model] Pipeline parallel support for Qwen2 by @xuyi in https://github.com/vllm-project/vllm/pull/6924
• [Bugfix][TPU] Do not use torch.Generator for TPUs by @WoosukKwon in https://github.com/vllm-project/vllm/pull/6981
• [Bugfix][Model] Skip loading lm_head weights if using tie_word_embeddings by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/6758
• PP comm optimization: replace send with partial send + allgather by @aurickq in https://github.com/vllm-project/vllm/pull/6695
• [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user by @zifeitong in https://github.com/vllm-project/vllm/pull/6954
• [core][scheduler] simplify and improve scheduler by @youkaichao in https://github.com/vllm-project/vllm/pull/6867
• [Build/CI] Fixing Docker Hub quota issue. by @Alexei-V-Ivanov-AMD in https://github.com/vllm-project/vllm/pull/7043
• [CI/Build] Update torch to 2.4 by @SageMoore in https://github.com/vllm-project/vllm/pull/6951
• [Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm by @Isotr0py in https://github.com/vllm-project/vllm/pull/6992
• [CI/Build] Remove sparseml requirement from testing by @mgoin in https://github.com/vllm-project/vllm/pull/7037
• [Bugfix] Lower gemma's unloaded_params exception to warning by @mgoin in https://github.com/vllm-project/vllm/pull/7002
• [Models] Support Qwen model with PP by @andoorve in https://github.com/vllm-project/vllm/pull/6974
• Update run-amd-test.sh by @okakarpa in https://github.com/vllm-project/vllm/pull/7044
• [Misc] Support attention logits soft-capping with flash-attn by @WoosukKwon in https://github.com/vllm-project/vllm/pull/7022
• [CI/Build][Bugfix] Fix CUTLASS header-only line by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/7034
• [Performance] Optimize get_seqs by @WoosukKwon in https://github.com/vllm-project/vllm/pull/7051
• [Kernel] Fix input for flashinfer prefill wrapper. by @LiuXiaoxuanPKU in https://github.com/vllm-project/vllm/pull/7008
• [mypy] Speed up mypy checking by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/7056
• [ci][distributed] try to fix pp test by @youkaichao in https://github.com/vllm-project/vllm/pull/7054
• Fix tracing.py by @bong-furiosa in https://github.com/vllm-project/vllm/pull/7065
• [cuda][misc] remove error_on_invalid_device_count_status by @youkaichao in https://github.com/vllm-project/vllm/pull/7069
• [Core] Comment out unused code in sampler by @peng1999 in https://github.com/vllm-project/vllm/pull/7023
• [Hardware][Intel CPU] Update torch 2.4.0 for CPU backend by @DamonFool in https://github.com/vllm-project/vllm/pull/6931
• [ci] set timeout for test_oot_registration.py by @youkaichao in https://github.com/vllm-project/vllm/pull/7082
• [CI/Build] Add support for Python 3.12 by @mgoin in https://github.com/vllm-project/vllm/pull/7035
• [Misc] Disambiguate quantized types via a new ScalarType by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/6396
• [Core] Pipeline parallel with Ray ADAG by @ruisearch42 in https://github.com/vllm-project/vllm/pull/6837
• [Misc] Revive to use loopback address for driver IP by @ruisearch42 in https://github.com/vllm-project/vllm/pull/7091
• [misc] add a flag to enable compile by @youkaichao in https://github.com/vllm-project/vllm/pull/7092
• [ Frontend ] Multiprocessing for OpenAI Server with zeromq by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/6883
• [ci][distributed] shorten wait time if server hangs by @youkaichao in https://github.com/vllm-project/vllm/pull/7098
• [Frontend] Factor out chat message parsing by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/7055
• [ci][distributed] merge distributed test commands by @youkaichao in https://github.com/vllm-project/vllm/pull/7097
• [ci][distributed] disable ray dag tests by @youkaichao in https://github.com/vllm-project/vllm/pull/7099
• [Model] Refactor and decouple weight loading logic for InternVL2 model by @Isotr0py in https://github.com/vllm-project/vllm/pull/7067
• [Bugfix] Fix block table for seqs that have prefix cache hits by @zachzzc in https://github.com/vllm-project/vllm/pull/7018
• [LoRA] ReplicatedLinear support LoRA by @jeejeelee in https://github.com/vllm-project/vllm/pull/7081
• [CI] Temporarily turn off H100 performance benchmark by @KuntaiDu in https://github.com/vllm-project/vllm/pull/7104
• [ci][test] finalize fork_new_process_for_each_test by @youkaichao in https://github.com/vllm-project/vllm/pull/7114
• [Frontend] Warn if user max_model_len is greater than derived max_model_len by @fialhocoelho in https://github.com/vllm-project/vllm/pull/7080
• Support for guided decoding for offline LLM by @kevinbu233 in https://github.com/vllm-project/vllm/pull/6878
• [misc] add zmq in collect env by @youkaichao in https://github.com/vllm-project/vllm/pull/7119
• [core][misc] simply output processing with shortcut for non-parallel sampling and non-beam search usecase by @youkaichao in https://github.com/vllm-project/vllm/pull/7117
• [Model]Refactor MiniCPMV by @jeejeelee in https://github.com/vllm-project/vllm/pull/7020
• [Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator by @tdoublep in https://github.com/vllm-project/vllm/pull/7105
• [misc][distributed] improve libcudart.so finding by @youkaichao in https://github.com/vllm-project/vllm/pull/7127
• Clean up remaining Punica C information by @jeejeelee in https://github.com/vllm-project/vllm/pull/7027
• [Model] Add multi-image support for minicpmv offline inference by @HwwwwwwwH in https://github.com/vllm-project/vllm/pull/7122
• [Frontend] Reapply "Factor out code for running uvicorn" by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/7095
• [Model] SiglipVisionModel ported from transformers by @ChristopherCho in https://github.com/vllm-project/vllm/pull/6942
• [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification by @cadedaniel in https://github.com/vllm-project/vllm/pull/6963
• [SpecDecode] Support FlashInfer in DraftModelRunner by @bong-furiosa in https://github.com/vllm-project/vllm/pull/6926
• [BugFix] Use IP4 localhost form for zmq bind by @njhill in https://github.com/vllm-project/vllm/pull/7163
• [BugFix] Use args.trust_remote_code by @VastoLorde95 in https://github.com/vllm-project/vllm/pull/7121
• [Misc] Fix typo in GroupCoordinator.recv() by @ruisearch42 in https://github.com/vllm-project/vllm/pull/7167
• [Kernel] Update CUTLASS to 3.5.1 by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/7085
• [CI/Build] Suppress divide-by-zero and missing return statement warnings by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/7001
• [Bugfix][CI/Build] Fix CUTLASS FetchContent by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/7171
• bump version to v0.5.4 by @simon-mo in https://github.com/vllm-project/vllm/pull/7139

New Contributors

• @yecohn made their first contribution in https://github.com/vllm-project/vllm/pull/6652
• @thesues made their first contribution in https://github.com/vllm-project/vllm/pull/5753
• @luizanao made their first contribution in https://github.com/vllm-project/vllm/pull/6748
• @ezliu made their first contribution in https://github.com/vllm-project/vllm/pull/6626
• @HwwwwwwwH made their first contribution in https://github.com/vllm-project/vllm/pull/4087
• @LucasWilkinson made their first contribution in https://github.com/vllm-project/vllm/pull/6798
• @qingquansong made their first contribution in https://github.com/vllm-project/vllm/pull/6793
• @eaplatanios made their first contribution in https://github.com/vllm-project/vllm/pull/6770
• @gurpreet-dhami made their first contribution in https://github.com/vllm-project/vllm/pull/6847
• @omrishiv made their first contribution in https://github.com/vllm-project/vllm/pull/6844
• @cw75 made their first contribution in https://github.com/vllm-project/vllm/pull/6857
• @zeyugao made their first contribution in https://github.com/vllm-project/vllm/pull/6871
• @etwk made their first contribution in https://github.com/vllm-project/vllm/pull/6891
• @fzyzcjy made their first contribution in https://github.com/vllm-project/vllm/pull/6949
• @FeiDeng made their first contribution in https://github.com/vllm-project/vllm/pull/6927
• @HandH1998 made their first contribution in https://github.com/vllm-project/vllm/pull/5218
• @xuyi made their first contribution in https://github.com/vllm-project/vllm/pull/6924
• @bong-furiosa made their first contribution in https://github.com/vllm-project/vllm/pull/7065
• @zachzzc made their first contribution in https://github.com/vllm-project/vllm/pull/7018
• @fialhocoelho made their first contribution in https://github.com/vllm-project/vllm/pull/7080
• @ChristopherCho made their first contribution in https://github.com/vllm-project/vllm/pull/6942
• @VastoLorde95 made their first contribution in https://github.com/vllm-project/vllm/pull/7121

Full Changelog: https://github.com/vllm-project/vllm/compare/v0.5.3...v0.5.4