Major changes

Models

• New models: Command+R(#3433), Qwen2 MoE(#3346), DBRX(#3660), XVerse (#3610), Jais (#3183).
• New vision language model: LLaVA (#3042)

Production features

• Automatic prefix caching (#2762, #3703) supporting long system prompt to be automatically cached across requests. Use the flag --enable-prefix-caching to turn it on.
• Support json_object in OpenAI server for arbitrary JSON, --use-delay flag to improve time to first token across many requests, and min_tokens to EOS suppression.
• Progress in chunked prefill scheduler (#3236, #3538), and speculative decoding (#3103).
• Custom all reduce kernel has been re-enabled after more robustness fixes.
• Replaced cupy dependency due to its bugs.

Hardware

• Improved Neuron support for AWS Inferentia.
• CMake based build system for extensibility.

Ecosystem

• Extensive serving benchmark refactoring (#3277)
• Usage statistics collection (#2852)

What's Changed

• allow user chose log level by --log-level instead of fixed 'info'. by @AllenDou in https://github.com/vllm-project/vllm/pull/3109
• Reorder kv dtype check to avoid nvcc not found error on AMD platform by @cloudhan in https://github.com/vllm-project/vllm/pull/3104
• Add Automatic Prefix Caching by @SageMoore in https://github.com/vllm-project/vllm/pull/2762
• Add vLLM version info to logs and openai API server by @jasonacox in https://github.com/vllm-project/vllm/pull/3161
• [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3158
• Make it easy to profile workers with nsight by @pcmoritz in https://github.com/vllm-project/vllm/pull/3162
• [DOC] add setup document to support neuron backend by @liangfu in https://github.com/vllm-project/vllm/pull/2777
• [Minor Fix] Remove unused code in benchmark_prefix_caching.py by @gty111 in https://github.com/vllm-project/vllm/pull/3171
• Add document for vllm paged attention kernel. by @pian13131 in https://github.com/vllm-project/vllm/pull/2978
• enable --gpu-memory-utilization in benchmark_throughput.py by @AllenDou in https://github.com/vllm-project/vllm/pull/3175
• [Minor fix] The domain dns.google may cause a socket.gaierror exception by @ttbachyinsda in https://github.com/vllm-project/vllm/pull/3176
• Push logprob generation to LLMEngine by @Yard1 in https://github.com/vllm-project/vllm/pull/3065
• Add health check, make async Engine more robust by @Yard1 in https://github.com/vllm-project/vllm/pull/3015
• Fix the openai benchmarking requests to work with latest OpenAI apis by @wangchen615 in https://github.com/vllm-project/vllm/pull/2992
• [ROCm] enable cupy in order to enable cudagraph mode for AMD GPUs by @hongxiayang in https://github.com/vllm-project/vllm/pull/3123
• Store eos_token_id in Sequence for easy access by @njhill in https://github.com/vllm-project/vllm/pull/3166
• [Fix] Avoid pickling entire LLMEngine for Ray workers by @njhill in https://github.com/vllm-project/vllm/pull/3207
• [Tests] Add block manager and scheduler tests by @rkooo567 in https://github.com/vllm-project/vllm/pull/3108
• [Testing] Fix core tests by @cadedaniel in https://github.com/vllm-project/vllm/pull/3224
• A simple addition of dynamic_ncols=True by @chujiezheng in https://github.com/vllm-project/vllm/pull/3242
• Add GPTQ support for Gemma by @TechxGenus in https://github.com/vllm-project/vllm/pull/3200
• Update requirements-dev.txt to include package for benchmarking scripts. by @wangchen615 in https://github.com/vllm-project/vllm/pull/3181
• Separate attention backends by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3005
• Measure model memory usage by @mgoin in https://github.com/vllm-project/vllm/pull/3120
• Possible fix for conflict between Automated Prefix Caching (#2762) and multi-LoRA support (#1804) by @jacobthebanana in https://github.com/vllm-project/vllm/pull/3263
• Fix auto prefix bug by @ElizaWszola in https://github.com/vllm-project/vllm/pull/3239
• Connect engine healthcheck to openai server by @njhill in https://github.com/vllm-project/vllm/pull/3260
• Feature add lora support for Qwen2 by @whyiug in https://github.com/vllm-project/vllm/pull/3177
• [Minor Fix] Fix comments in benchmark_serving by @gty111 in https://github.com/vllm-project/vllm/pull/3252
• [Docs] Fix Unmocked Imports by @ywang96 in https://github.com/vllm-project/vllm/pull/3275
• [FIX] Make flash_attn optional by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3269
• Move model filelocks from /tmp/ to ~/.cache/vllm/locks/ dir by @mgoin in https://github.com/vllm-project/vllm/pull/3241
• [FIX] Fix prefix test error on main by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3286
• [Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling by @cadedaniel in https://github.com/vllm-project/vllm/pull/3103
• Enhance lora tests with more layer and rank variations by @tterrysun in https://github.com/vllm-project/vllm/pull/3243
• [ROCM] Fix blockReduceSum to use correct warp counts for ROCm and CUDA by @dllehr-amd in https://github.com/vllm-project/vllm/pull/3262
• [BugFix] Fix get tokenizer when using ray by @esmeetu in https://github.com/vllm-project/vllm/pull/3301
• [Fix] Fix best_of behavior when n=1 by @njhill in https://github.com/vllm-project/vllm/pull/3298
• Re-enable the 80 char line width limit by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3305
• [docs] Add LoRA support information for models by @pcmoritz in https://github.com/vllm-project/vllm/pull/3299
• Add distributed model executor abstraction by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3191
• [ROCm] Fix warp and lane calculation in blockReduceSum by @kliuae in https://github.com/vllm-project/vllm/pull/3321
• Support Mistral Model Inference with transformers-neuronx by @DAIZHENWEI in https://github.com/vllm-project/vllm/pull/3153
• docs: Add BentoML deployment doc by @Sherlock113 in https://github.com/vllm-project/vllm/pull/3336
• Fixes #1556 double free by @br3no in https://github.com/vllm-project/vllm/pull/3347
• Add kernel for GeGLU with approximate GELU by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3337
• [Fix] fix quantization arg when using marlin by @DreamTeamWangbowen in https://github.com/vllm-project/vllm/pull/3319
• add hf_transfer to requirements.txt by @RonanKMcGovern in https://github.com/vllm-project/vllm/pull/3031
• fix bias in if, ambiguous by @hliuca in https://github.com/vllm-project/vllm/pull/3259
• [Minor Fix] Use cupy-cuda11x in CUDA 11.8 build by @chenxu2048 in https://github.com/vllm-project/vllm/pull/3256
• Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when using Multi-LoRA. by @orsharir in https://github.com/vllm-project/vllm/pull/3350
• Add batched RoPE kernel by @tterrysun in https://github.com/vllm-project/vllm/pull/3095
• Fix lint by @Yard1 in https://github.com/vllm-project/vllm/pull/3388
• [FIX] Simpler fix for async engine running on ray by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3371
• [Hotfix] [Debug] test_openai_server.py::test_guided_regex_completion by @simon-mo in https://github.com/vllm-project/vllm/pull/3383
• allow user to chose which vllm's merics to display in grafana by @AllenDou in https://github.com/vllm-project/vllm/pull/3393
• [Kernel] change benchmark script so that result can be directly used; tune moe kernel in A100/H100 with tp=2,4,8 by @youkaichao in https://github.com/vllm-project/vllm/pull/3389
• Install flash_attn in Docker image by @tdoublep in https://github.com/vllm-project/vllm/pull/3396
• Add args for mTLS support by @declark1 in https://github.com/vllm-project/vllm/pull/3410
• [issue templates] add some issue templates by @youkaichao in https://github.com/vllm-project/vllm/pull/3412
• Fix assertion failure in Qwen 1.5 with prefix caching enabled by @chenxu2048 in https://github.com/vllm-project/vllm/pull/3373
• fix marlin config repr by @qeternity in https://github.com/vllm-project/vllm/pull/3414
• Feature: dynamic shared mem moe_align_block_size_kernel by @akhoroshev in https://github.com/vllm-project/vllm/pull/3376
• [Misc] add HOST_IP env var by @youkaichao in https://github.com/vllm-project/vllm/pull/3419
• Add chat templates for Falcon by @Dinghow in https://github.com/vllm-project/vllm/pull/3420
• Add chat templates for ChatGLM by @Dinghow in https://github.com/vllm-project/vllm/pull/3418
• Fix dist.broadcast stall without group argument by @GindaChen in https://github.com/vllm-project/vllm/pull/3408
• Fix tie_word_embeddings for Qwen2. by @fyabc in https://github.com/vllm-project/vllm/pull/3344
• [Fix] Add args for mTLS support by @declark1 in https://github.com/vllm-project/vllm/pull/3430
• Fixes the misuse/mixuse of time.time()/time.monotonic() by @sighingnow in https://github.com/vllm-project/vllm/pull/3220
• [Misc] add error message in non linux platform by @youkaichao in https://github.com/vllm-project/vllm/pull/3438
• Fix issue templates by @hmellor in https://github.com/vllm-project/vllm/pull/3436
• fix document error for value and v_vec illustration by @laneeeee in https://github.com/vllm-project/vllm/pull/3421
• Asynchronous tokenization by @Yard1 in https://github.com/vllm-project/vllm/pull/2879
• Removed Extraneous Print Message From OAI Server by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/3440
• [Misc] PR templates by @youkaichao in https://github.com/vllm-project/vllm/pull/3413
• Fixes the incorrect argument in the prefix-prefill test cases by @sighingnow in https://github.com/vllm-project/vllm/pull/3246
• Replace lstrip() with removeprefix() to fix Ruff linter warning by @ronensc in https://github.com/vllm-project/vllm/pull/2958
• Fix Baichuan chat template by @Dinghow in https://github.com/vllm-project/vllm/pull/3340
• [Misc] fix line length for entire codebase by @simon-mo in https://github.com/vllm-project/vllm/pull/3444
• Support arbitrary json_object in OpenAI and Context Free Grammar by @simon-mo in https://github.com/vllm-project/vllm/pull/3211
• Fix setup.py neuron-ls issue by @simon-mo in https://github.com/vllm-project/vllm/pull/2671
• [Misc] Define from_dict and to_dict in InputMetadata by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3452
• [CI] Shard tests for LoRA and Kernels to speed up by @simon-mo in https://github.com/vllm-project/vllm/pull/3445
• [Bugfix] Make moe_align_block_size AMD-compatible by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3470
• CI: Add ROCm Docker Build by @simon-mo in https://github.com/vllm-project/vllm/pull/2886
• [Testing] Add test_config.py to CI by @cadedaniel in https://github.com/vllm-project/vllm/pull/3437
• [CI/Build] Fix Bad Import In Test by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/3473
• [Misc] Fix PR Template by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3478
• Cmake based build system by @bnellnm in https://github.com/vllm-project/vllm/pull/2830
• [Core] Zero-copy asdict for InputMetadata by @Yard1 in https://github.com/vllm-project/vllm/pull/3475
• [Misc] Update README for the Third vLLM Meetup by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3479
• [Core] Cache some utils by @Yard1 in https://github.com/vllm-project/vllm/pull/3474
• [Core] print error before deadlock by @youkaichao in https://github.com/vllm-project/vllm/pull/3459
• [Doc] Add docs about OpenAI compatible server by @simon-mo in https://github.com/vllm-project/vllm/pull/3288
• [BugFix] Avoid initializing CUDA too early by @njhill in https://github.com/vllm-project/vllm/pull/3487
• Update dockerfile with ModelScope support by @ifsheldon in https://github.com/vllm-project/vllm/pull/3429
• [Doc] minor fix to neuron-installation.rst by @jimburtoft in https://github.com/vllm-project/vllm/pull/3505
• Revert "[Core] Cache some utils" by @simon-mo in https://github.com/vllm-project/vllm/pull/3507
• [Doc] minor fix of spelling in amd-installation.rst by @jimburtoft in https://github.com/vllm-project/vllm/pull/3506
• Use lru_cache for some environment detection utils by @simon-mo in https://github.com/vllm-project/vllm/pull/3508
• [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled by @ElizaWszola in https://github.com/vllm-project/vllm/pull/3357
• [Core] Add generic typing to LRUCache by @njhill in https://github.com/vllm-project/vllm/pull/3511
• [Misc] Remove cache stream and cache events by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3461
• Abort when nvcc command is not found in the PATH by @AllenDou in https://github.com/vllm-project/vllm/pull/3527
• Check for _is_cuda() in compute_num_jobs by @bnellnm in https://github.com/vllm-project/vllm/pull/3481
• [Bugfix] Fix ROCm support in CMakeLists.txt by @jamestwhedbee in https://github.com/vllm-project/vllm/pull/3534
• [1/n] Triton sampling kernel by @Yard1 in https://github.com/vllm-project/vllm/pull/3186
• [1/n][Chunked Prefill] Refactor input query shapes by @rkooo567 in https://github.com/vllm-project/vllm/pull/3236
• Migrate logits computation and gather to model_runner by @esmeetu in https://github.com/vllm-project/vllm/pull/3233
• [BugFix] Hot fix in setup.py for neuron build by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3537
• [PREFIX CACHING FOLLOW UP] OrderedDict-based evictor by @ElizaWszola in https://github.com/vllm-project/vllm/pull/3431
• Fix 1D query issue from _prune_hidden_states by @rkooo567 in https://github.com/vllm-project/vllm/pull/3539
• [🚀 Ready to be merged] Added support for Jais models by @grandiose-pizza in https://github.com/vllm-project/vllm/pull/3183
• [Misc][Log] Add log for tokenizer length not equal to vocabulary size by @esmeetu in https://github.com/vllm-project/vllm/pull/3500
• [Misc] Bump up transformers to v4.39.0 & Remove StarCoder2Config by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3551
• [BugFix] gemma loading after quantization or LoRA. by @taeminlee in https://github.com/vllm-project/vllm/pull/3553
• [Bugfix][Model] Fix Qwen2 by @esmeetu in https://github.com/vllm-project/vllm/pull/3554
• [Hardware][Neuron] Refactor neuron support by @zhuohan123 in https://github.com/vllm-project/vllm/pull/3471
• Some fixes for custom allreduce kernels by @hanzhi713 in https://github.com/vllm-project/vllm/pull/2760
• Dynamic scheduler delay to improve ITL performance by @tdoublep in https://github.com/vllm-project/vllm/pull/3279
• [Core] Improve detokenization performance for prefill by @Yard1 in https://github.com/vllm-project/vllm/pull/3469
• [Bugfix] use SoftLockFile instead of LockFile by @kota-iizuka in https://github.com/vllm-project/vllm/pull/3578
• [Misc] Fix BLOOM copyright notice by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3591
• [Misc] Bump transformers version by @ywang96 in https://github.com/vllm-project/vllm/pull/3592
• [BugFix] Fix Falcon tied embeddings by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3590
• [BugFix] 1D query fix for MoE models by @njhill in https://github.com/vllm-project/vllm/pull/3597
• [CI] typo fix: is_hip --> is_hip() by @youkaichao in https://github.com/vllm-project/vllm/pull/3595
• [CI/Build] respect the common environment variable MAX_JOBS by @youkaichao in https://github.com/vllm-project/vllm/pull/3600
• [CI/Build] fix flaky test by @youkaichao in https://github.com/vllm-project/vllm/pull/3602
• [BugFix] minor fix: method typo in rotary_embedding.py file, get_device() -> device by @jikunshang in https://github.com/vllm-project/vllm/pull/3604
• [Bugfix] Revert "[Bugfix] use SoftLockFile instead of LockFile (#3578)" by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3599
• [Model] Add starcoder2 awq support by @shaonianyr in https://github.com/vllm-project/vllm/pull/3569
• [Core] Refactor Attention Take 2 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3462
• [Bugfix] fix automatic prefix args and add log info by @gty111 in https://github.com/vllm-project/vllm/pull/3608
• [CI] Try introducing isort. by @rkooo567 in https://github.com/vllm-project/vllm/pull/3495
• [Core] Adding token ranks along with logprobs by @SwapnilDreams100 in https://github.com/vllm-project/vllm/pull/3516
• feat: implement the min_tokens sampling parameter by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/3124
• [Bugfix] API stream returning two stops by @dylanwhawk in https://github.com/vllm-project/vllm/pull/3450
• hotfix isort on logprobs ranks pr by @simon-mo in https://github.com/vllm-project/vllm/pull/3622
• [Feature] Add vision language model support. by @xwjiang2010 in https://github.com/vllm-project/vllm/pull/3042
• Optimize _get_ranks in Sampler by @Yard1 in https://github.com/vllm-project/vllm/pull/3623
• [Misc] Include matched stop string/token in responses by @njhill in https://github.com/vllm-project/vllm/pull/2976
• Enable more models to inference based on LoRA by @jeejeelee in https://github.com/vllm-project/vllm/pull/3382
• [Bugfix] Fix ipv6 address parsing bug by @liiliiliil in https://github.com/vllm-project/vllm/pull/3641
• [BugFix] Fix ipv4 address parsing regression by @njhill in https://github.com/vllm-project/vllm/pull/3645
• [Kernel] support non-zero cuda devices in punica kernels by @jeejeelee in https://github.com/vllm-project/vllm/pull/3636
• [Doc]add lora support by @jeejeelee in https://github.com/vllm-project/vllm/pull/3649
• [Misc] Minor fix in KVCache type by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3652
• [Core] remove cupy dependency by @youkaichao in https://github.com/vllm-project/vllm/pull/3625
• [Bugfix] More faithful implementation of Gemma by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3653
• [Bugfix] [Hotfix] fix nccl library name by @youkaichao in https://github.com/vllm-project/vllm/pull/3661
• [Model] Add support for DBRX by @megha95 in https://github.com/vllm-project/vllm/pull/3660
• [Misc] add the "download-dir" option to the latency/throughput benchmarks by @AmadeusChan in https://github.com/vllm-project/vllm/pull/3621
• feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark by @ywang96 in https://github.com/vllm-project/vllm/pull/3277
• Add support for Cohere's Command-R model by @zeppombal in https://github.com/vllm-project/vllm/pull/3433
• [Docs] Add Command-R to supported models by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3669
• [Model] Fix and clean commandr by @esmeetu in https://github.com/vllm-project/vllm/pull/3671
• [Model] Add support for xverse by @hxer7963 in https://github.com/vllm-project/vllm/pull/3610
• [CI/Build] update default number of jobs and nvcc threads to avoid overloading the system by @youkaichao in https://github.com/vllm-project/vllm/pull/3675
• [Kernel] Add Triton MoE kernel configs for DBRX + A100 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3679
• [Core] [Bugfix] Refactor block manager subsystem for better testability by @cadedaniel in https://github.com/vllm-project/vllm/pull/3492
• [Model] Add support for Qwen2MoeModel by @wenyujin333 in https://github.com/vllm-project/vllm/pull/3346
• [Kernel] DBRX Triton MoE kernel H100 by @ywang96 in https://github.com/vllm-project/vllm/pull/3692
• [2/N] Chunked prefill data update by @rkooo567 in https://github.com/vllm-project/vllm/pull/3538
• [Bugfix] Update neuron_executor.py to add optional vision_language_config. by @adamrb in https://github.com/vllm-project/vllm/pull/3695
• fix benchmark format reporting in buildkite by @simon-mo in https://github.com/vllm-project/vllm/pull/3693
• [CI] Add test case to run examples scripts by @simon-mo in https://github.com/vllm-project/vllm/pull/3638
• [Core] Support multi-node inference(eager and cuda graph) by @esmeetu in https://github.com/vllm-project/vllm/pull/3686
• [Kernel] Add MoE Triton kernel configs for A100 40GB by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3700
• [Bugfix] Set enable_prefix_caching=True in prefix caching example by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3703
• fix logging msg for block manager by @simon-mo in https://github.com/vllm-project/vllm/pull/3701
• [Core] fix del of communicator by @youkaichao in https://github.com/vllm-project/vllm/pull/3702
• [Benchmark] Change mii to use persistent deployment and support tensor parallel by @IKACE in https://github.com/vllm-project/vllm/pull/3628
• bump version to v0.4.0 by @simon-mo in https://github.com/vllm-project/vllm/pull/3705
• Revert "bump version to v0.4.0" by @youkaichao in https://github.com/vllm-project/vllm/pull/3708
• [Test] Make model tests run again and remove --forked from pytest by @rkooo567 in https://github.com/vllm-project/vllm/pull/3631
• [Misc] Minor type annotation fix by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3716
• [Core][Test] move local_rank to the last arg with default value to keep api compatible by @youkaichao in https://github.com/vllm-project/vllm/pull/3711
• add ccache to docker build image by @simon-mo in https://github.com/vllm-project/vllm/pull/3704
• Usage Stats Collection by @yhu422 in https://github.com/vllm-project/vllm/pull/2852
• [BugFix] Fix tokenizer out of vocab size by @esmeetu in https://github.com/vllm-project/vllm/pull/3685
• [BugFix][Frontend] Fix completion logprobs=0 error by @esmeetu in https://github.com/vllm-project/vllm/pull/3731
• [Bugfix] Command-R Max Model Length by @ywang96 in https://github.com/vllm-project/vllm/pull/3727
• bump version to v0.4.0 by @simon-mo in https://github.com/vllm-project/vllm/pull/3712
• [ROCm][Bugfix] Fixed several bugs related to rccl path and attention selector logic by @hongxiayang in https://github.com/vllm-project/vllm/pull/3699
• usage lib get version another way by @simon-mo in https://github.com/vllm-project/vllm/pull/3735
• [BugFix] Use consistent logger everywhere by @njhill in https://github.com/vllm-project/vllm/pull/3738
• [Core][Bugfix] cache len of tokenizer by @youkaichao in https://github.com/vllm-project/vllm/pull/3741
• Fix build when nvtools is missing by @bnellnm in https://github.com/vllm-project/vllm/pull/3698
• CMake build elf without PTX by @simon-mo in https://github.com/vllm-project/vllm/pull/3739

New Contributors

• @cloudhan made their first contribution in https://github.com/vllm-project/vllm/pull/3104
• @SageMoore made their first contribution in https://github.com/vllm-project/vllm/pull/2762
• @jasonacox made their first contribution in https://github.com/vllm-project/vllm/pull/3161
• @gty111 made their first contribution in https://github.com/vllm-project/vllm/pull/3171
• @pian13131 made their first contribution in https://github.com/vllm-project/vllm/pull/2978
• @ttbachyinsda made their first contribution in https://github.com/vllm-project/vllm/pull/3176
• @wangchen615 made their first contribution in https://github.com/vllm-project/vllm/pull/2992
• @chujiezheng made their first contribution in https://github.com/vllm-project/vllm/pull/3242
• @TechxGenus made their first contribution in https://github.com/vllm-project/vllm/pull/3200
• @mgoin made their first contribution in https://github.com/vllm-project/vllm/pull/3120
• @jacobthebanana made their first contribution in https://github.com/vllm-project/vllm/pull/3263
• @ElizaWszola made their first contribution in https://github.com/vllm-project/vllm/pull/3239
• @DAIZHENWEI made their first contribution in https://github.com/vllm-project/vllm/pull/3153
• @Sherlock113 made their first contribution in https://github.com/vllm-project/vllm/pull/3336
• @br3no made their first contribution in https://github.com/vllm-project/vllm/pull/3347
• @DreamTeamWangbowen made their first contribution in https://github.com/vllm-project/vllm/pull/3319
• @RonanKMcGovern made their first contribution in https://github.com/vllm-project/vllm/pull/3031
• @hliuca made their first contribution in https://github.com/vllm-project/vllm/pull/3259
• @orsharir made their first contribution in https://github.com/vllm-project/vllm/pull/3350
• @youkaichao made their first contribution in https://github.com/vllm-project/vllm/pull/3389
• @tdoublep made their first contribution in https://github.com/vllm-project/vllm/pull/3396
• @declark1 made their first contribution in https://github.com/vllm-project/vllm/pull/3410
• @qeternity made their first contribution in https://github.com/vllm-project/vllm/pull/3414
• @akhoroshev made their first contribution in https://github.com/vllm-project/vllm/pull/3376
• @Dinghow made their first contribution in https://github.com/vllm-project/vllm/pull/3420
• @fyabc made their first contribution in https://github.com/vllm-project/vllm/pull/3344
• @laneeeee made their first contribution in https://github.com/vllm-project/vllm/pull/3421
• @bnellnm made their first contribution in https://github.com/vllm-project/vllm/pull/2830
• @ifsheldon made their first contribution in https://github.com/vllm-project/vllm/pull/3429
• @jimburtoft made their first contribution in https://github.com/vllm-project/vllm/pull/3505
• @grandiose-pizza made their first contribution in https://github.com/vllm-project/vllm/pull/3183
• @taeminlee made their first contribution in https://github.com/vllm-project/vllm/pull/3553
• @kota-iizuka made their first contribution in https://github.com/vllm-project/vllm/pull/3578
• @shaonianyr made their first contribution in https://github.com/vllm-project/vllm/pull/3569
• @SwapnilDreams100 made their first contribution in https://github.com/vllm-project/vllm/pull/3516
• @tjohnson31415 made their first contribution in https://github.com/vllm-project/vllm/pull/3124
• @xwjiang2010 made their first contribution in https://github.com/vllm-project/vllm/pull/3042
• @liiliiliil made their first contribution in https://github.com/vllm-project/vllm/pull/3641
• @AmadeusChan made their first contribution in https://github.com/vllm-project/vllm/pull/3621
• @zeppombal made their first contribution in https://github.com/vllm-project/vllm/pull/3433
• @hxer7963 made their first contribution in https://github.com/vllm-project/vllm/pull/3610
• @wenyujin333 made their first contribution in https://github.com/vllm-project/vllm/pull/3346
• @adamrb made their first contribution in https://github.com/vllm-project/vllm/pull/3695
• @IKACE made their first contribution in https://github.com/vllm-project/vllm/pull/3628
• @yhu422 made their first contribution in https://github.com/vllm-project/vllm/pull/2852

Full Changelog: https://github.com/vllm-project/vllm/compare/v0.3.3...v0.4.0