Highlights

• Serve the official release demo of LLaVA v1.6 blog
• Support Yi-VL example
• Faster JSON decoding blog
• Support QWen 2

What's Changed

• Fix the error message and dependency of openai backend by @merrymercy in https://github.com/sgl-project/sglang/pull/71
• Add an async example by @Ying1123 in https://github.com/sgl-project/sglang/pull/37
• Add a note about triton version for older GPUs by @merrymercy in https://github.com/sgl-project/sglang/pull/72
• Support load fine-tuned LLaVA model by @isaac-vidas in https://github.com/sgl-project/sglang/pull/80
• Suppport qwen model and solve some problems by @Arcmoon-Hu in https://github.com/sgl-project/sglang/pull/75
• Fix after QWen support by @merrymercy in https://github.com/sgl-project/sglang/pull/82
• Fix the chat template for QWen by @merrymercy in https://github.com/sgl-project/sglang/pull/83
• Fix SRT endpoint api json syntax by @CSWellesSun in https://github.com/sgl-project/sglang/pull/84
• Return logprob for choices by @merrymercy in https://github.com/sgl-project/sglang/pull/87
• Add health endpoint to SGLang runtime server by @isaac-vidas in https://github.com/sgl-project/sglang/pull/90
• Llava-hd Support by @caoshiyi in https://github.com/sgl-project/sglang/pull/92
• Bump the version to v0.1.8 by @merrymercy in https://github.com/sgl-project/sglang/pull/93
• Improve Chinese character streaming when the last char is half Chinese word. by @haotian-liu in https://github.com/sgl-project/sglang/pull/95
• Handle grayscale images in expand2square by @isaac-vidas in https://github.com/sgl-project/sglang/pull/97
• support speculative execution for openai API by @parasol-aser in https://github.com/sgl-project/sglang/pull/48
• fix batch error for llava-hd by @caoshiyi in https://github.com/sgl-project/sglang/pull/98
• Dynamic model class loading by @comaniac in https://github.com/sgl-project/sglang/pull/101
• Flush Cache API by @hnyls2002 in https://github.com/sgl-project/sglang/pull/103
• Fix Mistral model loading by @comaniac in https://github.com/sgl-project/sglang/pull/108
• Improve the control of streaming and improve the first token latency in streaming by @merrymercy in https://github.com/sgl-project/sglang/pull/117
• Add qwen2 by @JustinLin610 in https://github.com/sgl-project/sglang/pull/114
• Format code by @merrymercy in https://github.com/sgl-project/sglang/pull/118
• Update quick start examples by @merrymercy in https://github.com/sgl-project/sglang/pull/120
• Improve docs & Add JSON decode example by @merrymercy in https://github.com/sgl-project/sglang/pull/121
• [Feature] Adds basic support for image content in OpenAI chat routes by @fozziethebeat in https://github.com/sgl-project/sglang/pull/113
• [Feature] Allow specifying all ports to use in advance by @Ja1Zhou in https://github.com/sgl-project/sglang/pull/116
• Add cache metrics by @comaniac in https://github.com/sgl-project/sglang/pull/119
• Fix model loading & format code by @merrymercy in https://github.com/sgl-project/sglang/pull/125
• Add city doc benchmark mode by @hnyls2002 in https://github.com/sgl-project/sglang/pull/129
• Yi-VL Model by @BabyChouSr in https://github.com/sgl-project/sglang/pull/112
• Fix is_multimodal_model judge by @hnyls2002 in https://github.com/sgl-project/sglang/pull/132
• Add max_prefill_num_token into server arguments by @Ying1123 in https://github.com/sgl-project/sglang/pull/133
• Release 0.1.11 by @Ying1123 in https://github.com/sgl-project/sglang/pull/134

New Contributors

• @isaac-vidas made their first contribution in https://github.com/sgl-project/sglang/pull/80
• @Arcmoon-Hu made their first contribution in https://github.com/sgl-project/sglang/pull/75
• @CSWellesSun made their first contribution in https://github.com/sgl-project/sglang/pull/84
• @haotian-liu made their first contribution in https://github.com/sgl-project/sglang/pull/95
• @parasol-aser made their first contribution in https://github.com/sgl-project/sglang/pull/48
• @JustinLin610 made their first contribution in https://github.com/sgl-project/sglang/pull/114
• @fozziethebeat made their first contribution in https://github.com/sgl-project/sglang/pull/113
• @Ja1Zhou made their first contribution in https://github.com/sgl-project/sglang/pull/116

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.1.6...v0.1.11