Highlights

• Fast JSON Decoding (blog)
• Output logprobs for decoding tokens
• Multiple bug fixes

What's Changed

• Fix no-cache mode by @Ying1123 in https://github.com/sgl-project/sglang/pull/136
• Support Faster JSON decoding for llava by @hnyls2002 in https://github.com/sgl-project/sglang/pull/137
• fix undfined variable by @yaya-sy in https://github.com/sgl-project/sglang/pull/142
• jump-forward rename by @hnyls2002 in https://github.com/sgl-project/sglang/pull/144
• Add warmup to SRT server by @comaniac in https://github.com/sgl-project/sglang/pull/146
• add openai error handler with retry and logger by @ChuyueSun in https://github.com/sgl-project/sglang/pull/148
• Temporary fix OpenAI API for Pydantic v1/v2 by @comaniac in https://github.com/sgl-project/sglang/pull/153
• Add gptq quantization model support by @Arcmoon-Hu in https://github.com/sgl-project/sglang/pull/141
• Support decode token logprobs by @comaniac in https://github.com/sgl-project/sglang/pull/130
• Format code & move functions by @merrymercy in https://github.com/sgl-project/sglang/pull/155
• [Submodule] Change FlashInfer to import by @comaniac in https://github.com/sgl-project/sglang/pull/156
• add --disable-disk-cache by @hnyls2002 in https://github.com/sgl-project/sglang/pull/160
• Add Auth Token to RuntimeEndpoint by @nivibilla in https://github.com/sgl-project/sglang/pull/162
• Fix BaseCache metric by @comaniac in https://github.com/sgl-project/sglang/pull/170
• import outlines by @hnyls2002 in https://github.com/sgl-project/sglang/pull/168
• Fix token usage with jump forward by @comaniac in https://github.com/sgl-project/sglang/pull/174
• Support extra field regex in OpenAI API by @comaniac in https://github.com/sgl-project/sglang/pull/172
• Fix the chat template for llava-v1.6-34b & format code by @merrymercy in https://github.com/sgl-project/sglang/pull/177
• Update version to 0.1.12 by @merrymercy in https://github.com/sgl-project/sglang/pull/178

New Contributors

• @yaya-sy made their first contribution in https://github.com/sgl-project/sglang/pull/142
• @ChuyueSun made their first contribution in https://github.com/sgl-project/sglang/pull/148
• @nivibilla made their first contribution in https://github.com/sgl-project/sglang/pull/162

Full Changelog: https://github.com/sgl-project/sglang/compare/v0.1.11...v0.1.12