What's new in 0.14.2 (2024-08-16)

These are the changes in inference v0.14.2.

New features

• FEAT: add gemma-2-it 2b & internlm2.5-chat 1.8b and 20b & update video and sglang docs by @qinxuye in https://github.com/xorbitsai/inference/pull/2080
• FEAT: support FP8 for vllm & sglang engine by @qinxuye in https://github.com/xorbitsai/inference/pull/2069
• Feat: Support internvl2 and internvl stream by @amumu96 in https://github.com/xorbitsai/inference/pull/2079

Enhancements

• ENH: make MiniCPM v2.6 support video by @Minamiyama in https://github.com/xorbitsai/inference/pull/2068
• REF: Remove some builtin old models and ggmlv3 model format by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2086

Bug fixes

• BUG: limit AutoAWQ version to fix docker issue by @qinxuye in https://github.com/xorbitsai/inference/pull/2067
• BUG: Fix custom glm4 & remove tool calls of ChatGLM3 by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2081
• BUG: Infinited loop with login by @WalkerWang731 in https://github.com/xorbitsai/inference/pull/2039

Documentation

• DOC: update readme & add tips for large image models by @qinxuye in https://github.com/xorbitsai/inference/pull/2056

New Contributors

• @WalkerWang731 made their first contribution in https://github.com/xorbitsai/inference/pull/2039

Full Changelog: https://github.com/xorbitsai/inference/compare/v0.14.1...v0.14.2