Highlight

• LMDeploy support Llama3.1 and its Tool Calling. An example of calling "Wolfram Alpha" to perform complex mathematical calculations can be found from here

What's Changed

🚀 Features

• Support glm4 awq by @AllentDan in https://github.com/InternLM/lmdeploy/pull/1993
• Support llama3.1 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/2122
• Support Llama3.1 tool calling by @AllentDan in https://github.com/InternLM/lmdeploy/pull/2123

💥 Improvements

• Remove the triton inference server backend "turbomind_backend" by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/1986
• Remove kv cache offline quantization by @AllentDan in https://github.com/InternLM/lmdeploy/pull/2097
• Remove session_len and deprecated short names of the chat templates by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/2105
• clarify "n>1" in GenerationConfig hasn't been supported yet by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/2108

🐞 Bug fixes

• fix stop words for glm4 by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/2044
• Disable peer access code by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/2082
• set log level ERROR in benchmark scripts by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/2086
• raise thread exception by @irexyc in https://github.com/InternLM/lmdeploy/pull/2071
• Fix index error when profiling token generation with -ct 1 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/1898

🌐 Other

• misc: replace slow Jimver/cuda-toolkit by @zhyncs in https://github.com/InternLM/lmdeploy/pull/2065
• misc: update bug issue template by @zhyncs in https://github.com/InternLM/lmdeploy/pull/2083
• update daily testcase new by @zhulinJulia24 in https://github.com/InternLM/lmdeploy/pull/2035
• bump version to v0.5.2 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/2143

Full Changelog: https://github.com/InternLM/lmdeploy/compare/v0.5.1...v0.5.2