What's Changed

🚀 Features

• Add extra_requires to reduce dependencies by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/580
• TurboMind 2 by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/590
• Support loading hf model directly by @irexyc in https://github.com/InternLM/lmdeploy/pull/685
• convert model with hf repo_id by @irexyc in https://github.com/InternLM/lmdeploy/pull/774
• Support turbomind bf16 by @grimoire in https://github.com/InternLM/lmdeploy/pull/803
• support image_embs input by @irexyc in https://github.com/InternLM/lmdeploy/pull/799
• Add api.py by @AllentDan in https://github.com/InternLM/lmdeploy/pull/805

💥 Improvements

• Fix Tokenizer encode by @AllentDan in https://github.com/InternLM/lmdeploy/pull/645
• Optimize for throughput by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/701
• Replace mmengine with mmengine-lite by @zhouzaida in https://github.com/InternLM/lmdeploy/pull/715
• Set the default value of max_context_token_num 1 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/761
• add triton server test and workflow yml by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/760
• improvement(build): enable ninja and gold linker by @tpoisonooo in https://github.com/InternLM/lmdeploy/pull/767
• Report first-token-latency and token-latency percentiles by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/736
• Unify prefill & decode passes by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/775
• add cuda12.1 build check ci by @irexyc in https://github.com/InternLM/lmdeploy/pull/782
• auto upload cuda12.1 python pkg to release when create new tag by @irexyc in https://github.com/InternLM/lmdeploy/pull/784
• Report the inference benchmark of models with different size by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/794
• Simplify block manager by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/812
• Disable attention mask when it is not needed by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/813
• FIFO pipe strategy for api_server by @AllentDan in https://github.com/InternLM/lmdeploy/pull/795
• simplify the header of the benchmark table by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/820
• add encode for opencompass by @AllentDan in https://github.com/InternLM/lmdeploy/pull/828
• fix: awq should save bin files by @hscspring in https://github.com/InternLM/lmdeploy/pull/793
• Support building docker image manually in CI by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/825

🐞 Bug fixes

• Fix init of batch state by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/682
• fix turbomind stream canceling by @grimoire in https://github.com/InternLM/lmdeploy/pull/686
• [Fix] Fix load_checkpoint_in_model bug by @HIT-cwh in https://github.com/InternLM/lmdeploy/pull/690
• Fix wrong eos_id and bos_id obtained through grpc api by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/644
• Fix cache/output length calculation by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/738
• [Fix] Skip empty batch by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/747
• [Fix] build docker image failed since packaging is missing by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/753
• [Fix] Rollback the data type of input_ids to TYPE_UINT32 in preprocessor's proto by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/758
• fix turbomind build on sm<80 by @grimoire in https://github.com/InternLM/lmdeploy/pull/754
• Fix early-exit condition in attention kernel by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/788
• Fix missed arguments when benchmark static inference performance by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/787
• fix extra colon in InternLMChat7B template by @C1rN09 in https://github.com/InternLM/lmdeploy/pull/796
• Fix local kv head num by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/806
• Fix out-of-bound access by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/809
• Set smem size for repetition penalty kernel by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/818
• Fix cache verification by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/821
• fix finish_reason by @AllentDan in https://github.com/InternLM/lmdeploy/pull/816
• fix turbomind awq by @grimoire in https://github.com/InternLM/lmdeploy/pull/847
• Fix stop requests by await before turbomind queue.get() by @AllentDan in https://github.com/InternLM/lmdeploy/pull/850
• [Fix] Fix meta tensor error by @pppppM in https://github.com/InternLM/lmdeploy/pull/848
• Fix cuda reinitialization in a multiprocessing setting by @grimoire in https://github.com/InternLM/lmdeploy/pull/862
• launch gradio server directly with hf model by @AllentDan in https://github.com/InternLM/lmdeploy/pull/856
• fix typo by @grimoire in https://github.com/InternLM/lmdeploy/pull/769
• Add chat template for Yi by @AllentDan in https://github.com/InternLM/lmdeploy/pull/779
• fix api_server stop_session and end_session by @AllentDan in https://github.com/InternLM/lmdeploy/pull/835
• Return the iterator after erasing it from a map by @irexyc in https://github.com/InternLM/lmdeploy/pull/864

📚 Documentations

• [Docs] Update Supported Matrix by @pppppM in https://github.com/InternLM/lmdeploy/pull/679
• [Docs] Update KV8 Docs by @pppppM in https://github.com/InternLM/lmdeploy/pull/681
• [Doc] Update restful api doc by @AllentDan in https://github.com/InternLM/lmdeploy/pull/662
• Check-in user guide about turbomind config by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/680
• Update benchmark user guide by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/763
• [Docs] Fix typo in restful_api user guide by @maxchiron in https://github.com/InternLM/lmdeploy/pull/858
• [Docs] Fix typo in restful_api user guide by @maxchiron in https://github.com/InternLM/lmdeploy/pull/859

🌐 Other

• bump version to v0.1.0a0 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/709
• bump version to 0.1.0a1 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/776
• bump version to v0.1.0a2 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/807
• bump version to v0.1.0 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/834

New Contributors

• @zhouzaida made their first contribution in https://github.com/InternLM/lmdeploy/pull/715
• @C1rN09 made their first contribution in https://github.com/InternLM/lmdeploy/pull/796
• @maxchiron made their first contribution in https://github.com/InternLM/lmdeploy/pull/858

Full Changelog: https://github.com/InternLM/lmdeploy/compare/v0.0.14...v0.1.0