v0.1.0
版本发布时间: 2023-12-18 20:10:04
InternLM/lmdeploy最新发布版本:v0.6.0a0(2024-08-26 17:12:19)
What's Changed
🚀 Features
- Add extra_requires to reduce dependencies by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/580
- TurboMind 2 by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/590
- Support loading hf model directly by @irexyc in https://github.com/InternLM/lmdeploy/pull/685
- convert model with hf repo_id by @irexyc in https://github.com/InternLM/lmdeploy/pull/774
- Support turbomind bf16 by @grimoire in https://github.com/InternLM/lmdeploy/pull/803
- support image_embs input by @irexyc in https://github.com/InternLM/lmdeploy/pull/799
- Add api.py by @AllentDan in https://github.com/InternLM/lmdeploy/pull/805
💥 Improvements
- Fix Tokenizer encode by @AllentDan in https://github.com/InternLM/lmdeploy/pull/645
- Optimize for throughput by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/701
- Replace mmengine with mmengine-lite by @zhouzaida in https://github.com/InternLM/lmdeploy/pull/715
- Set the default value of
max_context_token_num
1 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/761 - add triton server test and workflow yml by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/760
- improvement(build): enable ninja and gold linker by @tpoisonooo in https://github.com/InternLM/lmdeploy/pull/767
- Report first-token-latency and token-latency percentiles by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/736
- Unify prefill & decode passes by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/775
- add cuda12.1 build check ci by @irexyc in https://github.com/InternLM/lmdeploy/pull/782
- auto upload cuda12.1 python pkg to release when create new tag by @irexyc in https://github.com/InternLM/lmdeploy/pull/784
- Report the inference benchmark of models with different size by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/794
- Simplify block manager by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/812
- Disable attention mask when it is not needed by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/813
- FIFO pipe strategy for api_server by @AllentDan in https://github.com/InternLM/lmdeploy/pull/795
- simplify the header of the benchmark table by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/820
- add encode for opencompass by @AllentDan in https://github.com/InternLM/lmdeploy/pull/828
- fix: awq should save bin files by @hscspring in https://github.com/InternLM/lmdeploy/pull/793
- Support building docker image manually in CI by @RunningLeon in https://github.com/InternLM/lmdeploy/pull/825
🐞 Bug fixes
- Fix init of batch state by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/682
- fix turbomind stream canceling by @grimoire in https://github.com/InternLM/lmdeploy/pull/686
- [Fix] Fix load_checkpoint_in_model bug by @HIT-cwh in https://github.com/InternLM/lmdeploy/pull/690
- Fix wrong eos_id and bos_id obtained through grpc api by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/644
- Fix cache/output length calculation by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/738
- [Fix] Skip empty batch by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/747
- [Fix] build docker image failed since
packaging
is missing by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/753 - [Fix] Rollback the data type of
input_ids
toTYPE_UINT32
in preprocessor's proto by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/758 - fix turbomind build on sm<80 by @grimoire in https://github.com/InternLM/lmdeploy/pull/754
- Fix early-exit condition in attention kernel by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/788
- Fix missed arguments when benchmark static inference performance by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/787
- fix extra colon in InternLMChat7B template by @C1rN09 in https://github.com/InternLM/lmdeploy/pull/796
- Fix local kv head num by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/806
- Fix out-of-bound access by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/809
- Set smem size for repetition penalty kernel by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/818
- Fix cache verification by @lzhangzz in https://github.com/InternLM/lmdeploy/pull/821
- fix finish_reason by @AllentDan in https://github.com/InternLM/lmdeploy/pull/816
- fix turbomind awq by @grimoire in https://github.com/InternLM/lmdeploy/pull/847
- Fix stop requests by await before turbomind queue.get() by @AllentDan in https://github.com/InternLM/lmdeploy/pull/850
- [Fix] Fix meta tensor error by @pppppM in https://github.com/InternLM/lmdeploy/pull/848
- Fix cuda reinitialization in a multiprocessing setting by @grimoire in https://github.com/InternLM/lmdeploy/pull/862
- launch gradio server directly with hf model by @AllentDan in https://github.com/InternLM/lmdeploy/pull/856
- fix typo by @grimoire in https://github.com/InternLM/lmdeploy/pull/769
- Add chat template for Yi by @AllentDan in https://github.com/InternLM/lmdeploy/pull/779
- fix api_server stop_session and end_session by @AllentDan in https://github.com/InternLM/lmdeploy/pull/835
- Return the iterator after erasing it from a map by @irexyc in https://github.com/InternLM/lmdeploy/pull/864
📚 Documentations
- [Docs] Update Supported Matrix by @pppppM in https://github.com/InternLM/lmdeploy/pull/679
- [Docs] Update KV8 Docs by @pppppM in https://github.com/InternLM/lmdeploy/pull/681
- [Doc] Update restful api doc by @AllentDan in https://github.com/InternLM/lmdeploy/pull/662
- Check-in user guide about turbomind config by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/680
- Update benchmark user guide by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/763
- [Docs] Fix typo in
restful_api
user guide by @maxchiron in https://github.com/InternLM/lmdeploy/pull/858 - [Docs] Fix typo in
restful_api
user guide by @maxchiron in https://github.com/InternLM/lmdeploy/pull/859
🌐 Other
- bump version to v0.1.0a0 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/709
- bump version to 0.1.0a1 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/776
- bump version to v0.1.0a2 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/807
- bump version to v0.1.0 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/834
New Contributors
- @zhouzaida made their first contribution in https://github.com/InternLM/lmdeploy/pull/715
- @C1rN09 made their first contribution in https://github.com/InternLM/lmdeploy/pull/796
- @maxchiron made their first contribution in https://github.com/InternLM/lmdeploy/pull/858
Full Changelog: https://github.com/InternLM/lmdeploy/compare/v0.0.14...v0.1.0
1、 lmdeploy-0.1.0-cp310-cp310-manylinux2014_x86_64.whl 91.11MB
2、 lmdeploy-0.1.0-cp310-cp310-win_amd64.whl 61.72MB
3、 lmdeploy-0.1.0-cp311-cp311-manylinux2014_x86_64.whl 91.13MB
4、 lmdeploy-0.1.0-cp311-cp311-win_amd64.whl 61.72MB
5、 lmdeploy-0.1.0-cp38-cp38-manylinux2014_x86_64.whl 91.12MB
6、 lmdeploy-0.1.0-cp38-cp38-win_amd64.whl 61.72MB
7、 lmdeploy-0.1.0-cp39-cp39-manylinux2014_x86_64.whl 91.11MB
8、 lmdeploy-0.1.0-cp39-cp39-win_amd64.whl 61.71MB