English Version

New Features:

1. Inference and deployment support for logprobs.
2. RLHF support for lazy_tokenize.
3. Multimodal model support for neftune.
4. dynamic_eos compatibility with glm4 series and other models.

New Models:

1. mplug-owl3, best practices can be found here.
2. yi-coder 1.5b, base/chat model of 9b.
3. minicpm3-4b.
4. reflection-llama3.1-70b.

中文版

新功能：

1. 推理和部署支持 logprobs。
2. RLHF支持lazy_tokenize。
3. 多模态模型支持neftune。
4. dynamic_eos兼容glm4系列等模型。

新模型：

1. mplug-owl3，最佳实践可以查看这里。
2. yi-coder 1.5b、9b 的base/chat模型。
3. minicpm3-4b。
4. reflection-llama3.1-70b。

What's Changed

• Fix push_to_hub when last-checkpoint by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1897
• support custom quantized dataset by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1893
• fix push_to_ms by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1901
• support logprobs by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1900
• deepspeed use cosine lr_schduler by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1907
• update docs by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1908
• fix web-ui push to hub strategy by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1909
• Refactor docs by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1912
• refactor docs by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1915
• [TorchAcc] perf: use xm.save instead of torch.save by @baoleai in https://github.com/modelscope/ms-swift/pull/1916
• update wechat by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1925
• update docs & fix bug by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1926
• [TorchAcc] fix: fix the judegement of fsdp_num by @baoleai in https://github.com/modelscope/ms-swift/pull/1903
• Support deploy & logprobs by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1833
• fix typing by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1933
• fix swift deploy by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1936
• update yi-coder by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1942
• fix lmdeploy seed by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1945
• fix do_sample by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1946
• refactor rlhf by @hjh0119 in https://github.com/modelscope/ms-swift/pull/1885
• fix file rename error in megatron when there are multi process by @Zhikaiiii in https://github.com/modelscope/ms-swift/pull/1948
• fix qwen2-vl & video by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1950
• support dynamic_eos by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1947
• fix rlhf by @hjh0119 in https://github.com/modelscope/ms-swift/pull/1949
• Support minicpm 3 by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1952
• Add lazy_tokenize to RLHF by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1956
• Fix data info print in rlhf by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1964
• Fix the lora hook by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1963
• fix bugs by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1959
• support mplug_owl3 by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1957
• update docs by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1970
• Add reflection model by @tastelikefeet in https://github.com/modelscope/ms-swift/pull/1973
• fix typo by @Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1980

Full Changelog: https://github.com/modelscope/ms-swift/compare/v2.4.0...v2.4.1