chatllama0.0.3
版本发布时间: 2023-03-27 16:16:55
nebuly-ai/nebuly最新发布版本:chatllama0.0.4(2023-03-27 21:47:20)
Release Notes - ChatLLaMA 0.0.3
Major release of ChatLLaMA solving multiple bugs and expanding the support to distrubuted training.
New Features
- Training produces a log file containing also the training stats
- Add Template during the dataset creation
- Changed default training parameters to InstructGPT paper ones
- Implemented Cosine Scheduler for LR
- Add Conversation Logs during RLHF
- Improved management of checkpoints
- Add support for Accelerate backend for distributed training
- Defer import of LLaMA modules, avoiding the LLaMA dependecy while another model is used
Bug Fix
- Fix bug causing crashes during training due to long sequences
- Fix multiple bugs on deepspeed MultiGPU training
- Fix bugs on RLHF training
- Fix bug with different versions of LangChain during synthetic data generation
New Contributors
- @PierpaoloSorbellini made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/185
- @theSekyi made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/187
- @egrefen made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/190
- @AAnirudh07 made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/202
- @dentathor made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/204
- @bzantium made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/212
- @pgzhang made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/230
- @sebastianschramm made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/237
- @HuangLK made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/253
- @zhzou2020 made their first contribution in https://github.com/nebuly-ai/nebullvm/pull/271