core_v0.7.0
版本发布时间: 2024-06-06 07:12:47
NVIDIA/Megatron-LM最新发布版本:core_r0.8.0(2024-08-13 20:12:40)
- MoE
- Token drop support
- Several efficiency optimizations
- Improved model parallelism
- Memory optimizations
- Distributed checkpointing
- Enabled for Retro
- Asynchronous checkpoint saving
- Several minor bug fixes, speed improvements, and memory optimizations