Features

• Megatron GPT pre-training with tensor model parallelism #2975
• NMT encoder and decoder with different hidden size #2856
• Logging timing of train/val/test steps #2936
• Logging NMT encoder and decoder timing #2956
• Logging timing per sentence length and tokenized text statistics #3004
• Upgrade to PyTorch Lightning 1.5.0, bfloat support #2975
• French Inverse Text Normalization #2921
• Bucketing of tarred datasets for ASR models #2999
• ASR with diarization #3007
• Adding parallel transcribe for ASR models - suppports multi-gpu/multi-node #3017

Documentation Updates

• RNNT

Contributors

@ericharper @michalivne @MaximumEntropy @VahidooX @titu1994 @blisc @okuchaiev @tango4j @erastorgueva-nv @fayejf @vadam5 @ekmb @yaoyu-33 @nithinraok @erhoo82 @tbartley94 @PeganovAnton @madhukarkm @yzhang123 (Please let us know if you have contributed to this release and we have missed you here.)