New Features

• [New Features][ESPnet1][ASR][RNNT] Auxiliary task #2951 by @b-flo
• [New Features][ESPnet1][Recipe] RTF calculation #2942 by @hirofumi0810
• [New Features][ESPnet2] Supporting multiple optimizers in the default trainer #3014 by @kamo-naoyuki
• [New Features][ESPnet2][ASR] Streaming Transformer ASR #2907 by @eml914
• [New Features][ESPnet2][ASR][Installation] add wav2vec_encoder #2889 by @simpleoier
• [New Features][ESPnet2][Documentation][Installation][README] Support sharded training of fairscale #2980 by @kamo-naoyuki
• [New Features][ESPnet2][SE] Add SeparateSpeech API in espnet2/bin/enh_inference.py #2878 by @Emrys365
• [New Features][ESPnet2][TTS][Installation][README] Support phonemizer for vairous language G2P #2959 by @kan-bayashi

Bugfix

• [Bugfix][CI][Installation] Install warp-ctc using pip>=21.0 #2999 by @ysk24ok
• [Bugfix][ESPnet1] Integration testing for asr_mix was using the wrong config. #3006 by @siddalmia
• [Bugfix][ESPnet1][ASR] Fix model averaging #2910 by @b-flo
• [Bugfix][ESPnet1][ASR] bug fixed for streaming transformer ASR #2981 by @eml914
• [Bugfix][ESPnet1][ASR] builtin ctc modification #3001 by @siddalmia
• [Bugfix][ESPnet1][ASR][CI] Fix transfer learning w/ pre-trained LM + finetuning tutorial #2967 by @b-flo
• [Bugfix][ESPnet1][ASR][RNNT] Fix a condition in TSD #2965 by @b-flo
• [Bugfix][ESPnet1][ASR][Recipe] fix egs/ljspeech/asr1 #2865 #2884 by @kan-bayashi
• [Bugfix][ESPnet1][ASR][Recipe][ST] Fix bug in How2 recipe #2933 by @hirofumi0810
• [Bugfix][ESPnet1][ASR][Refactoring] Fix data sorting in attention/CTC visualization #2883 by @hirofumi0810
• [Bugfix][ESPnet1][Docker] Fix docker error caused by BeamSearchTransducer #2973 by @b-flo
• [Bugfix][ESPnet1][ESPnet2] Fix bugs of our Conformer implementation. #2816 by @pengchengguo
• [Bugfix][ESPnet1][ESPnet2][Refactoring] Fix arguments in dynamic and lightweight conv #3004 by @hirofumi0810
• [Bugfix][ESPnet1][RNNT] fix out_dim definition #2915 by @b-flo
• [Bugfix][ESPnet1][TTS] Fix attention plot bug #2984 #2985 by @kan-bayashi
• [Bugfix][ESPnet1][mergify] swbd run.sh is including dev data in the training set #2977 by @brianyan918
• [Bugfix][ESPnet2] Fix sharded_ddp mode #3015 by @kamo-naoyuki
• [Bugfix][ESPnet2] bug fix for Wav2Vec encoder #2997 by @simpleoier
• [Bugfix][ESPnet2][Documentation] Fix for sharded training with amp #2993 by @kamo-naoyuki
• [Bugfix][ESPnet2][Documentation] Fix sharded training for multiple nodes #2994 by @kamo-naoyuki
• [Bugfix][ESPnet2][SE] quick fix for librimix (SE) data preparation #2982 by @LiChenda

Recipe

• [Recipe][ESPnet1][ASR] Fix dev set in IWSLT21 ASR recipe #3000 by @hirofumi0810
• [Recipe][ESPnet1][ASR] IWSLT'21 ASR recipe #2934 by @hirofumi0810
• [Recipe][ESPnet1][ASR] Update IWSLT21 ASR recipe #2987 by @hirofumi0810
• [Recipe][ESPnet1][ASR] Update the pre-trained Conformer model link of Aishell-1 corpus. #2924 by @pengchengguo
• [Recipe][ESPnet1][ASR] Update transformer training results on common vioce dataset #2927 by @wenjie-p
• [Recipe][ESPnet1][ASR][CI][Installation][Refactoring] Update IWSLT18 (ST-TED) ASR recipe #2916 by @hirofumi0810
• [Recipe][ESPnet1][ASR][MT][ST][README] Must-C v2 recipe #2963 by @hirofumi0810
• [Recipe][ESPnet1][ASR][MT][ST][Refactoring] Refactor Fisher-CallHome recipe #2904 by @hirofumi0810
• [Recipe][ESPnet1][ASR][MT][ST][Refactoring] Refactor How2 recipe #2906 by @hirofumi0810
• [Recipe][ESPnet1][ASR][MT][ST][Refactoring] Refactor Must-C recipe #2901 by @hirofumi0810
• [Recipe][ESPnet1][ASR][MT][ST][Refactoring] Refactor libri-trans recipe #2903 by @hirofumi0810
• [Recipe][ESPnet1][ASR][ST][Refactoring] Update IWSLT'19 recipe #2940 by @hirofumi0810
• [Recipe][ESPnet1][ST][CI][Refactoring] Refactor ST recipes #2975 by @hirofumi0810
• [Recipe][ESPnet1][ST][Refactoring] Refactor Mboshi-French corpus #2911 by @hirofumi0810
• [Recipe][ESPnet2][ASR] Open-li52(add language id scoring & text case align for test set) #2938 by @ftshijt
• [Recipe][ESPnet2][ASR][README] Add Russian open STT recipe for ESPnet2 #2972 by @akreal
• [Recipe][ESPnet2][ASR][README] MLS (multi-lingual librispeech) recipe #2869 by @ftshijt
• [Recipe][ESPnet2][ASR][README] Update espnet2 librispeech result #2966 by @kamo-naoyuki
• [Recipe][ESPnet2][ASR][README] added nsc results #2937 by @sw005320
• [Recipe][ESPnet2][ASR][README] fix librispeech model url #2976 by @kamo-naoyuki
• [Recipe][ESPnet2][ASR][README] minor fix of li52 and nsc recipes #2936 by @sw005320
• [Recipe][ESPnet2][ASR][README] update the results of open li52 recipe #2974 by @sw005320
• [Recipe][ESPnet2][SE] Librimix separation results for Conv-Tasnet, 8k, min #2928 by @anogkongda
• [Recipe][ESPnet2][SE][README] Espnet-SE, Speech enhancement recipes #2888 by @LiChenda

Enhancement

• [Enhancement][ESPnet1][ASR] Auto Resampling to 16khz for pretrained models #2969 by @siddalmia
• [Enhancement][ESPnet1][ASR][RNNT] Minor refactoring #2932 by @b-flo
• [Enhancement][ESPnet1][ASR][RNNT][README][CI][Documentation] Refactoring RNNT #2887 by @b-flo
• [Enhancement][ESPnet1][ESPnet2][ASR][LM][MT][TTS] Print total params and trainable params. #2996 by @siddalmia
• [Enhancement][ESPnet1][LM] Add LM options like embedding dropout and tie weights #3010 by @siddalmia
• [Enhancement][ESPnet1][ST][Refactoring] Add the latest RPE implementation to the ST task. #3005 by @pengchengguo

Other

• [CI][README][mergify] Stop circle ci #2978 by @kamo-naoyuki
• [Documentation] Update docs for ESPnet contributing (especially for recipes part) #2905 by @ftshijt
• [Documentation] fix a typo #3016 by @Huang17
• [Installation] Uninstall typing #2979 by @kamo-naoyuki

Acknowledgements

Special thanks to @Emrys365, @Huang17, @LiChenda, @akreal, @anogkongda, @b-flo, @brianyan918, @eml914, @ftshijt, @hirofumi0810, @kamo-naoyuki, @kan-bayashi, @pengchengguo, @siddalmia, @simpleoier, @sw005320, @wenjie-p, @ysk24ok.