v1.21.0

NVIDIA/NeMo

版本发布时间: 2023-10-26 07:27:38

NVIDIA/NeMo最新发布版本:r2.0.0rc1(2024-08-16 05:55:14)

Highlights

Models

NeMo ASR

Multi-lookahead cache-aware streaming
Speech enahncement tutorial #6492
Online code switching dataset #6579

NeMo TTS

AudioCodec: Training recipe for EnCodec #6852

NeMo Framework

GPT from Mcore #7093
GPT distributed checkpointing #7116
Hidden transformations #6332
LLama-2 #7299

NeMo Core

Update to PTL 2.0 #6433

NeMo Tools

Forced aligner tutorial #7210

Container

For additional information regarding NeMo containers, please visit: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo

docker pull nvcr.io/nvidia/nemo:23.08

ASR

Changelog

Fix require_grad typos by @kit1980 :: PR: #6930
rnnt_greedy_decoding.py: typos? auto-repressively -> auto-regressively by @vadimkantorov :: PR: #6989
Adding tutorial for confidence ensembles by @Kipok :: PR: #6932
Add support for Numba FP16 RNNT Loss by @titu1994 :: PR: #6991
fix install_beamsearch_decoders by @karpnv :: PR: #7011
rnnt and char utils by @karpnv :: PR: #6971
ASR Confidence update and tutorial by @GNroy :: PR: #6810
st standalone model by @AlexGrinch :: PR: #6969
Fix typo in ASR-TTS tutorial by @artbataev :: PR: #7049
Update Frame-VAD doc and fix onnx export by @stevehuang52 :: PR: #7076
Fast Conformer global token fix by @sam1373 :: PR: #7085
Added script to extract ASR CTC and RNNT models from ASR hybrid models by @trias702 :: PR: #7092
Fix absolute path in path join call by @kingjan1999 :: PR: #7099
NeMo ASR Demo by @lleaver :: PR: #7110
Fix plot function in vad_utils.py by @stevehuang52 :: PR: #7113
Fixed small bug with NoisePerturbationWithNormalization by @trias702 :: PR: #7118
Merge release r1.20.0 to main by @ericharper :: PR: #7167
minor fix for conformer subsampling docstring. by @XuesongYang :: PR: #7195
[ASR] Fix GPU memory leak in transcribe_speech.py by @rlangman :: PR: #7249
Adding Multilingual, Code-Switched, and Hybrid ASR models by @KunalDhawan :: PR: #7250
fix partial transcribe by @stevehuang52 :: PR: #7284
Conv1d subsampling by @burchim :: PR: #7294
add bf16 inference support and fix seq_len stft issue by @nithinraok :: PR: #7338
Add finetuning scripts by @nithinraok :: PR: #7263
Move parameter: trainer -> exp_manager (for PTL 2.0) by @artbataev :: PR: #7339
Fix typos by @omahs :: PR: #7361
Fix wrong calling of librosa.get_duration() in notebook by @RobinDong :: PR: #7376
RNN-T confidence and alignment bugfix (#7381) by @GNroy :: PR: #7459
update branch by @nithinraok :: PR: #7488
Replace strategy = None with strategy = auto for notebooks by @athitten :: PR: #7521
Fix PTL2.0 related ASR bugs in r1.21.0: Val metrics logging, None dataloader issue by @KunalDhawan :: PR: #7531
gpus -> devices by @nithinraok :: PR: #7542
[BugFix] Add missing quotes for auto strategy in tutorial notebooks by @athitten :: PR: #7541
Append output of val_step to self.validation_step_outputs in EncMaskDecAudioToAudioModel by @athitten :: PR: #7543
fix validation_step_outputs initialization for multi-dataloader by @KunalDhawan :: PR: #7546
Append val/test output to instance variable in EncDecSpeakerLabelModel by @athitten :: PR: #7562
update strategy by @nithinraok :: PR: #7577
Typo fixes by @Kipok :: PR: #7591
Fix metrics for SE tutorial by @anteju :: PR: #7604
fix ssl models ptl monitor val through logging by @nithinraok :: PR: #7608
Fix py3.11 dataclasses issue by @titu1994 :: PR: #7582
bugfix: trainer.gpus, trainer.strategy, trainer.accelerator by @XuesongYang :: PR: #7621
Safeguard nemo_text_processing installation on ARM (#7485) by @blisc :: PR: #7619
[ASR] Fix type error in jasper by @rlangman :: PR: #7636
Fix vad & speech command tutorial - onnx by @fayejf :: PR: #7671
Replace strategy='dp'/None with 'auto' by @athitten :: PR: #7681
Fix multi rank finetune for ASR by @titu1994 :: PR: #7684
fix ptl_bugs in slu_models.py by @jzi040941 :: PR: #7689
Add NLPDDPStrategyNotebook and change trainer gpus to devices by @athitten :: PR: #7741
Updated installation of ctc-decoders by @vsl9 :: PR: #7746
Fix bug wrt change decoding strategy for bpe models by @titu1994 :: PR: #7762

TTS

Changelog

[TTS] Add cosine distance option to TTS aligner by @rlangman :: PR: #6806
[TTS] Add tutorial for TTS data prep scripts by @rlangman :: PR: #6922
update TTS readme by @XuesongYang :: PR: #7088
[TTS] Create EnCodec training recipe by @rlangman :: PR: #6852
[TTS][ZH] add Chinese TTS recipes based on IPA symbol sets. by @XuesongYang :: PR: #6893
[TTS] Add output audio format to preprocessing by @rlangman :: PR: #6889
[TTS] Remove nested TTS configs by @rlangman :: PR: #7154
[TTS] Fix TTS recipes with PTL 2.0 by @rlangman :: PR: #7188
[TTS] Add license to ported EnCodec code by @rlangman :: PR: #7197
[Fix] Discriminator update in AudioCodecModel by @anteju :: PR: #7209
Adapter ipa Tutorial and config update by @styagi130 :: PR: #7260
[TTS] Audio codec fixes by @rlangman :: PR: #7266
[TTS] minor fix typos and input_types by @XuesongYang :: PR: #7272
specify explicitly to set pretrained model paths by @styagi130 :: PR: #7305
[TTS] Update AudioCodec API by @anteju :: PR: #7310
[TTS] Add additional config to preprocess_text and compute_feature_stats by @rlangman :: PR: #7321
[TTS] Change audio codec token type to TokenIndex by @rlangman :: PR: #7356
fixed trainer.strategy=auto from None. by @XuesongYang :: PR: #7369
[TTS] Added a callback for logging initial data by @anteju :: PR: #7384
[TTS] bugfix: trainer.accelerator=auto from None. by @XuesongYang :: PR: #7492
bugfix: specify trainer.strategy=auto when devices=1 by @XuesongYang :: PR: #7509
Fix dimensionality in get_dist function by @redoctopus :: PR: #7506
Fix TTS FastPitch tutorial by @hsiehjackson :: PR: #7494
[TTS] remove curly braces from in jupyer notebook cell. by @XuesongYang :: PR: #7554
[TTS] fixed trainer's accelerator and strategy. by @XuesongYang :: PR: #7569
Change hifigan finetune strategy to ddp_find_unused_parameters_true by @hsiehjackson :: PR: #7579
Fix validation in G2PModel and ThutmoseTaggerModel by @athitten :: PR: #7597
[TTS] Fix FastPitch data prep tutorial by @rlangman :: PR: #7602
[TTS] Add dataset to path of logged artifacts by @rlangman :: PR: #7651

NLP / NMT

Changelog

Minor MPT-7B fixes and creation script update by @trias702 :: PR: #6982
remove hard coded input and output fields by @arendu :: PR: #7008
RoPE length extrapolation with interpolation by @MaximumEntropy :: PR: #7005
add async + distopt to sft by @MaximumEntropy :: PR: #7018
ptuning inference table bug fix by @arendu :: PR: #7015
Fix missing import for GPT SFT by @MaximumEntropy :: PR: #7026
Add end_strings to SamplingParams by @markelsanz14 :: PR: #6986
Fix race condition for downloading cache when executing with multi-node by @findkim :: PR: #7016
added back the retro documents. by @yidong72 :: PR: #7033
remove pos emb from state dict for old models by @ekmb :: PR: #7068
memmap worker arg by @arendu :: PR: #7062
Disable distopt contiguous param buffer by default by @timmoon10 :: PR: #7095
[Fix] load_state_dict in nlp_model.py by @stevehuang52 :: PR: #7086
Fix tokenizer file caching where torch.distributed may not be initialized yet by @findkim :: PR: #7061
freeze base mode on init during peft by @arendu :: PR: #7152
Include the scripts for preprocessing OAST and unit tests for chat sft datasets by @yidong72 :: PR: #7112
T5 metrics fix by @jubick1337 :: PR: #7037
megatron gpt training fix by @anmolgupt :: PR: #7199
Fix T5 using FA by @hsiehjackson :: PR: #7196
fix-causal-fa-infer by @hsiehjackson :: PR: #7200
Fix gpt trainer test by @hsiehjackson :: PR: #6915
Load ub_cfg from hydra config by @jbaczek :: PR: #7003
Fixes for lightning 2.0 upgrade by @athitten :: PR: #7176
Fix which was off by one batch by @odelalleau :: PR: #7212
Start using ModelParallelConfig from Megatron Core by @ericharper :: PR: #6885
deprecation warning by @arendu :: PR: #7193
Fix attention mask inference by @hsiehjackson :: PR: #7213
Use GPTModel from mcore by @ericharper :: PR: #7093
Add bf16-mixed and 16-mixed in module.py by @athitten :: PR: #7227
Refactor LLM pretraining examples by @maanug-nv :: PR: #7159
Add only trainable parameters to optimizer group in PEFT by @guyueh1 :: PR: #7230
Dummy class for ModelParallelConfig by @ericharper :: PR: #7254
[TN][Docs] update language coverage matrix and refs by @mgrafu :: PR: #7247
tied weights for adapters by @arendu :: PR: #6928
Fix skip generation by @hsiehjackson :: PR: #7270
Hidden transforms model parallel config + CI with Perceiver by @michalivne :: PR: #7241
Fix restore sequence parallel by @hsiehjackson :: PR: #7273
fix ptuning and lora model_parallel_config by @blahBlahhhJ :: PR: #7287
Fix adapters and ptuning for amp O2 by @guyueh1 :: PR: #7285
remove additional line in peft state dict by @blahBlahhhJ :: PR: #7293
loss mask aware final layer applicaiton by @arendu :: PR: #7275
Adding server option to peft eval by @Davood-M :: PR: #7292
migrated class CSVFieldsMemmapDataset from BioNeMo by @dorotat-nv :: PR: #7314
remove old prompt table for storing cached ptunig representations by @arendu :: PR: #7295
Bugfix and optimization in by @odelalleau :: PR: #7267
Set a default value when getting by @yaox12 :: PR: #7115
Distributed checkpointing with mcore GPT by @ericharper :: PR: #7116
Fix activation checkpoint by @hsiehjackson :: PR: #7334
Replace prefetch with val iterator check in megatron models by @athitten :: PR: #7318
Fixing indentation bug in indexed_dataset memory deallocation by @michalivne :: PR: #7352
NeMo MCore llama2 support + MCore PEFT adapters by @blahBlahhhJ :: PR: #7299
Hiddens modules documentation by @michalivne :: PR: #7303
Support for flash attention 2.0 by @MaximumEntropy :: PR: #7063
multiple fields can form a context by @arendu :: PR: #7147
adding bias_dropout_add_fusion option for BERT by @clumsy :: PR: #7332
enable selective unfreeze by @arendu :: PR: #7326
Upgrade pytorch container to 23.08 by @ericharper :: PR: #7353
enable fp32 optimizer for output_layer in mcore by @lhb8125 :: PR: #7355
Revert comment by @ericharper :: PR: #7368
fix pipeline parallel inference by @blahBlahhhJ :: PR: #7367
fix for peft tied weights by @arendu :: PR: #7372
add O2 option in gpt eval by @blahBlahhhJ :: PR: #7358
Move model precision copy by @maanug-nv :: PR: #7336
Fix PEFT checkpoint loading by @blahBlahhhJ :: PR: #7388
Use distributed optimizer support for multiple dtypes by @timmoon10 :: PR: #7359
[PATCH] PEFT import mcore by @blahBlahhhJ :: PR: #7393
Use cfg attribute in bert by @maanug-nv :: PR: #7394
Add support for bias conversion in Swiglu models by @titu1994 :: PR: #7386
Update save_to and restore_from for dist checkpointing by @ericharper :: PR: #7343
fix forward for with mcore=false by @JimmyZhang12 :: PR: #7403
Fix logging to remove 's/it' from progress bar in Megatron models and add train_step_timing by @athitten :: PR: #7374
Set Activation Checkpointing Defaults by @aklife97 :: PR: #7404
Make loss mask default to false by @ericharper :: PR: #7407
Add dummy userbuffer config files by @erhoo82 :: PR: #7408
Add missing ubconf files by @aklife97 :: PR: #7412
Update ptl training ckpt conversion script to work with dist ckpt by @ericharper :: PR: #7416
Add strategy as ddp_find_unused_parameters_true for glue_benchmark.py by @athitten :: PR: #7454
fix bug when loading dist ckpt in peft by @lhb8125 :: PR: #7479
Fix CustomProgressBar for resume by @athitten :: PR: #7427
Append val output to self.validation_step_outputs in GLUEModel by @athitten :: PR: #7530
Cherry pick Fix sft dataset truncation (#7464) to r1.21.0 by @ericharper :: PR: #7550
Avoid duplicated dist checkpoint save by @mikolajblaz :: PR: #7555
layernorm1p fix by @dimapihtar :: PR: #7523
r1.21: SFT model parallel fix for dist ckpt by @aklife97 :: PR: #7520
PEFT needs mp config propagated for dist ckpt by @ericharper :: PR: #7589
Fix ptuning crash for llama 2 ckpt by @yuanzhedong :: PR: #7594
PEFT eval fix by @cuichenx :: PR: #7626
Propagate mp config for continue training by @ericharper :: PR: #7637
Add ddp_find_unused_parameters=True and change accelerator to auto by @athitten :: PR: #7623
Add find_unused_parameters_true for text_classiftn and punctuation_capitalization by @athitten :: PR: #7649
conversion issue fix by @dimapihtar :: PR: #7648
Fix a nlp nb onnx by @fayejf :: PR: #7703
Add activations_checkpoint related args for model cfg in lora.ipynb by @athitten :: PR: #7752
Change accelerator to 'auto' in nlp_checkpoint_port.py by @athitten :: PR: #7747
Add reconfigure microbatch calculator before inference and update GBS, MBS for inference by @athitten :: PR: #7763
Create PrecisionPlugin for megatron_ckpt_to_nemo.py trainer by @athitten :: PR: #7767

NeMo Tools

Changelog

Update doc, new tutorial on SDE by @Jorjeous :: PR: #7405
Fix branch version for SDE by @titu1994 :: PR: #7528

Export

Changelog

Added bool types to neural_types export by @tbartley94 :: PR: #7032

General Improvements

Changelog

Add migration guide for lightning 2.0 upgrade by @athitten :: PR: #7360
add support for max_total_length=4096 for 43b by @Zhilin123 :: PR: #6763
Change Jenkins timeout by @ericharper :: PR: #6997
Update SDP docs page with a new documentation link by @Kipok :: PR: #7029
Fixed tutorial's name by @vsl9 :: PR: #7047
Revert Fix import guard checks by @titu1994 :: PR: #7125
Fix import guard checks by @titu1994 :: PR: #7126
fix evaluator.py for various exceptions by ast by @stevehuang52 :: PR: #7150
NFA bugfix: remove any empty segments by @erastorgueva-nv :: PR: #7155
NFA subtitle file config - specify colors and vertical alignment by @erastorgueva-nv :: PR: #7160
add paths to labeler. by @XuesongYang :: PR: #7087
[Bugfix] Fix a bug in filtering checkpoints by @yaox12 :: PR: #6851
Update README.rst by @fayejf :: PR: #7175
Make NFA subtitles stay until end of video by @erastorgueva-nv :: PR: #7189
Uncomment removal of exp_dir in JenkinsFile by @athitten :: PR: #7198
NFA: replace ellipses in text with 3 periods by @erastorgueva-nv :: PR: #7208
NFA tutorial notebook by @erastorgueva-nv :: PR: #7210
NFA docs: update READMEs and links, add docs page by @erastorgueva-nv :: PR: #7219
Make image centering in NFA README actually work by @erastorgueva-nv :: PR: #7220
Add mcore installation to Dockerfile by @ericharper :: PR: #7237
Checkpoint averaging for model parallel by @Kipok :: PR: #7252
Upgrade hydra and omegaconf by @athitten :: PR: #7243
Update numba support in docker by @titu1994 :: PR: #7271
remove deprecated scripts from ci by @arendu :: PR: #7239
Logging model checkpoints as artifacts in MlFlow by @AlirezaMorsali :: PR: #7258
Adithyare/peft metric calculation by @arendu :: PR: #7304
Resume checkpoint priority by @maanug-nv :: PR: #7335
lora merge fix for O2 names by @arendu :: PR: #7325
Llama load buffers in checkpoint by @blahBlahhhJ :: PR: #7357
pin numba=0.57.1 to fix reinstall.sh error by @XuesongYang :: PR: #7366
Update to core 23.08 branch ToT by @aklife97 :: PR: #7371
Upper bounding ptl by @ericharper :: PR: #7370
minor fix for llama ckpt conversion script by @blahBlahhhJ :: PR: #7387
Update Core Commit by @aklife97 :: PR: #7402
Fix resume from checkpoint in exp_manager by @athitten :: PR: #7424
add sleep by @gshennvm :: PR: #7498
Fix exp manager check for sleep by @titu1994 :: PR: #7503
unpin setuptools by @fayejf :: PR: #7534
Update FFMPEG version to fix issue with torchaudio by @titu1994 :: PR: #7551
fix typos in nfa and speech enhancement tutorials by @erastorgueva-nv :: PR: #7580
best ckpt fix by @dimapihtar :: PR: #7564
add build os key by @nithinraok :: PR: #7596
Fix issues with Dockerfile by @titu1994 :: PR: #7650
Change confidence parameters in the test by @Kipok :: PR: #7680
bugfix: pin nemo-text-process to fix Chinese normalizer error. by @XuesongYang :: PR: #7627
Remove PUBLICATIONS.md, point to github.io NeMo page instead by @erastorgueva-nv :: PR: #7694
Pin mcore to 0.3 by @ericharper :: PR: #7751
fix hybrid eval by @karpnv :: PR: #7759
Update Apex install command in Dockerfile by @ericharper :: PR: #7794

相关地址：原始地址下载(tar) 下载(zip)

1、 asset-post-2024-01-03-parakeet_arch.png 372.82KB

2、 asset-post-2024-01-03-parakeet_leaderboard.png 889.04KB

查看：2023-10-26发行的版本