版本发布时间: 2023-03-17 18:39:12

huggingface/trl最新发布版本:v0.9.6(2024-07-08 21:51:10)

Large models training, Naive Pipeline Parallelism, peft Data Parallelism support and distributed training bug fixes

This release includes a set of features and bug fixes to scale up your RLHF experiments for much larger models leveraging peft and bitsandbytes.

Naive Pipeline Parallelism support

We introduce a new paradigm in trl , termed as Naive Pipeline Parallelism, to fit large scale models on your training setup and apply RLHF on them. This feature uses peft to train adapters and bitsandbytes to reduce the memory foot print of your active model


peft Data Parallelism support

There were some bugs with respect to peft integration and DP. This release includes the bug fixes to enable multi-GPU training using accelerate + DDP (DIstributed Data Parallel)

Memory optimization

Your training runs can be now much more memory efficient thanks to few tricks / bug fixes: Now PPOConfig also supports the flag optimize_cuda_cache (set to False by default) to avoid increasing CUDA memory issues

Pytorch 2.0 fixes

This release also includes minor fixes related to PyTorch 2.0 release

What's Changed

New Contributors

Full Changelog:

相关地址:原始地址 下载(tar) 下载(zip)