Complete revamp of our initial release.

New features:

• Hydra models, 20x faster than vanilla PPO with minimal performance hits at large scales
• Massively revamped API, significantly less boiler plate.
• Save/load callbacks.
• Greatly improved orchestrator.
• Better commented RL code, easier to understand whats going on.
• Cool examples, including architext and simulacra.
• Better extendability, and standardized styling.

Features coming soon:

• Megatron support! we're already working on this.
• More interesting examples that are relevant to production use cases of TRLX.
• Better integration of W&B, including sweeps.
• Evaluation and benchmarking.

:)

Autogenerated release notes below:

What's Changed

• Fix typo by @mrm8488 in https://github.com/CarperAI/trlx/pull/2
• Create LICENSE by @LouisCastricato in https://github.com/CarperAI/trlx/pull/3
• QOL fixes by @LouisCastricato in https://github.com/CarperAI/trlx/pull/5
• stage ilql by @reciprocated in https://github.com/CarperAI/trlx/pull/6
• Adds style file and reward function capabilities to ppo orchestrator by @LouisCastricato in https://github.com/CarperAI/trlx/pull/8
• Update ppo value head + print logs by @Dahoas in https://github.com/CarperAI/trlx/pull/11
• Make ilql respect the config & remove sin by @reciprocated in https://github.com/CarperAI/trlx/pull/22
• Docs by @shahbuland in https://github.com/CarperAI/trlx/pull/31
• Implemented hydra heads + adaptive kl by @Dahoas in https://github.com/CarperAI/trlx/pull/33
• Add pre-commit with black by @cat-state in https://github.com/CarperAI/trlx/pull/36
• [update] Improve package setup by @jon-tow in https://github.com/CarperAI/trlx/pull/42
• Add initial issue templates by @jon-tow in https://github.com/CarperAI/trlx/pull/45
• Some readme improvements by @thedch in https://github.com/CarperAI/trlx/pull/44
• Add initial GitHub workflows by @jon-tow in https://github.com/CarperAI/trlx/pull/43
• [docs] Add CONTRIBUTING.md by @jon-tow in https://github.com/CarperAI/trlx/pull/52
• Simplify api by @reciprocated in https://github.com/CarperAI/trlx/pull/24

New Contributors

• @mrm8488 made their first contribution in https://github.com/CarperAI/trlx/pull/2
• @LouisCastricato made their first contribution in https://github.com/CarperAI/trlx/pull/3
• @reciprocated made their first contribution in https://github.com/CarperAI/trlx/pull/6
• @Dahoas made their first contribution in https://github.com/CarperAI/trlx/pull/11
• @shahbuland made their first contribution in https://github.com/CarperAI/trlx/pull/31
• @cat-state made their first contribution in https://github.com/CarperAI/trlx/pull/36
• @jon-tow made their first contribution in https://github.com/CarperAI/trlx/pull/42
• @thedch made their first contribution in https://github.com/CarperAI/trlx/pull/44

Full Changelog: https://github.com/CarperAI/trlx/commits/v0.2