What's new

Added 🎉

• GPT-based model.
• Tokenizer and data pre-processing pipeline.
• training script.
• Triton-based FlashAttention.

Commits

e801af8e add release proc e643f5ed update pyproject dbc81777 Bump version to v0.2.0 for release e99dbe5d Merge pull request #391 from allenai/hf-olmo-new a120ab24 Merge pull request #380 from allenai/shanea/storage-cleaner-download-upload 4e849e4b Merge pull request #390 from allenai/shanea/storage-cleaner-archive-fix-2 1dbc3461 Merge pull request #378 from allenai/shanea/storage-cleaner-cached-path 22cefa28 Merge pull request #389 from allenai/shanea/add-r2-scheme ac01778e fix 6c79c634 add option to only unshard model d1c185b6 Merge pull request #387 from allenai/epwalsh/dist-init e30d29f1 Merge pull request #364 from allenai/shanea/storage-cleaner ff883e5d Merge pull request #385 from allenai/epwalsh/max-duration-tokens e16e606c Merge pull request #383 from allenai/epwalsh/start-new-epoch