🚀 Composer v0.12.1

Composer v0.12.1 is released! Install via pip:

pip install --upgrade mosaicml==0.12.1

New Features

1. 📚 In-Context Learning (#1876)

   With Composer and MosaicML Cloud you can now evaluate LLMs on in-context learning tasks (LAMBADA, HellaSwag, PIQA, and more) hundreds of times faster than other evaluation harnesses. Please see our "Blazingly Fast LLM Evaluation for In-Context Learning" blog post for more details!

2. 💾 Added support for Coreweave Object Storage (#1915)

   Coreweave object store is compatible with boto3. Uploading objects to Coreweave object store is almost exactly like writing to using S3, except an endpoint_url must be set via the S3_ENDPOINT_URLenvironment variable. For example:

   import os
   os.environ['S3_ENDPOINT_URL'] = 'https://object.las1.coreweave.com'
   
   from composer.trainer import Trainer
   
   # Save checkpoints every epoch to s3://my_bucket/checkpoints
   trainer = Trainer(
       model=model,
       train_dataloader=train_dataloader,
       max_duration='10ep',
       save_folder='s3://my_bucket/checkpoints',
       save_interval='1ep',
       save_overwrite=True,
       save_filename='ep{epoch}.pt',
       save_num_checkpoints_to_keep=0,  # delete all checkpoints locally
    )
   
    trainer.fit()
   

   Please see our checkpointing documentation for more details.

3. 🪵 Automatic logging of Trainer hparams (#1855)

   Hyperparameter arguments passed to the Trainer are now automatically logged. Simply set the Trainer argument auto_log_hparams=True.

Bug Fixes

• Update Docker images to use ‘posix_prefix’ paths (#1854)
• Disable new notebook in CI (#1875)
• [Fix] Enable logging of metrics from Callbacks to ConsoleLogging (#1884)
• Ensure loggers run init event before callbacks in Engine (#1890)
• Raise an error in FSDP meta tensor initialization if there's no initialization functions, fix associated flaky FSDP test (#1905)
• Add primitive list support (#1906)
• Add logic for shifting labels before computing metrics (#1913)
• Fixes mis specified dependency (#1919)
• pin setuptools in build requirements (#1926)
• Pin pip<23 in Docker images (#1936)
• Fix bug in trainer.eval and add test cases for test_console_logger (#1937)

What's Changed

• Rename GradMonitor -> OptimizerMonitor; add functionality to log optimizer-specific metrics to assist loss spike investigation by @bmosaicml in https://github.com/mosaicml/composer/pull/1743
• Add GCS uri support for loading and saving checkpoints by @eracah in https://github.com/mosaicml/composer/pull/1833
• HF factory function tests by @dakinggg in https://github.com/mosaicml/composer/pull/1832
• Fix doc issue, Trainer hparam log_to_console defaults to False by @eracah in https://github.com/mosaicml/composer/pull/1840
• Removed YAHP references from Docs by @bandish-shah in https://github.com/mosaicml/composer/pull/1841
• Typo by @nguyenhoan1988 in https://github.com/mosaicml/composer/pull/1843
• Fix source code links in docs by @bandish-shah in https://github.com/mosaicml/composer/pull/1844
• add importorskip by @dakinggg in https://github.com/mosaicml/composer/pull/1847
• Update Docker images to use ‘posix_prefix’ paths by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1854
• Fix typo by @standardAI in https://github.com/mosaicml/composer/pull/1849
• ConsoleLogger: log first batch and first epoch when using console_log_interval by @eracah in https://github.com/mosaicml/composer/pull/1860
• Simpler auto log hparams by @eracah in https://github.com/mosaicml/composer/pull/1855
• Fix typos by @cclauss in https://github.com/mosaicml/composer/pull/1850
• Bump sphinxext-opengraph from 0.7.3 to 0.7.4 by @dependabot in https://github.com/mosaicml/composer/pull/1851
• Bump coverage[toml] from 6.5.0 to 7.0.1 by @dependabot in https://github.com/mosaicml/composer/pull/1853
• Bump traitlets from 5.7.0 to 5.8.0 by @dependabot in https://github.com/mosaicml/composer/pull/1852
• Bump ipython from 7.32.0 to 8.8.0 by @dependabot in https://github.com/mosaicml/composer/pull/1865
• Update monai requirement from <0.10,>=0.9.1 to >=0.9.1,<1.2 by @dependabot in https://github.com/mosaicml/composer/pull/1869
• Bump sphinxcontrib-katex from 0.9.3 to 0.9.4 by @dependabot in https://github.com/mosaicml/composer/pull/1868
• Bump coverage[toml] from 7.0.1 to 7.0.4 by @dependabot in https://github.com/mosaicml/composer/pull/1867
• Upgrade docker images to torch==1.13.1 by @abhi-mosaic in https://github.com/mosaicml/composer/pull/1863
• add more useful info to state by @dakinggg in https://github.com/mosaicml/composer/pull/1848
• Feature/lambada evaluator by @bmosaicml in https://github.com/mosaicml/composer/pull/1845
• multi-node distributed training, submitit & composer integration demo by @YilunKuang in https://github.com/mosaicml/composer/pull/1753
• Daily tests by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1870
• Disable new notebook in CI by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1875
• Update deepspeed by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1864
• fix fail fast in daily by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1880
• Fix getting started docs by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1878
• Speed up test_lm_task_evaluation by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1879
• Fix unprotected import by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1874
• add ignore_modules to fsdp by @vchiley in https://github.com/mosaicml/composer/pull/1877
• Change vision image by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1881
• Fix eval_forward in the ComposerModel ABC by @eracah in https://github.com/mosaicml/composer/pull/1871
• Fix fsdp weight tying by @bcui19 in https://github.com/mosaicml/composer/pull/1856
• Bump pytest from 7.2.0 to 7.2.1 by @dependabot in https://github.com/mosaicml/composer/pull/1886
• Bump ipykernel from 6.19.2 to 6.20.1 by @dependabot in https://github.com/mosaicml/composer/pull/1887
• Bump gitpython from 3.1.28 to 3.1.30 by @dependabot in https://github.com/mosaicml/composer/pull/1888
• Update Vision Image in Pytest by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1882
• Streaming data tests by @dakinggg in https://github.com/mosaicml/composer/pull/1842
• Add NLP Algorithms Tests by @nik-mosaic in https://github.com/mosaicml/composer/pull/1839
• rename HF notebook by @dakinggg in https://github.com/mosaicml/composer/pull/1873
• Ensure loggers run init event before callbacks in Engine by @eracah in https://github.com/mosaicml/composer/pull/1890
• [Fix] Enable logging of metrics from Callbacks to ConsoleLogging by @eracah in https://github.com/mosaicml/composer/pull/1884
• Updating how we load metrics in a state_dict so we don't add extra memory overhead by @bcui19 in https://github.com/mosaicml/composer/pull/1892
• Getting daily tests passing by @dakinggg in https://github.com/mosaicml/composer/pull/1893
• Bump nbsphinx from 0.8.10 to 0.8.12 by @dependabot in https://github.com/mosaicml/composer/pull/1897
• Fix docker image by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1894
• Add primitive list support by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1906
• Raise an error in FSDP meta tensor initialization if there's no initialization functions, fix associated flaky FSDP test by @bcui19 in https://github.com/mosaicml/composer/pull/1905
• Gpu Test by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1907
• Update docker with FFCV fix by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1908
• Restore GPU tests by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1909
• Update workflow names by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1910
• Enable daily gpu tests by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1911
• Tweak daily GPU tests by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1912
• Daily GPU Tests -- Change to Git Commit by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1914
• Add logic for shifting labels before computing metrics by @alextrott16 in https://github.com/mosaicml/composer/pull/1913
• Add coreweave object store support. by @eracah in https://github.com/mosaicml/composer/pull/1915
• Fixes mis specified dependency by @dakinggg in https://github.com/mosaicml/composer/pull/1919
• Bump coverage[toml] from 7.0.4 to 7.1.0 by @dependabot in https://github.com/mosaicml/composer/pull/1923
• Update importlib-metadata requirement from <6,>=5.0.0 to >=5.0.0,<7 by @dependabot in https://github.com/mosaicml/composer/pull/1921
• pin setuptools in build requirements by @dakinggg in https://github.com/mosaicml/composer/pull/1926
• Remove synthetic testing infrastructure for HF/NLP by @dakinggg in https://github.com/mosaicml/composer/pull/1895
• Add upgrade flags to pip installs by @dakinggg in https://github.com/mosaicml/composer/pull/1916
• Temporarily pin pip to <23 by @dakinggg in https://github.com/mosaicml/composer/pull/1930
• add link protection by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1927
• Cleaning up error checking for FSDP sharding strategies with fp32 precision by @bcui19 in https://github.com/mosaicml/composer/pull/1925
• Fix mcp script to avoid follow by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1932
• Emit Eval progress in console logging by @eracah in https://github.com/mosaicml/composer/pull/1917
• Remove Fused LayerNorm deprecation by @nik-mosaic in https://github.com/mosaicml/composer/pull/1931
• Add EFA Support for Multinode in AWS by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1891
• remove jenkins gpu tests by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1933
• Typo due to stale MCLI docs by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1934
• Pin pip<23 in Docker images by @bandish-shah in https://github.com/mosaicml/composer/pull/1936
• Fix bug in trainer.eval and add test cases for test_console_logger by @eracah in https://github.com/mosaicml/composer/pull/1937
• Add few shot and multiple choice to ICL evaluation by @bmosaicml in https://github.com/mosaicml/composer/pull/1876
• Disable test_streaming_datasets in pytest-daily by @bandish-shah in https://github.com/mosaicml/composer/pull/1939

New Contributors

• @bmosaicml made their first contribution in https://github.com/mosaicml/composer/pull/1743
• @nguyenhoan1988 made their first contribution in https://github.com/mosaicml/composer/pull/1843
• @standardAI made their first contribution in https://github.com/mosaicml/composer/pull/1849
• @cclauss made their first contribution in https://github.com/mosaicml/composer/pull/1850
• @YilunKuang made their first contribution in https://github.com/mosaicml/composer/pull/1753
• @vchiley made their first contribution in https://github.com/mosaicml/composer/pull/1877

Full Changelog: https://github.com/mosaicml/composer/compare/v0.12.0...v0.12.1