🚀 Composer v0.10.1

Composer v0.10.1 is released! Install via pip:

pip install --upgrade mosaicml==0.10.1

New Features

1. 𐄷 Weight Standardization

   Weight Standardization reparametrizes convolutional weights such that the fan-in dimensions have zero mean and unit standard deviation. This could slightly improve performance at the expensive of 5% lower throughput. This has been used in several papers to train with smaller batch sizes, with normalization layers besides batch norm, and for transfer learning.

   Using Weight Standardization with the Composer Trainer:

   import composer
   
   # Apply Weight Standardization (when training is initialized)
   weight_std = composer.algorithms.WeightStandardization()
   
   # Train with Weight Standardization
   trainer = composer.trainer.Trainer(
       ...
       algorithms=[weight_std]
   )
   trainer.fit()
   

   Using Weight Standardization with the Composer functional interface:

   import composer
   from torchvision.models import resnet50
   
   my_model = resnet50()
   
   # Apply weight standardization to model
   my_model = composer.functional.weight_standardization(my_model)
   

   Please see the Weight Standardization Method Card for more details.

Bug Fixes

• Fix for checkpoints not being saved automatically at the end of a run (#1552)
• Fix Onnx export for Composer HuggingFaceModels (#1557)
• Fix for MIoU metric producing NaN's (#1558)
• CometML logger documentation updates and fixes (#1567, #1570, #1571)
• WandB image visualizer fix (#1591)

What's Changed

• Update evaluate_periodically() when eval interval is of type Duration by @karan6181 in https://github.com/mosaicml/composer/pull/1523
• Quality of life updates to EMA by @coryMosaicML in https://github.com/mosaicml/composer/pull/1524
• Add ADE20K and COCO v2 dataset behind a version flag by @karan6181 in https://github.com/mosaicml/composer/pull/1528
• Pinned setuptools version to fix distutils version error by @karan6181 in https://github.com/mosaicml/composer/pull/1536
• Less strict name formatting by @hanlint in https://github.com/mosaicml/composer/pull/1535
• Defaulting streaming dataset version to 1 and add a deprecation warning by @karan6181 in https://github.com/mosaicml/composer/pull/1532
• Changing 'stable' to 'latest' in notebooks in examples by @bcui19 in https://github.com/mosaicml/composer/pull/1534
• Bump furo from 2022.6.21 to 2022.9.15 by @dependabot in https://github.com/mosaicml/composer/pull/1540
• Bump fasteners from 0.17.3 to 0.18 by @dependabot in https://github.com/mosaicml/composer/pull/1538
• Add Pandoc to Docker images, bump version to 2.19.2 by @bandish-shah in https://github.com/mosaicml/composer/pull/1550
• Removed streaming version 2 from yaml since version 1 is default by @karan6181 in https://github.com/mosaicml/composer/pull/1551
• Bump ipykernel from 6.15.2 to 6.15.3 by @dependabot in https://github.com/mosaicml/composer/pull/1548
• Bump yamllint from 1.27.1 to 1.28.0 by @dependabot in https://github.com/mosaicml/composer/pull/1546
• Bump traitlets from 5.3.0 to 5.4.0 by @dependabot in https://github.com/mosaicml/composer/pull/1539
• Object Store Logger Race Condition + EMA Fix by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1552
• Adding in erroring for when using GradMonitor and DeepSpeed by @bcui19 in https://github.com/mosaicml/composer/pull/1555
• Bump pypandoc from 1.8.1 to 1.9 by @dependabot in https://github.com/mosaicml/composer/pull/1559
• Update context to raise errror by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1561
• Fix MIoU metric when self.total_union==0 by @abhi-mosaic in https://github.com/mosaicml/composer/pull/1558
• Move dataloader initialize_object to factory methods by @hanlint in https://github.com/mosaicml/composer/pull/1510
• Weight Standardization method by @Landanjs in https://github.com/mosaicml/composer/pull/1562
• Update comet links to include query params and point to main site by @dakinggg in https://github.com/mosaicml/composer/pull/1567
• remove dead line in alibi by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1568
• GLU Fixes by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1564
• Add FSDP strategy by @abhi-mosaic in https://github.com/mosaicml/composer/pull/1553
• Comet example by @dakinggg in https://github.com/mosaicml/composer/pull/1570
• Add missing _enabled flag, post_close, and clean up comet ml tests by @dakinggg in https://github.com/mosaicml/composer/pull/1571
• Consistent Method Card Style by @growlix in https://github.com/mosaicml/composer/pull/1407
• add missing return in context by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1574
• Remove eval batch split by @mvpatel2000 in https://github.com/mosaicml/composer/pull/1576
• Fix Onnx Export for Composer HuggingFaceModels by @nik-mosaic in https://github.com/mosaicml/composer/pull/1557
• Revert checkpoint rename by @hanlint in https://github.com/mosaicml/composer/pull/1579

New Contributors

• @bcui19 made their first contribution in https://github.com/mosaicml/composer/pull/1534

Full Changelog: https://github.com/mosaicml/composer/compare/v0.10.0...v0.10.1