🚀 Composer v0.6.1

Composer v0.6.1 is released!

Go ahead and upgrade; it's fully backwards compatible with Composer v0.6.0.

Install via pip:

pip install --upgrade mosaicml==0.6.1

Alternatively, install Composer with Conda:

conda install -c mosaicml mosaicml=0.6.1

What's New?

1. 📎 Adaptive Gradient Clipping (AGC)

   Adaptive Gradient Clipping (AGC) clips gradients based on the ratio of their norms with weights' norms. This technique helps stabilize training with large batch sizes, especially for models without batchnorm layers.

2. 🚚 Exponential Moving Average (EMA)

   Exponential Moving Average (EMA) is a model averaging technique that maintains an exponentially weighted moving average of the model parameters during training. The averaged parameters are used for model evaluation. EMA typically results in less noisy validation metrics over the course of training, and sometimes increased generalization.

3. 🪵 Logger is available in the ComposerModel

   The Logger is bound to the ComposerModel via the self.logger attribute. It is available during training on all methods (other than __init__).

   For example, to log hidden activation:

   class Net(ComposerModel):
   
       def forward(self, x):
           x = F.relu(F.max_pool2d(self.conv1(x), 2))
           x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
           if self.logger:
               self.logger.data_batch({
                   "hidden_activation_norm": x.norm(2).item(),
               })
           x = x.view(-1, 320)
           x = F.relu(self.fc1(x))
           x = F.dropout(x, training=self.training)
           x = self.fc2(x)
           return F.log_softmax(x)
   

4. 🐛 Environment Collection Script

   Composer v0.6.1 includes an environment collection script which generates a printout of your system configuration and python environment. If you run into a bug, the results from this script will help us debug the issue and fix Composer.

   To collect your environment information:

   $ pip install mosaicml  # if composer is not already installed
   $ composer_collect_env
   

   Then, include the output in your GitHub Issue.

What's Improved?

1. 📜 TorchScriptable Algorithms

   BlurPool, Ghost BatchNorm, and Stochastic Depth are now TorchScript-compatible. Try exporting your models with these algorithms enabled!

2. 🏛️ ColOut on Segmentation

   ColOut now supports segmentation-style models.

What's Fixed?

1. 🚑️ Loggers capture the Traceback

   We fixed a bug so the Loggers, such as the Weights & Biases Logger and the File Logger, will capture the traceback any exception that crashes the training process.

2. 🏋️ Weights & Biases Logger Config

   We fixed a bug where the the Weights & Biases Logger was not properly recording the configuration.

Full Changelog

https://github.com/mosaicml/composer/compare/v0.6.0...v0.6.1