MyGit

v0.14.0

mosaicml/composer

版本发布时间: 2023-05-03 23:42:21

mosaicml/composer最新发布版本:v0.25.0(2024-09-25 04:56:05)

:rocket: Composer v0.14.0

Composer v0.14.0 is released! Install via pip:

pip install composer==0.14.0

The legacy package name still works via pip:

pip install mosaicml==0.14.0

New Features

  1. 🆕 PyTorch 2.0 Support (#2172)

    We're thrilled to announce official support for PyTorch 2.0! We've got all initial unit tests passing and run through our examples. We've also made some updates to start taking advantage of all the great new features.

    Initial support also includes:

    • Support for torch.compile

      Model Dataset Without compile thoughput/samples_per_sec With compile thoughput/samples_per_sec Performance %
      ResNet50 ImageNet 5557 7424 33.60%
      DeepLab V3 ADE20K 81.60 98.82 21.10%
      HF BERT C4 3360 4259 26.75%
      HF Causal LM C4 50.61 103.29 100.05%

      To start using, simply add compile_config argument to the Trainer:

        # To use default `torch.compile` config
  trainer = Trainer(
     ...,
     compile_config={},
  )

  # To use custom `torch.compile` config, provide an argument as a dictionary, for example:
  trainer = Trainer(
     ...,
     compile_config={'mode': 'reduce-overhead'},
  )

      The Trainer also supports pre-compiled models passed via the models argument. If the model has been pre-compiled, the compile_config argument is ignored if provided.

      Note: We recommend baselining your model with and without torch.compile as there are scenarios where enabling compile does not yield any throughput improvements and in some cases where this can lead to a regression.

    • PyTorch 2.0 Docker Images

      We've added the following new official MosaicML Docker Images with PyTorch 2.0 support:

      Linux Distro Flavor PyTorch Version CUDA Version Python Version Docker Tags
      Ubuntu 20.04 Base 2.0.0 11.7.1 (Infiniband) 3.10 mosaicml/pytorch:2.0.0_cu117-python3.10-ubuntu20.04
      Ubuntu 20.04 Base 2.0.0 11.7.1 (EFA) 3.10 mosaicml/pytorch:2.0.0_cu117-python3.10-ubuntu20.04-aws
      Ubuntu 20.04 Base 2.0.0 cpu 3.10 mosaicml/pytorch:2.0.0_cpu-python3.10-ubuntu20.04
      Ubuntu 20.04 Vision 2.0.0 11.7.1 (Infiniband) 3.10 mosaicml/pytorch_vision:2.0.0_cu117-python3.10-ubuntu20.04
      Ubuntu 20.04 Vision 2.0.0 cpu 3.10 mosaicml/pytorch_vision:2.0.0_cpu-python3.10-ubuntu20.04

  2. 🦾 New Callbacks

    • Activation monitor (#2066)

      Monitors activations in the network. Every interval batches it will attach a forwards hook and logs the max, average, l2 norm, and kurtosis for the input and output activations. To enable:

      from composer import Trainer
from composer.callbacks import ActivationMonitor

# Construct Trainer
trainer = Trainer(
   ...,
   callbacks=[ActivationMonitor()],
)

# Train!
trainer.fit()

    • Slack Logger (#2133)

      You can now send custom training metrics using Slack! To enable:

      from composer import Trainer
from composer.loggers import SlackLogger

transform = transforms.Compose([transforms.ToTensor()])


trainer = Trainer(
   ...
   loggers=[
       SlackLogger(
           log_interval="10ba", # or 1ep, 2ep 
           include_keys=["algorithm_traces*", "loss*"],
           formatter_func=(lambda data, **kwargs:
              [
                  {
                      "type": "section", "text": {"type": "mrkdwn", "text": f"*{k}:* {v}"}
                  }
                  for k, v in data.items()
              ])
       )
   ],
)

trainer.fit()

      Please see PR #2133 for additional details.

API changes

Deprecations

Bug Fixes

What's Changed

New Contributors

Full Changelog: https://github.com/mosaicml/composer/compare/v0.13.5...v0.14.0

相关地址：原始地址 下载(tar) 下载(zip)

查看：2023-05-03发行的版本

微信小程序
MyGit:GitHub仓库更新&通知小工具