What's Changed

Autoparallel

• [autoparallel] fix bugs caused by negative dim key (#1808) by YuliangLiu0306
• [autoparallel] fix bias addition module (#1800) by YuliangLiu0306
• [autoparallel] add batch norm metainfo (#1815) by Boyuan Yao
• [autoparallel] add conv metainfo class for auto parallel (#1796) by Boyuan Yao
• [autoparallel]add essential CommActions for broadcast oprands (#1793) by YuliangLiu0306
• [autoparallel] refactor and add rotorc. (#1789) by Super Daniel
• [autoparallel] add getattr handler (#1767) by YuliangLiu0306
• [autoparallel] added matmul handler (#1763) by Frank Lee
• [autoparallel] fix conv handler numerical test (#1771) by YuliangLiu0306
• [autoparallel] move ckpt solvers to autoparallel folder / refactor code (#1764) by Super Daniel
• [autoparallel] add numerical test for handlers (#1769) by YuliangLiu0306
• [autoparallel] update CommSpec to CommActions (#1768) by YuliangLiu0306
• [autoparallel] add numerical test for node strategies (#1760) by YuliangLiu0306
• [autoparallel] refactor the runtime apply pass and add docstring to passes (#1757) by YuliangLiu0306
• [autoparallel] added binary elementwise node handler (#1758) by Frank Lee
• [autoparallel] fix param hook issue in transform pass (#1755) by YuliangLiu0306
• [autoparallel] added addbmm handler (#1751) by Frank Lee
• [autoparallel] shard param and buffer as expected (#1753) by YuliangLiu0306
• [autoparallel] add sequential order to communication actions (#1735) by YuliangLiu0306
• [autoparallel] recovered skipped test cases (#1748) by Frank Lee
• [autoparallel] fixed wrong sharding strategy in conv handler (#1747) by Frank Lee
• [autoparallel] fixed wrong generated strategy for dot op (#1746) by Frank Lee
• [autoparallel] handled illegal sharding strategy in shape consistency (#1744) by Frank Lee
• [autoparallel] handled illegal strategy in node handler (#1743) by Frank Lee
• [autoparallel] handled illegal sharding strategy (#1728) by Frank Lee

Kernel

• [kernel] added jit warmup (#1792) by アマデウス
• [kernel] more flexible flashatt interface (#1804) by oahzxl
• [kernel] skip tests of flash_attn and triton when they are not available (#1798) by Jiarui Fang

Gemini

• [Gemini] make gemini usage simple (#1821) by Jiarui Fang

Checkpointio

• [CheckpointIO] a uniform checkpoint I/O module (#1689) by ver217

Doc

• [doc] polish diffusion README (#1840) by binmakeswell
• [doc] remove obsolete API demo (#1833) by binmakeswell
• [doc] add diffusion (#1827) by binmakeswell
• [doc] add FastFold (#1766) by binmakeswell

Example

• [example] remove useless readme in diffusion (#1831) by Jiarui Fang
• [example] add TP to GPT example (#1828) by Jiarui Fang
• [example] add stable diffuser (#1825) by Fazzie-Maqianli
• [example] simplify the GPT2 huggingface example (#1826) by Jiarui Fang
• [example] opt does not depend on Titans (#1811) by Jiarui Fang
• [example] add GPT by Jiarui Fang
• [example] add opt model in lauguage (#1809) by Jiarui Fang
• [example] add diffusion to example (#1805) by Jiarui Fang

Nfc

• [NFC] update gitignore remove DS_Store (#1830) by Jiarui Fang
• [NFC] polish type hint for shape consistency (#1801) by Jiarui Fang
• [NFC] polish tests/test_layers/test_3d/test_3d.py code style (#1740) by Ziheng Qin
• [NFC] polish tests/test_layers/test_3d/checks_3d/common.py code style (#1733) by lucasliunju
• [NFC] polish colossalai/nn/metric/_utils.py code style (#1727) by Sze-qq
• [NFC] polish tests/test_layers/test_3d/checks_3d/check_layer_3d.py code style (#1731) by Xue Fuzhao
• [NFC] polish tests/test_layers/test_sequence/checks_seq/check_layer_seq.py code style (#1723) by xyupeng
• [NFC] polish accuracy_2d.py code style (#1719) by Ofey Chan
• [NFC] polish .github/workflows/scripts/build_colossalai_wheel.py code style (#1721) by Arsmart1
• [NFC] polish _checkpoint_hook.py code style (#1722) by LuGY
• [NFC] polish test_2p5d/checks_2p5d/check_operation_2p5d.py code style (#1718) by Kai Wang (Victor Kai)
• [NFC] polish colossalai/zero/sharded_param/init.py code style (#1717) by CsRic
• [NFC] polish colossalai/nn/lr_scheduler/linear.py code style (#1716) by yuxuan-lou
• [NFC] polish tests/test_layers/test_2d/checks_2d/check_operation_2d.py code style (#1715) by binmakeswell
• [NFC] polish colossalai/nn/metric/accuracy_2p5d.py code style (#1714) by shenggan

Fx

• [fx] add a symbolic_trace api. (#1812) by Super Daniel
• [fx] skip diffusers unitest if it is not installed (#1799) by Jiarui Fang
• [fx] Add linear metainfo class for auto parallel (#1783) by Boyuan Yao
• [fx] support module with bias addition (#1780) by YuliangLiu0306
• [fx] refactor memory utils and extend shard utils. (#1754) by Super Daniel
• [fx] test tracer on diffuser modules. (#1750) by Super Daniel

Hotfix

• [hotfix] fix build error when torch version >= 1.13 (#1803) by xcnick
• [hotfix] polish flash attention (#1802) by oahzxl
• [hotfix] fix zero's incompatibility with checkpoint in torch-1.12 (#1786) by HELSON
• [hotfix] polish chunk import (#1787) by Jiarui Fang
• [hotfix] autoparallel unit test (#1752) by YuliangLiu0306

Pipeline

• [Pipeline]Adapt to Pipelinable OPT (#1782) by Ziyue Jiang

Ci

• [CI] downgrade fbgemm. (#1778) by Super Daniel

Compatibility

• [compatibility] ChunkMgr import error (#1772) by Jiarui Fang

Feat

• [feat] add flash attention (#1762) by oahzxl

Fx/profiler

• [fx/profiler] debug the fx.profiler / add an example test script for fx.profiler (#1730) by Super Daniel

Workflow

• [workflow] handled the git directory ownership error (#1741) by Frank Lee

Full Changelog: https://github.com/hpcaitech/ColossalAI/compare/v0.1.11rc2...v0.1.11rc1