v0.0.23
版本发布时间: 2023-12-07 00:05:54
facebookresearch/xformers最新发布版本:v0.0.28.post1(2024-09-13 23:52:20)
Pre-built binary wheels require PyTorch 2.1.1
Fixed
- fMHA: Fixed a bug in cutlass backend forward pass where the logsumexp was not correctly calculated, resulting in wrong results in the BW pass. This would happen with MQA when one sequence has a query with
length%64 == 1
- fMHA: Updated Flash-Attention to v2.3.6 - this fixes a performance regression in causal backward passes, and now supports
BlockDiagonalCausalWithOffsetPaddedKeysMask
Added
- fMHA: Added
LocalAttentionFromBottomRightMask
(local) - fMHA: Added
LowerTriangularFromBottomRightMask
(causal) - fMHA: Added
LowerTriangularFromBottomRightLocalAttentionMask
(local + causal)
Removed
- Removed
xformers.triton.sum_strided