v4.42.3
版本发布时间: 2024-06-28 23:35:36
huggingface/transformers最新发布版本:v4.46.3(2024-11-19 06:13:14)
Make sure we have attention softcapping for "eager" GEMMA2 model
After experimenting, we noticed that for the 27b model mostly, softcapping is a must. So adding it back (it should have been there, but an error on my side made it disappear) sorry all! 😭
- Gemma capping is a must for big models (#31698)