v0.13.4.rc2
版本发布时间: 2023-08-14 21:28:42
huggingface/tokenizers最新发布版本:v0.15.0(2023-11-15 03:06:30)
What's Changed
- Fix stride condition. by @Narsil in https://github.com/huggingface/tokenizers/pull/1321
Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.13.4.rc1...v0.13.4.rc2