python-v0.11.6
版本发布时间: 2022-02-28 17:22:18
huggingface/tokenizers最新发布版本:v0.15.0(2023-11-15 03:06:30)
- [#919] Fixing single_word AddedToken. (regression from 0.11.2)
- [#916] Deserializing faster
added_tokens
by loading them in batch.
版本发布时间: 2022-02-28 17:22:18
huggingface/tokenizers最新发布版本:v0.15.0(2023-11-15 03:06:30)
added_tokens
by loading them in batch.