What's new in 0.4.2 (2023-09-15)

These are the changes in inference v0.4.2.

New features

• FEAT: concurrent generation by @codingl2k1 in https://github.com/xorbitsai/inference/pull/417
• FEAT: Support gguf by @aresnow1 in https://github.com/xorbitsai/inference/pull/446
• FEAT: Support OpenBuddy by @codingl2k1 in https://github.com/xorbitsai/inference/pull/444

Enhancements

• ENH: client support desc model by @UranusSeven in https://github.com/xorbitsai/inference/pull/442
• ENH: caching from self-hosted storage by @UranusSeven in https://github.com/xorbitsai/inference/pull/419
• ENH: Assign worker sub pool at runtime instead of pre-allocated by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/437
• ENH: add benchmark script by @UranusSeven in https://github.com/xorbitsai/inference/pull/451

Bug fixes

• BUG: Fix restful client for embedding models by @aresnow1 in https://github.com/xorbitsai/inference/pull/439
• BUG: cmdline double line breaker by @UranusSeven in https://github.com/xorbitsai/inference/pull/441
• BUG: no error raised on unsupported fmt by @UranusSeven in https://github.com/xorbitsai/inference/pull/443
• BUG: Xinferecen list failed if embedding models are launched by @aresnow1 in https://github.com/xorbitsai/inference/pull/452

Tests

• TST: skip self-hosted storage tests by @UranusSeven in https://github.com/xorbitsai/inference/pull/453

Documentation

• DOC: fix baichuan-2 and make naming consistent by @UranusSeven in https://github.com/xorbitsai/inference/pull/432
• DOC: update hot topics by @UranusSeven in https://github.com/xorbitsai/inference/pull/456

Others

• CI: Fix Windows CI by @codingl2k1 in https://github.com/xorbitsai/inference/pull/440

New Contributors

• @ChengjieLi28 made their first contribution in https://github.com/xorbitsai/inference/pull/437

Full Changelog: https://github.com/xorbitsai/inference/compare/v0.4.1...v0.4.2