What's new in 0.0.5 (2023-07-19)

These are the changes in inference v0.0.5.

New features

• FEAT: support pytorch models by @pangyoki in https://github.com/xorbitsai/inference/pull/157
• FEAT: support vicuna-v1.3 33B by @Bojun-Feng in https://github.com/xorbitsai/inference/pull/192
• FEAT: support baichuan-chat pytorch model by @pangyoki in https://github.com/xorbitsai/inference/pull/190
• FEAT: pytorch model support MPS backend by @pangyoki in https://github.com/xorbitsai/inference/pull/198
• FEAT: Embedding by @jiayini1119 in https://github.com/xorbitsai/inference/pull/194
• FEAT: LLaMA-2 by @UranusSeven in https://github.com/xorbitsai/inference/pull/203

Enhancements

• ENH: Implement RESTful API stream generate by @jiayini1119 in https://github.com/xorbitsai/inference/pull/171
• ENH: set default device to mps on MacOS by @pangyoki in https://github.com/xorbitsai/inference/pull/205
• ENH: Set default mlock to true and mmap to false by @RayJi01 in https://github.com/xorbitsai/inference/pull/206
• ENH: add Gradio ChatInterface chatbot to example by @Bojun-Feng in https://github.com/xorbitsai/inference/pull/208

Bug fixes

• BUG: fix pytorch int8 by @pangyoki in https://github.com/xorbitsai/inference/pull/197
• BUG: RuntimeError when launching model using kwargs whose value is of type int by @jiayini1119 in https://github.com/xorbitsai/inference/pull/209
• BUG: Fix some gradio issues by @aresnow1 in https://github.com/xorbitsai/inference/pull/200

Documentation

• DOC: sphinx init by @UranusSeven in https://github.com/xorbitsai/inference/pull/189
• DOC: chinese readme by @UranusSeven in https://github.com/xorbitsai/inference/pull/191

Full Changelog: https://github.com/xorbitsai/inference/compare/v0.0.4...v0.0.5