v0.9.2

版本发布时间: 2024-03-08 14:09:49

xorbitsai/inference最新发布版本:v0.11.0(2024-05-11 17:41:09)

What's new in 0.9.2 (2024-03-08)

These are the changes in inference v0.9.2.

FEAT: Add a command / SDK interface to query which models are able to… by @hainaweiben in https://github.com/xorbitsai/inference/pull/1076
FEAT: add a docker-compose-distributed example with multiple workers by @bufferoverflow in https://github.com/xorbitsai/inference/pull/1064
FEAT: Support download and merge multiple parts of gguf files by @notsyncing in https://github.com/xorbitsai/inference/pull/1075
FEAT: Supports LoRA for LLM and image models by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1080

ENH: Supports n_gpu_layers parameter for llama-cpp-python by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1070
ENH: Add a dropdown to the web UI to support adjusting GPU offload layers for llama.cpp loader by @notsyncing in https://github.com/xorbitsai/inference/pull/1073
ENH: [UI] Show replica on running model page by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1093
ENH: Add "[DONE]" to the end of stream generation for better openai SDK compatibility by @ZhangTianrong in https://github.com/xorbitsai/inference/pull/1062
ENH: [UI] Support setting CPU when selecting n_gpu by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1096

Update llm_family.json to correct the context length of glaive coder by @mikeshi80 in https://github.com/xorbitsai/inference/pull/1083

Full Changelog: https://github.com/xorbitsai/inference/compare/v0.9.1...v0.9.2