v0.14.0

版本发布时间: 2024-08-02 16:42:16

xorbitsai/inference最新发布版本:v1.2.0(2025-01-10 17:34:30)

What's new in 0.14.0 (2024-08-02)

These are the changes in inference v0.14.0.

FEAT: Supports model_path input when launching models by @Valdanitooooo in https://github.com/xorbitsai/inference/pull/1918
FEAT: Support gte-Qwen2-7B-instruct and multi gpu deploy by @amumu96 in https://github.com/xorbitsai/inference/pull/1994

ENH: Add support of sglang for llama 3 qwen 2 by @luweizheng in https://github.com/xorbitsai/inference/pull/1947
ENH: add cache_limit_gb option for MLX by @qinxuye in https://github.com/xorbitsai/inference/pull/1954
ENH: [benchmark] Add api-key support by @frostyplanet in https://github.com/xorbitsai/inference/pull/1961
ENH: Support for Gemma 2 and Llama 3.1 Models for vllm & sglang by @vikrantrathore in https://github.com/xorbitsai/inference/pull/1929
ENH: [K8s] worker log dir name by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1997
ENH: support image_to_image by @qinxuye in https://github.com/xorbitsai/inference/pull/1986
REF: enable sglang by default by @qinxuye in https://github.com/xorbitsai/inference/pull/1953

BUG: Fix GLM chat by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1966
BUG: fix match for transformers from model registered by @qinxuye in https://github.com/xorbitsai/inference/pull/1955
BUG: Load llama.so failed in docker image by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1974
BUG: [UI]Modifying 'model format' again resulted in an error message. by @yiboyasss in https://github.com/xorbitsai/inference/pull/1990
BUG: fix loading multiple gguf parts by @qinxuye in https://github.com/xorbitsai/inference/pull/1987

@vikrantrathore made their first contribution in https://github.com/xorbitsai/inference/pull/1929
@Valdanitooooo made their first contribution in https://github.com/xorbitsai/inference/pull/1918

Full Changelog: https://github.com/xorbitsai/inference/compare/v0.13.3...v0.14.0