MyGit

v2.22.0

mudler/LocalAI

版本发布时间: 2024-10-12 21:09:59

mudler/LocalAI最新发布版本:v2.23.0(2024-11-11 01:07:39)

LocalAI v2.22.0 is out :partying_face:

:bulb: Highlights

:framed_picture: Multimodal vLLM

To use multimodal models with vLLM simply specify the model in the YAML file. Models however can differ if support multiple images or single images, along how they process internally placeholders for images.

Some models/libraries have different way to express images, videos or audio placeholders. For example, llama.cpp backend expects images within an [img-ID] tag, but other backends/models (e.g. vLLM) use a different notation ( <|image_|>).

For example, to override defaults, now it is possible to set in the model configuration the following:


template:
  video: "<|video_{{.ID}}|> {{.Text}}"
  image: "<|image_{{.ID}}|> {{.Text}}"
  audio: "<|audio_{{.ID}}|> {{.Text}}"

:video_camera: Video and Audio understanding

Some libraries might support both Video and Audio. Currently only vLLM supports Video understanding, and can be used in the API by "extending" the OpenAI API with audio and video type along images:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What'\''s in this video?"
          },
          {
            "type": "video_url",
            "video_url": {
              "url": "https://video-image-url"
            }
          }
        ]
      }
    ],
    "max_tokens": 300
  }'

:factory_worker: Work in progress

What's Changed

Bug fixes :bug:

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

New Contributors

Full Changelog: https://github.com/mudler/LocalAI/compare/v2.21.1...v2.22.0

相关地址：原始地址 下载(tar) 下载(zip)

1、 local-ai-Darwin-arm64 187.28MB

2、 local-ai-Darwin-arm64.sha256 96B

3、 local-ai-Darwin-x86_64 178.53MB

4、 local-ai-Darwin-x86_64.sha256 97B

5、 local-ai-Linux-arm64 142.5MB

6、 local-ai-Linux-arm64.sha256 95B

7、 local-ai-Linux-x86_64 1.25GB

8、 local-ai-Linux-x86_64.sha256 96B

9、 stablediffusion 6.52MB

查看：2024-10-12发行的版本

微信小程序
MyGit:GitHub仓库更新&通知小工具