What's Changed

• fix build on metal by returning Device by @rgbkrk in https://github.com/EricLBuehler/mistral.rs/pull/642
• Add invite to Matrix chatroom by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/644
• Make sure we don't have dead links by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/647
• Fix more links by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/648
• Throughput for interactive mode by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/655
• Implement tool calling by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/649
• Fix device map check for paged attn by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/656
• Fix for mistral nemo in gguf by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/657
• Fix check of cache config when device mapping + PA by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/658
• Biollama in tool calling example by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/659
• Biollama in tool calling example by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/660
• Examples for simple tool calling by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/661
• Bump version to 0.2.4 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/662

New Contributors

• @rgbkrk made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/642

Full Changelog: https://github.com/EricLBuehler/mistral.rs/compare/v0.2.3...v0.2.4

MSRV

MSRV is 1.75

Install mistralrs-server 0.2.4

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-installer.sh | sh

Download mistralrs-server 0.2.4