What's Changed

• Prometheus - custom metrics support + other improvements by @krrishdholakia in https://github.com/BerriAI/litellm/pull/7489
• (feat) POST /fine_tuning/jobs support passing vertex specific hyper params by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7490
• (Feat) - LiteLLM Use UsernamePasswordCredential for Azure OpenAI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7496
• (docs) Add docs on load testing benchmarks by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7499
• (Feat) Add support for reading secrets from Hashicorp vault by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7497
• Litellm dev 12 30 2024 p2 by @krrishdholakia in https://github.com/BerriAI/litellm/pull/7495
• Refactor Custom Metrics on Prometheus - allow setting k,v pairs on all metrics via config.yaml by @krrishdholakia in https://github.com/BerriAI/litellm/pull/7498
• (fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7500
• Litellm dev 01 01 2025 p3 by @krrishdholakia in https://github.com/BerriAI/litellm/pull/7503
• Litellm dev 01 02 2025 p2 by @krrishdholakia in https://github.com/BerriAI/litellm/pull/7512
• Revert "(fix) GCS bucket logger - apply truncate_standard_logging_payload_content to standard_logging_payload and ensure GCS flushes queue on fails" by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7515
• (perf) use aiohttp for custom_openai by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7514
• (perf) use threadpool executor - for sync logging integrations by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/7509

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.56.6...v1.56.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.56.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results