v1.34.8
版本发布时间: 2024-03-28 22:39:20
BerriAI/litellm最新发布版本:v1.44.15-stable(2024-09-04 00:07:25)
What's Changed
- Fix #2713 Remove duplicated "blocked" field on LiteLLM_TeamTable by @readevalprint in https://github.com/BerriAI/litellm/pull/2714
- Updating the default Claude3 max tokens by @rmann-nflx in https://github.com/BerriAI/litellm/pull/2701
- (fix) bump uvicorn on proxy docker builds by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/2719
- (fix) Proxy - remove background tasks by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/2721
- feat(auth_checks.py): enable admin to enforce 'user' param for all openai endpoints by @krrishdholakia in https://github.com/BerriAI/litellm/pull/2726
- feat(proxy_server.py): new
/spend/calculate
endpoint by @krrishdholakia in https://github.com/BerriAI/litellm/pull/2725 - [FEAT] Improve Proxy Perf - access router model names in constant time by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/2723
- [FEAT] Proxy - reduce deep copies by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/2728
New Contributors
- @readevalprint made their first contribution in https://github.com/BerriAI/litellm/pull/2714
- @rmann-nflx made their first contribution in https://github.com/BerriAI/litellm/pull/2701
Full Changelog: https://github.com/BerriAI/litellm/compare/v1.34.6...v1.34.8
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 39 | 47.89723312525114 | 1.6532145452982783 | 0.003339827364238946 | 495 | 1 | 35.51040400003558 | 1220.0857709999582 |
/health/liveliness | Passed ✅ | 26 | 28.17128257893658 | 15.633731892002507 | 0.006679654728477892 | 4681 | 2 | 23.512639999978546 | 1222.522328000025 |
/health/readiness | Passed ✅ | 26 | 30.12289661691968 | 15.396604149141542 | 0.0 | 4610 | 0 | 23.55434899999409 | 1311.0719019999806 |
Aggregated | Passed ✅ | 26 | 30.08843833568385 | 32.68355058644232 | 0.010019482092716837 | 9786 | 3 | 23.512639999978546 | 1311.0719019999806 |
1、 load_test.html 1.59MB
2、 load_test_stats.csv 905B