The OpenCompass team is thrilled to announce the release of OpenCompass v0.3.8!

🎉 Added Features:

-🆕 DLC runner Lark report functionality has been introduced to improve reporting mechanisms. ( #1735 ) -🆕 Chinese SimpleQA configuration has been added to extend language support. ( #1697 ) -🆕 Addition of OC academic 2412 evaluation example ( #1750 )

📖 Documentation -📚 Supplemented KOR-BENCH readme documentation for better user guidance. ( #1734 ) -📚 Updated README files for various datasets including Korbench to ensure up-to-date information. ( #1737 )

🐛 Bug Fixes -🔧 Corrected an error in the subjective default summarizer for more accurate results. ( #1740 ) -🔧 Adjusted the max_out_len parameter for ChineseSimpleQA to fix related issues. ( #1757 ) -🔧 Resolved a problem with the transfer of the vllm max_seq_len parameter for consistent behavior. ( #1745 )

⚙ Enhancements and Refactors -💪 Updated Manifest file to reflect the latest changes in the project. ( #1738 ) -💪 Modified Compassarena metric for enhanced performance measurement. ( #1749 ) -💪 Improved dataset configurations by removing max_out_len where not applicable. ( #1754 ) -💪 Upgraded requirement and deepseek configurations for better compatibility. ( #1764 )

🎉 Welcome New Contributors

We are pleased to welcome @OpenStellarTeam, who contributed to the addition of Chinese SimpleQA config. ( #1697 )

For a comprehensive overview of all changes, please refer to the Full Changelog.

Thank you for being part of the OpenCompass community! Your support and contributions make each release possible.