Welcome to the newest version of OpenCompass! v0.1.6 brings forth exciting dataset additions, crucial fixes, and enhanced documentation. We're confident that this release will provide a better and smoother experience for all users.

🆕 Highlights:

• Dataset Enrichment: Multiple additions, especially from the GLUE suite, to provide more versatility and better testing capabilities.
• Documentation Revamp: Fixed dead links and updated the 'get_started' section to assist our users in navigating OpenCompass effortlessly.
• Introducing New Faces: A warm welcome to our newest contributors. Your dedication and contributions are pivotal to our progress!

Dive into the details:

🌟 New Features:

• 📦 Datasets Galore:

  • Introduced WikiText-2&103 dataset (#397)
  • GLUE dataset additions:
    • CoLA (#406)
    • QQP (#438)
    • MRPC (#440)
  • Lawbench dataset addition (#460)

• 🛠 Utilities and Enhancements:

  • Re-implementation of ceval load dataset (#446)
  • Integrated turbomind inference through its RPC API (#414)
  • Moved fix_id_list to Retriever for better code organization (#442)

• 📖 Documentation and Syncs:

  • Updated dataset list and get_started section (#437, #435)
  • Resolved dead links in the readme (#455)
  • Enhancements to LongEval and subjective evaluation (#443, #475)

🐛 Bug Fixes:

• Addressed issues related to clp errors and support for bs>1 (#439)
• Resolved issues concerning jieba rouge (#459, #467)
• Enhanced EOS string detection for splitting (#477)
• Various other fixes for optimal performance.

🎉 Welcome New Contributors:

• A big shout-out to our new contributors:
  • @KevinNuNu (First PR)
  • @lvhan028 (First PR)

Huge thanks to all contributors! Your constant efforts make OpenCompass better with each release. 🙌 🎉

Changelog

• [SIG] add WikiText-2&103 by @KevinNuNu in https://github.com/open-compass/opencompass/pull/397
• [SIG] add GLUE_CoLA dataset by @KevinNuNu in https://github.com/open-compass/opencompass/pull/406
• [SIG] add GLUE QQP dataset by @KevinNuNu in https://github.com/open-compass/opencompass/pull/438
• [SIG] add GLUE_MRPC dataset by @KevinNuNu in https://github.com/open-compass/opencompass/pull/440
• [Doc] Update dataset list by @Leymore in https://github.com/open-compass/opencompass/pull/437
• [Fix] use eval field check by @Leymore in https://github.com/open-compass/opencompass/pull/441
• [Sync] Update LongEval by @philipwangOvO in https://github.com/open-compass/opencompass/pull/443
• [Fix] fix clp potential error and support bs>1 by @yingfhu in https://github.com/open-compass/opencompass/pull/439
• [Feature] re-implement ceval load dataset by @Leymore in https://github.com/open-compass/opencompass/pull/446
• Integrate turbomind inference via its RPC API instead of its python API by @lvhan028 in https://github.com/open-compass/opencompass/pull/414
• [Docs] update get_started by @gaotongxiao in https://github.com/open-compass/opencompass/pull/435
• [Refactor] Move fix_id_list to Retriever by @gaotongxiao in https://github.com/open-compass/opencompass/pull/442
• [Docs] Fix dead links in readme by @gaotongxiao in https://github.com/open-compass/opencompass/pull/455
• [Fix] Use jieba rouge in lcsts by @Leymore in https://github.com/open-compass/opencompass/pull/459
• [Fix] Fix jieba rouge with empty string by @Leymore in https://github.com/open-compass/opencompass/pull/467
• [Sync] Add subjective evaluation by @Leymore in https://github.com/open-compass/opencompass/pull/475
• [Feature] Add lawbench by @Leymore in https://github.com/open-compass/opencompass/pull/460
• [Fix] split if and only if complete eos string shows up by @Leymore in https://github.com/open-compass/opencompass/pull/477
• Bump version to 0.1.6 by @Leymore in https://github.com/open-compass/opencompass/pull/478

For a detailed overview, check out our Full Changelog.

If you find OpenCompass beneficial, kindly star 🌟 our GitHub repository! We value your feedback, reviews, and continued support.