🔆 Highlights

Argilla 1.15.0 comes with an enhanced FeedbackDataset settings page enabling the update of the dataset settings, an integration of the TRL package with the ArgillaTrainer, and continues adding improvements to the Python client for managing FeedbackDatasets.

⚙️ Update FeedbackDataset settings from the UI

FeedbackDataset settings page has been updated and now it allows to update the guidelines and some attributes of the fields and questions of the dataset. Did you misspell the title or description of a field or question? Well, you don't have to remove your dataset and create it again anymore! Just go to the settings page and fix it.

🤖 TRL integration with the ArgillaTrainer

The famous TRL package for training Transformers with Reinforcement Learning techniques has been integrated with the ArgillaTrainer, that comes with four new TrainingTask: SFT, Reward Modeling, PPO and DPO. Each training task expects a formatting function that will return the data in the expected format for training the model.

Check this 🆕 tutorial for training a Reward Model using the Argilla Trainer.

🐍 Filter FeedbackDataset and remove suggestions

In the 1.14.0 release we added many improvements for working with remote FeedbackDatasets. In this release, a new filter_by method has been added that allows to filter the records of a dataset from the Python client. For now, the records can be only filtered using the response_status, but we're planning adding more complex filters for the upcoming releases. In addition, new methods have been added allowing to remove the suggestions created for a record.

1.15.0

Added

• Added Enable to update guidelines and dataset settings for Feedback Datasets directly in the UI (#3489)
• Added ArgillaTrainer integration with TRL, allowing for easy supervised finetuning, reward modeling, direct preference optimization and proximal policy optimization (#3467)
• Added formatting_func to ArgillaTrainer for FeedbackDataset datasets add a custom formatting for the data (#3599).
• Added login function in argilla.client.login to login into an Argilla server and store the credentials locally (#3582).
• Added login command to login into an Argilla server (#3600).
• Added logout command to logout from an Argilla server (#3605).
• Added DELETE /api/v1/suggestions/{suggestion_id} endpoint to delete a suggestion given its ID (#3617).
• Added DELETE /api/v1/records/{record_id}/suggestions endpoint to delete several suggestions linked to the same record given their IDs (#3617).
• Added response_status param to GET /api/v1/datasets/{dataset_id}/records to be able to filter by response_status as previously included for GET /api/v1/me/datasets/{dataset_id}/records (#3613).
• Added list classmethod to ArgillaMixin to be used as FeedbackDataset.list(), also including the workspace to list from as arg (#3619).
• Added filter_by method in RemoteFeedbackDataset to filter based on response_status (#3610).
• Added list_workspaces function (to be used as rg.list_workspaces, but Workspace.list is preferred) to list all the workspaces from an user in Argilla (#3641).
• Added list_datasets function (to be used as rg.list_datasets) to list the TextClassification, TokenClassification, and Text2Text datasets in Argilla (#3638).
• Added RemoteSuggestionSchema to manage suggestions in Argilla, including the delete method to delete suggestios from Argilla via DELETE /api/v1/suggestions/{suggestion_id} (#3651).
• Added delete_suggestions to RemoteFeedbackRecord to remove suggestions from Argilla via DELETE /api/v1/records/{record_id}/suggestions (#3651).

Changed

• Changed Optional label for * mark for required question (#3608)
• Updated RemoteFeedbackDataset.delete_records to use batch delete records endpoint (#3580).
• Included allowed_for_roles for some RemoteFeedbackDataset, RemoteFeedbackRecords, and RemoteFeedbackRecord methods that are only allowed for users with roles owner and admin (#3601).
• Renamed ArgillaToFromMixin to ArgillaMixin (#3619).
• Move users CLI app under database CLI app (#3593).
• Move server Enum classes to argilla.server.enums module (#3620).

Fixed

• Fixed Filter by workspace in breadcrumbs (#3577)
• Fixed Filter by workspace in datasets table (#3604)
• Fixed Query search highlight for Text2Text and TextClassification (#3621)
• Fixed RatingQuestion.values validation to raise a ValidationError when values are out of range i.e. [1, 10] (#3626).

Removed

• Removed multi_task_text_token_classification from TaskType as not used (#3640).
• Removed argilla_id in favor of id from RemoteFeedbackDataset (#3663).
• Removed fetch_records from RemoteFeedbackDataset as now the records are lazily fetched from Argilla (#3663).
• Removed push_to_argilla from RemoteFeedbackDataset, as it just works when calling it through a FeedbackDataset locally, as now the updates of the remote datasets are automatically pushed to Argilla (#3663).
• Removed set_suggestions in favor of update(suggestions=...) for both FeedbackRecord and RemoteFeedbackRecord, as all the updates of any "updateable" attribute of a record will go through update instead (#3663).
• Remove unused owner attribute for client Dataset data model (#3665)

As always, thanks to our amazing contributors

• @peppinob-ol made their first contribution in https://github.com/argilla-io/argilla/pull/3472
• @eshwarhs made their first contribution in https://github.com/argilla-io/argilla/pull/3605

Full Changelog: https://github.com/argilla-io/argilla/compare/v1.14.1...v1.15.0