Highlights

1. Standardize the ModelOutput API:

• Remove ambiguous flags: ignore_masking and hf_format: https://github.com/NVIDIA-Merlin/Transformers4Rec/pull/543
• Introduce the testing flag to differentiate between evaluation (=True) and inference (=False) modes: https://github.com/NVIDIA-Merlin/Transformers4Rec/pull/543
• All prediction tasks return the same output
  https://github.com/NVIDIA-Merlin/Transformers4Rec/pull/546
  • During training and evaluation: the output is a dictionary with three elements: {"loss":torch.tensor, "labels": torch.tensor, "predictions": torch.tensor}
  • During inference: The output is the tensor of predictions.

2. Extend the Trainer class to support all prediction tasks:

https://github.com/NVIDIA-Merlin/Transformers4Rec/pull/564

• The trainer class is now accepting a T4Rec model defined with binary or regression tasks.
• Remove the HFWrapper class as the Trainer is now supporting the base T4Rec Model class.
• Set the default of the trainer's argument predict_top_k to 0 instead of 10.
  • Note that getting the top-k predictions is specific to NextItemPredictionTask and the user should explicitly set the parameter in the T4RecTrainingArguments object. If not specified, the method Trainer.predict() returns unsorted predictions for the whole item catalog.
• Support multi-task learning in the Trainer class: it accepts any T4Rec model defined with multiple tasks and/or multiple heads.

3. Fix the inference performance of the Transformer-based model trained with masked language modeling (MLM):

https://github.com/NVIDIA-Merlin/Transformers4Rec/pull/551

• At inference, the input sequence is extended by a [MASK] embedding after the last non-padded position to take into account the target position. The hidden representation of the [MASK] position is used to get the next-item prediction scores.
• With this fix, the user doesn't need to add a dummy position to the input test data when calling Trainer.predict() or model(test_batch, training=False, testing=False)

4. Update Transformers4Rec to use the new merlin-dataloader package: https://github.com/NVIDIA-Merlin/Transformers4Rec/pull/547

• The NVTabularDataLoader is renamed to MerlinDataLoader to use the loader from merlin-dataloader package.
• User can specify the argument data_loader_engine=‘merlin’ in the T4RecTrainingArguments object to use the merlin dataloader. It supports GPU and CPU environments. The alias nvtabular is also kept to ensure backward compatibility.

What’s Changed

⚠ Breaking Changes

• Extend trainer class to support all T4Rec prediction tasks @sararb (#564)
• Standardize prediction tasks' outputs @nzarif (#546)
• Uses merlin-dataloader package @edknv (#547)
• Refactoring part1- flags modification @nzarif (#543)

🐜 Bug Fixes

• Fix error raised by latest Torchmetrics (0.11.0) @sararb (#576)
• Fix the test data path in Trainer.predict() @sararb (#571)
• Fix discrepancy between evaluation and inference modes @sararb (#551)

🚀 Features

• Support to pre-trained embeddings initializer (trainable or not) @gabrielspmoreira (#572)
• Extend trainer class to support all T4Rec prediction tasks @sararb (#564)
• Standardize prediction tasks' outputs @nzarif (#546)
• Add music-streaming synthetic data to test the support of all predictions tasks with the Trainer class @sararb (#540)
• Refactoring part1- flags modification @nzarif (#543)

📄 Documentation

• Address review feedback @mikemckiernan (#562)
• Serving tfrec with pyt backend example @rnyak (#552)
• docs: Add basic SEO configuration @mikemckiernan (#518)
• docs: Add semver to calver banner @mikemckiernan (#520)
• Minor updates to notebook texts @bbozkaya (#548)

🔧 Maintenance

• Update mypy version from 0.971 to 0.991 @oliverholworthy (#574)
• Uses merlin-dataloader package @edknv (#547)
• fix drafter and update cpu ci to run on targeted branch @jperez999 (#549)
• Add lint workflow to run pre-commit on all files @oliverholworthy (#545)
• Specify packages to look for in setup.py to avoid publishing tests @oliverholworthy (#529)
• Cleanup tensorflow dependencies @oliverholworthy (#530)
• Add docs requirements to extras list in setup.py (#533)
• Remove stale documentation reviews (#531)
• Update branch name extraction for tag builds (#608)
• run github action tests and lint via tox, with upstream deps installed (#527)