Highlights

Note: BigDL v2.1.0 has been updated to include functional and security updates. Users should update to the latest version.

• Orca
  • Improve user experience and API consistency for Orca Estimators.
  • Support directly save and load TensorFlow model format in Orca TensorFlow2 Estimator.
  • Provide more examples (e.g. PyTorch brain image segmentation, XShards tutorials for distributed Python data processing), etc.
  • Support customized metrics in Orca PyTorch Estimator.
• Nano
  • New inference optimization pipelines, with more optimization methods and a new InferenceOptimizer
  • More training optimization methods (bf16, channel last)
  • Add TorchNano support for PyTorch model customized training loop
  • Auto-scale learning rate for multi-instance training
  • Built-in AutoML support through hyperparameter optimization
  • Support a wide range versions of pytorch (1.9-1.12) and tensorflow (2.7-2.9)
• DLlib
  • Add LightGBM support
  • Improve Keras-style model summary API
  • Add Python support for loading HDFS files
• Chronos
  • Add new Autoformer (https://arxiv.org/abs/2106.13008) Forecaster and pipeline that are optimized on CPU
  • Tensorflow 2 support for LSTM, Seq2Seq, TCN and MTNet Forecasters
  • Add light-weight (does not rely on Spark/Ray Tune) auto tunning
  • Better support on distributed workflow (spark df and distributed pandas processing)
  • Add more installation options is now supported to make the installation lighter
• Friesian:
  • Integration of DeepRec (https://github.com/alibaba/DeepRec) with Friesian.
  • Add more reference examples, e.g. multi-task recommendation, TFRS (https://www.tensorflow.org/recommenders) list-wise ranking, LightGBM training, etc.
  • Add a reference example for offline distributed similarity search (using FAISS)
  • More operations in FeatureTable (e.g. string embeddings with BERT, etc.).
• PPML
  • Upgrade BigDL PPML on Gramine.
  • Improve the attestation and key managing process
  • More Big Data frameworks on BigDL PPML (including spark, flink, hive, hdfs, etc.)
  • Add PPMLContext API for encryption IO and KMS, supports different file formats, encryption algorithms and KMS services
  • Support PSI, Pytorch NN, Keras NN, FGBoost (federated XGBoost) in VFL scenario, linear regression & logistic regression for VFL