Highlights

• Distributed TensorFlow (both training and inference) on Spark, which supports:

  • Data wrangling and analysis using PySpark
  • Deep learning model development using TensorFlow or Keras
  • Distributed training/inference on Spark and BigDL
  • All within a single unified pipeline and in a user-transparent fashion!

• More support for text processing and models, including:

  • Common feature engineering operations for text data (such as tokenization, normalization, padding, etc.)
  • Word Embedding layers that directly load pretrained GloVe model
  • Text matching models (such as KNRM)

• Various improvements and new features, such as:

  • Support for trainable variable (Parameter)
  • Support for Keras objectives (with zero-based label)
  • Improvements to model serving APIs
  • Improvements to example and use case documents