We are excited to announce the release of Delta Lake 1.0.1 on Apache Spark™ 3.1, which back-ports bug fixes from Delta Lake 1.1.0 to Delta Lake 1.0.0.

The details of the fixed bugs are as follows:

• Fix for rare data corruption issue on GCS - Experimental GCS support released in Delta Lake 1.0 has a rare bug that can lead to Delta tables being unreadable due to partially written transaction log files. This issue has now been fixed (1, 2).

• Fix for the incorrect return object in Python DeltaTable.convertToDelta() - This existing API now returns the correct Python object of type delta.tables.DeltaTable instead of an incorrectly-typed, and therefore unusable object.

• Fix for incorrect handling of special characters (e.g. spaces) in paths by MERGE/UPDATE/DELETE operations

• Fix for Hadoop configurations not being used to write checkpoints

• Improvements to DeltaTableBuilder API introduced in Delta 1.0.0

  • Fix for bug that prevented passing of multiple partition columns in Python DeltaTableBuilder.partitionBy.
  • Throw error when column data type is not specified.

Credits Jarred Parrett, Shixiong Zhu, Tathagata Das, Tom Lynch, Yijia Cui, Yaohua Zhao, gurunath