v2.10.0-rc0
版本发布时间: 2022-08-04 06:21:17
tensorflow/tensorflow最新发布版本:v2.17.0(2024-07-12 00:28:57)
Release 2.10.0
Breaking Changes
- Causal attention in
keras.layers.Attention
andkeras.layers.AdditiveAttention
is now specified in thecall()
method via theuse_causal_mask
argument (rather than in the constructor), for consistency with other layers. - Some files in
tensorflow/python/training
have been moved totensorflow/python/tracking
andtensorflow/python/checkpoint
. Please update your imports accordingly, the old files will be removed in Release 2.11. -
tf.keras.optimizers.experimental.Optimizer
will graduate in Release 2.11, which meanstf.keras.optimizers.Optimizer
will be an alias oftf.keras.optimizers.experimental.Optimizer
. The currenttf.keras.optimizers.Optimizer
will continue to be supported astf.keras.optimizers.legacy.Optimizer
, e.g.,tf.keras.optimizers.legacy.Adam
. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer totf.keras.optimizers.legacy.Optimizer
. - RNG behavior change for
tf.keras.initializers
. Keras initializers will now use stateless random ops to generate random numbers.- Both seeded and unseeded initializers will always generate the same
values every time they are called (for a given variable shape).
For unseeded initializers (
seed=None
), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds). - An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.
- Both seeded and unseeded initializers will always generate the same
values every time they are called (for a given variable shape).
For unseeded initializers (
Major Features and Improvements
-
tf.lite
:- New operations supported:
- tflite SelectV2 now supports 5D.
- tf.einsum is supported with multiple unknown shapes.
- tf.unsortedsegmentprod op is supported.
- tf.unsortedsegmentmax op is supported.
- tf.unsortedsegmentsum op is supported.
- Updates to existing operations:
- tfl.scatter_nd now supports I1 for update arg.
- Upgrade Flatbuffers v2.0.5 from v1.12.0
- New operations supported:
-
tf.keras
:-
EinsumDense
layer moved from experimental to core. Its import path moved fromtf.keras.layers.experimental.EinsumDense
totf.keras.layers.EinsumDense
. - Added
tf.keras.utils.audio_dataset_from_directory
utility to easily generate audio classification datasets from directories of.wav
files. - Added
subset="both"
support intf.keras.utils.image_dataset_from_directory
,tf.keras.utils.text_dataset_from_directory
, andaudio_dataset_from_directory
, to be used with thevalidation_split
argument, for returning both dataset splits at once, as a tuple. - Added
tf.keras.utils.split_dataset
utility to split aDataset
object or a list/tuple of arrays into twoDataset
objects (e.g. train/test). - Added step granularity to
BackupAndRestore
callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing. - Added
tf.keras.dtensor.experimental.optimizers.AdamW
. This optimizer is similar as the existingkeras.optimizers.experimental.AdamW
, and works in the DTensor training use case. - Improved masking support for tf.keras.layers.MultiHeadAttention.
- Implicit masks for
query
,key
andvalue
inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with anyattention_mask
passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding withmask_zero=True
to automatically infer a correct padding mask. - Added a
use_causal_mask
call time arugment to the layer. Passinguse_causal_mask=True
will compute a causal attention mask, and optionally combine it with anyattention_mask
passed in directly when calling the layer.
- Implicit masks for
- Added
ignore_class
argument in the lossSparseCategoricalCrossentropy
and metricsIoU
andMeanIoU
, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class). - Added
tf.keras.models.experimental.SharpnessAwareMinimization
. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.
-
-
tf.data
:- Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See https://www.tensorflow.org/api_docs/python/tf/data/experimental/service#sharing_tfdata_service_with_concurrent_trainers for more details.
- Added
dataset_id
totf.data.experimental.service.register_dataset
. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should callregister_dataset
with the samedataset_id
. - Added a new field,
inject_prefetch
, totf.data.experimental.OptimizationOptions
. If it is set toTrue
, tf.data will now automatically add aprefetch
transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, setinject_prefetch=True
intf.data.experimental.OptimizationOptions
. - Added a new value to
tf.data.Options.autotune.autotune_algorithm
: STAGE_BASED. If the autotune algorithm is set to STAGE_BASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage. - Added
tf.data.experimental.from_list
, a new API for creatingDataset
s from lists of elements.
-
tf.distribute
:- Added
tf.distribute.experimental.PreemptionCheckpointHandler
to handle worker preemption/maintenance and cluster-wise consistent error reporting fortf.distribute.MultiWorkerMirroredStrategy
. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.
- Added
-
tf.math
:- Added
tf.math.approx_max_k
andtf.math.approx_min_k
which are the optimized alternatives totf.math.top_k
on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.
- Added
-
tf.train
:- Added
tf.train.TrackableView
which allows users to inspect the TensorFlow Trackable object (e.g.tf.Module
, Keras Layers and models).
- Added
-
tf.vectorized_map
:- Added an optional parameter:
warn
. This parameter controls whether or not warnings will be printed when operations in the providedfn
fall back to a while loop.
- Added an optional parameter:
-
XLA:
- MWMS is now compilable with XLA.
Bug Fixes and Other Changes
-
New argument
experimental_device_ordinal
inLogicalDeviceConfiguration
to control the order of logical devices. (GPU only) -
tf.keras
:- Changed the TensorBoard tag names produced by the
tf.keras.callbacks.TensorBoard
callback, so that summaries logged automatically for model weights now include either a/histogram
or/image
suffix in their tag names, in order to prevent tag name collisions across summary types.
- Changed the TensorBoard tag names produced by the
-
When running on GPU (with cuDNN version 7.6.3 or later),
tf.nn.depthwise_conv2d
backprop tofilter
(and therefore alsotf.keras.layers.DepthwiseConv2D
) now operate deterministically (andtf.errors.UnimplementedError
is no longer thrown) when op-determinism has been enabled viatf.config.experimental.enable_op_determinism
. This closes issue 47174. -
tf.random
- Added
tf.random.experimental.stateless_shuffle
, a stateless version oftf.random.shuffle
.
- Added
Deprecations
- The C++
tensorflow::Code
andtensorflow::Status
will become aliases of respectivelyabsl::StatusCode
andabsl::Status
in some future release.- Use
tensorflow::OkStatus()
instead oftensorflow::Status::OK()
. - Stop constructing
Status
objects fromtensorflow::error::Code
. - One MUST NOT access
tensorflow::errors::Code
fields. Accessingtensorflow::error::Code
fields is fine.- Use the constructors such as
tensorflow::errors:InvalidArgument
to create status using an error code without accessing it. - Use the free functions such as
tensorflow::errors::IsInvalidArgument
if needed. - In the last resort, use e.g.
static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT)
orstatic_cast<int>(code)
for comparisons.
- Use the constructors such as
- Use
-
tensorflow::StatusOr
will also become in the future alias toabsl::StatusOr
, so useStatusOr::value
instead ofStatusOr::ConsumeValueOrDie
.
Thanks to our Contributors
This release contains contributions from many people at Google, as well as:
Abolfazl Shahbazi, Adam Lanicek, Amin Benarieb, andreii, Andrew Fitzgibbon, Andrew Goodbody, angerson, Ashiq Imran, Aurélien Geron, Banikumar Maiti (Intel Aipg), Ben Barsdell, Ben Mares, bhack, Bhavani Subramanian, Bill Schnurr, Byungsoo Oh, Chandra Sr Potula, Chengji Yao, Chris Carpita, Christopher Bate, chunduriv, Cliff Woolley, Cliffs Dover, Cloud Han, Code-Review-Doctor, DEKHTIARJonathan, Deven Desai, Djacon, Duncan Riach, fedotoff, fo40225, Frederic Bastien, gadagashwini, Gauri1 Deshpande, guozhong.zhuang, Hui Peng, James Gerity, Jason Furmanek, Jonathan Dekhtiar, Jueon Park, Kaixi Hou, Kanvi Khanna, Keith Smiley, Koan-Sin Tan, Kulin Seth, kushanam, Learning-To-Play, Li-Wen Chang, lipracer, liuyuanqiang, Louis Sugy, Lucas David, Lukas Geiger, Mahmoud Abuzaina, Marius Brehler, Maxiwell S. Garcia, mdfaijul, Meenakshi Venkataraman, Michal Szutenberg, Michele Di Giorgio, Mickaël Salamin, Nathan John Sircombe, Nathan Luehr, Neil Girdhar, Nils Reichardt, Nishidha Panpaliya, Nobuo Tsukamoto, Om Thakkar, Patrice Vignola, Philipp Hack, Pooya Jannaty, Prianka Liz Kariat, pshiko, Rajeshwar Reddy T, rdl4199, Rohit Santhanam, Rsanthanam-Amd, Sachin Muradi, Saoirse Stewart, Serge Panev, Shu Wang, Srinivasan Narayanamoorthy, Stella Stamenova, Stephan Hartmann, Sunita Nadampalli, synandi, Tamas Bela Feher, Tao Xu, Thibaut Goetghebuer-Planchon, Trevor Morris, Xiaoming (Jason) Cui, Yimei Sun, Yong Tang, Yuanqiang Liu, Yulv-Git, Zhoulong Jiang, ZihengJiang