We are excited to announce the release of PyG 2.1.0 🎉🎉🎉

• Highlights
• Breaking Changes
• Deprecations
• Features
• Bugfixes
• Full Changelog

PyG 2.1.0 is the culmination of work from over 60 contributors who have worked on features and bug-fixes for a total of over 320 commits since torch-geometric==2.0.4.

Highlights

Principled Aggregations

See here for the accompanying tutorial.

Aggregation functions play an important role in the message passing framework and the readout functions of Graph Neural Networks. Specifically, many works in the literature (Hamilton et al. (2017), Xu et al. (2018), Corso et al. (2020), Li et al. (2020), Tailor et al. (2021), Bartunov et al. (2022)) demonstrate that the choice of aggregation functions contributes significantly to the representational power and performance of the model.

To facilitate further experimentation and unify the concepts of aggregation within GNNs across both MessagePassing and global readouts, we have made the concept of Aggregation a first-class principle in PyG (#4379, #4522, #4687, #4721, #4731, #4762, #4749, #4779, #4863, #4864, #4865, #4866, #4872, #4927, #4934, #4935, #4957, #4973, #4973, #4986, #4995, #5000, #5021, #5034, #5036, #5039, #4522, #5033, #5085, #5097, #5099, #5104, #5113, #5130, #5098, #5191). As of now, PyG provides support for various aggregations — from simple ones (e.g., mean, max, sum), to advanced ones (e.g., median, var, std), learnable ones (e.g., SoftmaxAggregation, PowerMeanAggregation), and exotic ones (e.g., LSTMAggregation, SortAggregation, EquilibriumAggregation). Furthermore, multiple aggregations can be combined and stacked together:

from torch_geometric.nn import MessagePassing, SoftmaxAggregation

class MyConv(MessagePassing):
    def __init__(self, ...):
        # Combines a set of aggregations and concatenates their results.
        # The interface also supports automatic resolution.
        super().__init__(aggr=['mean', 'std', SoftmaxAggregation(learn=True)])

Link-level Neighbor Loader

We added a new LinkNeighborLoader class for training scalable GNNs that perform edge-level predictions on giant graphs (#4396, #4439, #4441, #4446, #4508, #4509, #4868). LinkNeighborLoader comes with automatic support for both homogeneous and heterogenous data, and supports link prediction via automatic negative sampling as well as edge-level classification and regression models:

from torch_geometric.loader import LinkNeighborLoader

loader = LinkNeighborLoader(
    data,
    num_neighbors=[30] * 2,  # Sample 30 neighbors for each node for 2 iterations
    batch_size=128,  # Use a batch size of 128 for sampling training links
    edge_label_index=data.edge_index,  # Use the entire graph for supervision
    negative_sampling_ratio=1.0,  # Sample negative edges
)

sampled_data = next(iter(loader))
print(sampled_data)
>>> Data(x=[1368, 1433], edge_index=[2, 3103], edge_label_index=[2, 256], edge_label=[256])

Neighborhood Sampling based on Temporal Constraints

Both NeighborLoader and LinkNeighborLoader now support temporal sampling via the time_attr argument (#4025, #4877, #4908, #5137, #5173). If set, temporal sampling will be used such that neighbors are guaranteed to fulfill temporal constraints, i.e. neighbors have an earlier timestamp than the center node:

from torch_geometric.loader import NeighborLoader

data['paper'].time = torch.arange(data['paper'].num_nodes)

loader = NeighborLoader(
    data,
    input_nodes='paper',
    time_attr='time',  # Only sample papers that appeared before the seed paper
    num_neighbors=[30] * 2,
    batch_size=128,
)

Note that this feature requires torch-sparse>=0.6.14.

Functional DataPipes

See here for the accompanying example.

PyG now fully supports data loading using the newly introduced concept of DataPipes in PyTorch for easily constructing flexible and performant data pipelines (#4302, #4345, #4349). PyG provides DataPipe support for batching multiple PyG data objects together and for applying any PyG transform:

datapipe = FileOpener(['SMILES_HIV.csv'])
datapipe = datapipe.parse_csv_as_dict()
datapipe = datapipe.parse_smiles(target_key='HIV_active')
datapipe = datapipe.in_memory_cache()  # Cache graph instances in-memory.
datapipe = datapipe.shuffle()
datapipe = datapipe.batch_graphs(batch_size=32)

datapipe = FileLister([root_dir], masks='*.off', recursive=True)
datapipe = datapipe.read_mesh()
datapipe = datapipe.in_memory_cache()  # Cache graph instances in-memory.
datapipe = datapipe.sample_points(1024)  # Use PyG transforms from here.
datapipe = datapipe.knn_graph(k=8)
datapipe = datapipe.shuffle()
datapipe = datapipe.batch_graphs(batch_size=32)

Breaking Changes

• The torch_geometric.utils.metric package has been removed. We now recommend to use the torchmetrics package instead.
• len(batch) of the data.Batch class will now return the number of graphs inside the batch, not the number of attributes (#4931)

Deprecations

• The usage of the torch_geometric.nn.glob package is now deprecated in favor of torch_geometric.nn.aggr
• The usage of RandomTranslate is now deprecated in favor of RandomJitter (#4828)

Features

Layers, Models and Examples

• Added the GroupAddRev module with support for reducing training GPU memory (#4671, #4701, #4715, #4730) [Example]
• Added the MaskLabel module for performing masked label propagation (#4197) **[Example]
• Added the DimeNetPlusPlus module (#4432, #4699, #4700, #4800)
• Added the MeanSubtractionNorm module (#5068)
• Added the DynamicBatchSampler for filling a mini-batch with a variable number of samples up to a maximum size (#4972)
• Added PyTorch Lightning support in GraphGym (#4511, #4516 #4531, #4689, #4843)
• Added an example of using PyG with PyTorch Ignite (#4487)
• Added an example for unsupervised heterogeneous graph learning (#3189)
• Added an example for unsupervised GraphSAGE on the PPI dataset (#4416)
• Added the EdgeCNN model (#4991)
• Added an example to load a trained PyG model in C++ (#4307)

Transformations

• Added the AddPositionalEncoding transforms with two implementations: AddLaplacianEigenvectorPE and AddRandomWalkPE (#4521)
• Added the Rooted transform with two implementations: RootedEgoNets and RootedRWSubgraph (#3926)
• Added support for computing weighted metapaths in AddMetapaths (#5049)

Datasets

• Added the Genius and Wiki datasets to the LINKXDataset (#4570, #4600)
• Added the AQSOL dataset (#4626)
• Added Geom-GCN splits to the Planetoid datasets (#4442)

General Improvements

• Added support for GATv2Conv in the GAT model (#4357)
• Added support for projecting features before propagation in SAGEConv (#4437)
• Added a MessagePassing.explain_message() method to customize making explanations on messages (#4278, #4448))
• Added the MLP.plain_last = False option (4652)
• Added support graph-level attributes in networkx conversion (#4343)
• Added Data.validate() and HeteroData.validate() functionality to validate the correctness of the data (#4885)
• Added TorchScript support to JumpingKnowledge module (#4805)
• Added predict() support to the LightningNodeData module (#4884)
• Added support for renaming node types via HeteroData.rename() (#4329)
• Added HeteroData.num_features functionality (#4504)
• Added HeteroData.subgraph, HeteroData.node_type_subgraph and HeteroData.edge_type_subgraph functionality (#4243)
• Added HeteroData support to the RemoveIsolatedNodes transform (#4479)
• Added support for graph-level outputs in to_hetero (#4582)
• Added HeteroData.is_undirected() support (#4604)
• Added HeteroData.node_items() and HeteroData.edge_items() functionality (#4644)
• Added HeteroData.subgraph() support (#4635)
• Added node-wise normalization mode in LayerNorm (#4944)
• Added utils.unbatch and utils.unbatch_edge_index functionality for splitting an edge_index tensor according to a batch vector (#4628, #4903)
• Added scalable inference mode in BasicGNN with layer-wise neighbor loading (#4977)
• Added fine grained options for setting bias and dropout per layer in the MLP model (#4981)
• Added support for BasicGNN models within to_hetero (#5091)
• Let ImbalancedSampler accept torch.Tensor as input (#5138)
• Allow edge_type == rev_edge_type argument in RandomLinkSplit (#4757)

Bugfixes

• Fixed a bug in RGATConv that produced device mismatches for "f-scaled" mode (#5187]
• Fixed a bug in GINEConv bug for non-Sequential neural network layers (#5154]
• Fixed a bug in HGTLoader which produced outputs with missing edge types, will require torch-sparse>=0.6.15 (#5067)
• Fixed a bug in load_state_dict for Linear with strict=False mode (5094)
• Fixed data.num_node_features computation for sparse matrices (5089)
• Fixed a bug in which GraphGym did not create new non-linearity functions but re-used existing ones (4978)
• Fixed BasicGNN for num_layers=1, which now respects a desired number of out_channels (#4943)
• Fixed a bug in data.subgraph for 0-dim tensors (#4932)
• Fixed a bug in InMemoryDataset inferring wrong length for lists of tensors (#4837)
• Fixed a bug in TUDataset where pre_filter was not applied whenever pre_transform was present (#4842)
• Fixed access of edge types in HeteroData via two node types when there exists multiple relations between them (#4782)
• Fixed a bug in HANConv in which destination node features rather than source node features were propagated (#4753)
• Fixed a ranking protocol bug in the RGCN link prediction example (#4688)
• Fixed the interplay between TUDataset and pre_transform transformations that modify node features (#4669)
• The bias argument in TAGConv is now correctly applied (#4597)
• Fixed filtering of attributes in samplers in case __cat_dim__ != 0 (#4629)
• Fixed SparseTensor support in NeighborLoader (#4320)
• Fixed average degree handling in PNAConv (#4312)
• Fixed a bug in from_networkx in case some attributes are PyTorch tensors (#4486)
• Fixed a missing clamp in the DimeNet model (#4506, #4562)
• Fixed the download link in DBP15K (#4428)
• Fixed an autograd bug in DimeNet when resetting parameters (#4424)
• Fixed bipartite message passing in case flow="target_to_source" (#4418)
• Fixed a bug in which num_nodes was not properly updated in the FixedPoints transform (#4394)
• Fixed a bug in which GATConv was not jittable (#4347)
• Fixed a bug in which nn.models.GAT did not produce out_channels many output channels (#4299)
• Fixed a bug in mini-batching with empty lists as attributes (#4293)
• Fixed a bug in which GCNConv could not be combined with to_hetero on heterogeneous graphs with one node type (#4279)

Full Changelog

Full commit list: https://github.com/pyg-team/pytorch_geometric/compare/2.0.4...2.1.0