TorchGeo 0.4.0 Release Notes

This is our biggest release yet, with improved support for pre-trained models, faster datamodules and transforms, and more powerful trainers. See the following sections for specific changes to each module:

• Backwards-incompatible changes
• Dependencies
• Datamodules
• Datasets
• Models
• Samplers
• Trainers
• Transforms
• Documentation

As always, thanks to our many contributors!

Backwards-incompatible changes

• Datasets: So2Sat bands were renamed (#735)
• Datasets: TropicalCycloneWindEstimation was renamed to TropicalCyclone (#815, #846)
• Datasets: VisionDataset and VisionClassificationDataset (deprecated in 0.3) have been removed (#627)
• Datamodules: many arguments have been renamed or reordered (#666, #730, #992)
• Datamodules: CycloneDataModule was renamed to TropicalCycloneDataModule (#815, #846)
• Models: resnet50 has a new multi-weight API (#917)
• Trainers: many arguments have been renamed (#916, #917, #918, #919, #920)
• Transforms: now take a single image as input instead of a sample dict (#999)

Dependencies

• Open3D replaced by PyVista (#663)
• Remove packaging dependency (#1019)
• Support einops 0.6 (#896)
• Support flake8 6 (#910)
• Support mypy 0.991 (#900)
• Support pytest-cov 4 (#801)
• Support pyupgrade 3 (#817)
• Support setuptools 66 (#1017)
• Support shapely 2 (#949)
• Support sphinx 6 (#990)
• Support timm 0.6 (#1002)
• Support torchmetrics 0.11 (#925)
• Support torchvision 0.14 (#875)

Datamodules

Our existing datamodules worked well, but suffered from several performance issues. For the average dataset with 3 splits (train/val/test), we were instantiating the dataset 10 times! All data augmentation was done on the CPU, one sample at a time. A multiprocessing bug prevented parallel data loading on macOS and Windows. And a serious bug was discovered in some of our datamodules that allowed training images to leak into the test set (only affected datamodules using torchgeo.datamodules.utils.dataset_split). All of these bugs have been fixed, and performance has been drastically improved. Datasets are only instantiated 3 times (once for each split). All data augmentation happens on the GPU, an entire batch at a time. And multiprocessing is now supported on all platforms. By refactoring our datamodules and adding new base classes, we were able to remove 1.6K lines of duplicated code in the process!

New datamodules:

• GID-15 (#928)
• SpaceNet1 (#965)

Changes to existing datamodules:

• Only instantiate dataset in prepare_data if download is requested (#967, #974)
• Only instantiate datasets needed for a given stage (#992)
• Use Kornia for all data augmentation (#992)
• Faster data augmentation (CPU → GPU, sample → batch) (#992)
• Fix macOS/Windows multiprocessing bug (#886, #992)
• Fix bug with train images leaking into test set (#992)
• Add plot method to all datamodules (#814, #992)
• torchgeo.datamodules.utils.dataset_split is deprecated, use torch.utils.data.random_split instead (#992)
• Pass kwargs directly to datasets (#666, #730)
• Add random cropping to several datamodules (#851, #853, #855, #876, #929)
• Inria Aerial Image Labeling: fix predict dimensions (#975)
• LandCover.ai: fix mIoU calculation and plotting (#959)
• Tropical Cyclone: CycloneDataModule was renamed to TropicalCycloneDataModule (#815, #846)

New base classes:

• Add GeoDataModule and NonGeoDataModule base classes (#992)

Datasets

This release adds a new Sentinel-1 dataset. Here is a scene taken over the Big Island of Hawai'i:

Additionally, all image datasets now have a plot method.

New datasets:

• Cloud Cover Detection (#510)
• Sentinel-1 (#821)
• SpaceNet 6 (#878)

Changes to existing datasets:

• Add default root argument to all datasets (#802)
• Consistent capitalization of band names (#778)
• Many datasets now return float images and int labels (#992)
• Chesapeake CVPR: add plot method (#820)
• ETCI 2021: fix data loading (#861)
• NASA Marine Debris: fix plot warning when model outputs no prediction boxes (#988)
• OSCD: images are now stacked channel-wise (#992)
• SEN12MS: mask is only single channel (#992)
• Sentinel-2: use 10,000 as scale factor (#1027)
• So2Sat: rename bands (#735)
• Tropical Cyclone: renamed from TropicalCycloneWindEstimation to TropicalCyclone (#815, #846)
• Tropical Cyclone: images are RGB, not grayscale (#992)
• VHR-10: add plot method (#847)
• xView2: remove labels folder (#787)

Changes to existing base classes:

• RasterDataset supports band indexing now (#687)
• UnionDataset actually works now (#769, #786)
• UnionDataset and IntersectionDataset support transforms (#867, #870)
• VectorDataset supports multi-label datasets (#862)

Models

Due to the nature of satellite imagery (different number of spectral bands for every satellite), it is impossible to have a single set of pre-trained weights for each model. TorchGeo has always had multi-weight support:

model = resnet50(sensor="sentinel2", bands="all", pretrained=True)

However, this is difficult to extend if you want more fine-grained control over model weights. More recently, torchvision introduced a new multi-weight support API:

• Introducing TorchVision's New Multi-Weight Support API
• Easily List and Initialize Models With New APIs in TorchVision

With the 0.4.0 release, TorchGeo has now adopted the same API:

model = resnet50(weights=ResNet50_Weights.SENTINEL2_ALL_MOCO)

We also support PyTorch Hub now:

>>> import torch
>>> from torchgeo.models import ResNet18_Weights
>>> torch.hub.list("microsoft/torchgeo", trust_repo=True)
Downloading: "https://github.com/microsoft/torchgeo/zipball/models/weights" to ~/.cache/torch/hub/models_weights.zip
['resnet18', 'resnet50', 'vit_small_patch16_224']
>>> model = torch.hub.load("microsoft/torchgeo", "resnet18")
Using cache found in ~/.cache/torch/hub/microsoft_torchgeo_models_weights
>>> model = torch.hub.load("microsoft/torchgeo", "resnet18", weights=ResNet18_Weights.SENTINEL2_RGB_MOCO)
Using cache found in ~/.cache/torch/hub/microsoft_torchgeo_models_weights

In our previous release, we had 1 model pre-trained on 1 satellite with 1 training procedure. We now have 3 models (ResNet-18, ResNet-50, ViT) trained on both Sentinel-1 and Sentinel-2 for all bands and RGB-only bands with 3 SSL techniques (MoCo, DINO, SeCo), and plans to expand this in the future. Shoutout to Zhu Lab and ServiceNow for publishing these weights!

New models:

• Add ResNet-18 and ViT models (#917)

Changes to existing models:

• Adopt torchvision's multi-weight support API (#762, #804, #917)
• Add support for torch.hub (#917)

New utility functions:

• Functions to list, query, and initialize models and weights (#917)

Samplers

Changes to existing samplers:

• All random samplers now have a default value for length (#755)

New utility functions:

• get_random_bounding_box and tile_to_chips are now public functions (#755)

Trainers

This release introduces a new trainer for object detection, one of our most highly requested features. All trainers now support prediction. Our old trainers only supported ResNet backbones. Our new trainers now support the 600+ backbones provided by the timm library. And all of the new pre-trained models mentioned above are now supported by our trainers as well.

New trainers:

• Object Detection: add trainer, add Faster R-CNN (#442, #758)
• Object Detection: add RetinaNet and FCOS (#984)

Changes to existing trainers:

• Add support for all timm backbones (#854, #918)
• Add support for more pretrained models (#917)
• Change model argument names (#916, #918, #919, #920)
• Support prediction (#790, #792, #813, #818, #819, #939)
• Fix plotting file handle leak (#825, #826)
• Multi-label Classification: replace softmax with sigmoid (#791)

Transforms

Whenever possible, we try to avoid reinventing the wheel. For data augmentation transforms that aren't specific to geospatial data or satellite imagery, we use existing implementations in popular libraries like:

• torchvision (PIL and PyTorch backends)
• albumentations (OpenCV backend)
• kornia (PyTorch backend)

Until now, we've been fairly agnostic towards data augmentation libraries. However, neither PIL nor OpenCV support multispectral imagery. Because of this, we've decided to use Kornia for all transforms.

Changes to existing transforms:

• All transforms are now compatible with kornia.augmentation.AugmentationSequential (#999)
• All transforms now take a single image as input instead of a sample dict (#999)
• torchgeo.transforms.AugmentationSequential is deprecated, use kornia.augmentation.AugmentationSequential instead (#992)

Documentation

• Add new tutorial for working with pretrained model weights (#693, #799, #917)
• Remove execution count from tutorials (#783)
• Remove __module__ hacks, fixing most documentation issues (#976)
• Use kornia for all transforms in tutorials (#999)
• Improve trainer API docs (#852)
• Add num classes to ReforeTree dataset (#907)
• Convert tensor to array in tutorials (#841, #845)
• Fix typo in USAVars documentation (#1038)
• Fix typos in TropicalCyclone and GID-15 documentation (#1011)
• Fix URL formatting in LoveDA documentation (#977)
• Fix Aster GDEM dataset name (#884)
• Fix dead link in Vaihingen2D documentation (#850)
• Fix link to iNaturalist in datasets table (#775)
• Fix link in GBIF dataset documentation (#774)

Contributors

This release is thanks to the following contributors:

@adamjstewart @ashnair1 @bugraaldal @calebrob6 @daiki-kimura @eltociear @fnands @isaaccorley @KennSmithDS @mgnolde @nilsleh @Niro4 @osgeokr @pmandiola @RitwikGupta