What's Changed

Some documentation is missing for some of these new features, but it will be added soon.

Multi-caption support

• Supported caption_strategy: parquet, textfile
  • textfiles with multiple captions split by newline
  • parquet backends can now have multiple caption columns, or, a column that contains a list of captions.

Default noise scheduler (inference)

The default value if none is supplied is now None, which uses the upstream model configuration from the repository.

Prefetch

Some minor bugfixes have gone into this, but it remains an experimental feature with uncertain gains in performance.

Minor features

• --torch_num_threads to stop torch from spawning too many threads on big systems
• CV2 is now used for image loading, which is also going to invoke libpng for png images, which is very chatty and spews lots of warnings, which we cannot control.

Other bugfixes

• Reworked the area resize code and cropping logic to strip out redundancy and improve clarity.
  • Ensures we do not see any squished images for a more broad range of aspect ratios, across every cropping and resizing configuration.
  • Remove your VAE and aspect bucket caches to take advantage of this.

Changes

• remove unneeded imports by @bghira in https://github.com/bghira/SimpleTuner/pull/544
• prefetch: minor bugfixes, epoch tracking
• add --torch_num_threads for very large systems
• catch delete failure when delete_problematic_images is set by @bghira in https://github.com/bghira/SimpleTuner/pull/547
• Fix bucket search for unseen images not containing the absolute path to the image by @bghira in https://github.com/bghira/SimpleTuner/pull/549
• multi-caption support for textfile and parquet backend by @bghira in https://github.com/bghira/SimpleTuner/pull/527
• Load images preferentially with CV2, fall back to PIL only if that fails by @AmericanPresidentJimmyCarter in https://github.com/bghira/SimpleTuner/pull/551
• bugfix: bucket search for unseen images should prepend the instance data root so that the images can actually be loaded from disk
• batch prefetch should be correctly destroyed/shutdown upon error or ctrl+c
• VAE embed inconsistency fixed by cloning latent before write by @bghira in https://github.com/bghira/SimpleTuner/pull/552
• Refactor save_hooks by @sayakpaul in https://github.com/bghira/SimpleTuner/pull/554
• cv2: error checking for image load when we hit grayscale images
• arguments: set --inference_noise_scheduler to None by default so that PixArt scheduler is uninterrupted by @bghira in https://github.com/bghira/SimpleTuner/pull/555
• refactor area resize for code clarity and fixing non-cropped / downsampled images by @bghira in https://github.com/bghira/SimpleTuner/pull/558
• area resize refactor by @bghira in https://github.com/bghira/SimpleTuner/pull/559

Full Changelog: https://github.com/bghira/SimpleTuner/compare/v0.9.7.5...v0.9.7.6