Flux

It's here! Runs on 24G cards using Quanto's 8bit quantisation, or 25.7G on a Macbook system (slowly)!

If you're after accuracy, a 40G card will do Just Fine, with 80G cards being somewhat of a sweet spot for larger training efforts.

What you get:

• LoRA, full tuning (but probably just don't do that)
• Documentation to get you started fast
• Probably better for just square crop training for now - might artifact for weird resolutions
• Quantised base model unlocking the ability to safely use Adafactor, Prodigy, and other neat optimisers as a consolation prize for losing access to full bf16 training (AdamWBF16 just won't work with Quanto)

What's Changed

• trainer: simplify check by @bghira in https://github.com/bghira/SimpleTuner/pull/592
• documentation updates, apple pytorch 2.4 by @bghira in https://github.com/bghira/SimpleTuner/pull/595
• staged storage for image embed support by @bghira in https://github.com/bghira/SimpleTuner/pull/596
• fix: loading default image embed backend by @bghira in https://github.com/bghira/SimpleTuner/pull/597
• fix: loading default image embed backend by @bghira in https://github.com/bghira/SimpleTuner/pull/598
• multi-gpu console output improvements by @bghira in https://github.com/bghira/SimpleTuner/pull/599
• vae cache: hash_filenames option for image sets by @bghira in https://github.com/bghira/SimpleTuner/pull/601
• multi-gpu console output reduction by @bghira in https://github.com/bghira/SimpleTuner/pull/602
• fix for relative cache directories with NoneType being unsubscriptable by @bghira in https://github.com/bghira/SimpleTuner/pull/603
• multigpu / relative path fixes for caching by @bghira in https://github.com/bghira/SimpleTuner/pull/604
• backend for csv based datasets by @bghira in https://github.com/bghira/SimpleTuner/pull/600
• CSV data backend by @bghira in https://github.com/bghira/SimpleTuner/pull/605
• config file versioning to allow updating defaults without breaking backwards compat by @bghira in https://github.com/bghira/SimpleTuner/pull/607
• config file versioning for backwards compat by @bghira in https://github.com/bghira/SimpleTuner/pull/608
• experiment: small DiT model by @bghira in https://github.com/bghira/SimpleTuner/pull/609
• merge by @bghira in https://github.com/bghira/SimpleTuner/pull/610
• Fix crash when using jsonl files by @swkang-rp in https://github.com/bghira/SimpleTuner/pull/611
• merge by @bghira in https://github.com/bghira/SimpleTuner/pull/612
• flux training by @bghira in https://github.com/bghira/SimpleTuner/pull/614
• update base_dir to output_dir by @bghira in https://github.com/bghira/SimpleTuner/pull/615
• merge by @bghira in https://github.com/bghira/SimpleTuner/pull/616
• flux: validations should ignore any custom schedulers by @bghira in https://github.com/bghira/SimpleTuner/pull/618
• release: flux by @bghira in https://github.com/bghira/SimpleTuner/pull/617
• bugfix: correctly set hash_filenames to true or false for an initial dataset creation by @bghira in https://github.com/bghira/SimpleTuner/pull/620
• release: minor follow-up fixes by @bghira in https://github.com/bghira/SimpleTuner/pull/628
• Flux: Fix random validation errors due to some tensors being on the cpu by @mhirki in https://github.com/bghira/SimpleTuner/pull/629
• Improve config support for transformers with accelerate by @touchwolf in https://github.com/bghira/SimpleTuner/pull/630
• quanto: exploring low-precision training. by @bghira in https://github.com/bghira/SimpleTuner/pull/622
• remove all text encoders from memory correctly by @bghira in https://github.com/bghira/SimpleTuner/pull/637

New Contributors

• @swkang-rp made their first contribution in https://github.com/bghira/SimpleTuner/pull/611
• @touchwolf made their first contribution in https://github.com/bghira/SimpleTuner/pull/630

Full Changelog: https://github.com/bghira/SimpleTuner/compare/v0.9.7.8...v0.9.8