MyGit

keijiro/UnityMLStableDiffusion

Fork: 54 Star: 598 (更新于 2024-11-02 16:35:38)

license: NOASSERTION

Language: C# .

Core ML Stable Diffusion on Unity

最后发布版本: 1.5.0 ( 2024-09-21 15:00:05)

GitHub网址

Unity Core ML Stable Diffusion Plugin

gif

Stable Diffusion plugin for Unity, based on Apple's Core ML port. You can run the model on-editor and at-runtime without needing any extra components.

System Requirements

  • Unity 2023.1 or later
  • Apple Silicon Mac (editor/runtime support) with macOS 13.1 or later
  • iPad Pro with Apple silicon (runtime support) with iOS 16.2 or later

Although the plugin supports iOS, it requires huge amount of memory to run the model, so it only supports memory-rich iPad models.

How To Try

Before running the sample project, you must put the model files in the Assets/StreamingAssets directory.

It takes a long time (a few minutes) for the first run. After this initialization step, it only takes a few tens of seconds to generate an image.

Performance Considerations

You can change which processing unit it uses by switching the "Compute Units" property in the Tester component.

It depends on the device model to choose the best option:

  • M1/M2 Mac and iOS: GPUs in those devices are not powerful enough, so you should select "CPU and NE" (Neural Engine) or "All".
  • M1/M2 Pro/Max Mac: GPUs in those devices have enough processing power compared to NE, so "CPU and GPU" can be a better option.

When using "CPU and GPU" mode, you must use the "original" model instead of the "split_einsum" model. Please overwrite the StreamingAssets/StableDiffusion directory with the original/compiled directory.

LCM (SD-Turbo) Support

You can use SD-Turbo or other LCMs (latent consistency models) with setting Pipeline.Scheduler to Lcm. You might also have to change StepCount to 1~4 and GuidanceScale to 1~2. Please refer to the model description to know the correct settings.

You can download the pre-converted SD-Turbo model from my Hugging Face repository.

Sample Projects

gif

  • Flipbook3: Running the image-to-image pipeline with a real-time 3D scene.

最近版本更新:(数据更新于 2024-10-11 07:01:50)

2024-09-21 15:00:05 1.5.0

2023-12-04 10:38:46 1.4.0

2023-04-30 22:11:33 1.3.0

2023-04-29 16:44:51 1.2.0

2023-03-29 22:54:30 1.1.0

2023-03-29 09:46:06 1.0.0

keijiro/UnityMLStableDiffusion同语言 C#最近更新仓库

2024-11-04 20:01:02 Pik-4/HsMod

2024-11-03 23:57:50 jellyfin/jellyfin

2024-10-31 10:00:36 metatube-community/jellyfin-plugin-metatube

2024-10-30 03:58:07 AlchlcDvl/TownOfUsReworked

2024-10-26 07:18:02 Azure/azure-sdk-for-net

2024-10-23 06:18:57 PowerShell/PowerShell