diff --git a/README.md b/README.md index e7cf7cd..8996cb6 100644 --- a/README.md +++ b/README.md @@ -300,16 +300,53 @@ This mirrors how the resources are placed in the [HuggingFace Repo](https://hugg ### LoRA -MFLUX support loading precomputed [LoRA](https://huggingface.co/docs/diffusers/en/training/lora) weights (training support is coming soon). -By pointing out +MFLUX support loading trained [LoRA](https://huggingface.co/docs/diffusers/en/training/lora) adapters (actual training support is coming). + +The following example [The_Hound](https://huggingface.co/TheLastBen/The_Hound) LoRA from [@TheLastBen](https://github.com/TheLastBen): + +``` +python main.py --prompt "sandor clegane" --model dev --steps 20 --seed 43 -q 8 --lora-paths "sandor_clegane_single_layer.safetensors" +``` + +![image](src/flux_1/assets/lora1.jpg) +--- + +The following example is [Flux_1_Dev_LoRA_Paper-Cutout-Style](https://huggingface.co/Norod78/Flux_1_Dev_LoRA_Paper-Cutout-Style) LoRA from [@Norod78](https://github.com/TheLastBen): + +``` +python main.py --prompt "pikachu, Paper Cutout Style" --model schnell --steps 4 --seed 43 -q 8 --lora-paths "Flux_1_Dev_LoRA_Paper-Cutout-Style.safetensors" +``` +![image](src/flux_1/assets/lora2.jpg) + +*Note that LoRA trained weights are typically trained with a **trigger word or phrase**. For example, in the latter case, the sentence should include the phrase **"Paper Cutout Style"**.* + +*Also note that the same LoRA weights can work well with both the `schnell` and `dev` models. Refer to the original LoRA repository to see what mode it was trained for.* + +#### Multi-LoRA + +Multiple LoRAs can be sent in to combine the effects of the individual adapters. The following example combines both of the above LoRAs: + +``` +python main.py \ +--prompt "sandor clegane in a forest, Paper Cutout Style" \ +--model dev \ +--steps 20 \ +--seed 43 \ +--lora-paths sandor.safetensors paper.safetensors \ +--lora-scales 1.0 1.0 -q 8 \ +``` +![image](src/flux_1/assets/lora3.jpg) + +Just to see the difference, this image displays the four cases: One of having both adapters fully active, partially active and no LoRA at all. +The example above also show the usage of `--lora-scales` flag. ### Current limitations - Images are generated one by one. - Negative prompts not supported. +- LoRA weights are only supported for the transformer part of the network. ### TODO -- [ ] LoRA adapters - [ ] LoRA fine-tuning - [ ] Frontend support (Gradio/Streamlit/Other?) \ No newline at end of file diff --git a/src/flux_1/assets/lora1.jpg b/src/flux_1/assets/lora1.jpg new file mode 100644 index 0000000..46188d4 Binary files /dev/null and b/src/flux_1/assets/lora1.jpg differ diff --git a/src/flux_1/assets/lora2.jpg b/src/flux_1/assets/lora2.jpg new file mode 100644 index 0000000..b317be0 Binary files /dev/null and b/src/flux_1/assets/lora2.jpg differ diff --git a/src/flux_1/assets/lora3.jpg b/src/flux_1/assets/lora3.jpg new file mode 100644 index 0000000..f83edb6 Binary files /dev/null and b/src/flux_1/assets/lora3.jpg differ