Add some example runs

This commit is contained in:
filipstrand 2024-09-05 23:13:53 +02:00
parent a0ee9a7a8a
commit 1d8714d3b1
4 changed files with 40 additions and 3 deletions

View File

@ -300,16 +300,53 @@ This mirrors how the resources are placed in the [HuggingFace Repo](https://hugg
### LoRA
MFLUX support loading precomputed [LoRA](https://huggingface.co/docs/diffusers/en/training/lora) weights (training support is coming soon).
By pointing out
MFLUX support loading trained [LoRA](https://huggingface.co/docs/diffusers/en/training/lora) adapters (actual training support is coming).
The following example [The_Hound](https://huggingface.co/TheLastBen/The_Hound) LoRA from [@TheLastBen](https://github.com/TheLastBen):
```
python main.py --prompt "sandor clegane" --model dev --steps 20 --seed 43 -q 8 --lora-paths "sandor_clegane_single_layer.safetensors"
```
![image](src/flux_1/assets/lora1.jpg)
---
The following example is [Flux_1_Dev_LoRA_Paper-Cutout-Style](https://huggingface.co/Norod78/Flux_1_Dev_LoRA_Paper-Cutout-Style) LoRA from [@Norod78](https://github.com/TheLastBen):
```
python main.py --prompt "pikachu, Paper Cutout Style" --model schnell --steps 4 --seed 43 -q 8 --lora-paths "Flux_1_Dev_LoRA_Paper-Cutout-Style.safetensors"
```
![image](src/flux_1/assets/lora2.jpg)
*Note that LoRA trained weights are typically trained with a **trigger word or phrase**. For example, in the latter case, the sentence should include the phrase **"Paper Cutout Style"**.*
*Also note that the same LoRA weights can work well with both the `schnell` and `dev` models. Refer to the original LoRA repository to see what mode it was trained for.*
#### Multi-LoRA
Multiple LoRAs can be sent in to combine the effects of the individual adapters. The following example combines both of the above LoRAs:
```
python main.py \
--prompt "sandor clegane in a forest, Paper Cutout Style" \
--model dev \
--steps 20 \
--seed 43 \
--lora-paths sandor.safetensors paper.safetensors \
--lora-scales 1.0 1.0 -q 8 \
```
![image](src/flux_1/assets/lora3.jpg)
Just to see the difference, this image displays the four cases: One of having both adapters fully active, partially active and no LoRA at all.
The example above also show the usage of `--lora-scales` flag.
### Current limitations
- Images are generated one by one.
- Negative prompts not supported.
- LoRA weights are only supported for the transformer part of the network.
### TODO
- [ ] LoRA adapters
- [ ] LoRA fine-tuning
- [ ] Frontend support (Gradio/Streamlit/Other?)

BIN
src/flux_1/assets/lora1.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 806 KiB

BIN
src/flux_1/assets/lora2.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 789 KiB

BIN
src/flux_1/assets/lora3.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 770 KiB