Small additions before 0.8.0 (#209)

This commit is contained in:
Filip Strand 2025-06-14 10:02:04 +02:00 committed by GitHub
parent dac08fdfdc
commit 9a20e83331
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 88 additions and 19 deletions

View File

@ -7,7 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.8.0] - 2025-01-20
## [0.8.0] - 2025-06-14
# MFLUX v.0.8.0 Release Notes
@ -58,6 +58,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Better quality variations with adjustable parameters for more predictable results
- Refined Redux algorithm for more natural image variations
### 📥 Stdin Prompt Support
- **LLM Integration Ready**: Added support for providing prompts via stdin using `--prompt -`
- Enables seamless integration with LLMs and other text generation tools
- Supports both single-line and multi-line prompts through stdin
- Perfect for automation workflows and dynamic prompt generation
- Example usage: `echo "A beautiful landscape" | mflux-generate --prompt -`
## Developer Experience
### 🔧 LORA_LIBRARY_PATH Improvements
@ -78,6 +85,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Enhanced documentation structure for better organization and user navigation
- Improved categorization of experimental vs stable features
### 🔄 Code Architecture Changes
- **Class rename**: `Flux1InContextLora` is now `Flux1InContextDev` to better reflect the dev model variant
- **Module reorganization**: Moved from `mflux.community.in_context_lora.flux_in_context_lora` to `mflux.community.in_context.flux_in_context_dev`
- **Breaking change for library users**: If you import the class directly, update your imports accordingly
### ⚡ Performance Optimizations
- Updated MLX dependency to latest version for improved performance and stability
- Removed PyTorch dependency for DepthPro model, significantly reducing installation requirements
@ -90,8 +103,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## Contributors
Special thanks to the following contributors for their exceptional work since v0.7.1:
- **Anthony Wu (@anthonywu)**: Battery Saver implementation, Prompt File Support, LORA_LIBRARY_PATH improvements
- **Alessandro (@alessandro)**: Redux Function Balancing enhancements
- **Anthony Wu (@anthonywu)**: Battery Saver implementation, Prompt File Support, Stdin Prompt Support, LORA_LIBRARY_PATH improvements
- **Alessandro (@azrahello)**: Redux Function Balancing enhancements
- **Filip Strand (@filipstrand)**: Core development, experimental features integration, infrastructure improvements
## [0.7.1] - 2025-05-06

View File

@ -28,9 +28,12 @@ Run the powerful [FLUX](https://blackforestlabs.ai/#get-flux) models from [Black
* [LoRA Library Path](#lora-library-path)
* [Supported LoRA formats (updated)](#supported-lora-formats-updated)
- [🎭 In-Context Generation](#-in-context-generation)
* [Available Styles](#available-styles)
* [How It Works](#how-it-works)
* [Tips for Best Results](#tips-for-best-results)
* [🎨 In-Context LoRA](#-in-context-lora)
+ [Available Styles](#available-styles)
+ [How It Works](#how-it-works)
+ [Tips for Best Results](#tips-for-best-results)
* [👕 CatVTON (Virtual Try-On)](#-catvton-virtual-try-on)
* [✏️ IC-Edit (In-Context Editing)](#%EF%B8%8F-ic-edit-in-context-editing)
- [🛠️ Flux Tools](#%EF%B8%8F-flux-tools)
* [🖌️ Fill](#%EF%B8%8F-fill)
+ [Inpainting](#inpainting)
@ -958,19 +961,42 @@ CatVTON enables virtual try-on capabilities using in-context learning. This appr
The implementation is based on [@nftblackmagic/catvton-flux](https://github.com/nftblackmagic/catvton-flux) and uses the model weights from [xiaozaa/catvton-flux-beta](https://huggingface.co/xiaozaa/catvton-flux-beta) (approximately 24 GB download).
##### How to Use CatVTON
![catvton example](src/mflux/assets/catvton_example.jpg)
```sh
##### How it works
CatVTON uses a sophisticated in-context learning approach that creates a side-by-side composition for virtual try-on:
1. **Image Layout**: The garment image is placed on the left side, while the person image is positioned on the right side of the composition.
2. **Selective Processing**: The model treats each side differently:
- **Left side (garment)**: The model can only *read* information from this area - no inpainting or modifications occur here. This serves as a reference for the garment's appearance, texture, and style.
- **Right side (person)**: Only the areas highlighted in the person mask are inpainted. The mask determines exactly which parts of the person image should be modified to "wear" the garment.
3. **Contextual Understanding**: By reading the garment characteristics from the left and selectively applying them to the masked regions on the right, the model creates a realistic virtual try-on that maintains proper lighting, shadows, and fabric behavior.
This approach allows the model to understand the garment's properties while precisely controlling which parts of the person image should be transformed, resulting in natural-looking virtual clothing fitting.
![catvton how it works](src/mflux/assets/catvton_how_it_works.jpg)
##### Example Command
```bash
mflux-generate-in-context-catvton \
--person-image "person.jpg" \
--person-mask "person_mask.png" \
--garment-image "garment.jpg" \
--person-image "person.png" \
--person-mask "mask.png" \
--garment-image "garment.png" \
--prompt "The pair of images highlights a clothing and its styling on a model, high resolution, 4K, 8K; [IMAGE1] Detailed product shot of a light blue shirt with designer details such as white and pink patterns; [IMAGE2] The *exact* same cloth (the light blue shirt with designer details such as white and pink patterns) is worn by a model in a lifestyle setting." \
--steps 20 \
--seed 42 \
--seed 6269363 \
--guidance 30 \
--height 1024 \
--width 1024
--width 891 \
-q 8
```
##### Required Inputs
- **Person Image**: A photo of the person who will "wear" the garment
@ -990,6 +1016,8 @@ mflux-generate-in-context-catvton \
2. **Accurate Masks**: Ensure the person mask precisely covers the areas where the garment should appear
3. **Consistent Lighting**: Match lighting conditions between person and garment images when possible
4. **Garment Type**: Works best with clearly defined clothing items (shirts, dresses, jackets, etc.)
5. **Pattern Limitations**: Very intricate patterns, fine details, and complex textures may not transfer well - simpler, solid colors and basic patterns typically work better
6. **Try Multiple Seeds**: Generate several variations with different seeds to find the best result, as outcomes can vary significantly
#### ✏️ IC-Edit (In-Context Editing)
@ -999,16 +1027,41 @@ IC-Edit provides intuitive image editing capabilities using natural language ins
The implementation is based on [@River-Zhang/ICEdit](https://github.com/River-Zhang/ICEdit).
##### How to Use IC-Edit
![ic-edit example](src/mflux/assets/ic_edit_example.jpg)
*Original images credit: [Jimmy Chang (flower)](https://unsplash.com/photos/closeup-photography-of-pink-lotus-flower-LP5WXkrnxX0), [Kar Ming Moo (tiger)](https://unsplash.com/photos/brown-and-black-tiger-lying-on-ground-Q_3WmguWgYg), [Martin Baron (car)](https://unsplash.com/photos/a-shiny-black-car-parks-on-a-street-8JoIWt2KzYI) on Unsplash*
```sh
##### How it works
IC-Edit employs a streamlined diptych approach for natural language image editing:
1. **Image Layout**: The reference image is placed on the left side, while the right side is reserved for the edited result.
2. **Automatic Masking**: Unlike CatVTON, no user-provided mask is required. The system automatically creates a simple but effective mask:
- **Left side (reference)**: Completely black mask - no denoising occurs here, preserving the original image as reference
- **Right side (edit area)**: Completely white mask - the entire area is available for denoising and modification
3. **Instruction Processing**: Your natural language instruction is automatically wrapped in a diptych template that tells the model to create the desired changes on the right side while using the left side as reference.
4. **Guided Generation**: The model reads the visual information from the left side and applies the requested modifications to generate the edited result on the right side, maintaining visual coherence and context.
This simplified approach removes the complexity of mask creation while still providing precise control through natural language instructions, making image editing accessible and intuitive.
![ic-edit how it works](src/mflux/assets/ic_edit_how_it_works.jpg)
##### Example Command
```bash
mflux-generate-in-context-edit \
--reference-image "original.jpg" \
--instruction "make the hair black" \
--reference-image "flower.jpg" \
--prompt "two images of the exact same flower in two different styles: On the left the photo has is in bright colors showing green leaves and a pink flower. On the right, the *exact* same photo (same flower, same leaves, same background, identical etc). but with where everything is black and white except for the flower which is still in color. The content is *exactly* the same between the left and right image, except only for the coloring (black and white for everything except for the colorful flower)" \
--steps 20 \
--seed 42
--seed 8570325 \
--guidance 30 \
-q 8
```
##### Key Features
- **Natural Language Instructions**: Use simple, descriptive instructions like "make the hair black", "add sunglasses", or "change the background to a beach"
@ -1038,6 +1091,9 @@ mflux-generate-in-context-edit \
2. **Single Changes**: Focus on one edit at a time for more predictable results
3. **Reference Quality**: Higher quality reference images generally produce better edits
4. **Iterative Editing**: Use the output of one edit as input for the next to build complex changes
5. **Try Multiple Seeds**: Generate several variations with different seeds to find the best result, as outcomes can vary significantly
⚠️ *Note: Using the IC-Edit tool requires an additional ~230MB download for the specialized LoRA weights. The download happens automatically on first use. This remarkably small LoRA (less than 0.7% of the base model's ~34GB) enables sophisticated image editing capabilities, demonstrating that the base FLUX model is already highly competent at style transfers and transformations - the LoRA simply provides the user control and interface to access these latent capabilities.*
---

View File

@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "mflux"
version = "0.7.1"
version = "0.8.0"
description = "A MLX port of FLUX based on the Huggingface Diffusers implementation."
readme = "README.md"
keywords = ["diffusers", "flux", "mlx"]

Binary file not shown.

After

Width:  |  Height:  |  Size: 429 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 488 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 619 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 828 KiB