Small additions before 0.8.0 (#209)
This commit is contained in:
parent
dac08fdfdc
commit
9a20e83331
19
CHANGELOG.md
19
CHANGELOG.md
@ -7,7 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.8.0] - 2025-01-20
|
||||
## [0.8.0] - 2025-06-14
|
||||
|
||||
# MFLUX v.0.8.0 Release Notes
|
||||
|
||||
@ -58,6 +58,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- Better quality variations with adjustable parameters for more predictable results
|
||||
- Refined Redux algorithm for more natural image variations
|
||||
|
||||
### 📥 Stdin Prompt Support
|
||||
- **LLM Integration Ready**: Added support for providing prompts via stdin using `--prompt -`
|
||||
- Enables seamless integration with LLMs and other text generation tools
|
||||
- Supports both single-line and multi-line prompts through stdin
|
||||
- Perfect for automation workflows and dynamic prompt generation
|
||||
- Example usage: `echo "A beautiful landscape" | mflux-generate --prompt -`
|
||||
|
||||
## Developer Experience
|
||||
|
||||
### 🔧 LORA_LIBRARY_PATH Improvements
|
||||
@ -78,6 +85,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- Enhanced documentation structure for better organization and user navigation
|
||||
- Improved categorization of experimental vs stable features
|
||||
|
||||
### 🔄 Code Architecture Changes
|
||||
- **Class rename**: `Flux1InContextLora` is now `Flux1InContextDev` to better reflect the dev model variant
|
||||
- **Module reorganization**: Moved from `mflux.community.in_context_lora.flux_in_context_lora` to `mflux.community.in_context.flux_in_context_dev`
|
||||
- **Breaking change for library users**: If you import the class directly, update your imports accordingly
|
||||
|
||||
|
||||
### ⚡ Performance Optimizations
|
||||
- Updated MLX dependency to latest version for improved performance and stability
|
||||
- Removed PyTorch dependency for DepthPro model, significantly reducing installation requirements
|
||||
@ -90,8 +103,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## Contributors
|
||||
|
||||
Special thanks to the following contributors for their exceptional work since v0.7.1:
|
||||
- **Anthony Wu (@anthonywu)**: Battery Saver implementation, Prompt File Support, LORA_LIBRARY_PATH improvements
|
||||
- **Alessandro (@alessandro)**: Redux Function Balancing enhancements
|
||||
- **Anthony Wu (@anthonywu)**: Battery Saver implementation, Prompt File Support, Stdin Prompt Support, LORA_LIBRARY_PATH improvements
|
||||
- **Alessandro (@azrahello)**: Redux Function Balancing enhancements
|
||||
- **Filip Strand (@filipstrand)**: Core development, experimental features integration, infrastructure improvements
|
||||
|
||||
## [0.7.1] - 2025-05-06
|
||||
|
||||
86
README.md
86
README.md
@ -28,9 +28,12 @@ Run the powerful [FLUX](https://blackforestlabs.ai/#get-flux) models from [Black
|
||||
* [LoRA Library Path](#lora-library-path)
|
||||
* [Supported LoRA formats (updated)](#supported-lora-formats-updated)
|
||||
- [🎭 In-Context Generation](#-in-context-generation)
|
||||
* [Available Styles](#available-styles)
|
||||
* [How It Works](#how-it-works)
|
||||
* [Tips for Best Results](#tips-for-best-results)
|
||||
* [🎨 In-Context LoRA](#-in-context-lora)
|
||||
+ [Available Styles](#available-styles)
|
||||
+ [How It Works](#how-it-works)
|
||||
+ [Tips for Best Results](#tips-for-best-results)
|
||||
* [👕 CatVTON (Virtual Try-On)](#-catvton-virtual-try-on)
|
||||
* [✏️ IC-Edit (In-Context Editing)](#%EF%B8%8F-ic-edit-in-context-editing)
|
||||
- [🛠️ Flux Tools](#%EF%B8%8F-flux-tools)
|
||||
* [🖌️ Fill](#%EF%B8%8F-fill)
|
||||
+ [Inpainting](#inpainting)
|
||||
@ -958,19 +961,42 @@ CatVTON enables virtual try-on capabilities using in-context learning. This appr
|
||||
|
||||
The implementation is based on [@nftblackmagic/catvton-flux](https://github.com/nftblackmagic/catvton-flux) and uses the model weights from [xiaozaa/catvton-flux-beta](https://huggingface.co/xiaozaa/catvton-flux-beta) (approximately 24 GB download).
|
||||
|
||||
##### How to Use CatVTON
|
||||

|
||||
|
||||
```sh
|
||||
##### How it works
|
||||
|
||||
CatVTON uses a sophisticated in-context learning approach that creates a side-by-side composition for virtual try-on:
|
||||
|
||||
1. **Image Layout**: The garment image is placed on the left side, while the person image is positioned on the right side of the composition.
|
||||
|
||||
2. **Selective Processing**: The model treats each side differently:
|
||||
- **Left side (garment)**: The model can only *read* information from this area - no inpainting or modifications occur here. This serves as a reference for the garment's appearance, texture, and style.
|
||||
- **Right side (person)**: Only the areas highlighted in the person mask are inpainted. The mask determines exactly which parts of the person image should be modified to "wear" the garment.
|
||||
|
||||
3. **Contextual Understanding**: By reading the garment characteristics from the left and selectively applying them to the masked regions on the right, the model creates a realistic virtual try-on that maintains proper lighting, shadows, and fabric behavior.
|
||||
|
||||
This approach allows the model to understand the garment's properties while precisely controlling which parts of the person image should be transformed, resulting in natural-looking virtual clothing fitting.
|
||||
|
||||

|
||||
|
||||
##### Example Command
|
||||
|
||||
```bash
|
||||
mflux-generate-in-context-catvton \
|
||||
--person-image "person.jpg" \
|
||||
--person-mask "person_mask.png" \
|
||||
--garment-image "garment.jpg" \
|
||||
--person-image "person.png" \
|
||||
--person-mask "mask.png" \
|
||||
--garment-image "garment.png" \
|
||||
--prompt "The pair of images highlights a clothing and its styling on a model, high resolution, 4K, 8K; [IMAGE1] Detailed product shot of a light blue shirt with designer details such as white and pink patterns; [IMAGE2] The *exact* same cloth (the light blue shirt with designer details such as white and pink patterns) is worn by a model in a lifestyle setting." \
|
||||
--steps 20 \
|
||||
--seed 42 \
|
||||
--seed 6269363 \
|
||||
--guidance 30 \
|
||||
--height 1024 \
|
||||
--width 1024
|
||||
--width 891 \
|
||||
-q 8
|
||||
```
|
||||
|
||||
|
||||
|
||||
##### Required Inputs
|
||||
|
||||
- **Person Image**: A photo of the person who will "wear" the garment
|
||||
@ -990,6 +1016,8 @@ mflux-generate-in-context-catvton \
|
||||
2. **Accurate Masks**: Ensure the person mask precisely covers the areas where the garment should appear
|
||||
3. **Consistent Lighting**: Match lighting conditions between person and garment images when possible
|
||||
4. **Garment Type**: Works best with clearly defined clothing items (shirts, dresses, jackets, etc.)
|
||||
5. **Pattern Limitations**: Very intricate patterns, fine details, and complex textures may not transfer well - simpler, solid colors and basic patterns typically work better
|
||||
6. **Try Multiple Seeds**: Generate several variations with different seeds to find the best result, as outcomes can vary significantly
|
||||
|
||||
#### ✏️ IC-Edit (In-Context Editing)
|
||||
|
||||
@ -999,16 +1027,41 @@ IC-Edit provides intuitive image editing capabilities using natural language ins
|
||||
|
||||
The implementation is based on [@River-Zhang/ICEdit](https://github.com/River-Zhang/ICEdit).
|
||||
|
||||
##### How to Use IC-Edit
|
||||

|
||||
*Original images credit: [Jimmy Chang (flower)](https://unsplash.com/photos/closeup-photography-of-pink-lotus-flower-LP5WXkrnxX0), [Kar Ming Moo (tiger)](https://unsplash.com/photos/brown-and-black-tiger-lying-on-ground-Q_3WmguWgYg), [Martin Baron (car)](https://unsplash.com/photos/a-shiny-black-car-parks-on-a-street-8JoIWt2KzYI) on Unsplash*
|
||||
|
||||
```sh
|
||||
##### How it works
|
||||
|
||||
IC-Edit employs a streamlined diptych approach for natural language image editing:
|
||||
|
||||
1. **Image Layout**: The reference image is placed on the left side, while the right side is reserved for the edited result.
|
||||
|
||||
2. **Automatic Masking**: Unlike CatVTON, no user-provided mask is required. The system automatically creates a simple but effective mask:
|
||||
- **Left side (reference)**: Completely black mask - no denoising occurs here, preserving the original image as reference
|
||||
- **Right side (edit area)**: Completely white mask - the entire area is available for denoising and modification
|
||||
|
||||
3. **Instruction Processing**: Your natural language instruction is automatically wrapped in a diptych template that tells the model to create the desired changes on the right side while using the left side as reference.
|
||||
|
||||
4. **Guided Generation**: The model reads the visual information from the left side and applies the requested modifications to generate the edited result on the right side, maintaining visual coherence and context.
|
||||
|
||||
This simplified approach removes the complexity of mask creation while still providing precise control through natural language instructions, making image editing accessible and intuitive.
|
||||
|
||||

|
||||
|
||||
##### Example Command
|
||||
|
||||
```bash
|
||||
mflux-generate-in-context-edit \
|
||||
--reference-image "original.jpg" \
|
||||
--instruction "make the hair black" \
|
||||
--reference-image "flower.jpg" \
|
||||
--prompt "two images of the exact same flower in two different styles: On the left the photo has is in bright colors showing green leaves and a pink flower. On the right, the *exact* same photo (same flower, same leaves, same background, identical etc). but with where everything is black and white except for the flower which is still in color. The content is *exactly* the same between the left and right image, except only for the coloring (black and white for everything except for the colorful flower)" \
|
||||
--steps 20 \
|
||||
--seed 42
|
||||
--seed 8570325 \
|
||||
--guidance 30 \
|
||||
-q 8
|
||||
```
|
||||
|
||||
|
||||
|
||||
##### Key Features
|
||||
|
||||
- **Natural Language Instructions**: Use simple, descriptive instructions like "make the hair black", "add sunglasses", or "change the background to a beach"
|
||||
@ -1038,6 +1091,9 @@ mflux-generate-in-context-edit \
|
||||
2. **Single Changes**: Focus on one edit at a time for more predictable results
|
||||
3. **Reference Quality**: Higher quality reference images generally produce better edits
|
||||
4. **Iterative Editing**: Use the output of one edit as input for the next to build complex changes
|
||||
5. **Try Multiple Seeds**: Generate several variations with different seeds to find the best result, as outcomes can vary significantly
|
||||
|
||||
⚠️ *Note: Using the IC-Edit tool requires an additional ~230MB download for the specialized LoRA weights. The download happens automatically on first use. This remarkably small LoRA (less than 0.7% of the base model's ~34GB) enables sophisticated image editing capabilities, demonstrating that the base FLUX model is already highly competent at style transfers and transformations - the LoRA simply provides the user control and interface to access these latent capabilities.*
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "mflux"
|
||||
version = "0.7.1"
|
||||
version = "0.8.0"
|
||||
description = "A MLX port of FLUX based on the Huggingface Diffusers implementation."
|
||||
readme = "README.md"
|
||||
keywords = ["diffusers", "flux", "mlx"]
|
||||
|
||||
BIN
src/mflux/assets/catvton_example.jpg
Normal file
BIN
src/mflux/assets/catvton_example.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 429 KiB |
BIN
src/mflux/assets/catvton_how_it_works.jpg
Normal file
BIN
src/mflux/assets/catvton_how_it_works.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 488 KiB |
BIN
src/mflux/assets/ic_edit_example.jpg
Normal file
BIN
src/mflux/assets/ic_edit_example.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 619 KiB |
BIN
src/mflux/assets/ic_edit_how_it_works.jpg
Normal file
BIN
src/mflux/assets/ic_edit_how_it_works.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 828 KiB |
Loading…
Reference in New Issue
Block a user