diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a53666..0d168e9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,79 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.11.0] - 2025-10-12 + +# MFLUX v.0.11.0 Release Notes + +### 🎨 New Model Support + +- **Qwen Image Support**: Added support for the Qwen Image text-to-image model, enabling a new generation of visual content creation +- **New command**: `mflux-generate` now supports Qwen models for image generation +- **Qwen-specific features**: Full LoRA support with Qwen naming conventions, img2img support, and optimized weight handling +- **Qwen-Image-mflux-6bit Model**: Added [filipstrand/Qwen-Image-mflux-6bit](https://huggingface.co/filipstrand/Qwen-Image-mflux-6bit) quantized model to HF + +### 🏗️ Major Architecture Improvements + +- **Package Restructure**: Complete reorganization of the codebase to support multiple model architectures + - Moved from flat structure to organized `models/` hierarchy (`models/flux/`, `models/qwen/`, `models/depth_pro/`) + - Better separation of concerns with dedicated model, variant, tokenizer, and weight handler modules + - Improved maintainability and extensibility for future model additions +- **Namespace Package**: Converted mflux to a namespace package (in preparation for mflux.mcp extension) +- **Common Module**: Extracted shared functionality into `models/common/` for better code reuse + - Unified LoRA handling across different model types + - Shared attention utilities + - Common download and weight management utilities + +### 📊 Metadata Enhancements + +- **XMP/IPTC Metadata Support**: Added comprehensive metadata support for professional workflows + - Write XMP and IPTC metadata to generated images + - Industry-standard metadata formats for better compatibility with professional image tools + - Enhanced metadata reading and writing capabilities +- **New `mflux-info` command**: Display detailed metadata information from generated images + - View generation parameters, model information, and settings + - Extract metadata from any mflux-generated image + - Professional-grade metadata inspection + +### 🔧 Scheduler System + +- **Scheduler Interface**: Introduced a new scheduler abstraction for better extensibility + - Clean interface for implementing custom sampling schedulers + - Foundation for future scheduler additions (Euler, DPM++, etc.) + - Current implementation: Linear scheduler (existing behavior preserved) +- **Scheduler Selection**: Added `--scheduler` command-line argument for choosing schedulers + +### 🐛 Bug Fixes + +- **Non-Quantized Model Loading**: Fixed critical bug where locally saved non-quantized models failed to load properly +- **Model Weight Handling**: Improved weight loading reliability for edge cases + +### 🔧 Developer Experience + +- **MLX 0.29.2 Support**: Updated MLX dependency to support the latest version (mlx>=0.27.0,<0.30.0) +- **Python 3.13 Support**: Unblocked sentencepiece and torch dependencies for Python 3.13 + - Updated dependency specifications for better Python 3.13 compatibility + - Ensured smooth experience on latest Python versions +- **Test Improvements**: Enhanced image comparison logic to allow similar images that are "close enough" + - More robust test suite that accommodates minor numerical differences + - Reduced false positives in image generation tests +- **CI Updates**: Removed Claude CI agent (replacement coming soon) + +### 🔄 Breaking Changes + +⚠️ **Import Path Changes**: Due to the package restructure, some internal import paths have changed. If you're using mflux as a library and importing internal modules directly, you may need to update your imports: +- Flux modules moved from `mflux.flux.*` to `mflux.models.flux.*` +- Common utilities moved to `mflux.models.common.*` +- CLI tools remain unchanged and fully backward compatible + +### 👩‍💻 Contributors + +- **Filip Strand (@filipstrand)**: Qwen model support, package restructure, core development +- **Alessandro Rizzo (@azrahello)**: XMP/IPTC metadata support, info command implementation +- **Anthony Wu (@anthonywu)**: Scheduler interface, namespace package conversion, Python 3.13 improvements, bug fixes + +--- + ## [0.10.0] - 2025-08-04 # MFLUX v.0.10.0 Release Notes diff --git a/Makefile b/Makefile index 67e227e..be58b5c 100644 --- a/Makefile +++ b/Makefile @@ -93,7 +93,7 @@ check: ensure-ruff .PHONY: test test: ensure-pytest # 🏗️ Running tests... - uv pip install mlx==0.27.1 # Install pinned MLX version specifically for testing + uv pip install mlx==0.29.2 # Install pinned MLX version specifically for testing $(PYTHON) -m pytest # ✅ Tests completed diff --git a/README.md b/README.md index ed790ee..cf0a0db 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,7 @@ All models are implemented from scratch in MLX and only the tokenizers are used [Huggingface Transformers](https://github.com/huggingface/transformers) library. Other than that, there are only minimal dependencies like [Numpy](https://numpy.org) and [Pillow](https://pypi.org/project/pillow/) for simple image post-processing. +As of v.0.11.0, MFLUX now supports the Qwen Image model. --- ### 💿 Installation @@ -152,6 +153,12 @@ This example uses the more powerful `dev` model with 25 time steps: mflux-generate --model dev --prompt "Luxury food photograph" --steps 25 --seed 2 -q 8 ``` +This example uses the `qwen` model with 20 time steps: + +```sh +mflux-generate --model qwen --prompt "Luxury food photograph" --steps 20 --seed 2 -q 6 +``` + You can also pipe prompts from other commands using stdin: ```sh @@ -168,8 +175,8 @@ from mflux.config.config import Config # Load the model flux = Flux1.from_name( - model_name="schnell", # "schnell", "dev", or "krea-dev" - quantize=8, # 3, 4, 5, 6, or 8 + model_name="schnell", # "schnell", "dev", "krea-dev", or "qwen" + quantize=8, # 3, 4, 5, 6, or 8 ) # Generate an image @@ -177,7 +184,7 @@ image = flux.generate_image( seed=2, prompt="Luxury food photograph", config=Config( - num_inference_steps=2, # "schnell" works well with 2-4 steps, "dev" and "krea-dev" work well with 20-25 steps + num_inference_steps=2, # "schnell" works well with 2-4 steps, "dev", "krea-dev", and "qwen" work well with 20-25 steps height=1024, width=1024, ) @@ -188,7 +195,7 @@ image.save(path="image.png") For more advanced Python usage and additional configuration options, you can explore the entry point files in the source code, such as [`generate.py`](src/mflux/generate.py), [`generate_controlnet.py`](src/mflux/generate_controlnet.py), [`generate_fill.py`](src/mflux/generate_fill.py), and others in the [`src/mflux/`](src/mflux/) directory. These files demonstrate how to use the Python API for various features and provide examples of advanced configurations. -⚠️ *If the specific model is not already downloaded on your machine, it will start the download process and fetch the model weights (~34GB in size for the Schnell or Dev model respectively). See the [quantization](#%EF%B8%8F-quantization) section for running compressed versions of the model.* ⚠️ +⚠️ *If the specific model is not already downloaded on your machine, it will start the download process and fetch the model weights (~34GB for Schnell/Dev models, ~58GB for Qwen). See the [quantization](#%EF%B8%8F-quantization) section for running compressed versions of the model.* ⚠️ *By default, mflux caches files in `~/Library/Caches/mflux/`. The Hugging Face model files themselves are cached separately in the Hugging Face cache directory (e.g., `~/.cache/huggingface/`).* @@ -228,9 +235,9 @@ mflux-generate \ - **`--prompt`** (required, `str`): Text description of the image to generate. Use `-` to read the prompt from stdin (e.g., `echo "A beautiful sunset" | mflux-generate --prompt -`). -- **`--model`** or **`-m`** (required, `str`): Model to use for generation. Can be one of the official models (`"schnell"`, `"dev"`, or `"krea-dev"`) or a HuggingFace repository ID for a compatible third-party model (e.g., `"Freepik/flux.1-lite-8B-alpha"`). +- **`--model`** or **`-m`** (required, `str`): Model to use for generation. Can be one of the official models (`"schnell"`, `"dev"`, `"krea-dev"`, or `"qwen"`) or a HuggingFace repository ID for a compatible third-party model (e.g., `"Freepik/flux.1-lite-8B-alpha"`). -- **`--base-model`** (optional, `str`, default: `None`): Specifies which base architecture a third-party model is derived from (`"schnell"`, `"dev"`, or `"krea-dev"`). Required when using third-party models from HuggingFace. +- **`--base-model`** (optional, `str`, default: `None`): Specifies which base architecture a third-party model is derived from (`"schnell"`, `"dev"`, `"krea-dev"`, or `"qwen"`). Required when using third-party models from HuggingFace. - **`--output`** (optional, `str`, default: `"image.png"`): Output image filename. If `--seed` or `--auto-seeds` establishes multiple seed values, the output filename will automatically be modified to include the seed value (e.g., `image_seed_42.png`). @@ -244,7 +251,7 @@ mflux-generate \ - **`--steps`** (optional, `int`, default: `4`): Number of inference steps. -- **`--guidance`** (optional, `float`, default: `3.5`): Guidance scale (only used for `"dev"` model). +- **`--guidance`** (optional, `float`, default: `3.5`): Guidance scale (only used for `"dev"`, `"krea-dev"` and `qwen` models). - **`--path`** (optional, `str`, default: `None`): Path to a local model on disk. @@ -745,6 +752,8 @@ mflux-save \ --quantize 8 ``` +The `mflux-save` command works with both Flux and Qwen models. + *Note that when saving a quantized version, you will need the original huggingface weights.* It is also possible to specify [LoRA](#-lora) adapters when saving the model, e.g @@ -792,6 +801,7 @@ In other words, you can reclaim the 34GB diskspace (per model) by deleting the f - [dhairyashil/FLUX.1-dev-mflux-4bit](https://huggingface.co/dhairyashil/FLUX.1-dev-mflux-4bit) - [akx/FLUX.1-Kontext-dev-mflux-4bit](https://huggingface.co/akx/FLUX.1-Kontext-dev-mflux-4bit) - [filipstrand/FLUX.1-Krea-dev-mflux-4bit](https://huggingface.co/filipstrand/FLUX.1-Krea-dev-mflux-4bit) + - [filipstrand/Qwen-Image-mflux-6bit](https://huggingface.co/filipstrand/Qwen-Image-mflux-6bit) Using the [community model support](#-third-party-huggingface-model-support), the quantized weights can be also be automatically downloaded when running the generate command: @@ -940,7 +950,7 @@ mflux-generate --prompt "pikachu, Paper Cutout Style" --model schnell --steps 4 *Note that LoRA trained weights are typically trained with a **trigger word or phrase**. For example, in the latter case, the sentence should include the phrase **"Paper Cutout Style"**.* -*Also note that the same LoRA weights can work well with both the `schnell` and `dev` models. Refer to the original LoRA repository to see what mode it was trained for.* +*Also note that the same LoRA weights can work well with both the `schnell` and `dev` models. Qwen models support LoRA but may require Qwen-specific LoRA weights. Refer to the original LoRA repository to see what mode it was trained for.* #### Multi-LoRA @@ -1863,6 +1873,7 @@ See `uv run tools/rename_images.py --help` for full CLI usage help. - Set up shell aliases for required args examples: - shortcut for dev model: `alias mflux-dev='mflux-generate --model dev'` - shortcut for schnell model *and* always save metadata: `alias mflux-schnell='mflux-generate --model schnell --metadata'` + - shortcut for qwen model: `alias mflux-qwen='mflux-generate --model qwen'` - For systems with limited memory, use the `--low-ram` flag to reduce memory usage by constraining the MLX cache size and releasing components after use - On battery-powered Macs, use `--battery-percentage-stop-limit` (or `-B`) to prevent your laptop from shutting down during long generation sessions - When generating multiple images with different seeds, use `--seed` with multiple values or `--auto-seeds` to automatically generate a series of random seeds diff --git a/pyproject.toml b/pyproject.toml index 987f132..3e34f9c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -17,7 +17,7 @@ source-exclude = [ [project] name = "mflux" -version = "0.10.0" +version = "0.11.0" description = "A MLX port of FLUX based on the Huggingface Diffusers implementation." readme = "README.md" keywords = ["diffusers", "flux", "mlx"] @@ -30,7 +30,7 @@ dependencies = [ "filelock>=3.18.0", "huggingface-hub>=0.24.5,<1.0", "matplotlib>=3.9.2,<4.0", - "mlx>=0.27.0,<0.28.0", + "mlx>=0.27.0,<0.30.0", "numpy>=2.0.1,<3.0", "opencv-python>=4.10.0,<5.0", "piexif>=1.1.3,<2.0", @@ -68,7 +68,7 @@ dev = [ "matplotlib>3.10,<4.0", "pytest>=8.3.0,<9.0", "pytest-timer>=1.0,<2.0", - "mlx==0.27.1", # Used ONLY during test runs to ensure deterministic test results + "mlx==0.29.2", # Used ONLY during test runs to ensure deterministic test results ] [project.urls] diff --git a/src/mflux/config/model_config.py b/src/mflux/config/model_config.py index 092122e..87e7e90 100644 --- a/src/mflux/config/model_config.py +++ b/src/mflux/config/model_config.py @@ -128,7 +128,9 @@ class ModelConfig: default_base = next((b for b in base_models if base_model == b.model_name or base_model in b.aliases), None) else: # Infer from model_name substring - prefer longer matches (more specific) - matching_bases = [(b, alias) for b in base_models for alias in b.aliases if alias and alias in model_name] + # Use case-insensitive matching for better compatibility + model_name_lower = model_name.lower() + matching_bases = [(b, alias) for b in base_models for alias in b.aliases if alias and alias.lower() in model_name_lower] if matching_bases: # Sort by alias length descending, then by priority ascending diff --git a/src/mflux/info.py b/src/mflux/info.py index d88b37b..591a291 100644 --- a/src/mflux/info.py +++ b/src/mflux/info.py @@ -56,7 +56,7 @@ def format_metadata(metadata: dict) -> str: if lora_paths := exif.get("lora_paths"): lines.append("") lines.append(f"LoRAs ({len(lora_paths)}):") - lora_scales = exif.get("lora_scales", []) + lora_scales = exif.get("lora_scales") or [] for i, lora in enumerate(lora_paths): scale = lora_scales[i] if i < len(lora_scales) else 1.0 lora_name = Path(lora).name diff --git a/src/mflux/models/common/lora/mapping/lora_loader.py b/src/mflux/models/common/lora/mapping/lora_loader.py index 9632447..4f15826 100644 --- a/src/mflux/models/common/lora/mapping/lora_loader.py +++ b/src/mflux/models/common/lora/mapping/lora_loader.py @@ -174,33 +174,49 @@ class LoRALoader: effective_scale = scale # Create new LoRA layer - if hasattr(current_module, 'weight'): - # Create LoRA layer - lora_layer = LoRALinear.from_linear( - current_module, - r=lora_A.shape[1], - scale=effective_scale - ) - - # Set the LoRA matrices - use the correct dimensions from the LoRA file - lora_layer.lora_A = lora_A - lora_layer.lora_B = lora_B - - # Apply alpha scaling to the matrices if present - if "alpha" in lora_data: - lora_layer.lora_B = lora_layer.lora_B * alpha_scale - + # Check if it's a linear layer (either nn.Linear, LoRALinear, or FusedLoRALinear) + is_linear = hasattr(current_module, 'weight') + is_lora_linear = isinstance(current_module, LoRALinear) + is_fused_linear = isinstance(current_module, FusedLoRALinear) + + if is_linear or is_lora_linear or is_fused_linear: # Handle fusion: if the current module is already a LoRA layer, fuse them - if isinstance(current_module, LoRALinear): + if is_lora_linear: print(f" 🔀 Fusing with existing LoRA at {target_path}") + # Create a temporary LoRA layer from the base linear of the existing LoRA + lora_layer = LoRALinear.from_linear( + current_module.linear, + r=lora_A.shape[1], + scale=effective_scale + ) + # Set the LoRA matrices + lora_layer.lora_A = lora_A + lora_layer.lora_B = lora_B + # Apply alpha scaling to the matrices if present + if "alpha" in lora_data: + lora_layer.lora_B = lora_layer.lora_B * alpha_scale + # Create fused layer with the existing LoRA and the new one fused_layer = FusedLoRALinear( base_linear=current_module.linear, loras=[current_module, lora_layer] ) replacement_layer = fused_layer - elif isinstance(current_module, FusedLoRALinear): + elif is_fused_linear: print(f" 🔀 Adding to existing fusion at {target_path}") + # Create a temporary LoRA layer from the base linear + lora_layer = LoRALinear.from_linear( + current_module.base_linear, + r=lora_A.shape[1], + scale=effective_scale + ) + # Set the LoRA matrices + lora_layer.lora_A = lora_A + lora_layer.lora_B = lora_B + # Apply alpha scaling to the matrices if present + if "alpha" in lora_data: + lora_layer.lora_B = lora_layer.lora_B * alpha_scale + # Add to existing fusion fused_layer = FusedLoRALinear( base_linear=current_module.base_linear, @@ -209,6 +225,19 @@ class LoRALoader: replacement_layer = fused_layer else: # First LoRA on this layer + # Create LoRA layer + lora_layer = LoRALinear.from_linear( + current_module, + r=lora_A.shape[1], + scale=effective_scale + ) + # Set the LoRA matrices - use the correct dimensions from the LoRA file + lora_layer.lora_A = lora_A + lora_layer.lora_B = lora_B + # Apply alpha scaling to the matrices if present + if "alpha" in lora_data: + lora_layer.lora_B = lora_layer.lora_B * alpha_scale + replacement_layer = lora_layer # Replace the layer in the parent module diff --git a/src/mflux/models/qwen/variants/txt2img/qwen_image.py b/src/mflux/models/qwen/variants/txt2img/qwen_image.py index 0d681f1..546fde5 100644 --- a/src/mflux/models/qwen/variants/txt2img/qwen_image.py +++ b/src/mflux/models/qwen/variants/txt2img/qwen_image.py @@ -13,6 +13,7 @@ from mflux.models.qwen.model.qwen_text_encoder.qwen_text_encoder import QwenText from mflux.models.qwen.model.qwen_transformer.qwen_transformer import QwenTransformer from mflux.models.qwen.model.qwen_vae.qwen_vae import QwenVAE from mflux.models.qwen.qwen_initializer import QwenImageInitializer +from mflux.models.qwen.weights.qwen_model_saver import QwenModelSaver from mflux.post_processing.array_util import ArrayUtil from mflux.post_processing.generated_image import GeneratedImage from mflux.post_processing.image_util import ImageUtil @@ -168,6 +169,9 @@ class QwenImage(nn.Module): negative_prompt=negative_prompt, ) + def save_model(self, base_path: str) -> None: + QwenModelSaver.save_model(self, self.bits, base_path) + @staticmethod def _compute_guided_noise( noise: mx.array, diff --git a/src/mflux/models/qwen/weights/qwen_model_saver.py b/src/mflux/models/qwen/weights/qwen_model_saver.py new file mode 100644 index 0000000..be98a63 --- /dev/null +++ b/src/mflux/models/qwen/weights/qwen_model_saver.py @@ -0,0 +1,56 @@ +from pathlib import Path + +import mlx.core as mx +from mlx import nn +from mlx.utils import tree_flatten +from transformers import Qwen2Tokenizer + +from mflux.utils.version_util import VersionUtil + + +class QwenModelSaver: + @staticmethod + def save_model(model, bits: int, base_path: str): + # Save the tokenizer + QwenModelSaver._save_tokenizer(base_path, model.qwen_tokenizer.tokenizer, "tokenizer") + + # Save the models + QwenModelSaver.save_weights(base_path, bits, model.vae, "vae") + QwenModelSaver.save_weights(base_path, bits, model.transformer, "transformer") + QwenModelSaver.save_weights(base_path, bits, model.text_encoder, "text_encoder") + + @staticmethod + def _save_tokenizer(base_path: str, tokenizer: Qwen2Tokenizer, subdir: str): + path = Path(base_path) / subdir + tokenizer.save_pretrained(path) + + @staticmethod + def save_weights(base_path: str, bits: int, model: nn.Module, subdir: str): + path = Path(base_path) / subdir + path.mkdir(parents=True, exist_ok=True) + weights = QwenModelSaver._split_weights(base_path, dict(tree_flatten(model.parameters()))) + for i, weight in enumerate(weights): + mx.save_safetensors( + str(path / f"{i}.safetensors"), + weight, + { + "quantization_level": str(bits), + "mflux_version": VersionUtil.get_mflux_version(), + }, + ) + + @staticmethod + def _split_weights(base_path: str, weights: dict, max_file_size_gb: int = 2) -> list: + # Copied from mlx-examples repo + max_file_size_bytes = max_file_size_gb << 30 + shards = [] + shard, shard_size = {}, 0 + for k, v in weights.items(): + if shard_size + v.nbytes > max_file_size_bytes: + shards.append(shard) + shard, shard_size = {}, 0 + shard[k] = v + shard_size += v.nbytes + shards.append(shard) + return shards + diff --git a/src/mflux/models/qwen/weights/qwen_weight_handler.py b/src/mflux/models/qwen/weights/qwen_weight_handler.py index b70e04d..39b3235 100644 --- a/src/mflux/models/qwen/weights/qwen_weight_handler.py +++ b/src/mflux/models/qwen/weights/qwen_weight_handler.py @@ -56,18 +56,80 @@ class QwenWeightHandler: @staticmethod def load_transformer(root_path: Path) -> tuple[dict, int | None, str | None]: flat = QwenWeightHandler._load_safetensors_shards(root_path / "transformer", loading_mode="multi_glob") + + # Check if this is a saved quantized model (with metadata) or HuggingFace weights + # Saved models from mflux-save are already in MLX structure and don't need manual mapping + quantization_level = None + mflux_version = None + + # Try to get metadata from the first weight shard + import mlx.core as mx + from mlx.utils import tree_unflatten + + file_glob = sorted((root_path / "transformer").glob("*.safetensors")) + if file_glob: + data = mx.load(str(file_glob[0]), return_metadata=True) + if len(data) > 1: + quantization_level = data[1].get("quantization_level") + mflux_version = data[1].get("mflux_version") + + # If this is a saved model (has metadata), use tree_unflatten directly + if quantization_level is not None or mflux_version is not None: + return tree_unflatten(list(flat.items())), quantization_level, mflux_version + + # Otherwise, it's HuggingFace weights that need manual mapping mapped_weights = QwenWeightHandler._manual_transformer_mapping(flat) return mapped_weights, None, None @staticmethod def _load_qwen_text_encoder(root_path: Path) -> tuple[dict, int | None, str | None]: + import mlx.core as mx + from mlx.utils import tree_unflatten + + # Check for saved model metadata FIRST to determine loading mode + quantization_level = None + mflux_version = None + file_glob = sorted((root_path / "text_encoder").glob("*.safetensors")) + if file_glob: + data = mx.load(str(file_glob[0]), return_metadata=True) + if len(data) > 1: + quantization_level = data[1].get("quantization_level") + mflux_version = data[1].get("mflux_version") + + # If this is a saved model, load without expecting index.json + if quantization_level is not None or mflux_version is not None: + all_weights = QwenWeightHandler._load_safetensors_shards( + root_path / "text_encoder", loading_mode="multi_glob" + ) + return tree_unflatten(list(all_weights.items())), quantization_level, mflux_version + + # Otherwise, it's HuggingFace weights that need manual mapping all_weights = QwenWeightHandler._load_safetensors_shards(root_path / "text_encoder", loading_mode="multi_json") mapped_weights = QwenWeightHandler._manual_text_encoder_mapping(all_weights) return mapped_weights, None, None @staticmethod def _load_vae(root_path: Path) -> tuple[dict, int | None, str | None]: + import mlx.core as mx + from mlx.utils import tree_unflatten + weights = QwenWeightHandler._load_safetensors_shards(root_path / "vae", loading_mode="single") + + # Check for saved model metadata + quantization_level = None + mflux_version = None + file_glob = sorted((root_path / "vae").glob("*.safetensors")) + if file_glob: + data = mx.load(str(file_glob[0]), return_metadata=True) + if len(data) > 1: + quantization_level = data[1].get("quantization_level") + mflux_version = data[1].get("mflux_version") + + # If this is a saved model, use tree_unflatten directly + if quantization_level is not None or mflux_version is not None: + return tree_unflatten(list(weights.items())), quantization_level, mflux_version + + # Otherwise, it's HuggingFace weights that need manual mapping and reshaping reshaped_weights = [QwenWeightUtil.reshape_weights(k, v) for k, v in weights.items()] reshaped_weights = QwenWeightUtil.flatten(reshaped_weights) weights = dict(reshaped_weights) diff --git a/src/mflux/post_processing/metadata_builder.py b/src/mflux/post_processing/metadata_builder.py index a0dffbd..9818f86 100644 --- a/src/mflux/post_processing/metadata_builder.py +++ b/src/mflux/post_processing/metadata_builder.py @@ -214,7 +214,8 @@ class MetadataBuilder: Returns: Comma-separated string of LoRA names and scales, or empty string """ - if "lora_paths" not in metadata or not metadata["lora_paths"]: + # Check if lora_paths exists and is not None/empty + if "lora_paths" not in metadata or metadata["lora_paths"] is None or not metadata["lora_paths"]: return "" lora_list = [] diff --git a/src/mflux/save.py b/src/mflux/save.py index 13790b4..2c6ab7d 100644 --- a/src/mflux/save.py +++ b/src/mflux/save.py @@ -1,23 +1,27 @@ from mflux.config.model_config import ModelConfig from mflux.models.flux.variants.txt2img.flux import Flux1 +from mflux.models.qwen.variants.txt2img.qwen_image import QwenImage from mflux.ui.cli.parsers import CommandLineParser def main(): # 0. Parse command line arguments - parser = CommandLineParser(description="Save a quantized version of Flux.1 to disk.") # fmt: off + parser = CommandLineParser(description="Save a quantized version of a model to disk.") # fmt: off parser.add_model_arguments(path_type="save", require_model_arg=True) parser.add_lora_arguments() args = parser.parse_args() - # 1. Load, quantize and save the model - flux = Flux1( + # 1. Determine model class based on model name + model_class = QwenImage if "qwen" in args.model.lower() else Flux1 + + # 2. Load, quantize and save the model + model = model_class( model_config=ModelConfig.from_name(args.model, base_model=args.base_model), quantize=args.quantize, lora_paths=args.lora_paths, lora_scales=args.lora_scales, ) - flux.save_model(args.path) + model.save_model(args.path) if __name__ == "__main__": diff --git a/tests/resources/reference_controlnet_dev_lora_depth.png b/tests/resources/reference_controlnet_dev_lora_depth.png index 8ea75a6..0cd2da4 100644 Binary files a/tests/resources/reference_controlnet_dev_lora_depth.png and b/tests/resources/reference_controlnet_dev_lora_depth.png differ diff --git a/uv.lock b/uv.lock index 76bb96b..2b00033 100644 --- a/uv.lock +++ b/uv.lock @@ -755,7 +755,7 @@ wheels = [ [[package]] name = "mflux" -version = "0.10.0" +version = "0.11.0" source = { editable = "." } dependencies = [ { name = "accelerate" }, @@ -799,8 +799,8 @@ requires-dist = [ { name = "huggingface-hub", specifier = ">=0.24.5,<1.0" }, { name = "matplotlib", specifier = ">=3.9.2,<4.0" }, { name = "matplotlib", marker = "extra == 'dev'", specifier = ">3.10,<4.0" }, - { name = "mlx", specifier = ">=0.27.0,<0.28.0" }, - { name = "mlx", marker = "extra == 'dev'", specifier = "==0.27.1" }, + { name = "mlx", specifier = ">=0.27.0,<0.30.0" }, + { name = "mlx", marker = "extra == 'dev'", specifier = "==0.29.2" }, { name = "numpy", specifier = ">=2.0.1,<3.0" }, { name = "opencv-python", specifier = ">=4.10.0,<5.0" }, { name = "piexif", specifier = ">=1.1.3,<2.0" }, @@ -827,38 +827,38 @@ requires-dist = [ [[package]] name = "mlx" -version = "0.27.1" +version = "0.29.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "mlx-metal", marker = "platform_system == 'Darwin'" }, ] wheels = [ - { url = "https://files.pythonhosted.org/packages/c9/ed/07a7145c500914ed8ab704a7a3828a5134e85f8b4458e1fee887306867d6/mlx-0.27.1-cp310-cp310-macosx_13_0_arm64.whl", hash = "sha256:a033b65fe46425ad5032867d5a71a556a5168108d89aa7092b457556c70d84fc", size = 557679 }, - { url = "https://files.pythonhosted.org/packages/aa/ae/b3e46904ea04b5badb77195169880733f09976a3dfc81c954b99f9adbbc4/mlx-0.27.1-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:2836c6fd9803dc0c6cd06f204e31b3e0191e6c5b6bc8570b28661d926908cba3", size = 530556 }, - { url = "https://files.pythonhosted.org/packages/6e/24/85f5a7dfcf0b4974aa43a2f6dc0d59dfc04e4a1051aacebd7e7810b823ff/mlx-0.27.1-cp310-cp310-macosx_15_0_arm64.whl", hash = "sha256:8b0566054c46d84c470cc99cda2afc3914ad6c7808fbb724dc1ec235e5b2a98c", size = 530559 }, - { url = "https://files.pythonhosted.org/packages/14/bd/91ad900837a3760ba056863c1350f0deabc3e462c3999b4d5ea875368177/mlx-0.27.1-cp310-cp310-manylinux_2_35_x86_64.whl", hash = "sha256:91ef93ce09900c9a8ca662cf34e3c39ab5af2762822ecd6b12fecae518be167f", size = 628183 }, - { url = "https://files.pythonhosted.org/packages/ac/63/88cb9a5019bea21244385eacc4c64deb4d074a8bc3b4b6d2d97cdacf97a2/mlx-0.27.1-cp311-cp311-macosx_13_0_arm64.whl", hash = "sha256:d2e5dedbfbcbe558e51a5c476ca6a18e307676f9e49854eb27e53778bc474699", size = 557582 }, - { url = "https://files.pythonhosted.org/packages/fa/87/c2d7343e7b054481c52e23a190911c40aed4f8c77a8dfeda1f48d5cb520a/mlx-0.27.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:9f04b9778897a879c9ca22e5413dfa1efc192d86d7211b184e079efec49dfb8b", size = 530820 }, - { url = "https://files.pythonhosted.org/packages/06/bf/20497ca7411028baa56372c20e94a3eaddac973155b415f08fc12592c2cf/mlx-0.27.1-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:01d794f9e390438ab4f942a18d9a8ca65bef10c2c2007ef38ca988d039d6d9d3", size = 530821 }, - { url = "https://files.pythonhosted.org/packages/48/68/234a0d846576502ea1bf8ea478c764d87eec9042349e298d2aea58c4e8e9/mlx-0.27.1-cp311-cp311-manylinux_2_35_x86_64.whl", hash = "sha256:fae11432d0639789f1e172b19b35ac8987c8ab9716e55a23fc7a170d6545fc33", size = 628500 }, - { url = "https://files.pythonhosted.org/packages/65/43/125102bbb2be6825880ae2dc8d8702f99cfa7753407f574457b36e422218/mlx-0.27.1-cp312-cp312-macosx_13_0_arm64.whl", hash = "sha256:0c570c9afb57c697bd864504115be8a7c4de97f0b80557a597d496ee426a6812", size = 549869 }, - { url = "https://files.pythonhosted.org/packages/f3/79/0bf681700fc8b165517e907f9ec777b5a5d628004a65a777148f68c6baa0/mlx-0.27.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:ccff7bbbd9df302b26e79013ef6d0c3531c9ba5963ead521e2d85856811b86a0", size = 531671 }, - { url = "https://files.pythonhosted.org/packages/ec/97/f1367b4892bef7f78e38737d3a28094e93124f11684a28a9e92ed5a13b2b/mlx-0.27.1-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:9ccadaed449c07dfeae484620992b904c17dfea7564f8df63095c60eed3af02b", size = 531672 }, - { url = "https://files.pythonhosted.org/packages/9c/0f/3ebec0806ed2c5ba8ba151394cd62a46b4d83ea347da33af91b86d8969d4/mlx-0.27.1-cp312-cp312-manylinux_2_35_x86_64.whl", hash = "sha256:742c413e75605b71db69379a176da63e32ba19b9e9ad03b8763adbd1fcfcd394", size = 621661 }, - { url = "https://files.pythonhosted.org/packages/86/f6/4324386b0764deb692e14a97282a348a9a938aa8b441bf8b6c7599f418d4/mlx-0.27.1-cp313-cp313-macosx_13_0_arm64.whl", hash = "sha256:803669a28031766c2b0fe94c0a3bfd030184e706092f0a831b33620c1e2ef865", size = 549847 }, - { url = "https://files.pythonhosted.org/packages/cf/4b/3194ccb03527a050c04d837d731a11599f8620e6ce16d3971798caae1d44/mlx-0.27.1-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:e9649b3d86ce564000797384510c9d07af38a9ce2a07df8e2f7c6a3e0f0f059e", size = 531664 }, - { url = "https://files.pythonhosted.org/packages/cc/57/a6e0d8dc6e7ba08a64d71fb89d743e77446040113ea1dbb7950be8f60f39/mlx-0.27.1-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:5c501ceec7c6aa2ea1c850dd4e14e679f5416a142264b9d5d405a4e0aeb991b2", size = 531663 }, - { url = "https://files.pythonhosted.org/packages/48/96/6932e0fa866d08cdf77a3fc37cf624e4cac8960d2e297c3fd80afb49417d/mlx-0.27.1-cp313-cp313-manylinux_2_35_x86_64.whl", hash = "sha256:42bfbabdcd0b7aa1c94748c8963e66680a3cf3f599b01036db14e496bf276eb8", size = 621609 }, + { url = "https://files.pythonhosted.org/packages/3f/f0/2c2f99a91ed9dfcc78d31d9e5d3bb2f5305a8d65953cbc41f34f8056c49a/mlx-0.29.2-cp310-cp310-macosx_13_0_arm64.whl", hash = "sha256:b46c1a24b9b8f7145e4d84410552ddfa03f40f9afdbe8f819f6b4b52b4db5d30", size = 547369 }, + { url = "https://files.pythonhosted.org/packages/3b/06/0edddf0a5facb58c17616cd33da6de2722636da8d8d3927272a5a88658e4/mlx-0.29.2-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:18c5b63c6e4b35f75f477f74d3942870e0bc9c9f1c6a071d8a058bbd46681bf4", size = 547367 }, + { url = "https://files.pythonhosted.org/packages/e1/cd/8c089cf1678a752ed4175a7ad5f08823ceef75d797217e12a44a71d6c062/mlx-0.29.2-cp310-cp310-macosx_15_0_arm64.whl", hash = "sha256:ed53b8383ad4ee4311400558e0a5ec61105fc4553950b5732c66e7081cd1a9e8", size = 547365 }, + { url = "https://files.pythonhosted.org/packages/8b/cc/9a14ac57a251f0ee5047f42214fa50d1d5223d805f0b8b6627639c3692eb/mlx-0.29.2-cp310-cp310-manylinux_2_35_x86_64.whl", hash = "sha256:fff27af8dca74546422e8400d32ab848baf42405123cbcfdf62674a9c0560ebe", size = 652302 }, + { url = "https://files.pythonhosted.org/packages/cb/f0/f57349f37cf5dd53f95127e141fc59fc435e4b6bfabba5a84c65de4d3597/mlx-0.29.2-cp311-cp311-macosx_13_0_arm64.whl", hash = "sha256:e74965369227230374b3e8e8c8d46e209e5221a9b76bbb0fa788617e2c68f73c", size = 547581 }, + { url = "https://files.pythonhosted.org/packages/66/04/e016ca28dc9e0738a2541581420125cfe6bba24466a64420600bdd6fd52c/mlx-0.29.2-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:0f79194eeac78e85b96439d3bbc17aae5aba045a2af083c000b4fbbc501f253e", size = 547581 }, + { url = "https://files.pythonhosted.org/packages/1f/b3/e2595e70ef8d4438dff694857745b0e108911e5b5fb83259dde6e5dc5bd1/mlx-0.29.2-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:33bbbb0fd24895d5ff080bb4d10e3e77017bba675d9a12466c8866eaf9b47854", size = 547578 }, + { url = "https://files.pythonhosted.org/packages/11/8c/5d51543ab128c2dff5e4b44ca799db8db5aa4f4ffc34af6531fd73627b54/mlx-0.29.2-cp311-cp311-manylinux_2_35_x86_64.whl", hash = "sha256:32e159f2772be893bec580d2d50c0e6b32ad71a19ded7307bf6c871c8aaa9cf2", size = 651900 }, + { url = "https://files.pythonhosted.org/packages/f3/84/7250237039e91d8e44ca0cf3522f189164844c196f262509afd29ef54710/mlx-0.29.2-cp312-cp312-macosx_13_0_arm64.whl", hash = "sha256:eec950bf7118ad0865d0fc4686bd85d99bf8463fc717d836a5132e1a08b4f129", size = 548336 }, + { url = "https://files.pythonhosted.org/packages/13/47/428ac8d9b0cb5c136e5ce6c726cfdd55caa5b9497dafb6221acfee18f145/mlx-0.29.2-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:bef7333268d6d02e50a9ac6b10f661b711cd02da4a5e2d7619cf198a7e530308", size = 548334 }, + { url = "https://files.pythonhosted.org/packages/14/f0/7d5d3527ca3fdc664c900b4b822028691739e58c8e8f7975b33df4d3536e/mlx-0.29.2-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:f622fc6a84542a08ad2136e9251822d2c08106e5a1a0bd5d249a2d72bccd6577", size = 548330 }, + { url = "https://files.pythonhosted.org/packages/09/18/e202e0f6232822f6768995cdbf50eda202137bb6547368f6e3993dbee00b/mlx-0.29.2-cp312-cp312-manylinux_2_35_x86_64.whl", hash = "sha256:a1aa1aee8e1b6bd1e51361e6b692c70d281b8187b2e859e70ecc11daab306dac", size = 648728 }, + { url = "https://files.pythonhosted.org/packages/a0/9a/91f6f5d031f109fa8c00ba9dd4f7a3fc42e1097a57c26783ce000069c264/mlx-0.29.2-cp313-cp313-macosx_13_0_arm64.whl", hash = "sha256:05ea54173f4bde11b2c93e673d65d72523f5d850f5112d3874156a6fc74ca591", size = 548297 }, + { url = "https://files.pythonhosted.org/packages/2b/2d/dae7ca0b7fa68c6c1f2b896dfe1b8060647f144d5c5da2d53388e38809b1/mlx-0.29.2-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:199dd029b5e55b6d94f1ce366d0137824e46e4333891424dd00413c739f50ae9", size = 548305 }, + { url = "https://files.pythonhosted.org/packages/b1/56/f02f5c9e1fc11c020982501a763fa92b497ea50671a587760543987ba8c8/mlx-0.29.2-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:b6dd4e5f227414882b1676d99250d99389228d1bdc14e4e4e88c95d4903810b7", size = 548302 }, + { url = "https://files.pythonhosted.org/packages/4b/b4/b61eeb92c424947675492dec3a411bdbeae307dfd78162d65ab47e8c3b4f/mlx-0.29.2-cp313-cp313-manylinux_2_35_x86_64.whl", hash = "sha256:c3b9a9aee13f346d060966472954eebe99d9f1b295c9a237c9a000f1ef9adf2c", size = 648709 }, ] [[package]] name = "mlx-metal" -version = "0.27.1" +version = "0.29.2" source = { registry = "https://pypi.org/simple" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8b/77/89fa3327011f018638c9e943e1edc081ce9ca0ed296fe64d6daf93c6ff51/mlx_metal-0.27.1-py3-none-macosx_13_0_arm64.whl", hash = "sha256:c66d9b1adb3c0ea19492fba6493f672bc7542e65dd65f7e2995918815fbeb907", size = 33523035 }, - { url = "https://files.pythonhosted.org/packages/d7/a8/ac706ad6ce834834762d5146d791f77710efc896c13ef47fd7d672099056/mlx_metal-0.27.1-py3-none-macosx_14_0_arm64.whl", hash = "sha256:fe4415ddd242974d91c7ca0699cd01507d17da8a5ba304122ef137cdb5e7fff4", size = 32926383 }, - { url = "https://files.pythonhosted.org/packages/78/77/6963681fb54ecaa0ae5de4209c15504a803a0edd1a33fd074e6c558fd5e0/mlx_metal-0.27.1-py3-none-macosx_15_0_arm64.whl", hash = "sha256:d025dea30bda8baa32c928cfa333eac64a5adc8d07656f8fc55072d99403ebc9", size = 32897065 }, + { url = "https://files.pythonhosted.org/packages/31/a5/a045006546fed791f6e9a74ed4451dac871d3c35f9e54a3a25d820668a85/mlx_metal-0.29.2-py3-none-macosx_13_0_arm64.whl", hash = "sha256:cf8f83a521e620357185c57945142718d526b9312ee112e5a89eb5600480f4d6", size = 35056194 }, + { url = "https://files.pythonhosted.org/packages/4c/8c/4bdd3a7d04ed477b32aec30d30236dfca9f9ac27706cb309511278ddd281/mlx_metal-0.29.2-py3-none-macosx_14_0_arm64.whl", hash = "sha256:fa944001970813b296e8aff5616f2fa9daeda6bc1d190c17fbe8a7ca838ecef0", size = 34791708 }, + { url = "https://files.pythonhosted.org/packages/b1/11/12e158848fe4d3316c999ffb6c2d88f554bde98d69022b3385e25ece997e/mlx_metal-0.29.2-py3-none-macosx_15_0_arm64.whl", hash = "sha256:08d8b7fe305425a14b74ebf36cee176575bfd4cd8d34a2aaae8f05b9983d2d71", size = 34784506 }, ] [[package]]