Front-end clients for AI image models

3.1.1 Walkthrough — adding a CivitAI LoRA on top of Flux

Suppose we want to try YFG Patents, the patent-figure Flux LoRA from the niche-LoRA menu. The full path from a clean install:

1. Install the app — one click from the Mac App Store. The binary is signed, sandboxed, and has a download menu for the models it supports.
2. Pull a base model — in the Settings tab, click Manage to open the Models panel and pick FLUX.1 [dev] from the built-in list; it downloads on selection. The weights arrive pre-converted to the app’s own internal format (a few GB), with no safetensors → diffusers → CoreML dance like Mochi Diffusion demands.
3. Pull the LoRA — in that same Models panel, click Import Model → Enter URL… and paste the CivitAI link, setting the type to LoRA. The URL route sometimes fails; if it does, download the .safetensors from CivitAI and use Select from Files instead. Either way the app converts it to its internal format and tags it as a LoRA against the matching backbone (Flux dev in this case). LoRAs from other backbones don’t appear when a Flux model is loaded — fewer ways to wire it up wrong.
4. Generate — the LoRA appears in the LoRA panel with a strength slider. Set it around 0.7–1.0 to start, write a prompt (an exploded view of a kettle, technical diagram, numbered callouts), hit generate.

The same pattern works for Z-Image Turbo LoRAs like Perfect Ink Drawing, with the added payoff that Z-Image Turbo sounds like it might be faster.

Hugging Face offers a slicker route for anything hosted there: a compatible model page has a Use this model → Draw Things button (part of HF’s local-apps integration) that deeplinks the weights straight into the app, skipping the manual import above. CivitAI has no such hook, so its models and LoRAs go through Import Model by hand.

Backbones: SD 1.5, SDXL, SD3, Flux dev/schnell, FLUX.1 Kontext for instruction edits, Qwen-Image-Edit 2509/2511 and Qwen-Image-Layered, Wan video (multiple variants), HiDream, Z-Image Turbo.

Affordances: LoRA loading and local LoRA training, ControlNet, inpainting, outpainting, infinite canvas. Memory pressure is managed automatically — Draw Things falls back to a smaller working set rather than OOM, at some cost to throughput. 5–7 GB of working memory typical; runs comfortably on a 16 GB Mac.

Failure mode: lags ComfyUI by weeks, not months, on genuinely novel architectures. We pay for the polish with closed source.

3.2.1 Two installation paths

There are two ways onto the platform: a desktop app and a Python source install. They run the same engine and we can share models between them; the difference is in how much of the Python plumbing we touch.

The ComfyUI Desktop DMG is an Electron wrapper around the upstream project. On first launch it asks where to put the install directory, then sets up a self-contained Python environment inside it, downloads PyTorch with MPS support, and brings up the node editor. It is not a sealed bundle — there is an in-app terminal, and we can drop into that environment later to add packages if we ever need to. Models sit wherever we pointed the installer; the install wizard explicitly offers to import an existing source install’s models/, workflows/, and settings/ so a desktop install and a source install can sit side by side without duplicating multi-GB checkpoints. An extra_models_config.yaml lets us point either install at additional model directories anywhere on disk, so several ComfyUI instances can share one model tree and never duplicate a multi-GB checkpoint. That sharing stops at the ComfyUI boundary, though: Draw Things and Mochi Diffusion convert models to their own internal formats on import and keep the converted copies inside their app containers, so anything used in both Draw Things and ComfyUI is stored on disk twice — there is no shared directory bridging a safetensors-native client and a converting one.

The Python source install suits us when we want hands-on control of the Python environment — to install experimental custom nodes from git, pin a particular torch build, or untangle a dependency clash by hand. The desktop app keeps that environment hidden and managed for us; the source install hands over the keys, at the price of having to manage it ourselves. If the words virtual environment and dependency resolver already sound like a threat, the Python packaging notebook is the place to start; the short version is that uv makes this about as painless as it gets on a fresh Mac:

brew install uv
git clone https://github.com/comfyanonymous/ComfyUI && cd ComfyUI
uv venv --python 3.13
source .venv/bin/activate
uv pip install torch torchvision torchaudio
uv pip install -r requirements.txt
uv run main.py

If a custom-node pack refuses to build, dropping to the previous Python minor version usually clears it; stable PyTorch has solid Apple Silicon support now, so there’s no need to chase nightly builds.

Both paths support ComfyUI Manager for installing custom node packs from the UI. Install it once, restart, and the Manager button appears in the node editor.

3.2.2 Example one — Flux schnell via MLX, with a CivitAI LoRA

Say we want YFG Patents running on top of Flux schnell, accelerated through Apple MLX rather than PyTorch+MPS.

1. Add the MLX node pack. Open Manager → search “ComfyUI-MLX” (thoddnn/ComfyUI-MLX) → install → restart. This pack uses Apple’s DiffusionKit underneath and tends to give ~30–70% wall-clock speedups on Flux compared to PyTorch+MPS. Mflux-ComfyUI is the older alternative; it still works, and is friendlier for people who’d rather not touch the terminal, but ComfyUI-MLX has more momentum as of 2026.
2. Fetch the base model. MLX-flavoured Flux weights are hosted under mlx-community on HuggingFace, separately quantized for the framework — these are not the same files as the regular safetensors Flux releases. Drop them in ComfyUI/models/diffusion_models/ (the MLX loader nodes know where to look).
3. Fetch the LoRA. Download the YFG Patents .safetensors from CivitAI into ComfyUI/models/loras/. A LoRA file is small (often <300 MB) because it only stores low-rank deltas against the backbone’s weights.
4. Wire the workflow. In the editor: MLX Model Loader → MLX LoRA Loader (point at the YFG file, strength ≈ 0.8) → MLX Sampler → VAE Decode → Save Image. Prompt the patent-figure idiom (exploded view, numbered callouts, dashed hidden lines, technical patent figure) and queue.

One quirk: with mflux, LoRA loading and on-the-fly weight quantization are mutually exclusive — if we want a quantized MLX model and a LoRA, we have to bake the LoRA into the weights ahead of time (see the Mflux-ComfyUI readme for the recipe). ComfyUI-MLX handles this more gracefully but is younger.

3.2.3 Example two — Flux dev via GGUF, on a 16 GB Mac

Flux dev at full precision is 23.8 GB in fp16, which does not fit a 16 GB machine. GGUF quantization is what makes it fit at all. DiT-backbone models (the diffusion-transformer architecture: Flux, SD3, Qwen-Image) compress well under GGUF; the older UNet-backbone models (SD 1.5, SDXL) do not benefit as much.

1. Add the GGUF node pack via Manager (search “ComfyUI-GGUF”, city96/ComfyUI-GGUF). The manual route works too:

   cd ComfyUI/custom_nodes
   git clone https://github.com/city96/ComfyUI-GGUF
   uv pip install gguf

2. Download a quantized checkpoint from city96’s HuggingFace repo. The trade-off is roughly: Q4_K_S (6.8 GB, noticeably softer outputs), Q5_K_M (8.3 GB, the usual sweet spot), Q6_K (9.8 GB, near-fp16 quality). Drop the file in ComfyUI/models/unet/. The T5 text encoder is a separate ~9 GB component and can be quantized independently with the matching CLIP loader nodes — we’ll often want a Q5 T5 alongside a Q5 DiT.

3. Wire the workflow. Unet Loader (GGUF) (category bootleg) → DualCLIPLoader (GGUF) → standard KSampler and VAE Decode. A regular CivitAI LoRA (e.g., YFG Patents again) loads on top through the normal LoraLoader node — no bake-in dance needed here, because we’re back in PyTorch+MPS land.

4. On a 32 GB+ machine GGUF is optional — full fp16 fits, and GGUF buys nothing there (its win is memory, not speed). Older PyTorch builds had an MPS buffer bug that broke GGUF on Apple Silicon; current stable resolves it, so no version pin is needed.

The first two examples contrast deliberately: MLX gives speed on Apple Silicon but limits which weights we can load and how LoRAs compose; GGUF gives memory headroom for bigger models on smaller machines at the cost of staying in the PyTorch lane. Sometimes we want both, sometimes neither.

3.2.4 Example three — emoji assets: generate, matte, and export

The niche-LoRA menu notes that no current model emits a clean cut-out icon with transparency: we generate on a flat background and remove it as a separate step. ComfyUI does the whole chain in one graph, which is what makes it the right tool for minting a set of emoji rather than a one-off.

1. Add a background-removal node pack via Manager — 1038lab/ComfyUI-RMBG bundles RMBG-2.0, BiRefNet and SAM, and its GroundingDINO option can cut out a subject named by text.
2. Generate on a flat field. Load Checkpoint (SDXL) → LoraLoader (fofr/sdxl-emoji, strength ≈ 0.8) → KSampler → VAE Decode. Prompt the subject, not a person: an emoji of a teapot, centered, plain white background.
3. Matte. Feed the decoded image into the RMBG node; it returns the subject on transparency. Emoji cut out cleanly because the edges are crisp and the background is flat — the hard cases that defeat background removal do not arise here.
4. Resize and export. Image Resize to the target (128 px for Slack or Discord custom emoji) → Save Image, which preserves the alpha channel as an RGBA PNG. Queue the prompt over a list of subjects to mint a whole pack at once.

For exotic targets — multi-size icon sets, WebP, SVG, .icns — hand the matted PNGs to vips or ImageMagick as a final non-diffusion step.

Backbones: SD 1.5, SDXL, SD3/3.5, Flux dev/schnell/2, FLUX.1 Kontext, Qwen-Image, Qwen-Image-Edit, Wan 2.1/2.2, HiDream E1.1, LTX-Video, Hunyuan Video/3D, Omnigen 2, ACE Step audio. LoRA, ControlNet, inpainting, img2img, regional prompting all native.

Affordances: training is out of scope (we might use kohya for that?) Failure mode for unknown CivitAI checkpoints: usually “find the right node pack in the manager”, and they almost always load eventually.

3.3.1 Walkthrough — emoji-style edits on a portrait

A workflow that exercises what InvokeAI is for: pull a portrait onto the canvas, mask a region, and apply fofr/sdxl-emoji (from the niche-LoRA menu) to that region only. This is the kind of selective re-style that is awkward in Draw Things and tedious to wire by hand in ComfyUI.

1. Install via the official launcher. The launcher is a small Electron app that manages the underlying Python install for us — it provisions a venv, pulls the right PyTorch wheel for Apple Silicon (MPS), and tracks updates separately from the app itself. Source install works too (git clone then uv pip install -e .), but the launcher is what the team supports and where the model importer is.
2. Pull a base model through the launcher’s built-in Model Manager. SDXL base is the sensible match for the fofr LoRA, since the LoRA was trained on SDXL. An SDXL LoRA cannot be stacked on a Flux base; the rank-decomposed weights only make sense against the backbone they were trained on. The Model Manager downloads HuggingFace diffusers-format weights directly; CivitAI safetensors import works through URL paste.
3. Pull the LoRA. Either drop fofr’s pytorch_lora_weights.safetensors into ~/invokeai/models/sdxl/lora/, or paste the HuggingFace URL into the Model Manager. InvokeAI identifies the architecture from the file’s metadata and rejects it cleanly if it doesn’t match the loaded base.
4. Use the canvas. Open the Unified Canvas tab, drag a portrait onto it, draw a mask around the face, and add the LoRA to the prompt stack (strength ~0.8 to start). The inpaint sampler respects the mask boundary while letting the LoRA drive style inside it — the canvas affordance Draw Things lacks.

The same pattern handles outpainting (extend canvas, mask the new area, generate), Kontext-based instruction edits (make this a watercolour), and SAM/SAM2-driven auto-segmentation (Segment Anything, Meta’s click-to-select segmentation model) when we can’t be bothered to draw the mask ourselves.

Backbones: SD 1.5/2.0, SDXL, SD 3.5 Medium/Large, Flux dev/schnell/Kontext/Krea/Redux/Fill, Flux 2 Klein 4B/9B, Qwen Image, Qwen Image Edit, CogView 4, Z-Image. GGUF, ckpt, and diffusers formats all loadable.

Affordances: LoRA + embeddings, SAM/SAM2 segmentation, unified canvas inpainting/outpainting. No video, no training.

Failure mode: the model picker rejects an unknown architecture cleanly rather than crashing — we know immediately whether a CivitAI download will work, which is more than ComfyUI offers when a custom node pack is missing.

3.4.1 Walkthrough — running a pre-converted SDXL bundle

The supported path is to take someone else’s CoreML conversion off the shelf. Self-conversion is its own adventure, covered in CoreML conversion in practice below; we won’t need it for this walkthrough.

1. Install the app by downloading the latest .dmg from the GitHub releases page and dragging it to Applications. The binary is unsigned outside the App Store route, so the first launch requires a right-click → Open to get past Gatekeeper.
2. Grab a pre-converted bundle. The CoreML format is not interchangeable with safetensors; a CivitAI checkpoint simply won’t load. The community curates pre-converted bundles at coreml-community on HuggingFace — ~145 of them covering SD 1.5, SD 2.1, SDXL and its art-style forks (Animagine XL, AnythingXL, CounterfeitXL), plus ControlNets in both attention modes. For an official SDXL base, apple/coreml-stable-diffusion-xl-base is the canonical source.
3. Pick the right attention mode. Each bundle ships in one of two flavours: ORIGINAL targets CPU+GPU (Metal) and is the right choice for Macs; SPLIT_EINSUM targets the Neural Engine and only makes sense on iPhone/iPad or for very small models. For SDXL on Mac, always pick ORIGINAL — the SDXL UNet is too large for the Neural Engine (ANE) to hold the weights anyway.
4. Install the bundle by copying the bundle directory (the one containing Resources/) into Mochi’s models folder — by default ~/Documents/MochiDiffusion/models/.
5. Generate. The bundle now appears in Mochi’s model dropdown. The Neural Engine inference is fast and the RAM footprint stays tiny; we can run it alongside a heavier ComfyUI workflow without pushing the machine into swap.

The reasons not to pick this client for general work are stacked up against it: no LoRA loading, no native inpainting, no Flux dev/schnell, no Kontext, no Qwen, no SD3, no video, no direct CivitAI import. We come here when RAM headroom matters more than any of those. Self-conversion of fresh CivitAI checkpoints is doable for SDXL fine-tunes but rough; for anything Flux-class or newer, the converter doesn’t know the architecture and we should use Draw Things instead.

Backbones: SD 1.5, SD 2.x, SDXL, plus Flux 2 Klein (4B/9B distilled, pre-built bundles available).

Affordances: ControlNet, RealESRGAN upscaling, EXIF metadata preservation in generated PNGs.