Front-end clients for AI image models

1.1 Niceties of Apple Silicon

My choice of client is constrained by how I wish to execute the model — itself a trade-off between performance and compatibility. The runtimes (PyTorch + MPS versus MLX) and the weight formats (GGUF versus MLX builds) are the same as for local text models; the twin lays them out as compute backends and storage backends. Among image clients, ComfyUI, InvokeAI and the A1111-family forks take the PyTorch + MPS path, while MLX shows up as add-on node packs (ComfyUI-MLX, mflux-ComfyUI, mflux standalone).

Two backends matter here that barely figure for text generation:

• Apple Core ML — lowest RAM footprint, runs on the Neural Engine, but every model needs offline conversion via coremltools. Mochi Diffusion is the only mainstream consumer AFAIK.
• Custom Swift + Metal kernels — Draw Things sidesteps all of this with its own implementation. Faster than MPS, broader coverage than CoreML, but locked in to a vendor and their assumptions.

3.1 Client-level content filters

Base models and fine-tunes have safety/censorship baked in.

Some clients add a content filter on top of the model’s own behaviour. You may or may not wish this; I never do, but maybe you want to put your laptop in the hands of a pre-schooler and don’t want them generating NSFW images?

The safety_checker used by the diffusers-based clients is a separate model — a small CLIP classifier not baked into the generation model’s weights. Removing it removes the post-generation output filter.

4.1 Walkthrough — adding a CivitAI LoRA on top of Flux

Suppose we want to try YFG Patents, the patent-figure Flux LoRA from the niche-LoRA menu. The full path from a clean install:

1. Install the app — one click from the Mac App Store.
2. Pull a base model — in the Settings tab, click Manage to open the Models panel and select FLUX.1 [dev] from the built-in list; it downloads on selection. The weights arrive pre-converted to the app’s own internal format (a few GB).
3. Pull the LoRA — in that same Models panel, click Import Model → Enter URL… and paste the CivitAI link, setting the type to LoRA. The URL route sometimes fails; if it does, download the .safetensors from CivitAI and use Select from Files instead. Either way the app converts it to its internal format and tags it as a LoRA against the matching backbone (Flux dev in this case). LoRAs from other backbones don’t appear when a Flux model is loaded — fewer ways to wire it up wrong.
4. Generate — the LoRA appears in the LoRA panel with a strength slider. Set it around 0.7–1.0 to start, write a prompt (an exploded view of a kettle, technical diagram, numbered callouts), hit generate.

The same pattern works for Z-Image Turbo LoRAs like Perfect Ink Drawing, with the added payoff that Z-Image Turbo sounds like it might be faster.

5.1 Installation paths

There are two ways onto the platform: a desktop app and a Python source install. They run the same engine and we can share models between them; the difference is in how much of the Python plumbing we touch.

The ComfyUI Desktop DMG is an Electron wrapper around the upstream project. On first launch it asks where to put the install directory, then sets up a self-contained Python environment inside it, downloads PyTorch with MPS support, and brings up the node editor. It is not a sealed bundle — there is an in-app terminal, and we can drop into that environment later to add packages if we ever need to. Models sit wherever we pointed the installer; the install wizard explicitly offers to import an existing source install’s models/, workflows/, and settings/ so a desktop install and a source install can sit side by side without duplicating multi-GB checkpoints. An extra_models_config.yaml lets us point either install at additional model directories anywhere on disk, so several ComfyUI instances can share one model tree and never duplicate a multi-GB checkpoint. That sharing stops at the ComfyUI boundary, though: Draw Things and Mochi Diffusion convert models to their own internal formats on import and keep the converted copies inside their app containers, so anything used in both Draw Things and ComfyUI is stored on disk twice — there is no shared directory bridging a safetensors-native client and a converting one.

The Python source install suits when we want hands-on control of the Python environment — to install experimental custom nodes from git, pin a particular torch build, or untangle a dependency clash by hand. The desktop app keeps that environment hidden; the source install hands over the keys, at the price of having to manage it ourselves. uv makes this latter option relatively easy:

brew install uv
git clone https://github.com/comfyanonymous/ComfyUI && cd ComfyUI
uv venv --python 3.13
source .venv/bin/activate
uv pip install torch torchvision torchaudio
uv pip install -r requirements.txt
uv run main.py

Both paths support ComfyUI Manager for installing custom node packs from the UI. Install it once, restart, and the Manager button appears in the node editor.

The first two examples contrast between speed and flexibility: MLX gives us speed on Apple Silicon but limits which weights we can load and how LoRAs compose; GGUF gives memory headroom for bigger models on smaller machines at the cost of performance.

5.2 Example one — Flux schnell via MLX, with a CivitAI LoRA

Say we want YFG Patents running on top of Flux schnell, accelerated through Apple MLX rather than PyTorch+MPS.

1. Add the MLX node pack. Open Manager → search “ComfyUI-MLX” (thoddnn/ComfyUI-MLX) → install → restart. This pack uses Apple’s DiffusionKit underneath and tends to give ~30–70% wall-clock speedups on Flux compared to PyTorch+MPS. Mflux-ComfyUI is the older alternative; it still works, and is friendlier for people who’d rather not touch the terminal, but ComfyUI-MLX has more momentum as of 2026.
2. Fetch the base model. MLX-flavoured Flux weights are hosted under mlx-community on HuggingFace, separately quantized for the framework — these are not the same files as the regular safetensors Flux releases. Drop them in ComfyUI/models/diffusion_models/ (the MLX loader nodes know where to look).
3. Fetch the LoRA. Download the YFG Patents .safetensors from CivitAI into ComfyUI/models/loras/. A LoRA file is small (often <300 MB) because it only stores low-rank deltas against the backbone’s weights.
4. Wire the workflow. In the editor: MLX Model Loader → MLX LoRA Loader (point at the YFG file, strength ≈ 0.8) → MLX Sampler → VAE Decode → Save Image. Prompt the patent-figure idiom (exploded view, numbered callouts, dashed hidden lines, technical patent figure) and queue.

One quirk: with mflux, LoRA loading and on-the-fly weight quantization are mutually exclusive — if we want a quantized MLX model and a LoRA, we have to bake the LoRA into the weights ahead of time (see the Mflux-ComfyUI readme for a recipe). ComfyUI-MLX handles this more gracefully but is younger.

5.3 Example two — Flux dev via GGUF, on a 16 GB Mac

Flux dev at full precision is 23.8 GB in fp16, which does not fit a 16 GB machine. DiT-backbone models (the diffusion-transformer architecture: Flux, SD3, Qwen-Image) compress well under GGUF, and some fit.

1. Add the GGUF node pack via Manager (search “ComfyUI-GGUF”, city96/ComfyUI-GGUF). The manual route works too:

   cd ComfyUI/custom_nodes
   git clone https://github.com/city96/ComfyUI-GGUF
   uv pip install gguf

2. Download a quantized checkpoint from city96’s HuggingFace repo. The trade-off is roughly: Q4_K_S (6.8 GB, noticeably softer outputs), Q5_K_M (8.3 GB, the usual sweet spot), Q6_K (9.8 GB, near-fp16 quality). Drop the file in ComfyUI/models/unet/. The T5 text encoder is a separate ~9 GB component and can be quantized independently with the matching CLIP loader nodes — we’ll often want a Q5 T5 alongside a Q5 DiT.

3. Wire the workflow. Unet Loader (GGUF) (category bootleg) → DualCLIPLoader (GGUF) → standard KSampler and VAE Decode. A regular CivitAI LoRA (e.g., YFG Patents again) loads on top through the normal LoraLoader node — no bake-in dance needed here, because we’re back in PyTorch+MPS land.

4. On a 32 GB+ machine GGUF is optional — full fp16 fits, and GGUF buys nothing there (its win is memory, not speed). Older PyTorch builds had an MPS buffer bug that broke GGUF on Apple Silicon; current stable resolves it, so no version pin is needed.

5.4 Example three — emoji assets: generate, matte, and export

The niche-LoRA menu notes that no current model emits a clean cut-out icon with transparency: we generate on a flat background and remove it as a separate step. ComfyUI does the whole chain in one pipeline, so we can create an ad hoc emoji factory.

1. Add a background-removal node pack via Manager — 1038lab/ComfyUI-RMBG bundles RMBG-2.0, BiRefNet and SAM, and its GroundingDINO option can cut out a subject named by text.
2. Generate on a flat field. Load Checkpoint (SDXL) → LoraLoader (fofr/sdxl-emoji, strength ≈ 0.8) → KSampler → VAE Decode. Prompt the subject, not a person: an emoji of a teapot, centered, plain white background.
3. Matte. Feed the decoded image into the RMBG node; it returns the subject on transparency. Emoji cut out cleanly because the edges are crisp and the background is flat — the hard cases that defeat background removal do not arise here.
4. Resize and export. Image Resize to the target (128 px for Slack or Discord custom emoji) → Save Image, which preserves the alpha channel as an RGBA PNG. Queue the prompt over a list of subjects to mint a whole pack at once.

For exotic targets — multi-size icon sets, WebP, SVG, .icns — postprocess the matted PNGs using vips or ImageMagick.

5.5 Example four — pose and scribble control (ControlNet)

You want to get the model to do something like “turn my sketch of a castle into a fantasy world”? That is ControlNet. ControlNet conditions the generation of images on various special inputs: given a stick-figure skeleton, or a rough scribble, or a Canny edge map — and it fleshes out a finished image that holds that structure. ComfyUI is the most complete client for it, achieving early support for hip ControlNets first and even supporting multiple controls.

1. Add the preprocessor pack via Manager — ComfyUI-controlnet-aux bundles the hint extractors (DWPose/OpenPose, Canny, HED/soft-edge, lineart, scribble). Skip it if we are drawing the control image by hand.
2. Fetch a ControlNet that matches the base into ComfyUI/models/controlnet/ — for Flux, InstantX Union-Pro (canny / soft-edge / depth / pose in one file); for Qwen-Image, InstantX Qwen Union; for SDXL, the xinsir Union.
3. Make the control image — author it (a stick figure from a pose editor, a scribble) or run a reference photo through the preprocessor node (DWPose Estimator → skeleton, Canny → edges).
4. Wire it. Load Checkpoint → Load ControlNet Model → Apply ControlNet (fed the conditioning, the ControlNet, and the control image; strength ≈ 0.6–0.9, with an end-percent if we want the control to relax late in the denoise) → KSampler → VAE Decode. Prompt the content and style; the skeleton determines the pose.

This strength parameter controls how strong an influence a ControlNet has — low lets the model reinterpret, high forces rigid literalism. On a Mac, Draw Things and ComfyUI both support these controls.

Backbones: SD 1.5, SDXL, SD3/3.5, Flux dev/schnell/2, FLUX.1 Kontext, Qwen-Image 2512, Qwen-Image-Edit 2511, Wan 2.1/2.2, HiDream E1.1, LTX-Video, Hunyuan Video/3D, Omnigen 2, ACE Step audio. LoRA, ControlNet, inpainting, img2img, regional prompting all native.

6.1 Walkthrough — emoji-style edits on a portrait

A workflow that exercises what InvokeAI is for: pull a portrait onto the canvas, mask a region, and apply fofr/sdxl-emoji (from the niche-LoRA menu) to that region only. This is the kind of selective re-style that is awkward in Draw Things and tedious to wire by hand in ComfyUI.

1. Install via the official launcher. The launcher is a small Electron app that manages the underlying Python install for us — it provisions a venv, pulls the right PyTorch wheel for Apple Silicon (MPS), and tracks updates separately from the app itself. Source install works too (git clone then uv pip install -e .), but the launcher is what the team supports and where the model importer is.
2. Pull a base model through the launcher’s built-in Model Manager. SDXL base is the sensible match for the fofr LoRA, since the LoRA was trained on SDXL. An SDXL LoRA cannot be stacked on a Flux base; the rank-decomposed weights only make sense against the backbone they were trained on. The Model Manager downloads HuggingFace diffusers-format weights directly; CivitAI safetensors import works through URL paste.
3. Pull the LoRA. Either drop fofr’s pytorch_lora_weights.safetensors into ~/invokeai/models/sdxl/lora/, or paste the HuggingFace URL into the Model Manager. InvokeAI identifies the architecture from the file’s metadata and rejects it cleanly if it doesn’t match the loaded base.
4. Use the canvas. Open the Unified Canvas tab, drag a portrait onto it, draw a mask around the face, and add the LoRA to the prompt stack (strength ~0.8 to start). The inpaint sampler respects the mask boundary while letting the LoRA drive style inside it — the canvas affordance Draw Things lacks.

The same pattern handles outpainting (extend canvas, mask the new area, generate), Kontext-based instruction edits (make this a watercolour), and SAM/SAM2-driven auto-segmentation (Segment Anything, Meta’s click-to-select segmentation model) when we can’t be bothered to draw the mask ourselves.

9.1 ChaiNNer

A node-based GUI specifically for non-diffusion image-processing pipelines: Real-ESRGAN → GFPGAN → format conversion → batch over a folder. Electron app + Python sidecar with PyTorch+MPS; works on Apple Silicon. Maintained.

Pareto dominates when: we want repeatable upscale/restore pipelines without writing a script. Complementary to ComfyUI rather than competitive — if the task is diffusion, this is the wrong tool.

Affordances: Spandrel-supported PyTorch arches (super-resolution, face restore, denoise, JPEG-deartefact, dehaze, low-light, colourisation), plus NCNN/ONNX/TensorRT model files. Loads .pth, .pt, .safetensors, some .ckpt.

Source: chaiNNer-org/chaiNNer.

9.2 Spandrel

ChaiNNer’s Python core, also usable on its own. Spandrel on PyPI. Useful if we want ChaiNNer’s arch coverage from a script rather than a node graph.

Catalogue: ESRGAN family, SwinIR, HAT, Real-ESRGAN, AuraSR plus ~20 other super-resolution arches; GFPGAN/CodeFormer/RestoreFormer face restorers; LaMa/MAT inpainting; NAFNet/SCUNet/Restormer denoisers; DDColor; DeJPEG. No diffusion, no Flux, no Qwen.

10.1 Runway

Runway pivoted to video around 2024 and the image-generation product line is no longer a marketed offering. Current focus: Gen-4.5 video, GWM-1 world models, real-time video agents (Runway Characters). The Photoshop and Blender plugins still exist.

Pareto dominates when: we are doing video and world-model work; for image-only workflows there are cheaper and more focused options elsewhere.

10.2 Midjourney

Midjourney — currently spanning V7 and the V8 generation (V8 shipped 2026: claimed ~5× faster than V7, native 2K output, image and video). Browser/Discord interface; AFAICT still no public API. $10–$120/month subscription.

Pareto dominates when: we want the prompt-craft aesthetic frontier. It is addictive in that we can get better at it, which feels like mastering a real skill. Unsuitable for programmatic pipelines because there’s no API.