Skip to content

FLUX IP-Adapter

Video Lecture

Section Video Links
Flux IP-Adapter Flux IP-Adapter Flux IP-Adapter
Video Outline Introduces IP Adapter with Flux for image-guided generation.
Explains installing the ComfyUI IP adapter Flux custom node.
Guides downloading and placing the `ip_adapter.bin` model file.
Shows improving text prompts for better initial results.
Demonstrates integrating the IP adapter into the workflow.
Mentions first-time download of additional 3.5GB SIGLIP files.
Notes GPU/CPU options for Flux model loading.
Shows how IP adapter improves image styling significantly.
Advises using Flux-generated images as input for best results.
Explores generating images with or without an open pose.

Description

Using the IP-Adapter with Flux Dev & Schnell.

Install the Custom Node

Install the custom node ComfyUI-IPAdapter-Flux using the manager, or you can use your command/terminal prompt.

  1. Navigate to your ComfyUI/custom_nodes folder. (On Windows & macOS Desktop Versions Documents/ComfyUI/custom_nodes)
  2. Run, 
    git clone https://github.com/Shakker-Labs/ComfyUI-IPAdapter-Flux.git
  3. Restart ComfyUI

Install the IP-Adapter model

Download ip-adapter.bin.

Create a new folder under ComfyUI/models/ named ipadapter-flux.

Save ip-adapter.bin in the new folder.

📂 ComfyUI/
├── 📂 models/
│   ├── 📂 ipadapter-flux/
│   │   └── ip-adapter.bin

Workflow Assets

Start Workflow Image Prompt

Poses

Pose Image Prompt
A highly fashionable model standing in front of a white wall, wearing nice shoes, looking happy, relaxed and optimistic.
A highly fashionable model smiling at the camera showing their hands folded into fists.
4 pictures in a grid with 4 distinct poses, top left smile, top right peace sign, bottom left silly, bottom right fun

Tip

The Flux IP-Adapter does not produce perfect results based on the input image. But you can get quite close. You will need to work on your prompt as best you can. Once you have a good prompt, continue to generate images using random seeds, until you get the best result you can. Once you produce a good match, then take note of that seed, set the KSampler to this seed, and set it as fixed, and then continue to reuse this same seed in future generations. If you don't like the output at any time, don't change the seed, but try tweaking the prompt instead. For best consistency in your images, keep your seed fixed once you've found a good reliable one.

WGET Commands

If you are using Runpod, or a similar hosted GPU service, then you can access your running pod/instance using a terminal.

#
#
#
#
# CD into ./ComfyUI/models/ folder
mkdir ipadapter-flux
cd ipadapter-flux
wget https://huggingface.co/InstantX/FLUX.1-dev-IP-Adapter/resolve/main/ip-adapter.bin

Wait for files to download fully before running your workflows.

Troubleshooting

Doesn't work on Windows Desktop version

These instructions are for Windows Desktop only and will only work if you've installed python and git on your system.

Open a Windows cmd into C:\Users\[your username]\Documents\ComfyUI\custom_nodes

git clone https://github.com/Shakker-Labs/ComfyUI-IPAdapter-Flux.git

Restart ComfyUI. Toggle bottom panel. If you see an error about diffusers, then open a Windows cmd into C:\Users\[your username]\Documents\ComfyUI:

.venv\Scripts\activate.bat

You should get no error and your console prompt should have the (ComfyUI) label visible:

(ComfyUI) C:\Users\[your username]\Documents\ComfyUI>

Now in the same folder enter:

python -m pip install -r ./custom_nodes/ComfyUI-IPAdapter-Flux/requirements.txt

Restart ComfyUI and you should be able to double click workspace and add search for a node named Apply IPAdapter Flux Model

AttributeError: 'DoubleStreamBlock' object has no attribute 'flipped_img_txt'

If you get an error, 'DoubleStreamBlock' object has no attribute 'flipped_img_txt' when trying to generate, then you can:

  1. Open ComfyUI/custom_nodes/ComfyUI-IPAdapter-Flux/flux/layers.py

  2. Change line 31 from

self.flipped_img_txt = original_block.flipped_img_txt

to

self.flipped_img_txt = getattr(original_block, 'flipped_img_txt', False)
  1. Restart ComfyUI and try again

TypeError: forward_orig_ipa() got an unexpected keyword argument 'timestep_zero_index'

  1. Open ComfyUI/custom_nodes/ComfyUI-IPAdapter-Flux/utils.py

  2. Add these lines to the forward_orig_ipa definition like this below, (do not add the + symbol)

def forward_orig_ipa(
    self,
    img: Tensor,
    img_ids: Tensor,
    txt: Tensor,
    txt_ids: Tensor,
    timesteps: Tensor,
    y: Tensor,
    guidance: Tensor|None = None,
    control=None,
+   timestep_zero_index=None,
    transformer_options={},
    attn_mask: Tensor = None,
+   **kwargs,
) -> Tensor:
  1. Restart ComfyUI and try again