Canny ControlNet
Video Lecture
| Section | Video Links |
|---|---|
| Canny ControlNet |   |
Video Timings
00:00 Canny ControlNet creates outlines for new images.00:30 Canny edge algorithm detects outlines, best for clear structures.
01:00 In Comfy UI, connect Canny output to Apply ControlNet.
01:30 Use SD1.5 scribble ControlNet for initial image generation.
02:00 Generate reliable 1024x1024 images using ControlNet.
02:30 Experiment with various checkpoints for different output styles.
03:00 Canny on house image; adjust thresholds for outline detail.
03:30 SDXL checkpoints require specific Lora Canny ControlNet.
04:00 Flux needs Flux Canny ControlNet and strength adjustment.
04:30 Pre-generated Canny images can be reused for new creations.
Description
We will use the "Canny" node to create an outline of an image and then generate a new image based on that outline.
Then Canny Edge Detection algorithm, was developed by John F. Canny in 1986. It's a widely-used technique in computer vision to detect edges (outlines) in images.
The Canny edge detector is best suited for images with clear structure, strong contrasts, and well-defined shapes.
Generate A Canny Image
To start, we can get an image of something, and then generate a Canny output of it.
| Original | Canny |
|---|---|
 |
 |
 |
 |
 |
 |
Note that we can tweak the low threshold and high threshold values for stronger or lighter outlines.
Use the Canny Image
Next, create a new workflow and we will use this Canny image to guide the generation of new images.
We will start by using models based on SD1.5.
We need to use the "Apply ControlNet" node, and that will also need to reference a ControlNet model.
| Base | ControlNet Model (Canny) |
|---|---|
| SD1.5 | control_v11p_sd15_scribble_fp16.safetensors |
| SDXL | control-lora-canny-rank256.safetensors |
| Flux | flux-canny-controlnet_v3.safetensors |
Download the required ControlNet model and place it into your ComfyUI controlnet folder.
Optional Extra Checkpoints
SD1.5 Based
dreamCreationVirtual3DECommerce_v10 (civitai)
dreamCreationVirtual3DECommerce_v10 (huggingface)