Skip to content

Conversation

@Pizzawookiee
Copy link

@Pizzawookiee Pizzawookiee commented Oct 27, 2025

Adds the nodes 'NunchakuSDXLUNetLoader', 'NunchakuSDXLLoraLoader' and 'NunchakuSDXLLoraStack'. Does not modify the base nunchaku library.

The file models/sdxl.py converts the state dictionary from the existing nunchaku SDXL checkpoint on huggingface into one that is compatible with ComfyUI's UNetModel, which is distinct from the diffusers 2DUNetConditionModel.

Compare this image generated with the new node (should be able to reproduce my workflow by dragging it into ComfyUI):
Test_image

With one copied from https://comfyanonymous.github.io/ComfyUI_examples/sdturbo/:
sdxlturbo_example

Update 25/10/29:
LoRA nodes implemented. Includes "NunchakuSDXLLoraLoader" and "NunchakuSDXLLoraStack". Because I converted everything to work with ComfyUI's UNetModel (and not diffusers 2DUNetConditionModel used in the base nunchaku repo), my LoRA implementation does not require any update to nunchaku library.

LoRA and LyCORIS should work. DoRA is not supported right now, I'm not sure how to apply the magnitude_vector DoRA layer to the quantized qweight Nunchaku layer.

From my testing, most LoRAs work but some don't. For example both the Lightning and LCM LoRAs cause horrible image quality.

Additionally, it appears ControlNet sort of works (can use standard ComfyUI Controlnet nodes, and sometimes get okay results), but see Known Issues at bottom.

Update 25/10/30:
First-block cache implemented.

Known issues:

  1. In most ControlNet workflows I run into a "CUDA error: an illegal memory access was encountered".

@Pizzawookiee Pizzawookiee changed the title SDXL + SDXL Turbo implementation SDXL + SDXL Turbo UNetLoader implementation Oct 27, 2025
@Pizzawookiee Pizzawookiee changed the title SDXL + SDXL Turbo UNetLoader implementation SDXL + SDXL Turbo UNetLoader support Oct 27, 2025
@Pizzawookiee Pizzawookiee changed the title SDXL + SDXL Turbo UNetLoader support SDXL + SDXL Turbo UNetLoader, LoRA support Oct 29, 2025
@Pizzawookiee Pizzawookiee changed the title SDXL + SDXL Turbo UNetLoader, LoRA support SDXL + SDXL Turbo UNetLoader, LoRA, ControlNet support Oct 30, 2025
@Pizzawookiee Pizzawookiee changed the title SDXL + SDXL Turbo UNetLoader, LoRA, ControlNet support SDXL + SDXL Turbo UNetLoader, LoRA, FB Cache, ControlNet support Oct 30, 2025
Due to the way I modified variables, the garbage collecting broke within the sdxl loading node. So the temp fix is to comment it out.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant