Add Timestep shift, SGM Uniform and Simple scheduler and support for NitroFusion #675
+240
−44
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Tested simple in isolation, and it works 👍 |
Green-Sky
reviewed
Jul 27, 2025
|
|
||
| result_sigmas.reserve(n + 1); | ||
|
|
||
| int model_sigmas_len = TIMESTEPS; |
|
Please rebase and update this pr on master. It is not nice to test right now. I am currently trying to get sgm to work. :) |
|
fixes #198 |
I'll try to do it next week. I think the changes are small enough to be integrated into a single PR, but if that's not preferred, I can make a separate PR for the new schedulers and the timestep shift |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for the timestep-shift technique required for inference with NitroFusion models and Diff-Instruct* and other one-step models. It also adds support for two schedulers: SGM Uniform and Simple, because the existing scheduler, for a mysterious reason, fails at step 2 calculation and produces an output similar to step 1.
NitroFusion is one of the best models for single-step inference, making it useful for inference on compute-constrained devices like mobile phones or CPUs.
Example command:
./bin/sd -m nitrosd-realism_f16.gguf -v -p "cute cat" --cfg-scale 1 --steps 1 --timestep-shift 250 -H 1024 -W 1024 --seed 2024 --schedule sgm_uniformThe recommended timestep-shift values by the authors are 250 for NitroSD-Realism and 500 for NitroSD-Vibrant and 400 for Diff-Instruct*.
I created GGUF versions of NitroFusion that already include the fixed SDXL VAE, available for download here.
The authors mentioned it's possible to extract LoRA weights from these models and apply them to other checkpoints. I’ll try to do that in the future.
EDIT : Just add Diff-Instruct* GGUF
References:
Timestep-shift implementation: node.py
SGM Uniform: sd_schedulers
Simple: samplers.py