Seems like stable-diffusion.cpp does not support sdxl_lightning_1step.safetensors right now.

./sd -m SDXL-Lightning/sdxl_lightning_1step.safetensors --vae SDXL-Lightning/sdxl_vae.safetensors -p "1girl, extremely beautiful, miniskirt, nsfw" -H 1024 -W 1024 --cfg-scale 1 --steps 1 -s -1 -v -o test.png

Log:

Option: 
    n_threads:         2
    mode:              txt2img
    model_path:        SDXL-Lightning/sdxl_lightning_1step.safetensors
    wtype:             unspecified
    vae_path:          SDXL-Lightning/sdxl_vae.safetensors
    taesd_path:        
    esrgan_path:       
    controlnet_path:   
    embeddings_path:   
    output_path:       test.png
    init_img:          
    control_image:     
    controlnet cpu:    false
    strength(control): 0.90
    prompt:            1girl, extremely beautiful, miniskirt, nsfw
    negative_prompt:   
    min_cfg:           1.00
    cfg_scale:         1.00
    clip_skip:         -1
    width:             1024
    height:            1024
    sample_method:     euler_a
    schedule:          default
    sample_steps:      1
    strength(img2img): 0.75
    rng:               cuda
    seed:              1768825772
    batch_count:       1
    vae_tiling:        false
    upscale_repeats:   1
System Info: 
    BLAS = 1
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
[DEBUG] stable-diffusion.cpp:122  - Using CUDA backend
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 CUDA devices:
  Device 0: Tesla T4, compute capability 7.5, VMM: yes
[INFO ] stable-diffusion.cpp:142  - loading model from 'SDXL-Lightning/sdxl_lightning_1step.safetensors'
[WARN ] model.cpp:682  - unknown format SDXL-Lightning/sdxl_lightning_1step.safetensors
[ERROR] stable-diffusion.cpp:148  - init model loader from file failed: 'SDXL-Lightning/sdxl_lightning_1step.safetensors'
new_sd_ctx_t failed

This may be because SDXL Lightning 1step version needs "sample" prediction type.

See: https://colab.research.google.com/github/camenduru/SDXL-Lightning-jupyter/blob/main/SDXL_Lightning_gradio_jupyter.ipynb

def generate_image(prompt, ckpt):

    checkpoint = checkpoints[ckpt][0]
    num_inference_steps = checkpoints[ckpt][1] 

    if num_inference_steps==1:
        # Ensure sampler uses "trailing" timesteps and "sample" prediction type for 1-step inference.
        pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing", prediction_type="sample")
    else:
        # Ensure sampler uses "trailing" timesteps.
        pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
        
    pipe.unet.load_state_dict(torch.load(hf_hub_download(repo, checkpoint), map_location="cuda"))
    image = pipe(prompt, num_inference_steps=num_inference_steps, guidance_scale=0).images[0]
    return image

SDXL-Lightning support #198

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions