non-square aspect ratio results in failure (although have installed all the suitable version)

[solitaryTian]

I have installed all the package and the version is the same as you show. But I still have this error.

(hi_diff) (base) [root@node58 HiDiffusion-main]# conda list

packages in environment at /root/miniconda3/envs/hi_diff:
Name Version Build Channel
_libgcc_mutex 0.1 main https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
_openmp_mutex 5.1 1_gnu https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
accelerate 0.18.0 pypi_0 pypi
ca-certificates 2024.3.11 h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
certifi 2024.2.2 pypi_0 pypi
charset-normalizer 3.3.2 pypi_0 pypi
diffusers 0.27.0 pypi_0 pypi
filelock 3.14.0 pypi_0 pypi
fsspec 2024.3.1 pypi_0 pypi
hidiffusion 0.1.6 pypi_0 pypi
huggingface-hub 0.22.2 pypi_0 pypi
idna 3.7 pypi_0 pypi
importlib-metadata 7.1.0 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi 3.4.4 h6a678d5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgcc-ng 11.2.0 h1234567_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgomp 11.2.0 h1234567_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libstdcxx-ng 11.2.0 h1234567_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mypy-extensions 1.0.0 pypi_0 pypi
ncurses 6.4 h6a678d5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy 1.24.4 pypi_0 pypi
nvidia-cublas-cu11 11.10.3.66 pypi_0 pypi
nvidia-cuda-nvrtc-cu11 11.7.99 pypi_0 pypi
nvidia-cuda-runtime-cu11 11.7.99 pypi_0 pypi
nvidia-cudnn-cu11 8.5.0.96 pypi_0 pypi
openssl 3.0.13 h7f8727e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
packaging 24.0 pypi_0 pypi
pillow 10.3.0 pypi_0 pypi
pip 23.3.1 py38h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
psutil 5.9.8 pypi_0 pypi
pyre-extensions 0.0.23 pypi_0 pypi
python 3.8.19 h955ad1f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h5eee18b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
regex 2024.4.28 pypi_0 pypi
requests 2.31.0 pypi_0 pypi
safetensors 0.4.3 pypi_0 pypi
setuptools 68.2.2 py38h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite 3.41.2 h5eee18b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tk 8.6.12 h1ccaba5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tokenizers 0.13.3 pypi_0 pypi
torch 1.13.1 pypi_0 pypi
tqdm 4.66.2 pypi_0 pypi
transformers 4.27.4 pypi_0 pypi
typing-extensions 4.11.0 pypi_0 pypi
typing-inspect 0.9.0 pypi_0 pypi
urllib3 2.2.1 pypi_0 pypi
wheel 0.41.2 py38h06a4308_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
xformers 0.0.16rc425 pypi_0 pypi
xz 5.4.6 h5eee18b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zipp 3.18.1 pypi_0 pypi
zlib 1.2.13 h5eee18b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main

error
File "test.py", line 53, in
image = pipe(prompt, guidance_scale=7.5, height=h, width=w, eta=1.0, generator=generator).images[0]
File "/root/miniconda3/envs/hi_diff/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/hi_diff/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 971, in call
noise_pred = self.unet(
File "/root/miniconda3/envs/hi_diff/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1212, in _call_impl
result = forward_call(*input, **kwargs)
File "/root/miniconda3/envs/hi_diff/lib/python3.8/site-packages/diffusers/models/unets/unet_2d_condition.py", line 1284, in forward
sample = upsample_block(
File "/root/miniconda3/envs/hi_diff/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/hi_diff/lib/python3.8/site-packages/diffusers/models/unets/unet_2d_blocks.py", line 2512, in forward
hidden_states = torch.cat([hidden_states, res_hidden_states], dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 16 but got size 15 for tensor number 1 in the list.

[ShenZhang-Shin]

I see, man. Could you let me know the resolution and model you are using?

[solitaryTian]

I see, man. Could you let me know the resolution and model you are using?

pretrain_model = "/dfs/comicai/zhiyuan.shi/models/stable-diffusion-v1-5"
h, w = 960, 512

the whole code is as follows:

from hidiffusion import apply_hidiffusion, remove_hidiffusion
from diffusers import DiffusionPipeline, DDIMScheduler, EulerAncestralDiscreteScheduler
import torch

pretrain_model = "/dfs/comicai/bin.hu/hub/darkSushiMixMix_225D"

pretrain_model = "/dfs/comicai/zhiyuan.shi/models/stable-diffusion-v1-5"
scheduler = EulerAncestralDiscreteScheduler.from_pretrained(pretrain_model, subfolder="scheduler")
pipe = DiffusionPipeline.from_pretrained(pretrain_model, scheduler = scheduler, torch_dtype=torch.float16, load_safety_checker=False).to("cuda")
pipe.safety_checker = None

# Optional. enable_xformers_memory_efficient_attention can save memory usage and increase inference speed. enable_model_cpu_offload and enable_vae_tiling can save memory usage.

pipe.enable_xformers_memory_efficient_attention()

pipe.enable_model_cpu_offload()

pipe.enable_vae_tiling()

Apply hidiffusion with a single line of code.

apply_hidiffusion(pipe)

prompt_list = [prompt1,prompt2,prompt3,prompt4,prompt5,prompt6,prompt7,prompt8,prompt9,prompt10,prompt11,prompt12,prompt13,prompt14,prompt15,prompt16,prompt17,prompt18]

seed=100
generator = torch.Generator(device=pipe.device).manual_seed(seed)
h, w = 960, 512
steps = 30
for i, prompt in enumerate(prompt_list):
image = pipe(prompt, guidance_scale=7.5, height=h, width=w, eta=1.0, generator=generator, num_inference_steps = steps).images[0]
image.save(f'pictures/h{h}w{w}{i}_seed{seed}_step{steps}.png')

[ShenZhang-Shin]

Ok, I will be going to reproduce the problem and fix it.

[solitaryTian]

Ok, I will be going to reproduce the problem and fix it.

I am very interested in this work. Have you reproduced this bug? Do you know the reason for the error and if there is any way to solve it?

[ShenZhang-Shin]

Problem solved. install the latest version
pip3 install hidiffusion==0.1.8
here is a 960x512 example with SD 1.5

[solitaryTian]

pip3 install hidiffusion==0.1.8

I have upgraded by 'pip install hidiffusion==0.1.8'. But I still get the same error

[ShenZhang-Shin]

I retest hidffusion==0.1.8 and it works fine for 960x512 resolution.
you might check your hidiffusion version,
pip3 show hidiffusion

and this my inference process

if hidiffusion==0.1.8 still gets the error. You can further check diffusers==0.27.0 or ==0.25.0