[bug][regression?]: Depth (Zoe) processor fails with TypeError

[keturn]

Is there an existing issue for this?

• I have searched the existing issues

OS

Linux

GPU

cuda

VRAM

12GB

What version did you experience this issue on?

1c7ea57

What happened?

• add t2i-adapter zoedepth-sdxl (but probably any adapter will do, as we don't get as far as running the adapter)
• add reference image to adapter (1024px square)
• choose processor: Depth (Zoe)

Traceback:

  File "site-packages/controlnet_aux/zoe/zoedepth/models/base_models/midas.py", line 263, in forward
    x = self.prep(x)
       ^^^^^^^^^^^^   
  File "site-packages/controlnet_aux/zoe/zoedepth/models/base_models/midas.py", line 187, in __call__
    return self.normalization(self.resizer(x))
                              ^^^^^^^^^^^^^^^
  File "site-packages/controlnet_aux/zoe/zoedepth/models/base_models/midas.py", line 174, in __call__
    return nn.functional.interpolate(x, (height, width), mode='bilinear', align_corners=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/torch/nn/functional.py", line 3924, in interpolate
    raise TypeError(                                                                                                                                                        
TypeError: expected size to be one of int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int], but got size with types [<class 'numpy.int64'>, <class 'numpy.int64'>]

though this traceback says it is in resizer, it does not change when I set Resize Mode to Crop or Fill instead of Resize.

The Midas depth processor works without error.

[keturn]

This must be a regression, since I'm sure we had a working zoe depth processor at some point.

[Millu]

Other UIs are reporting the same error:

• Error occurred when executing Zoe-DepthMapPreprocessor Fannovel16/comfyui_controlnet_aux#2
• [Bug]: TypeError: expected size to be one of int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int] Mikubill/sd-webui-controlnet#2113 (comment)

The A111 issue references torch 2.1 as a possible cause 😬

[psychedelicious]

Yeah, it's a torch 2.1 issue, introduced in this PR: pytorch/pytorch#99243 (comment)

I was about to create a PR to fix it, but I see it was actually already done: pytorch/pytorch#110778

So I think we will just need to wait for the next release of torch.

[psychedelicious]

A quicker fix might be to update controlnet-aux to use plain integers. It's a good handful of spots where this would be needed: https://github.com/search?q=repo%3Apatrickvonplaten%2Fcontrolnet_aux+interpolate%28&type=code

[psychedelicious]

torch 2.1.1 was released yesterday - it does not include the fix. Here's the 2.1.1 tag with that code: https://github.com/pytorch/pytorch/blob/v2.1.1/torch/nn/functional.py#L3777-L3783

[Vargol]

Look like it going to be fixed in 2.2, the merge has been pulled to the 2.2 release branch..

https://github.com/pytorch/pytorch/blob/e57f089704ae9fa0f57f18c6122ce645457b183a/torch/nn/functional.py#L3797

[psychedelicious]

This is fixed in the just-released torch 2.2.