Example code only works for square image [bug]

[hungyiwu]

Expected behavior

segmentify should work for non-square images (width != length, ex. 512x512 is square, 511x512 is not square)

Potentially relevant issues/PR

#11 #13

How to reproduce the bug

Below is a slightly modified version of https://github.com/transformify-plugins/segmentify/blob/master/examples/launch.py

1. Run the code as is, popping the napari window; click execute button, confirm working
2. Un-comment the two lines right before napari.gui_qt; run the code, popping the napari window, click execute button, and confirm it crashes

import os
import napari
from segmentify import segmentation
import numpy as np
from skimage import io


# parse input file
example_image = os.path.join(os.path.abspath(os.path.dirname(__file__)), "hpa.png")
example_labels = os.path.join(os.path.abspath(os.path.dirname(__file__)), "hpa_labels.tif")

example_image = io.imread(example_image)
example_labels = io.imread(example_labels)

### WARNING: uncomment these two lines below to reproduce bug ###
#example_image = example_image[:, 1:]
#example_labels = example_labels[:, 1:]
##########################################################

with napari.gui_qt():
    viewer = napari.Viewer()

    # instantiate the widget
    gui = segmentation.Gui()

    # add our new widget to the napari viewer
    viewer.window.add_dock_widget(gui)

    # keep the dropdown menus in the gui in sync with the layer model
    viewer.layers.events.changed.connect(lambda x: gui.refresh_choices())

    gui.refresh_choices()

    # load data
    viewer.add_image(example_image)
    viewer.add_labels(example_labels)

Full error message

Segmenting...
WARNING: Traceback (most recent call last):
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/magicgui-0.1.5-py3.8.egg/magicgui/core.py", line 199, in <lambda>
    self.call_button.clicked.connect(lambda checked: self.__call__())
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/magicgui-0.1.5-py3.8.egg/magicgui/core.py", line 526, in __call__
    value = self.func(**_kwargs)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/gui.py", line 29, in segmentation
    clf, features = fit(data, initial_labels.data, featurizer=featurizer.value)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/semantic/main.py", line 129, in fit
    padded_features = unet_featurize(padded_image, featurizer)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/semantic/main.py", line 59, in unet_featurize
    features = model(image)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/torch-1.5.1-py3.8-linux-x86_64.egg/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/model/unet.py", line 57, in forward
    x = self.decode_4(x, skip_4)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/torch-1.5.1-py3.8-linux-x86_64.egg/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/model/layers/unet_layers.py", line 63, in forward
    output += skip
RuntimeError: The size of tensor a (62) must match the size of tensor b (63) at non-singleton dimension 3
Aborted (core dumped)

Environment, version, and other information

python 3.8.2
napari 0.3.5
napari-plugin-engine 0.1.6
napari-svg 0.1.3
segmentify 0.1.1 (installed by cloning this repo and python setup.py install the master branch)
Ubuntu 20.04
python venv
pip freeze > requirements.txt

[sofroniewn]

segmentify should work for non-square images (width != length, ex. 512x512 is square, 511x512 is not square)

If you wanted to work on a fix for this @hungyiwu I'd be happy to review and merge! It likely involves some of the padding code

[hungyiwu]

I looked into the models and the issue is not strictly square vs. non-square images, but images of shape 512x512 vs. otherwise, as this following didn't work either

example_image = example_image[1:, 1:]
example_labels = example_labels[1:, 1:]

I realized the original model was trained using 512x512 images and likely only support predictions of this input shape. So perhaps not really a bug but we can add documentation to remind users this pre-trained model supports this shape, and if they want something else they can train their own model for that.

Any thoughts? @sofroniewn

[sofroniewn]

I realized the original model was trained using 512x512 images and likely only support predictions of this input shape. So perhaps not really a bug but we can add documentation to remind users this pre-trained model supports this shape, and if they want something else they can train their own model for that.

Ah, makes sense! That sounds like a good addition to the docs. Thanks!!