Python: Unable to perform optical flow interpolation with SparseMatchInterpolator (OpenCV Contrib)

[conradorg]

System information (version)

• OpenCV => 4.5.0 (built from source)
• Operating System / Platform => Debian GNU/Linux 10 (buster)
• Compiler => Python 3.7 Interpreter

Detailed description

I tried to perform an interpolation of a sparse optical flow with the subclasses of SparseMatchInterpolator (RICInterpolator, EdgeAwareInterpolator) and got the following error (with respect to the steps to reproduce):

dense_flow = interpolator.interpolate(frame1, from_points, frame2, to_points)

# for RICInterpolator
cv2.error: OpenCV(4.5.0) [...]/opencv/opencv_contrib-4.5.0/modules/ximgproc/src/sparse_match_interpolators.cpp:1105: error: (-215:Assertion failed) !from_points.empty() && from_points.isVector() in function 'interpolate'

# for EdgeAwareInterpolator
cv2.error: OpenCV(4.5.0) [...]/opencv/opencv_contrib-4.5.0/modules/ximgproc/src/sparse_match_interpolators.cpp:178: error: (-215:Assertion failed) !from_points.empty() && from_points.isVector() && !to_points .empty() && to_points .isVector() && from_points.sameSize(to_points) in function 'interpolate'

• I think this is a bug and has something to do with the conversion from python ndarray to an OpenCV matrix which is passed as InputArray.
• In the addressed interpolate() functions it is asserted, that from_points and to_points are vectors.
• This conforms the documentation which states it as Point2f vectors.
• So the problem is, that from python no Point2f vectors and hence no suitable input data can be passed.

I found just one post in the forum which addressed this earlier: https://answers.opencv.org/question/229942/_inputarrayisvector-with-python-data/

Steps to reproduce

Here a simple example which reproduces either one or the other error:

import numpy as np
import cv2

# interpolator = cv2.ximgproc.createRICInterpolator()
interpolator = cv2.ximgproc.createEdgeAwareInterpolator()

frame1 = np.array([[255,  0], [  0,  0], [  0,255]], dtype=np.uint8)
frame2 = np.array([[  0,255], [  0,  0], [255,  0]], dtype=np.uint8)
from_points = np.array([[0,0],[1,2]], dtype=np.float32)
to_points =   np.array([[1,0],[0,2]], dtype=np.float32)

dense_flow = interpolator.interpolate(frame1, from_points, frame2, to_points)

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues,
  answers.opencv.org, Stack Overflow, etc and have not found solution
• I updated to latest OpenCV version and the issue is still there
• There is reproducer code and related data files: videos, images, onnx, etc

[alalek]

Thank you for the report!

isVector() is misused in this code.
checkVector() should be used instead.

BTW, current implementation of isVector():

inline bool _InputArray::isVector() const { return kind() == _InputArray::STD_VECTOR ||
                                                   kind() == _InputArray::STD_BOOL_VECTOR ||
                                                   kind() == _InputArray::STD_ARRAY; }