Python: Possible memory corruption in optical flow interpolation with RICInterpolator

[conradorg]

System information (version)

• OpenCV => 4.5.4-dev (built from source)
• Operating System / Platform => Debian GNU/Linux 10 (buster)
• Compiler => Python 3.7.3 Interpreter

Detailed description

I tried out differently encoded videos with the interpolation with the RICInterpolator. In certain specific cases the interpolation crashes (deterministically for certain videos with certain encoding). This only happens if I use the same instance of the RICInterpolator for every interpolation. If I create a new instance every time, this does not happen.

Error messages

The error messages seem to indicate a memory corruption. Maybe an experienced C++ developer has an idea what causes this and could have a look.

The error message differs from execution to execution. I get the following errors:

• malloc_consolidate(): invalid chunk size
• double free or corruption (out)
• munmap_chunk(): invalid pointer
• corrupted size vs. prev_size in fastbins
• an memory access error

Debugging

I tried to debug it with gdb but had no success yet. I appended some of the backtraces. Apparently, the crash happens at two different lines in the code:

• sparse_match_interpolators.cpp:452 - in the new operator
• sparse_match_interpolators.cpp:1497 - in push_back() (only happens sometimes if the std:vector needs a reallocation)
  So, the bug seems to be related to the node struct and/or the nodeHeap struct. I wonder, in which cases a program crashes if the new operator is used.

Steps to reproduce

I attached some frames (.png) and motion vectors (.npy) which reproduce the bug with the following python code.

# issue.py
import numpy as np
import cv2

interpolator = cv2.ximgproc.createRICInterpolator()
for i in range(5):
    directory = "data/%02d/" % (i)
    src = np.load(directory + "src.npy")
    dst = np.load(directory + "dst.npy")
    frame_curr = cv2.imread("data/%02d/frame.png" % (i))
    frame_next = cv2.imread("data/%02d/frame.png" % (i+1))

    dense_flow = interpolator.interpolate(frame_curr, np.float32(src), frame_next, np.float32(dst))

Attachments

data.zip
backtrace_gdb.zip

For debugging with gdb run:

$ gdb python
(gdb) run issue.py

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues,
  answers.opencv.org, Stack Overflow, etc and have not found solution
• I updated to latest OpenCV version and the issue is still there
• There is reproducer code and related data files: videos, images, onnx, etc

[alalek]

/cc @tsenst