Image convolution with a 4D Kernel in opencv

Question

Given an image, I'm trying to apply convolution with a (3 x 3 x 3 x 64) kernel:

cv2.filter2D(img, -1, np.random.rand(3,3,3,64))

Gives:

error: /Users/travis/build/skvark/opencv-python/opencv/modules/imgproc/src/filterengine.hpp:363: error: (-215) anchor.inside(Rect(0, 0, ksize.width, ksize.height)) in function normalizeAnchor

In fact in the documentation it says:

kernel – convolution kernel (or rather a correlation kernel), a single-channel floating point matrix; if you want to apply different kernels to different channels, split the image into separate color planes using split() and process them individually.

Is there any other opencv function that can convolve a > 2D kernel? Or do I have to do two for loops applying filter2d?

Answer 1

There is no such functionality in OpenCV - not in any of the interfaces nor the core C++ library itself. If you want to do 4D convolution, you'll either have to use cv2.filter2D looping over 2D submatrices of your 4D kernel, write it yourself manually, or use something else that supports it, like a deep learning package, or SciPy.

The easiest solution I can suggest without you having to write one or hack at one yourself is to use SciPy's scipy.signal.convolve which performs N-dimensional convolution: https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.convolve.html. Bear in mind that both image and kernel need to have the same number of dimensions, so it's expected that your image is 4D as well.

Answer 2

The OpenCV function cv2.filter2D(), as the name implies, assumes a 2D img and a 2D kernel. For more than that, you have to use loops. For example, the following runs with no errors,

import cv2
import numpy as np

# read an rgb image
img = cv2.imread('fig1.png')

# filter the first channel (blue)
out0 = cv2.filter2D( img[:,:,0], -1, np.random.rand(3,3))

See the documentation at cv2.filter2D()