What convolution operation is used for image recognition?

Question

All of the contrived examples that I've found of convolution in feature recognition "simplify" the convolution operation by having pixel values of either 1 or -1. It makes for a really simple operation (multiply input pixel by filter pixel, sum results, then divide by number of pixels):

basic convolution

However, it's not really helpful for most images where pixel values will have ranges. E.g. (0.0-1.0), or (0-255).

I can't find an example anywhere of which algorithm to use for these input values. I tried summing the difference for each pixel, then dividing by the number of pixels to get an overall "error". Activation then equals max - error. E.g. 255 - error, or 1.0 - error.

It will never output a negative value though so I don't see the need for a ReLU layer. This makes me suspect it's a naive approach and wouldn't actually work, but I'm not sure why.

So what is the operation used when the input data is something other than 1/-1?

EDIT Here's the example I've been looking at: http://brohrer.github.io/how_convolutional_neural_networks_work.html

And the convolution operation it describes:

To calculate the match of a feature to a patch of the image, simply multiply each pixel in the feature by the value of the corresponding pixel in the image. Then add up the answers and divide by the total number of pixels in the feature. If both pixels are white (a value of 1) then 1 * 1 = 1. If both are black, then (-1) * (-1) = 1. Either way, every matching pixel results in a 1. Similarly, any mismatch is a -1.

A concrete example of why I don't think this works for pixels with a value of [0.0,1.0]. Say we have a 1x1 filter with the value [0.5]. If we run that over an input pixel whose value is 0.5 then we get 0.25.

Similarly, if we're using color ranges of [0,255] then we easily end up with values > 255. Although I'm not sure that matters as it's no longer pixel data; it's activation in a feature map, right?

Tatarize Tatarize · Accepted Answer · 2016-10-01T08:57:34

Typically you use the convolution for each pixel. So each pixel is the sum of all of those pixels with all of those weights. So it's 0.77 * that corner pixel, 0.33 * that other corner, and all of those values are added up and put into the center. Usually crimped. Then the same is done for all the other pixels without overlapping any of the data.

The value of the next version of that pixel is the sum of several pixels. Sometimes this is given with weights. So rather than multiply by -1, we multiply by the weight of each of those pixels.

You would typically normalize the convolution weights. In this case 9.444444444444444 and divide the sum of the various pixels at various weights by that amount. But, this is clearly Canny Edge Detection which means the point is to exceed the given range and divide up just the edges. Which means depending on the kernel it might well allow for a max of 9.44x the top range. And a bottom range of a negative value. Then you crimp the routine and have the given pixel truncated into the range of 0-255 or 0.0-1.0 (depending on what you're using). This loses a lot of data but that's the point, it wants to lose the noise and keep the edges.

What convolution operation is used for image recognition?

2 Answers