Horn-Schunck optical flow calculations

Question

I was studying the Horn-schunck method for calculating optical flow in videos. My code is in C, which would mean i am implementing all of the algorithms from scratch including gray-scaling the image, computing derivatives etc. I am not able to completely absorb the essence of the method. The final flow matrix that I get would contain displacement vectors for each pixel, right? Meaning for each pixel, the value in the flow matrix would indicate the amount by which it is displaced in the next image.

How does this work out when I have all pixel values between 0-255, all my calculations are done on these pixel values and the resulting output gives displacement in, say, a 1920 X 1080 image.

Answer 1

The result of your method would be a matrix with two channels or two matrix, one for the u (or dx) direction/displacement and the other for the v (or dy) direction/displacement. That means you have a vector field

[u(x,y) v(x,y] = optical flow for each position (x,y) in your image

this vector field (the values of this field) have floating precision. That means e.g. u(0,0) = 0.2 v(0,0) = 0.13. Consequently in one part of your coud you have transform the gray-values of your input image into floating values. This is mostly done when you are compution the gradients e.g. with the sobel operator. The OpenCV library has a Horn-Schunk implementation. While reading the code takes some time, but you can be assured that this a very efficient way to implement this method.