MFT Custom image Filters

Question

I am currently developing a Metro Style App which uses an MFT (Media Foundation Transform) to filter the webcam's video stream into grayscale, as demonstrated in this sample.

However, now I want to apply other types of filters, such as exposure, hue, luminance, texture, vignette, etc. This answer says I am supposed to modify the TransformChroma method in order to achieve this. Unfortunately, I can't figure out how to get the Y value, I can only get the U and V. How do I get the Y value in the formats NV12, YUY2, and UYVY?

All help is greatly appreciated and I always accept an answer!

Answer 1

You would need to change the signature of the method (an poto take another parameter and modify the TransformImage_UYVY, TransformImage_YUY2 and TransformImage_NV12 methods to pass that parameter into the updated method. You would need to figure out how to extract that value for yourself though though. For example looking at this piece of code below you can see how the U and V values are extracted and that the Y value is split in two bytes - you would need to do some bit logic to join these. You can find descriptions of these formats online, e.g. here.

// Byte order is U0 Y0 V0 Y1
// Each WORD is a byte pair (U/V, Y)
// Windows is little-endian so the order appears reversed.
BYTE u = pSrc_Pixel[x] & 0x00FF;
BYTE v = pSrc_Pixel[x+1] & 0x00FF;

TransformChroma(mat, &u, &v);