What, if any, are the differences in MediaCodec encoders (I420, NV12, Planar, Semi-Planar, etc)?

0
votes

Referring to this page: http://bigflake.com/mediacodec/

A5. The color formats for the camera output and the MediaCodec encoder input are different. Camera supports YV12 (planar YUV 4:2:0) and NV21 (semi-planar YUV 4:2:0). The MediaCodec encoders support one or more of:

#19 COLOR_FormatYUV420Planar (I420)

#20 COLOR_FormatYUV420PackedPlanar (also I420)

#21 COLOR_FormatYUV420SemiPlanar (NV12)

#39 COLOR_FormatYUV420PackedSemiPlanar (also NV12)

#0x7f000100 COLOR_TI_FormatYUV420PackedSemiPlanar (also also NV12)

In my application, I am capturing frames from an external camera in YUY2 format, converting them to a usable format, and feeding them to a MediaMuxer.

Based on what I've read here, this means that I need to query what the device supports with MediaCodecInfo.CodeCapabilities. Then, based on that, do a conversion from YUY2 to the appropriate format. At least this is my understanding.

In the sea of codec formats, I am unsure of the differences with these and whether or not I need to account for all of them in my application. If so, I need to know the byte layout of these formats. I've filled in the ones I think are correct. Starting from the top and moving down:

FormatYUV420Planar - YYYY YYYY UU VV

FormatYUVPackedPlanar - ???

FormatYUV420SemiPlanar -- YYYY YYYY VU VU

FormatYUV420PackedSemiPlanar -- ???

COLOR_TI_FormatYUV420PackedSemiPlanar -- ???

1
androidandroid-mediacodecrenderscriptmediamuxer
One useful "reference" is the CTS test referenced from bigflake that generates frames in the various formats: android.googlesource.com/platform/cts/+/lollipop-release/tests/… . This must work on all devices, so if you do what it does you should be fine. (Bear in mind, though, that it only tests specific resolutions.) – fadden
From the point of view of the CTS, which these devices must pass, FormatYUVPackedPlanar is equal to FormatYUV420Planar, and COLOR_TI_FormatYUV420PackedSemiPlanar is equal to FormatYUV420PackedSemiPlanar and FormatYUV420PackedSemiPlanar. (e.g. COLOR_TI_FormatYUV420PackedSemiPlanar has got some weird details only in the form that the decoder outputs it, but when feeding data to an encoder you can do it exactly as for FormatYUV420SemiPlanar). – mstorsjo

1 Answers

3
votes

Packed formats are those formats in which all 3 components are packed together in one plane. Please refer to the following links for more information on the different color formats which are widely employed in visual pipelines.

  1. fourcc

  2. videoLan

  3. Microsoft

For the COLOR_TI_FormatYUV420PackedSemiPlanar, I would recommend to refer the color conversion function in ColorConverteras here. It is similar to YUV420SemiPlanar, but has some specific differences in the way the data is picked up.