SPS/PPS VUI Not Used By Android MediaCodec NDK

Question

I'm trying to decode video with non-default colorimetry using MediaCodec NDK. I provide the SPS and PPS into the csd-0 and csd-1 buffers respectively, but that information does not seem to affect how the decoded video looks.

First, I initialize the AMediaFormat

AMediaFormat * format = AMediaFormat_new ();
AMediaFormat_setString (format, AMEDIAFORMAT_KEY_MIME, "video/avc");
AMediaFormat_setInt32 (format, AMEDIAFORMAT_KEY_WIDTH, this->width);
AMediaFormat_setInt32 (format, AMEDIAFORMAT_KEY_HEIGHT, this->height);
AMediaFormat_setInt32 (format, AMEDIAFORMAT_KEY_FRAME_RATE, this->fps_n);

Then I provide the SPS and PPS buffers for my video stream

uint8_t sps[] = { 0,0,0,1,103,100,0,52,172,43,64,8,0,24,54,2,220,4,32,6,148,0,0,15,160,0,7,83,2,61,42,128 };
uint8_t pps[] = { 0,0,0,1,104,238,60,176 };
const size_t sps_len = 32;
const size_t pps_len = 8;
AMediaFormat_setBuffer (format, "csd-0", sps, sps_len);
AMediaFormat_setBuffer (format, "csd-1", pps, pps_len);

And finally, I configure and start the codec

AMediaCodec_configure (codec, format, window, NULL, 0);
AMediaCodec_start (codec);
AMediaFormat_delete (format);

I would now begin queueing input buffers for decompression as usual. This runs, without any error in the logs, but the decoded video looks exactly the same, regardless of what I have set for the transfer characteristics (above it's set to '8' for linear gamma).

Does anyone have any suggestions on why the media codec doesn't seem to be actually using the colorimetry data that I have provided?

Markus Schumann Markus Schumann · Accepted Answer · 2019-04-25T03:09:11

The color-space information in the H.264 stream is informational metadata only. So your observation is correct and the decompressor works as it should. You will get the decompressed bitmap in the same color-space as it was encoded. Usually the decompressor doesn't do or care about color-spaces. You have to do a color-space conversation after decompression.

SPS/PPS VUI Not Used By Android MediaCodec NDK

1 Answers