python - How to convert sRGB to NV12 format using NumPy?

Question

NV12 format defines specific color channels ordering of YUV color space with 420 subsampling.
NV12 format is mostly used in video encoding/decoding pipeline.

libyuv description of NV12:

NV12 is a biplanar format with a full sized Y plane followed by a single chroma plane with weaved U and V values. NV21 is the same but with weaved V and U values. The 12 in NV12 refers to 12 bits per pixel. NV12 has a half width and half height chroma channel, and therefore is a 420 subsampling.

In context of NV12, YUV format is mainly referred as YCbCr color space.
NV12 elements are 8 bits per element (uint8 type).
In the context of the post, YUV elements are in "limited range" standard: Y range is [16, 235], U,V range is [16, 240].

sRGB (standard Red Green Blue) is a standard color space used by PC systems.
In the context of the post, sRGB color components range is [0, 255] (uint8 type).
RGB elements ordering is not relevant to the post (assume 3 color planes).

There are currently at least 2 possible YCbCr formats applying NV12:

BT.601 - Applies SDTV.
BT.709 - Applies HDTV.

Example for NV12 elements ordering:
YYYYYY
YYYYYY
UVUVUV

RGB to NV12 conversion can be described by the following stages:

Color space conversion - convert from sRGB to YUV color space.
Chroma downsampling - shrink U,V channels by a factor of x2 in each axis (converting from YUV444 to YUV420).
Chroma elements interleaving - arrange U,V elements as U,V,U,V...

Following figure illustrates the conversion stages applying image size of 6x6 pixels:

How can we convert sRGB to NV12 using NumPy?

Note:
The question refers Python implementation that demonstrates the conversion process (post is not intended for existing function like OpenCV implementation).

Rotem Rotem · Accepted Answer · 2019-07-13T20:33:38

Converting sRGB to NV12 format using NumPy

The purpose of the post is demonstrating the conversion process.
The Python implementation below uses NumPy, and deliberately avoids using OpenCV.

RGB to NV12 conversion stages:

Color space conversion - convert from sRGB to YUV color space:
Use sRGB to YCbCr conversion formula.
Multiply each RGB triple by 3x3 conversion matrix, and add a vector of 3 offsets.
The post shows both BT.709 and BT.601 conversions (the only difference is the coefficients matrix).
Chroma downsampling - shrink U,V channels by a factor of x2 in each axis (converting from YUV444 to YUV420).
The implementation resizes U,V by factor of 0.5 in each axis using bi-linear interpolation.
Note: bi-linear interpolation is not the optimal downsampling method, but it's usually good enough.
Instead of using cv2.resize, code uses average of every 2x2 pixels (result is equivalent to bi-linear interpolation).
Note: implementation fails in case input resolution is not even in both dimensions.
Chroma elements interleaving - arrange U,V elements as U,V,U,V...
Implemented by array indexing manipulation.

Here is a Python code sample for converting RGB to NV12 standard:

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

do_use_bt709 = True; # True for BT.709, False for BT.601

RGB = mpimg.imread('rgb_input.png')*255.0     # Read RGB input image, multiply by 255 (set RGB range to [0, 255]).
R, G, B = RGB[:, :, 0], RGB[:, :, 1], RGB[:, :, 2]  # Split RGB to R, G and B numpy arrays.
rows, cols = R.shape

# I. Convert RGB to YUV (convert sRGB to YUV444)
#################################################
if do_use_bt709:
    # Convert sRGB to YUV, BT.709 standard
    # Conversion formula used: 8 bit sRGB to "limited range" 8 bit YUV (BT.709).
    Y =  0.18258588*R + 0.61423059*G + 0.06200706*B + 16.0
    U = -0.10064373*R - 0.33857195*G + 0.43921569*B + 128.0
    V =  0.43921569*R - 0.39894216*G - 0.04027352*B + 128.0
else:
    # Convert sRGB to YUV, BT.601 standard.
    # Conversion formula used: 8 bit sRGB to "limited range" 8 bit YUV (BT.601).
    Y =  0.25678824*R + 0.50412941*G + 0.09790588*B + 16.0
    U = -0.14822290*R - 0.29099279*G + 0.43921569*B + 128.0
    V =  0.43921569*R - 0.36778831*G - 0.07142737*B + 128.0


# II. U,V Downsampling (convert YUV444 to YUV420)
##################################################
# Shrink U and V channels by a factor of x2 in each axis (use bi-linear interpolation).
#shrunkU = cv2.resize(U, dsize=(cols//2, rows//2), interpolation=cv2.INTER_LINEAR)
#shrunkV = cv2.resize(V, dsize=(cols//2, rows//2), interpolation=cv2.INTER_LINEAR)

# Each element of shrunkU is the mean of 2x2 elements of U
# Result is equvalent to resize by a factor of 0.5 with bi-linear interpolation.
shrunkU = (U[0: :2, 0::2] + U[1: :2, 0: :2] + U[0: :2, 1: :2] + U[1: :2, 1: :2]) * 0.25
shrunkV = (V[0: :2, 0::2] + V[1: :2, 0: :2] + V[0: :2, 1: :2] + V[1: :2, 1: :2]) * 0.25


# III. U,V Interleaving
########################
# Size of UV plane is half the number of rows, and same number of columns as Y plane.
UV = np.zeros((rows//2, cols))  # Use // for integer division.

# Interleave shrunkU and shrunkV and build UV palne (each row of UV plane is u,v,u,u,v...)
UV[:, 0 : :2] = shrunkU
UV[:, 1 : :2] = shrunkV

# Place Y plane at the top, and UV plane at the bottom (number of rows NV12 matrix is rows*1.5)
NV12 = np.vstack((Y, UV))

# Round NV12, and cast to uint8 (use floor(x+0.5) instead of round to avoid "bankers rounding").
NV12 = np.floor(NV12 + 0.5).astype('uint8')


# Write NV12 array to binary file
NV12.tofile('nv12_output.raw')

# Display NV12 result (display as Grayscale image).
plt.figure()
plt.axis('off')
plt.imshow(NV12, cmap='gray', interpolation='nearest')
plt.show()

Sample RGB input image:

NV12 Result (displayed as Grayscale image):

python - How to convert sRGB to NV12 format using NumPy?

1 Answers

Converting sRGB to NV12 format using NumPy