YCbCr

RGB to YCbCr conversion

Y′CbCr signals (prior to scaling and offsets to place the signals into digital form) are called YPbPr, and are created from the corresponding gamma-adjusted RGB (red, green and blue) source using three defined constants K_R, K_G, and K_B as follows:

${\displaystyle {\begin{aligned}Y'&=K_{R}\cdot R'+K_{G}\cdot G'+K_{B}\cdot B'\\P_{B}&={\frac {1}{2}}\cdot {\frac {B'-Y'}{1-K_{B}}}\\P_{R}&={\frac {1}{2}}\cdot {\frac {R'-Y'}{1-K_{R}}}\end{aligned}}}$

where K_R, K_G, and K_B are ordinarily derived from the definition of the corresponding RGB space, and required to satisfy ${\displaystyle K_{R}+K_{G}+K_{B}=1}$.

The equivalent matrix manipulation is often referred to as the "color matrix":

${\displaystyle {\begin{bmatrix}Y'\\P_{B}\\P_{R}\end{bmatrix}}={\begin{bmatrix}K_{R}&K_{G}&K_{B}\\-{\frac {1}{2}}\cdot {\frac {K_{R}}{1-K_{B}}}&-{\frac {1}{2}}\cdot {\frac {K_{G}}{1-K_{B}}}&{\frac {1}{2}}\\{\frac {1}{2}}&-{\frac {1}{2}}\cdot {\frac {K_{G}}{1-K_{R}}}&-{\frac {1}{2}}\cdot {\frac {K_{B}}{1-K_{R}}}\end{bmatrix}}{\begin{bmatrix}R'\\G'\\B'\end{bmatrix}}}$

And its inverse:

${\displaystyle {\begin{bmatrix}R'\\G'\\B'\end{bmatrix}}={\begin{bmatrix}1&0&2-2\cdot K_{R}\\1&-{\frac {K_{B}}{K_{G}}}\cdot (2-2\cdot K_{B})&-{\frac {K_{R}}{K_{G}}}\cdot (2-2\cdot K_{R})\\1&2-2\cdot K_{B}&0\end{bmatrix}}{\begin{bmatrix}Y'\\P_{B}\\P_{R}\end{bmatrix}}}$

Here, the prime (′) symbols mean gamma correction is being used; thus R′, G′ and B′ nominally range from 0 to 1, with 0 representing the minimum intensity (e.g., for display of the color black) and 1 the maximum (e.g., for display of the color white). The resulting luma (Y) value will then have a nominal range from 0 to 1, and the chroma (P_B and P_R) values will have a nominal range from -0.5 to +0.5. The reverse conversion process can be readily derived by inverting the above equations.

When representing the signals in digital form, the results are scaled and rounded, and offsets are typically added. For example, the scaling and offset applied to the Y′ component per specification (e.g. MPEG-2[2]) results in the value of 16 for black and the value of 235 for white when using an 8-bit representation. The standard has 8-bit digitized versions of C_B and C_R scaled to a different range of 16 to 240. Consequently, rescaling by the fraction (235-16)/(240-16) = 219/224 is sometimes required when doing color matrixing or processing in YCbCr space, resulting in quantization distortions when the subsequent processing is not performed using higher bit depths.

The scaling that results in the use of a smaller range of digital values than what might appear to be desirable for representation of the nominal range of the input data allows for some "overshoot" and "undershoot" during processing without necessitating undesirable clipping. This "headroom" and "toeroom"[3] can also be used for extension of the nominal color gamut, as specified by xvYCC.

The value 235 accommodates a maximum overshoot of (255 - 235) / (235 - 16) = 9.1%, which is slightly larger than the theoretical maximum overshoot (Gibbs' Phenomenon) of about 8.9% of the maximum (black-to-white) step. The toeroom is smaller, allowing only 16 / 219 = 7.3% overshoot, which is less than the theoretical maximum overshoot of 8.9%. In addition, because values 0 and 255 are reserved in HDMI, the room is actually slightly less.

Since the equations defining Y'CbCr are formed in a way that rotates the entire nominal RGB color cube and scales it to fit within a (larger) YCbCr color cube, there are some points within the Y'CbCr color cube that cannot be represented in the corresponding RGB domain (at least not within the nominal RGB range). This causes some difficulty in determining how to correctly interpret and display some Y'CbCr signals. These out-of-range Y'CbCr values are used by xvYCC to encode colors outside the BT.709 gamut.

The form of Y′CbCr that was defined for standard-definition television use in the ITU-R BT.601 (formerly CCIR 601) standard for use with digital component video is derived from the corresponding RGB space (ITU-R BT.470-6 System M primaries) as follows:

${\displaystyle {\begin{aligned}K_{R}&=0.299\\K_{G}&=0.587\\K_{B}&=0.114\end{aligned}}}$

From the above constants and formulas, the following can be derived for ITU-R BT.601.

Analog YPbPr from analog R'G'B' is derived as follows:

${\displaystyle {\begin{aligned}Y'&=&0.299\cdot R'&+&0.587\cdot G'&+&0.114\cdot B'\\P_{B}&=-&0.168736\cdot R'&-&0.331264\cdot G'&+&0.5\cdot B'\\P_{R}&=&0.5\cdot R'&-&0.418688\cdot G'&-&0.081312\cdot B'\end{aligned}}}$

Digital Y′CbCr (8 bits per sample) is derived from analog R'G'B' as follows:

${\displaystyle {\begin{aligned}Y'&=&16&+&(65.481\cdot R'&+&128.553\cdot G'&+&24.966\cdot B')\\C_{B}&=&128&+&(-37.797\cdot R'&-&74.203\cdot G'&+&112.0\cdot B')\\C_{R}&=&128&+&(112.0\cdot R'&-&93.786\cdot G'&-&18.214\cdot B')\end{aligned}}}$

or simply componentwise

${\displaystyle {\begin{aligned}(Y',C_{B},C_{R})&=&(16,128,128)+(219\cdot Y,224\cdot P_{B},224\cdot P_{R})\\\end{aligned}}}$

The resultant signals range from 16 to 235 for Y′ (Cb and Cr range from 16 to 240); the values from 0 to 15 are called footroom, while the values from 236 to 255 are called headroom.

Alternatively, digital Y′CbCr can derived from digital R'dG'dB'd (8 bits per sample, each using the full range with zero representing black and 255 representing white) according to the following equations:

${\displaystyle {\begin{aligned}Y'&=&16&+&{\frac {65.481\cdot R'_{D}}{255}}&+&{\frac {128.553\cdot G'_{D}}{255}}&+&{\frac {24.966\cdot B'_{D}}{255}}\\C_{B}&=&128&-&{\frac {37.797\cdot R'_{D}}{255}}&-&{\frac {74.203\cdot G'_{D}}{255}}&+&{\frac {112.0\cdot B'_{D}}{255}}\\C_{R}&=&128&+&{\frac {112.0\cdot R'_{D}}{255}}&-&{\frac {93.786\cdot G'_{D}}{255}}&-&{\frac {18.214\cdot B'_{D}}{255}}\end{aligned}}}$

In the formula below, the scaling factors are multiplied by ${\displaystyle {\frac {256}{255}}}$. This allows for the value 256 in the denominator, which can be calculated by a single bitshift.

${\displaystyle {\begin{aligned}Y'&=&16&+&{\frac {65.738\cdot R'_{D}}{256}}&+&{\frac {129.057\cdot G'_{D}}{256}}&+&{\frac {25.064\cdot B'_{D}}{256}}\\C_{B}&=&128&-&{\frac {37.945\cdot R'_{D}}{256}}&-&{\frac {74.494\cdot G'_{D}}{256}}&+&{\frac {112.439\cdot B'_{D}}{256}}\\C_{R}&=&128&+&{\frac {112.439\cdot R'_{D}}{256}}&-&{\frac {94.154\cdot G'_{D}}{256}}&-&{\frac {18.285\cdot B'_{D}}{256}}\end{aligned}}}$

If the R'd G'd B'd digital source includes footroom and headroom, the footroom offset 16 needs to be subtracted first from each signal, and a scale factor of ${\displaystyle {\frac {255}{219}}}$ needs to be included in the equations.

The inverse transform is:

${\displaystyle {\begin{aligned}R'_{D}&=&{\frac {298.082\cdot Y'}{256}}&&&+&{\frac {408.583\cdot C_{R}}{256}}&-&222.921\\G'_{D}&=&{\frac {298.082\cdot Y'}{256}}&-&{\frac {100.291\cdot C_{B}}{256}}&-&{\frac {208.120\cdot C_{R}}{256}}&+&135.576\\B'_{D}&=&{\frac {298.082\cdot Y'}{256}}&+&{\frac {516.412\cdot C_{B}}{256}}&&&-&276.836\end{aligned}}}$

The inverse transform without any roundings (using values coming directly from ITU-R BT.601 recommendation) is:

${\displaystyle {\begin{aligned}R'_{D}={\frac {255}{219}}\cdot (Y'-16)&&&&&&&+{\frac {255}{224}}\cdot 1.402\cdot (C_{R}-128)\\G'_{D}={\frac {255}{219}}\cdot (Y'-16)&-&{\frac {255}{224}}\cdot 1.772&&\cdot {\frac {0.114}{0.587}}&&\cdot (C_{B}-128)&-{\frac {255}{224}}\cdot 1.402\cdot {\frac {0.299}{0.587}}\cdot (C_{R}-128)\\B'_{D}={\frac {255}{219}}\cdot (Y'-16)&+&{\frac {255}{224}}\cdot 1.772&&&&\cdot (C_{B}-128)\end{aligned}}}$

This form of Y′CbCr is used primarily for older standard-definition television systems, as it uses an RGB model that fits the phosphor emission characteristics of older CRTs.

Rec. 709 compared with Rec. 2020

A different form of Y′CbCr is specified in the ITU-R BT.709 standard, primarily for HDTV use. The newer form is also used in some computer-display oriented applications, as sRGB (though the matrix used for sRGB form of YCbCr, sYCC, is still BT.601). In this case, the values of Kb and Kr differ, but the formulas for using them are the same. For ITU-R BT.709, the constants are:

${\displaystyle {\begin{aligned}K_{B}&=0.0722\\K_{R}&=0.2126\\(K_{G}&=1-K_{B}-K_{R}=0.7152)\end{aligned}}}$

This form of Y′CbCr is based on an RGB model that more closely fits the phosphor emission characteristics of newer CRTs and other modern display equipment. The conversion matrices for BT.709 are these:

${\displaystyle {\begin{aligned}{\begin{bmatrix}Y'\\C_{B}\\C_{R}\end{bmatrix}}&={\begin{bmatrix}0.2126&0.7152&0.0722\\-0.1146&-0.3854&0.5\\0.5&-0.4542&-0.0458\end{bmatrix}}{\begin{bmatrix}R'\\G'\\B'\end{bmatrix}}\\{\begin{bmatrix}R'\\G'\\B'\end{bmatrix}}&={\begin{bmatrix}1&0&1.5748\\1&-0.1873&-0.4681\\1&1.8556&0\end{bmatrix}}{\begin{bmatrix}Y'\\C_{B}\\C_{R}\end{bmatrix}}\end{aligned}}}$

The definitions of the R', G', and B' signals also differ between BT.709 and BT.601, and differ within BT.601 depending on the type of TV system in use (625-line as in PAL and SECAM or 525-line as in NTSC), and differ further in other specifications. In different designs there are differences in the definitions of the R, G, and B chromaticity coordinates, the reference white point, the supported gamut range, the exact gamma pre-compensation functions for deriving R', G' and B' from R, G, and B, and in the scaling and offsets to be applied during conversion from R'G'B' to Y′CbCr. So proper conversion of Y′CbCr from one form to the other is not just a matter of inverting one matrix and applying the other. In fact, when Y′CbCr is designed ideally, the values of K_B and K_R are derived from the precise specification of the RGB color primary signals, so that the luma (Y′) signal corresponds as closely as possible to a gamma-adjusted measurement of luminance (typically based on the CIE 1931 measurements of the response of the human visual system to color stimuli).[4]

The ITU-R BT.2020 standard defines both BT.709 gamma corrected Y′CbCr and using the same gamma correction (except for Y', that is calculated differently), but with constant luminance Cb, Cr called YcCbcCrc.[5]

For both, the coefficients are:

${\displaystyle {\begin{aligned}K_{B}&=0.0593\\K_{R}&=0.2627\end{aligned}}}$

The decoding matrix for BT.2020-NCL is this with 14 decimal places:

${\displaystyle {\begin{aligned}{\begin{bmatrix}R\\G\\B\end{bmatrix}}&={\begin{bmatrix}1&0&1.4746\\1&-0.16455312684366&-0.57135312684366\\1&1.8814&0\end{bmatrix}}{\begin{bmatrix}Y'\\C_{B}\\C_{R}\end{bmatrix}}\end{aligned}}}$

The smaller values in the matrix are not rounded, they are precise values. For systems with limited precision (8 or 10 bit, for example) a lower precision of the above matrix could be used, for example, retaining only 6 digits after decimal point.[6]

YcCbcCrc may be used when the top priority is the most accurate retention of luminance information.[5] Color representation has true constant luminance (CL) when the luma channel (Y’ of Y′CbCr encoded with BT.709 transfer function or PQ, for example) matches with encoded luminance (BT.709 or PQ encoded luminance Y of XYZ), nevertheless YcCbcCrc does not provide for constant intensity (CI), that is done in IC_TC_P.[7][8] BT.2020 does not define PQ and thus HDR, it is further defined in SMPTE ST 2084 and BT.2100.

The derivation of BT.2020 coefficients from BT.2020 primaries further changes the space.[9]

The SMPTE 240M standard (used on the MUSE analog HD television system) defines YCC with these coefficients:

${\displaystyle {\begin{aligned}K_{B}&=0.087\\K_{R}&=0.212\end{aligned}}}$

The coefficients are derived from SMPTE 170M primaries and white point, as used in 240M standard.

JFIF usage of JPEG supports a modified Rec. 601 Y′CbCr where Y′, C_B and C_R have the full 8-bit range of [0...255].[10] Below are the conversion equations expressed to six decimal digits of precision. (For ideal equations, see ITU-T T.871.[11]) Note that for the following formulae, the range of each input (R,G,B) is also the full 8-bit range of [0...255].

${\displaystyle {\begin{aligned}Y'&=&0&+(0.299&\cdot R'_{D})&+(0.587&\cdot G'_{D})&+(0.114&\cdot B'_{D})\\C_{B}&=&128&-(0.168736&\cdot R'_{D})&-(0.331264&\cdot G'_{D})&+(0.5&\cdot B'_{D})\\C_{R}&=&128&+(0.5&\cdot R'_{D})&-(0.418688&\cdot G'_{D})&-(0.081312&\cdot B'_{D})\end{aligned}}}$

And back:

${\displaystyle {\begin{aligned}R'_{D}&=&Y'&&&+1.402&\cdot (C_{R}-128)\\G'_{D}&=&Y'&-0.344136&\cdot (C_{B}-128)&-0.714136&\cdot (C_{R}-128)\\B'_{D}&=&Y'&+1.772&\cdot (C_{B}-128)&\end{aligned}}}$

The above conversion is identical to sYCC when the input is given as sRGB, except that IEC 61966-2-1:1999/Amd1:2003 only gives four decimal digits.

JPEG also defines a "YCCK" format from Adobe for CMYK input. In this format, the "K" value is passed as-is, while CMY are used to derive YCbCr with the above matrix by assuming R=1-C, G=1-M, and B=1-Y. As a result, a similar set of subsampling techniques can be used.[12]

${\displaystyle {\begin{aligned}K_{B}&=0.0713\\K_{R}&=0.2220\end{aligned}}}$

These coefficients are not in use and were never in use.[13]

H.273 also describes constant and non-constant luminance systems which are derived strictly from primaries and white point, so that situations like sRGB/BT.709 default primaries of JPEG that use BT.601 matrix (that is derived from BT.470-6 System M) do not happen.

These matrices round all factors to the closest 1/256 unit. As a result, only one 16-bit intermediate value is formed for each component, and a simple right-shift with rounding (x + 128) >> 8 can take care of the division.[14]

For studio-swing:

${\displaystyle {\begin{bmatrix}Y'\\C_{B}\\C_{R}\end{bmatrix}}={\frac {1}{256}}{\begin{bmatrix}66&129&25\\-38&-74&112\\112&-94&-18\end{bmatrix}}{\begin{bmatrix}R'\\G'\\B'\end{bmatrix}}+{\begin{bmatrix}16\\128\\128\end{bmatrix}}}$

For full-swing:

${\displaystyle {\begin{bmatrix}Y'\\C_{B}\\C_{R}\end{bmatrix}}={\frac {1}{256}}{\begin{bmatrix}77&150&29\\-43&-84&127\\127&-106&-21\end{bmatrix}}{\begin{bmatrix}R'\\G'\\B'\end{bmatrix}}+{\begin{bmatrix}0\\128\\128\end{bmatrix}}}$

Google's Skia used to use the above 8-bit full-range matrix, resulting in a slight greening effect on JPEG images encoded by Android devices, more noticeable on repeated saving. The issue was fixed in 2016, when the more accurate version was used instead. Due to optimizations in libjpeg-turbo, the accurate version was actually faster.[15]

See § RGB conversion.

Input: Read 4 bytes of Y′UV (u, y1, v, y2)

Output: Writes 6 bytes of RGB (R, G, B, R, G, B)

u  = yuv[0];
y1 = yuv[1];
v  = yuv[2];
y2 = yuv[3];

${\displaystyle {YUV}_{422}={\begin{bmatrix}u&y_{1}\\v&y_{2}\end{bmatrix}}}$

${\displaystyle {YUV}_{444}={\begin{bmatrix}y_{1}&u&v\\y_{2}&u&v\end{bmatrix}}}$

${\displaystyle {RGB}=M^{-1}{YUV}_{444}}$

Using this information it could be parsed as regular Y′UV444 format to get 2 RGB pixels info:

rgb1 = Y′UV444toRGB888(y1, u, v);
rgb2 = Y′UV444toRGB888(y2, u, v);

Y′UV422 can also be expressed with the values in an alternative order, e.g. for the FourCC format code YUY2.

Input: Read 4 bytes of Y′UV (y1, u, y2, v), (y1, y2, u, v) or (u, v, y1, y2)

${\displaystyle {YUY}_{422}={\begin{bmatrix}y_{1}&u\\y_{2}&v\end{bmatrix}}}$

${\displaystyle {YYU}_{422}={\begin{bmatrix}y_{1}&y_{2}\\u&v\end{bmatrix}}}$

${\displaystyle {UVY}_{422}={\begin{bmatrix}u&v\\y_{1}&y_{2}\end{bmatrix}}}$

Input: Read 6 bytes of Y′UV

Output: Writes 12 bytes of RGB

// Extract YUV components
u  = yuv[0];
y1 = yuv[1];
y2 = yuv[2];
v  = yuv[3];
y3 = yuv[4];
y4 = yuv[5];

rgb1 = Y′UV444toRGB888(y1, u, v);
rgb2 = Y′UV444toRGB888(y2, u, v);
rgb3 = Y′UV444toRGB888(y3, u, v);
rgb4 = Y′UV444toRGB888(y4, u, v);

${\displaystyle {YUV}_{411}={\begin{bmatrix}u&y_{1}&y_{2}\\v&y_{3}&y_{4}\end{bmatrix}}}$

${\displaystyle {YUV}_{444}={\begin{bmatrix}y_{1}&u&v\\y_{2}&u&v\\y_{3}&u&v\\y_{4}&u&v\end{bmatrix}}}$

So the result is we are getting 4 RGB pixels values (4*3 bytes) from 6 bytes. This means reducing the size of transferred data to half, with a loss of quality.

Y′UV420p is a planar format, meaning that the Y′, U, and V values are grouped together instead of interspersed. The reason for this is that by grouping the U and V values together, the image becomes much more compressible. When given an array of an image in the Y′UV420p format, all the Y′ values come first, followed by all the U values, followed finally by all the V values.

The Y′V12 format is essentially the same as Y′UV420p, but it has the U and V data switched: the Y′ values are followed by the V values, with the U values last. As long as care is taken to extract U and V values from the proper locations, both Y′UV420p and Y′V12 can be processed using the same algorithm.

As with most Y′UV formats, there are as many Y′ values as there are pixels. Where X equals the height multiplied by the width, the first X indices in the array are Y′ values that correspond to each individual pixel. However, there are only one fourth as many U and V values. The U and V values correspond to each 2 by 2 block of the image, meaning each U and V entry applies to four pixels. After the Y′ values, the next X/4 indices are the U values for each 2 by 2 block, and the next X/4 indices after that are the V values that also apply to each 2 by 2 block.

As shown in the above image, the Y′, U and V components in Y′UV420 are encoded separately in sequential blocks. A Y′ value is stored for every pixel, followed by a U value for each 2×2 square block of pixels, and finally a V value for each 2×2 block. Corresponding Y′, U and V values are shown using the same color in the diagram above. Read line-by-line as a byte stream from a device, the Y′ block would be found at position 0, the U block at position x×y (6×4 = 24 in this example) and the V block at position x×y + (x×y)/4 (here, 6×4 + (6×4)/4 = 30).

This format (NV21) is the standard picture format on Android camera preview. YUV 4:2:0 planar image, with 8 bit Y samples, followed by interleaved V/U plane with 8bit 2x2 subsampled chroma samples.[18]

C++ code used on Android to convert pixels of YUVImage:[19]

void YUVImage::yuv2rgb(uint8_t yValue, uint8_t uValue, uint8_t vValue,
        uint8_t *r, uint8_t *g, uint8_t *b) const {
    int rTmp = yValue + (1.370705 * (vValue-128)); 
    // or fast integer computing with a small approximation
    // rTmp = yValue + (351*(vValue-128))>>8;
    int gTmp = yValue - (0.698001 * (vValue-128)) - (0.337633 * (uValue-128)); 
    // gTmp = yValue - (179*(vValue-128) + 86*(uValue-128))>>8;
    int bTmp = yValue + (1.732446 * (uValue-128));
    // bTmp = yValue + (443*(uValue-128))>>8;
    *r = clamp(rTmp, 0, 255);
    *g = clamp(gTmp, 0, 255);
    *b = clamp(bTmp, 0, 255);
}