Mapping pixel formats from CVPixelBuffer to equivalent V4L

Question

I need to display the range of OSX CoreVideo pixel formats listed in CVPixelBuffer.h , their equivalents in V4L . For example, kCVPixelFormatType_24RGB will map to V4L2_PIX_FMT_RGB24 .

I tried matching with fourcc , but the definitions for OSX and V4L do not match. In addition, to check their exact location and combine them manually, is there a way programmatically or an information table that will show me, for example, where kCVPixelFormatType_422YpCbCr8 will be displayed on?

Answer 1

Core Video was definitely not designed to interact with systems other than Apple, and compatibility with Core Video was never a goal for V4L, so there are some pixel formats that exist in only one library. Some formats, such as RGB24, exist in both, because they are common formats that Linux and OS X / iOS have to deal with. To confirm that the two pixel formats are the same, you will need to wait for their layout and match them manually. There is no function in any library to perform the mapping for you.

However, there are good comments in CVPixelBuffer.h and videodev2.h that allow you to confidently map pixel formats. The key to displaying some formats is understanding that YUV and Y'CbCr are often used interchangeably. According to Wikipedia ,

Scope of terms Y'UV, YUV, YCbCr, YPbPr, etc. sometimes ambiguous and overlapping. Historically, the terms YUV and Y'UV have been used to specifically encode color information in television systems, while YCbCr has been used to digitally encode color information suitable for compressing and transmitting video and still images such as MPEG and JPEG. Today, the term YUV is commonly used in the computer industry to describe file formats that are encoded using YCbCr.

V4L also has excellent documentation in its pixel formats, which can help match comments in CVPixelBuffer.h with V4L formats.

You specifically asked a question about what kCVPixelFormatType_422YpCbCr8 will be displayed kCVPixelFormatType_422YpCbCr8 . Given this information, it should be displayed on V4L2_PIX_FMT_UYVY . Below is a map of the formats of the main video formats to the V4L formats built using this information. If the format does not appear in the table, this means that it is not supported in both libraries. This information is not based on any actual testing, therefore it may be incorrect.

Note that only 8-bit formats appear to be shared between the two libraries. Even in cases where they seem to support the same 16 or 32-bit formats, they are not related to the differences between them. For example, kCVPixelFormatType_16Gray does not map to V4L2_PIX_FMT_Y16 because the former is big-endian and the latter is little-endian.

| Core Video | Video4Linux | | kCVPixelFormatType_16BE555 | V4L2_PIX_FMT_RGB555X | | kCVPixelFormatType_16LE555 | V4L2_PIX_FMT_RGB555 | | kCVPixelFormatType_16BE565 | V4L2_PIX_FMT_RGB565X | | kCVPixelFormatType_16LE565 | V4L2_PIX_FMT_RGB565 | | kCVPixelFormatType_24RGB | V4L2_PIX_FMT_RGB24 | | kCVPixelFormatType_24BGR | V4L2_PIX_FMT_BGR24 | | kCVPixelFormatType_32ARGB | V4L2_PIX_FMT_RGB32 | | kCVPixelFormatType_32BGRA | V4L2_PIX_FMT_BGR32 | | kCVPixelFormatType_422YpCbCr8 | V4L2_PIX_FMT_UYVY | | kCVPixelFormatType_420YpCbCr8Planar* | V4L2_PIX_FMT_YUV420 | | kCVPixelFormatType_422YpCbCr8_yuvs | V4L2_PIX_FMT_YUYV |

* The Core Video version contains a header with a large end, consisting of the location of the planes and the number of bytes in each row of each plane. The V4L version does not have this, so you will need to remove it from CoreVideo to V4L and add it to switch from V4L to Core Video.