How to read a frame from a YUV file in OpenCV?

Question

How to read a frame from a YUV file in OpenCV?

Answer 1

As already mentioned, there are many types of YUV formats:

http://www.fourcc.org/yuv.php

To convert to RGB from YUV to OpenCV is very simple:

• Create a one-dimensional OpenCV layout of the appropriate size for the data in this frame
• Create an empty Mat for RGB data with the desired size AND with 3 channels
• Finally, use cvtColor to convert between two mats, using the correct conversion flag enumeration

Here is an example of a YUV buffer in YV12 format:

Mat mYUV(height + height/2, width, CV_8UC1, (void*) frameData); Mat mRGB(height, width, CV_8UC3); cvtColor(mYUV, mRGB, CV_YUV2RGB_YV12, 3); 

The main trick is sizing your RGB Mat before converting.

Answer 2

UPDATE has a new version of the code: https://github.com/chelyaev/opencv-yuv

I am sending a code that will read a single YUV 4: 2: 0 file. You can directly apply this to most YUV files (just keep reading from the same FILE object). The exception to this applies to working with YUV files with a header (usually they have the extension *.y4m ). If you want to deal with such files, you have two options:

• Write your own function to use the header data from the FILE object before using the code below
• Separate captions from * .y4m images (using ffmpeg or a similar tool). This is the option that I prefer, since it is the simplest.

It will also not work for any other form of YUV format (non-planar miscellaneous thinning of color). As @Stephane noted, there are many such formats (and most of them do not have identification headers), so OpenCV probably does not support them out of the box.

But working with them is quite simple:

• Start with the image and its size (this is necessary when reading the YUV file).
• Read brightness and color in 3 separate images.
• High-quality color images 2 times to compensate for thinning color. Note that there are actually several ways to compensate for color thinning. Upsampling is simply the easiest
• Combine in YUV image. If you want to use RGB, you can use cvCvtColor .

Finally, the code:

 IplImage * cvLoadImageYUV(FILE *fin, int w, int h) { assert(fin); IplImage *py = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1); IplImage *pu = cvCreateImage(cvSize(w/2,h/2), IPL_DEPTH_8U, 1); IplImage *pv = cvCreateImage(cvSize(w/2,h/2), IPL_DEPTH_8U, 1); IplImage *pu_big = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1); IplImage *pv_big = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1); IplImage *image = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 3); IplImage *result = NULL; assert(py); assert(pu); assert(pv); assert(pu_big); assert(pv_big); assert(image); for (int i = 0; i < w*h; ++i) { int j = fgetc(fin); if (j < 0) goto cleanup; py->imageData[i] = (unsigned char) j; } for (int i = 0; i < w*h/4; ++i) { int j = fgetc(fin); if (j < 0) goto cleanup; pu->imageData[i] = (unsigned char) j; } for (int i = 0; i < w*h/4; ++i) { int j = fgetc(fin); if (j < 0) goto cleanup; pv->imageData[i] = (unsigned char) j; } cvResize(pu, pu_big, CV_INTER_NN); cvResize(pv, pv_big, CV_INTER_NN); cvMerge(py, pu_big, pv_big, NULL, image); result = image; cleanup: cvReleaseImage(&pu); cvReleaseImage(&pv); cvReleaseImage(&py); cvReleaseImage(&pu_big); cvReleaseImage(&pv_big); if (result == NULL) cvReleaseImage(&image); return result; } 

Answer 3

I wrote very simple python code to read the YUV NV21 stream from a binary file.

 import cv2 import numpy as np class VideoCaptureYUV: def __init__(self, filename, size): self.height, self.width = size self.frame_len = self.width * self.height * 3 / 2 self.f = open(filename, 'rb') self.shape = (int(self.height*1.5), self.width) def read_raw(self): try: raw = self.f.read(self.frame_len) yuv = np.frombuffer(raw, dtype=np.uint8) yuv = yuv.reshape(self.shape) except Exception as e: print str(e) return False, None return True, yuv def read(self): ret, yuv = self.read_raw() if not ret: return ret, yuv bgr = cv2.cvtColor(yuv, cv2.COLOR_YUV2BGR_NV21) return ret, bgr if __name__ == "__main__": #filename = "data/20171214180916RGB.yuv" filename = "data/20171214180916IR.yuv" size = (480, 640) cap = VideoCaptureYUV(filename, size) while 1: ret, frame = cap.read() if ret: cv2.imshow("frame", frame) cv2.waitKey(30) else: break 

Answer 4

I do not think this is possible, at least with the current version. Of course, this would not be so difficult, but it is not such an interesting feature as:

• OpenCV typically works with a webcam stream that is in RGB format, or on encoded files that are directly decoded in RGB for display.
• OpenCV is dedicated to Computer Vision, where YUV is a less common format than, for example, in the coding community;
• There are many different YUV formats, which will mean a lot of work to implement them.

Conversions are still possible using cvCvtColor() , which means this is of some interest.

Answer 5

I ran into the same problem. My solution 1. read one yuv frame (for example, I420) for the string object "yuv". 2. Convert the yuv frame to BGR24 format. I am using libyuv for this. It's easy to write a python shell for libyuv functions. now you get another string object "bgr" with the format BGR24. 3. use numpy.fromstring to get the image object from the string object "bgr". you need to change the shape of the image object.

Below is a simple yuv viewer for your reference.

 import cv2 # below is the extension wrapper for libyuv import yuvtorgb import numpy as np f = open('i420_cif.yuv', 'rb') w = 352 h = 288 size = 352*288*3/2 while True: try: yuv = f.read(size) except: break if len(yuv) != size: f.seek(0, 0) continue bgr = yuvtorgb.i420_to_bgr24(yuv, w, h) img = np.fromstring(bgr, dtype=np.uint8) img.shape = h,w,3 cv2.imshow('img', img) if cv2.waitKey(50) & 0xFF == ord('q'): break cv2.destroyAllWindows()