What does line alignment mean?

Question

I follow the ffmpeg tutorial at http://dranger.com/ffmpeg/tutorial01.html .

I just found that the avpicture_get_size function avpicture_get_size deprecated.

So, I checked the ffmpeg document ( https://www.ffmpeg.org/doxygen/3.0/group__lavu__picture.html#ga24a67963c3ae0054a2a4bab35930e694 ) and found the replacement av_image_get_buffer_size .

But I can not understand the align parameter, meaning "line alignment" ......

What does it mean?

Answer 1

Some parts of FFmpeg, especially libavcodec, require aligned lines [], which means that this requires:

 assert(linesize[0] % 32 == 0); assert(linesize[1] % 32 == 0); assert(linesize[2] % 32 == 0); 

This allows you to use fast / aligned SIMD routines (for example, movdqa or vmovdqa SSE2 / AVX2 vmovdqa ) to access data instead of slower inconsistent copies.

The align parameter for this function av_image_get_buffer_size is the alignment of the string, and you need it because it affects the size of the buffer. For example, the size of the Y plane in the YUV buffer is not really the width * of the height, it aligns the height [0] *. You will see this (especially for image sizes that are not a multiple of 16 or 32), as align increases to higher degrees 2, the return value slowly increases.

Practically speaking, if you are going to use this image as an output buffer for calls, for example. avcodec_decode_video2 , this should be 32. For swscale / avfilter, I believe there is no absolute requirement, but 32 is still recommended for you.

Answer 2

My practice:

1 . avpicture is an outdated problem. I am replacing autoobject functions with AVFrame and imgutils . sample code:

  //AVPicture _picture; AVFrame *_pictureFrame; uint8_t *_pictureFrameData; 

...

 //_pictureValid = avpicture_alloc(&_picture, // AV_PIX_FMT_RGB24, // _videoCodecCtx->width, // _videoCodecCtx->height) == 0; _pictureFrame = av_frame_alloc(); _pictureFrame->width = _videoCodecCtx->width; _pictureFrame->height = _videoCodecCtx->height; _pictureFrame->format = AV_PIX_FMT_RGB24; int size = av_image_get_buffer_size(_pictureFrame->format, _pictureFrame->width, _pictureFrame->height, 1); //dont forget to free _pictureFrameData at last _pictureFrameData = (uint8_t*)av_malloc(size); av_image_fill_arrays(_pictureFrame->data, _pictureFrame->linesize, _pictureFrameData, _pictureFrame->format, _pictureFrame->width, _pictureFrame->height, 1); 

...

 if (_pictureFrame) { av_free(_pictureFrame); if (_pictureFrameData) { free(_pictureFrameData); } } 

2.align parameter

at first I set align to 32 , but for some video streams it did not work, causing distorted images. Then I set it to 16 (my environment: mac, Xcode, iPhone6), some threads work fine. But finally, I set align to 1 because I found this

 Fill in the AVPicture fields, always assume a linesize alignment of 1. 

Answer 3

If you look at the definition of avpicture_get_size in version 3.2, you will see the following code:

 int avpicture_get_size(enum AVPixelFormat pix_fmt, int width, int height) { return av_image_get_buffer_size(pix_fmt, width, height, 1); } 

It simply calls the proposed function: av_image_get_buffer_size with the align parameter set to 1 . I did not go further to find out the full significance of why 1 used for the amortized function. As usual with ffmpeg, you can probably figure this out by reading the correct code and enough code (with some code experiments).