Explanation of JavaCV FFmpegFrameRecorder Properties Required

Question

I use FFmpegFrameRecorder to get video input from my webcam and record it in a video file. The problem is that I build my application using several different source codes that I found, and I use properties, some of which are not entirely clear to me.

Firstly, here is my code snippet:

 FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(FILENAME, grabber.getImageWidth(),grabber.getImageHeight()); recorder.setVideoCodec(13); recorder.setFormat("mp4"); recorder.setPixelFormat(avutil.PIX_FMT_YUV420P); recorder.setFrameRate(30); recorder.setVideoBitrate(10 * 1024 * 1024); recorder.start(); 

• setVideoCodec (13) - What is the point of this (13) , how can I understand what actual codec is behind any number?
• setPixelFormat - just get it, don't know what it does at all
• setFrameRate (30) . I think this should be pretty clear, but still what is the logic of what frame rate we choose (not the highest, the better?)
• setVideoBitrate (10 * 1024 * 1024) - again almost does not understand what it does and what is the number behind the numbers?

In the end, I just want to mention one last issue in which I get a video like this. If the actual video length can be called 20 seconds. When I play a video file created from a program, it works much faster. I can’t say if it is exactly 2 times faster than it should be, but in general, if I record a 20-second video, then it plays for about 10 seconds. What can cause this and how can I fix it?

Answer 1

VideoCodec can be selected from this list found in avcodec.h / avcodec.java (As you can see, number 13 gets us MPEG4, but there are others, but FFmpeg does not have an encoder for all of them):

  AV_CODEC_ID_MPEG1VIDEO = 1, /** preferred ID for MPEG-1/2 video decoding */ AV_CODEC_ID_MPEG2VIDEO = 2, AV_CODEC_ID_MPEG2VIDEO_XVMC = 3, AV_CODEC_ID_H261 = 4, AV_CODEC_ID_H263 = 5, AV_CODEC_ID_RV10 = 6, AV_CODEC_ID_RV20 = 7, AV_CODEC_ID_MJPEG = 8, AV_CODEC_ID_MJPEGB = 9, AV_CODEC_ID_LJPEG = 10, AV_CODEC_ID_SP5X = 11, AV_CODEC_ID_JPEGLS = 12, AV_CODEC_ID_MPEG4 = 13, AV_CODEC_ID_RAWVIDEO = 14, AV_CODEC_ID_MSMPEG4V1 = 15, AV_CODEC_ID_MSMPEG4V2 = 16, AV_CODEC_ID_MSMPEG4V3 = 17, AV_CODEC_ID_WMV1 = 18, AV_CODEC_ID_WMV2 = 19, AV_CODEC_ID_H263P = 20, AV_CODEC_ID_H263I = 21, AV_CODEC_ID_FLV1 = 22, AV_CODEC_ID_SVQ1 = 23, AV_CODEC_ID_SVQ3 = 24, AV_CODEC_ID_DVVIDEO = 25, AV_CODEC_ID_HUFFYUV = 26, AV_CODEC_ID_CYUV = 27, AV_CODEC_ID_H264 = 28, AV_CODEC_ID_INDEO3 = 29, AV_CODEC_ID_VP3 = 30, AV_CODEC_ID_THEORA = 31, AV_CODEC_ID_ASV1 = 32, AV_CODEC_ID_ASV2 = 33, AV_CODEC_ID_FFV1 = 34, AV_CODEC_ID_4XM = 35, AV_CODEC_ID_VCR1 = 36, AV_CODEC_ID_CLJR = 37, AV_CODEC_ID_MDEC = 38, AV_CODEC_ID_ROQ = 39, AV_CODEC_ID_INTERPLAY_VIDEO = 40, AV_CODEC_ID_XAN_WC3 = 41, AV_CODEC_ID_XAN_WC4 = 42, AV_CODEC_ID_RPZA = 43, AV_CODEC_ID_CINEPAK = 44, AV_CODEC_ID_WS_VQA = 45, AV_CODEC_ID_MSRLE = 46, AV_CODEC_ID_MSVIDEO1 = 47, AV_CODEC_ID_IDCIN = 48, AV_CODEC_ID_8BPS = 49, AV_CODEC_ID_SMC = 50, AV_CODEC_ID_FLIC = 51, AV_CODEC_ID_TRUEMOTION1 = 52, AV_CODEC_ID_VMDVIDEO = 53, AV_CODEC_ID_MSZH = 54, AV_CODEC_ID_ZLIB = 55, AV_CODEC_ID_QTRLE = 56, AV_CODEC_ID_TSCC = 57, AV_CODEC_ID_ULTI = 58, AV_CODEC_ID_QDRAW = 59, AV_CODEC_ID_VIXL = 60, AV_CODEC_ID_QPEG = 61, AV_CODEC_ID_PNG = 62, AV_CODEC_ID_PPM = 63, AV_CODEC_ID_PBM = 64, AV_CODEC_ID_PGM = 65, AV_CODEC_ID_PGMYUV = 66, AV_CODEC_ID_PAM = 67, AV_CODEC_ID_FFVHUFF = 68, AV_CODEC_ID_RV30 = 69, AV_CODEC_ID_RV40 = 70, AV_CODEC_ID_VC1 = 71, AV_CODEC_ID_WMV3 = 72, AV_CODEC_ID_LOCO = 73, AV_CODEC_ID_WNV1 = 74, AV_CODEC_ID_AASC = 75, AV_CODEC_ID_INDEO2 = 76, AV_CODEC_ID_FRAPS = 77, AV_CODEC_ID_TRUEMOTION2 = 78, AV_CODEC_ID_BMP = 79, AV_CODEC_ID_CSCD = 80, AV_CODEC_ID_MMVIDEO = 81, AV_CODEC_ID_ZMBV = 82, AV_CODEC_ID_AVS = 83, AV_CODEC_ID_SMACKVIDEO = 84, AV_CODEC_ID_NUV = 85, AV_CODEC_ID_KMVC = 86, AV_CODEC_ID_FLASHSV = 87, AV_CODEC_ID_CAVS = 88, AV_CODEC_ID_JPEG2000 = 89, AV_CODEC_ID_VMNC = 90, AV_CODEC_ID_VP5 = 91, AV_CODEC_ID_VP6 = 92, AV_CODEC_ID_VP6F = 93, AV_CODEC_ID_TARGA = 94, AV_CODEC_ID_DSICINVIDEO = 95, AV_CODEC_ID_TIERTEXSEQVIDEO = 96, AV_CODEC_ID_TIFF = 97, AV_CODEC_ID_GIF = 98, AV_CODEC_ID_DXA = 99, AV_CODEC_ID_DNXHD = 100, AV_CODEC_ID_THP = 101, AV_CODEC_ID_SGI = 102, AV_CODEC_ID_C93 = 103, AV_CODEC_ID_BETHSOFTVID = 104, AV_CODEC_ID_PTX = 105, AV_CODEC_ID_TXD = 106, AV_CODEC_ID_VP6A = 107, AV_CODEC_ID_AMV = 108, AV_CODEC_ID_VB = 109, AV_CODEC_ID_PCX = 110, AV_CODEC_ID_SUNRAST = 111, AV_CODEC_ID_INDEO4 = 112, AV_CODEC_ID_INDEO5 = 113, AV_CODEC_ID_MIMIC = 114, AV_CODEC_ID_RL2 = 115, AV_CODEC_ID_ESCAPE124 = 116, AV_CODEC_ID_DIRAC = 117, AV_CODEC_ID_BFI = 118, AV_CODEC_ID_CMV = 119, AV_CODEC_ID_MOTIONPIXELS = 120, AV_CODEC_ID_TGV = 121, AV_CODEC_ID_TGQ = 122, AV_CODEC_ID_TQI = 123, AV_CODEC_ID_AURA = 124, AV_CODEC_ID_AURA2 = 125, AV_CODEC_ID_V210X = 126, AV_CODEC_ID_TMV = 127, AV_CODEC_ID_V210 = 128, AV_CODEC_ID_DPX = 129, AV_CODEC_ID_MAD = 130, AV_CODEC_ID_FRWU = 131, AV_CODEC_ID_FLASHSV2 = 132, AV_CODEC_ID_CDGRAPHICS = 133, AV_CODEC_ID_R210 = 134, AV_CODEC_ID_ANM = 135, AV_CODEC_ID_BINKVIDEO = 136, AV_CODEC_ID_IFF_ILBM = 137, AV_CODEC_ID_IFF_BYTERUN1 = 138, AV_CODEC_ID_KGV1 = 139, AV_CODEC_ID_YOP = 140, AV_CODEC_ID_VP8 = 141, AV_CODEC_ID_PICTOR = 142, AV_CODEC_ID_ANSI = 143, AV_CODEC_ID_A64_MULTI = 144, AV_CODEC_ID_A64_MULTI5 = 145, AV_CODEC_ID_R10K = 146, AV_CODEC_ID_MXPEG = 147, AV_CODEC_ID_LAGARITH = 148, AV_CODEC_ID_PRORES = 149, AV_CODEC_ID_JV = 150, AV_CODEC_ID_DFA = 151, AV_CODEC_ID_WMV3IMAGE = 152, AV_CODEC_ID_VC1IMAGE = 153, AV_CODEC_ID_UTVIDEO = 154, AV_CODEC_ID_BMV_VIDEO = 155, AV_CODEC_ID_VBLE = 156, AV_CODEC_ID_DXTORY = 157, AV_CODEC_ID_V410 = 158, AV_CODEC_ID_XWD = 159, AV_CODEC_ID_CDXL = 160, AV_CODEC_ID_XBM = 161, AV_CODEC_ID_ZEROCODEC = 162, AV_CODEC_ID_MSS1 = 163, AV_CODEC_ID_MSA1 = 164, AV_CODEC_ID_TSCC2 = 165, AV_CODEC_ID_MTS2 = 166, AV_CODEC_ID_CLLC = 167, AV_CODEC_ID_MSS2 = 168, AV_CODEC_ID_VP9 = 169, AV_CODEC_ID_AIC = 170, // etc 

PixelFormat can be selected from this list in pixfmt.h / avutil.java , but each codec only supports some of them (most of them support at least AV_PIX_FMT_YUV420P ):

  /** planar YUV 4:2:0, 12bpp, (1 Cr & Cb sample per 2x2 Y samples) */ AV_PIX_FMT_YUV420P = 0, /** packed YUV 4:2:2, 16bpp, Y0 Cb Y1 Cr */ AV_PIX_FMT_YUYV422 = 1, /** packed RGB 8:8:8, 24bpp, RGBRGB... */ AV_PIX_FMT_RGB24 = 2, /** packed RGB 8:8:8, 24bpp, BGRBGR... */ AV_PIX_FMT_BGR24 = 3, /** planar YUV 4:2:2, 16bpp, (1 Cr & Cb sample per 2x1 Y samples) */ AV_PIX_FMT_YUV422P = 4, /** planar YUV 4:4:4, 24bpp, (1 Cr & Cb sample per 1x1 Y samples) */ AV_PIX_FMT_YUV444P = 5, /** planar YUV 4:1:0, 9bpp, (1 Cr & Cb sample per 4x4 Y samples) */ AV_PIX_FMT_YUV410P = 6, /** planar YUV 4:1:1, 12bpp, (1 Cr & Cb sample per 4x1 Y samples) */ AV_PIX_FMT_YUV411P = 7, /** Y , 8bpp */ AV_PIX_FMT_GRAY8 = 8, /** Y , 1bpp, 0 is white, 1 is black, in each byte pixels are ordered from the msb to the lsb */ AV_PIX_FMT_MONOWHITE = 9, /** Y , 1bpp, 0 is black, 1 is white, in each byte pixels are ordered from the msb to the lsb */ AV_PIX_FMT_MONOBLACK = 10, /** 8 bit with PIX_FMT_RGB32 palette */ AV_PIX_FMT_PAL8 = 11, /** planar YUV 4:2:0, 12bpp, full scale (JPEG), deprecated in favor of PIX_FMT_YUV420P and setting color_range */ AV_PIX_FMT_YUVJ420P = 12, /** planar YUV 4:2:2, 16bpp, full scale (JPEG), deprecated in favor of PIX_FMT_YUV422P and setting color_range */ AV_PIX_FMT_YUVJ422P = 13, /** planar YUV 4:4:4, 24bpp, full scale (JPEG), deprecated in favor of PIX_FMT_YUV444P and setting color_range */ AV_PIX_FMT_YUVJ444P = 14, /** XVideo Motion Acceleration via common packet passing */ AV_PIX_FMT_XVMC_MPEG2_MC = 15, AV_PIX_FMT_XVMC_MPEG2_IDCT = 16; /** packed YUV 4:2:2, 16bpp, Cb Y0 Cr Y1 */ AV_PIX_FMT_UYVY422 = 17, /** packed YUV 4:1:1, 12bpp, Cb Y0 Y1 Cr Y2 Y3 */ AV_PIX_FMT_UYYVYY411 = 18, /** packed RGB 3:3:2, 8bpp, (msb)2B 3G 3R(lsb) */ AV_PIX_FMT_BGR8 = 19, /** packed RGB 1:2:1 bitstream, 4bpp, (msb)1B 2G 1R(lsb), a byte contains two pixels, the first pixel in the byte is the one composed by the 4 msb bits */ AV_PIX_FMT_BGR4 = 20, /** packed RGB 1:2:1, 8bpp, (msb)1B 2G 1R(lsb) */ AV_PIX_FMT_BGR4_BYTE = 21, /** packed RGB 3:3:2, 8bpp, (msb)2R 3G 3B(lsb) */ AV_PIX_FMT_RGB8 = 22, /** packed RGB 1:2:1 bitstream, 4bpp, (msb)1R 2G 1B(lsb), a byte contains two pixels, the first pixel in the byte is the one composed by the 4 msb bits */ AV_PIX_FMT_RGB4 = 23, /** packed RGB 1:2:1, 8bpp, (msb)1R 2G 1B(lsb) */ AV_PIX_FMT_RGB4_BYTE = 24, /** planar YUV 4:2:0, 12bpp, 1 plane for Y and 1 plane for the UV components, which are interleaved (first byte U and the following byte V) */ AV_PIX_FMT_NV12 = 25, /** as above, but U and V bytes are swapped */ AV_PIX_FMT_NV21 = 26, /** packed ARGB 8:8:8:8, 32bpp, ARGBARGB... */ AV_PIX_FMT_ARGB = 27, /** packed RGBA 8:8:8:8, 32bpp, RGBARGBA... */ AV_PIX_FMT_RGBA = 28, /** packed ABGR 8:8:8:8, 32bpp, ABGRABGR... */ AV_PIX_FMT_ABGR = 29, /** packed BGRA 8:8:8:8, 32bpp, BGRABGRA... */ AV_PIX_FMT_BGRA = 30, /** Y , 16bpp, big-endian */ AV_PIX_FMT_GRAY16BE = 31, /** Y , 16bpp, little-endian */ AV_PIX_FMT_GRAY16LE = 32, /** planar YUV 4:4:0 (1 Cr & Cb sample per 1x2 Y samples) */ AV_PIX_FMT_YUV440P = 33, /** planar YUV 4:4:0 full scale (JPEG), deprecated in favor of PIX_FMT_YUV440P and setting color_range */ AV_PIX_FMT_YUVJ440P = 34, /** planar YUV 4:2:0, 20bpp, (1 Cr & Cb sample per 2x2 Y & A samples) */ AV_PIX_FMT_YUVA420P = 35, /** H.264 HW decoding with VDPAU, data[0] contains a vdpau_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VDPAU_H264 = 36, /** MPEG-1 HW decoding with VDPAU, data[0] contains a vdpau_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VDPAU_MPEG1 = 37, /** MPEG-2 HW decoding with VDPAU, data[0] contains a vdpau_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VDPAU_MPEG2 = 38, /** WMV3 HW decoding with VDPAU, data[0] contains a vdpau_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VDPAU_WMV3 = 39, /** VC-1 HW decoding with VDPAU, data[0] contains a vdpau_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VDPAU_VC1 = 40, /** packed RGB 16:16:16, 48bpp, 16R, 16G, 16B, the 2-byte value for each R/G/B component is stored as big-endian */ AV_PIX_FMT_RGB48BE = 41, /** packed RGB 16:16:16, 48bpp, 16R, 16G, 16B, the 2-byte value for each R/G/B component is stored as little-endian */ AV_PIX_FMT_RGB48LE = 42, /** packed RGB 5:6:5, 16bpp, (msb) 5R 6G 5B(lsb), big-endian */ AV_PIX_FMT_RGB565BE = 43, /** packed RGB 5:6:5, 16bpp, (msb) 5R 6G 5B(lsb), little-endian */ AV_PIX_FMT_RGB565LE = 44, /** packed RGB 5:5:5, 16bpp, (msb)1A 5R 5G 5B(lsb), big-endian, most significant bit to 0 */ AV_PIX_FMT_RGB555BE = 45, /** packed RGB 5:5:5, 16bpp, (msb)1A 5R 5G 5B(lsb), little-endian, most significant bit to 0 */ AV_PIX_FMT_RGB555LE = 46, /** packed BGR 5:6:5, 16bpp, (msb) 5B 6G 5R(lsb), big-endian */ AV_PIX_FMT_BGR565BE = 47, /** packed BGR 5:6:5, 16bpp, (msb) 5B 6G 5R(lsb), little-endian */ AV_PIX_FMT_BGR565LE = 48, /** packed BGR 5:5:5, 16bpp, (msb)1A 5B 5G 5R(lsb), big-endian, most significant bit to 1 */ AV_PIX_FMT_BGR555BE = 49, /** packed BGR 5:5:5, 16bpp, (msb)1A 5B 5G 5R(lsb), little-endian, most significant bit to 1 */ AV_PIX_FMT_BGR555LE = 50, /** HW acceleration through VA API at motion compensation entry-point, Picture.data[3] contains a vaapi_render_state struct which contains macroblocks as well as various fields extracted from headers */ AV_PIX_FMT_VAAPI_MOCO = 51, /** HW acceleration through VA API at IDCT entry-point, Picture.data[3] contains a vaapi_render_state struct which contains fields extracted from headers */ AV_PIX_FMT_VAAPI_IDCT = 52, /** HW decoding through VA API, Picture.data[3] contains a vaapi_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VAAPI_VLD = 53, /** planar YUV 4:2:0, 24bpp, (1 Cr & Cb sample per 2x2 Y samples), little-endian */ AV_PIX_FMT_YUV420P16LE = 54, /** planar YUV 4:2:0, 24bpp, (1 Cr & Cb sample per 2x2 Y samples), big-endian */ AV_PIX_FMT_YUV420P16BE = 55, /** planar YUV 4:2:2, 32bpp, (1 Cr & Cb sample per 2x1 Y samples), little-endian */ AV_PIX_FMT_YUV422P16LE = 56, /** planar YUV 4:2:2, 32bpp, (1 Cr & Cb sample per 2x1 Y samples), big-endian */ AV_PIX_FMT_YUV422P16BE = 57, /** planar YUV 4:4:4, 48bpp, (1 Cr & Cb sample per 1x1 Y samples), little-endian */ AV_PIX_FMT_YUV444P16LE = 58, /** planar YUV 4:4:4, 48bpp, (1 Cr & Cb sample per 1x1 Y samples), big-endian */ AV_PIX_FMT_YUV444P16BE = 59, /** MPEG4 HW decoding with VDPAU, data[0] contains a vdpau_render_state struct which contains the bitstream of the slices as well as various fields extracted from headers */ AV_PIX_FMT_VDPAU_MPEG4 = 60, /** HW decoding through DXVA2, Picture.data[3] contains a LPDIRECT3DSURFACE9 pointer */ AV_PIX_FMT_DXVA2_VLD = 61, /** packed RGB 4:4:4, 16bpp, (msb)4A 4R 4G 4B(lsb), little-endian, most significant bits to 0 */ AV_PIX_FMT_RGB444LE = 62, /** packed RGB 4:4:4, 16bpp, (msb)4A 4R 4G 4B(lsb), big-endian, most significant bits to 0 */ AV_PIX_FMT_RGB444BE = 63, /** packed BGR 4:4:4, 16bpp, (msb)4A 4B 4G 4R(lsb), little-endian, most significant bits to 1 */ AV_PIX_FMT_BGR444LE = 64, /** packed BGR 4:4:4, 16bpp, (msb)4A 4B 4G 4R(lsb), big-endian, most significant bits to 1 */ AV_PIX_FMT_BGR444BE = 65, /** 8bit gray, 8bit alpha */ AV_PIX_FMT_YA8 = 66, // etc 

FrameRate indicates the number of frames per second that the video should play (it has nothing to do with the number or display time of the images you actually record, although this provides the basis for encoding the bitrate). So, in the case of 30 FPS, to cover 20 seconds of the video, you need to call record() 30 * 20 = 600 times. If you do not call record() 600 times, this is the cause of your problem.

VideoBitrate provides the video bitrate (in bits per second) over which the video stream should be encoded. Wikipedia is a good article about this .