JavaCV Android, weird colors when overlapping while streaming to rtmp server

Question

I want to play facebook live with Android. I was able to adapt an existing example for streaming in FB.

This first step works more or less (sound is still a problem, but not in its scope). I can transfer to FB.

Now I want to overlay the stream with a transparent png image.

I create FFmpegFrameFilter at startup:

try{ filter = new FFmpegFrameFilter("movie="+path+"/image.png [logo];[in][logo]overlay=0:0:format=rgb [out]",imageWidth, imageHeight); filter.start(); }catch (FrameFilter.Exception e){ Log.e(CLASS_LABEL,"Error while starting filter: "+e.getMessage()); e.printStackTrace(); } 

and on each frame I do the following:

 filter.push(yuvImage); Frame frame; while ((frame = filter.pull()) != null) { recorder.record(frame,avutil.AV_PIX_FMT_NV21); } 

The problem is that I have no idea which pixel format I should use. My overlay image has rgb colors ( https://postimg.org/image/f1ri3vj43/ )

In the above pixel format, I get something like this: https://postimg.org/image/45ha64q9z/

I am very upset since I already tried a lot of pixels. All with a different exit, sometimes the logo appears several times.

Is there a way to find out which one should I choose from the capabilities of avutil.java?

EDIT: you can find all the code https://github.com/y4nnick/android_streaming/blob/master/app/src/main/java/com/example/yannick/olay1/RecordActivity.java

EDIT: I have already tried the following formats:

 // AV_PIX_FMT_ARGB --> 4 at once, all black/white // AV_PIX_FMT_0RGB --> 4 at once, all black/white // AV_PIX_FMT_BGR8 --> 1 a bit to big, strange colors // AV_PIX_FMT_BGR4_BYTE --> 1 a bit to big, stranger blue tint // AV_PIX_FMT_YUVA422P_LIBAV --> error: Cannot initialize the conversion context. // AV_PIX_FMT_FLAG_ALPHA --> error: Cannot initialize the conversion context. // AV_PIX_FMT_FLAG_PLANAR --> error: Cannot initialize the conversion context. // AV_PIX_FMT_RGB4 --> error: Cannot initialize the conversion context. // AV_PIX_FMT_RGB32_1 --> 4 at a time, all black/white // AV_PIX_FMT_0BGR --> 4 at a time, all black/white // AV_PIX_FMT_YVYU422 --> 2 side by side, gruen, purple tint // AV_PIX_FMT_YUVJ422P --> Fatal signal 11 (SIGSEGV) at 0x61f7xf000 (code=1), thread 18401 (e.yannick.olay1) // AV_PIX_FMT_BAYER_BGGR8 --> 1 a bit to big, black/white // AV_PIX_FMT_BAYER_GBRG8 --> 1 a bit to big, black/white // AV_PIX_FMT_FLAG_RGB --> 2 a bit to big, black/white // AV_PIX_FMT_RGB555LE --> 2 a bit to big, strange colors // AV_PIX_FMT_RGB555BE --> 2 a bit to big, strange colors // AV_PIX_FMT_RGB555 --> 2 a bit to big, strange colors // AV_PIX_FMT_RGB4_BYTE --> 1 a bit to big, orange tint // AV_PIX_FMT_RGBA64 --> 8 side by side, black/white // AV_PIX_FMT_RGB24 --> 3 side by side, red tint 

Answer 1

I understood:

• Find out which formats your Android camera supports by running

   mCamera = Camera.open(); Camera.Parameters params = mCamera.getParameters(); for(int i: params.getSupportedPreviewFormats()) { Log.e(TAG, "preview format supported are = "+i); } 

  You will get a list of integers. In my case, 17 and 842094169. I will go with 842094169 (YV12). So set your camera for her by calling

   mCamera.getParameters().setPreviewFormat(ImageFormat.YV12); 

• Open https://developer.android.com/reference/android/graphics/ImageFormat.html and check how your supported formats are precisely defined. 842094169 YV12 is a 4:2:0 YCrCb planar format comprised of a WxH Y plane followed by (W/2) x (H/2) Cr and Cb planes.

• Open https://ffmpeg.org/ffmpeg-filters.html#overlay-1 and check your filter you want to use, my overlap filter. This filter provides a parameter called “format” that sets its closest (YV12 is not listed) that you get from your camera. In my case, "yuv420". Now the filter looks like this:

   filter = new FFmpegFrameFilter("movie="+path+"/image.png [logo];[in][logo]overlay=0:0:format=yuv420 [out]",imageWidth, imageHeight); 

• Set the correct pixel format during frame recording:

   Frame frame; while ((frame = filter.pull()) != null) { Log.i(LOG_TAG,"In record-pull"); recorder.record(frame,avutil.AV_PIX_FMT_YUV420P); } 

In fact, you should install all your pixel formatters the same way. It's a little hard to find all sites to find a format that supports all your components.

A suggested alternative would be to convert the YUV image from the camera to an RGB image. See Convert YUV-> RGB (image processing) → YUV during onPreviewFrame in android?

I have not tried this because I taught that it should be slow and try a simpler solution.