Highlighting an OpenCV Background Image

Question

I am working on a robotics project using an Android phone as the main processor and camera for motion detection. I received the Android binary package from OpenCV and installed it correctly. I can record images using the built-in OpenCV camera and display them on the screen. However, I am having problems using the background subtraction class. I can create a new BackgroundSubtractorMOG object in the constructor, but when I try to run the code below, it forces the shutdown. I get the error message "Only single and three-channel 8-bit images are supported in BackgroundSubtractorMOG" from the native code. I tried changing Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA to Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGB and then it doesn’t force shutdown, but all I get is a black screen. I am sure bmp is still null with FRAME_RGB because the screen remains black and the fps counter that I drew right after the bitmap (removed from the code below for clarity and as a troubleshooting step) is not displayed.

I looked at the OpenCV C ++ code for this function ( line 388 here ), and the image type error occurs if the image type is not CV_8UC1 or CV_8UC3, so I tried using java CvType.CV_8UC3 instead of Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA in capture.retrieve (), but it is forcibly closed, and I got the error "Output frame format is not supported."

I assume that I just had a problem with type conversion, but I can’t understand in what life image types compatible with OpenCV for Android change with their usual image types that are documented. Any help would be appreciated.

Variables:

private SurfaceHolder mHolder; private VideoCapture mCamera; private Mat mRgba; private Mat mFGMask; private BackgroundSubtractorMOG mBGSub; 

My SurfaceView run () function:

 public void run() { Bitmap bmp = null; synchronized (this) { if (mCamera == null) break; if (!mCamera.grab()) { Log.e(TAG, "mCamera.grab() failed"); break; } processFrame(mCamera); bmp = Bitmap.createBitmap(mFGMask.cols(), mFGMask.rows(), Bitmap.Config.ARGB_8888); Utils.matToBitmap(mFGMask, bmp); } if (bmp != null) { Canvas canvas = mHolder.lockCanvas(); if (canvas != null) { canvas.drawBitmap(bmp, (canvas.getWidth() - bmp.getWidth()) / 2, (canvas.getHeight() - bmp.getHeight()) / 2, null); mHolder.unlockCanvasAndPost(canvas); } bmp.recycle(); } } 

The processFrame () function specified in run ():

 protected void processFrame(VideoCapture capture) { capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA); mBGSub.apply(mRgba, mFGMask); } 

Edit:

Solution that ended:

 protected void processFrame(VideoCapture capture) { capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGB); //GREY_FRAME also works and exhibits better performance //capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_GREY_FRAME); mBGSub.apply(mRgba, mFGMask, 0.1); Imgproc.cvtColor(mFGMask, mRgba, Imgproc.COLOR_GRAY2BGRA, 4); } 

Answer 1

You tried using cvtColor with CV_RGB2RGBA and CV_RGBA2RGB . So, maybe try converting the RGBA frame to RGB, and then do a background subtraction. Something like that:

 protected void processFrame(VideoCapture capture) { capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA); Mat rgb; Imgproc.cvtColor(mRgba, rgb, Imgproc.COLOR_RGBA2RGB); mBGSub.apply(rgb, mFGMask); } 

EDIT: You can check the OpenCV test module for BackgroundSubtractorMOG located here . However, the test has fail("Not yet implemented"); in the main test case.

I am not sure if this means that the test is not completed or that the BackgroundSubtractorMOG support is not implemented. You can try running the code contained in this unit test to see if it really works.

In addition, a C ++ usage example segment_objects.cpp may be useful as a usage example.

Hope this helps! :)

Answer 2

Thank you guys! And for future viewers who come to this page, you may need to tweak this knowledge to make it work. In the SDK v2.4.4, I applied this in the onCameraFrame method. Recall that the method takes an input frame from the camera. You use the input and return the frame that should be displayed on the screen of your Android device. Here is an example:

 //Assume appropriate imports private BackgroundSubtractorMOG sub = new BackgroundSubtractorMOG(3, 4, 0.8); private Mat mGray = new Mat(); private Mat mRgb = new Mat(); private Mat mFGMask = new Mat(); public Mat onCameraFrame(CvCameraViewFrame inputFrame) { mGray = inputFrame.gray(); //I chose the gray frame because it should require less resources to process Imgproc.cvtColor(mGray, mRgb, Imgproc.COLOR_GRAY2RGB); //the apply function will throw the above error if you don't feed it an RGB image sub.apply(mRgb, mFGMask, learningRate); //apply() exports a gray image by definition return mFGMask; } 

To talk about the gray image that comes out of apply (), if you want to make the RGBA version, you may need to use cvtcolor after calling apply ():

 private Mat mRgba = new Mat(); public Mat onCameraFrame(CvCameraViewFrame inputFrame) { mRgba = inputFrame.rgba(); Imgproc.cvtColor(mRgba, mRgb, Imgproc.COLOR_RGBA2RGB); //the apply function will throw the above error if you don't feed it an RGB image sub.apply(mRgb, mFGMask, learningRate); //apply() exports a gray image by definition Imgproc.cvtColor(mFGMask, mRgba, Imgproc.COLOR_GRAY2RGBA); return mRgba; } 

Answer 3

Also with the latest openCV you need to initialize with:

private BackgroundSubtractorMOG2 Sub = Video.createBackgroundSubtractorMOG2 ();