OpenCV Android Performance Improvement - Quick Object Tracking

Question

I am trying to implement an application for quick tracking of objects on Android

My logic is as follows

• Remove all colors except the desired color range.
• Smooth image using GaussianBlur
• Find the largest circle radius with HoughCircles

The view of the application works fine, but the performance is poor, and I would like to speed up my work at least 5 times faster. I borrowed most of the logic from this link.

Example of quick tracking of objects

public void apply(Mat src, Mat dst) { Mat mIntermediateMat = new Mat(src.rows(), src.cols(), CvType.CV_8UC1); Mat mHsv = new Mat(src.size(), CvType.CV_8UC3); Mat mHsv2 = new Mat(src.size(), CvType.CV_8UC3); Imgproc.cvtColor(src, mHsv, Imgproc.COLOR_RGB2HSV, 3); Core.inRange(mHsv, new Scalar(0, 86, 72), new Scalar(39, 255, 255), mHsv); // red Core.inRange(mHsv, new Scalar(150, 125, 100), new Scalar(180,255,255), mHsv2); // red Core.bitwise_or(mHsv, mHsv2, mHsv); /// Reduce the noise so we avoid false circle detection Imgproc.GaussianBlur(mHsv, mHsv, new Size(7, 7), 2); Imgproc.HoughCircles(mHsv, mIntermediateMat, Imgproc.CV_HOUGH_GRADIENT,2.0,100); int maxRadious = 0; Point pt = new Point(0,0); if (mIntermediateMat.cols() > 0) { for (int x = 0; x < mIntermediateMat.cols(); x++) { double vCircle[] = mIntermediateMat.get(0,x); if (vCircle == null) break; int radius = (int)Math.round(vCircle[2]); if (radius > maxRadious) { maxRadious = radius; pt = new Point(Math.round(vCircle[0]), Math.round(vCircle[1])); } } int iLineThickness = 5; Scalar red = new Scalar(255, 0, 0); // draw the found circle Core.circle(dst, pt, maxRadious, red, iLineThickness); } } 

I was thinking about how to increase my productivity, and I would like tips that can be viable and significant.

1) Using Multi Threading. I could use a stream to capture from the camera and one to process the image. From the OpenCV release notes for Android, I see "Enabled multithreaded support with TBB (only a few features have been optimized at the moment)." However, I do not understand this. Is TBB for Intel chips only? What features are available? Are there suitable examples for Android and OpenCV?

2) Using a more powerful Android device. I am currently working on a 2012 Nexus 7 using the front camera. I'm actually not very good at what specifications are important to me. Nexus 7 (2012) has a quad-core processor NVIDIA Tegra 3 with a clock frequency of 1.3 GHz; 416 MHz NVIDIA GeForce ULP GPU.

If I had to run the fastest Android mobile handset now, how much will it differ?

What specifications are most important for this type of application

• CPU.
• GPU
• The number of cores.
• Camera frame rate.

3) Does Native C ++ use code that positively affects my performance?

4) Are there alternatives to OpenCV that I could use?

Answer 1

0) I would profile (or measure the runtime) for all the functions that you use to check that you need to optimize, and then plan further optimization.

1) Multithreading can improve the frame rate, but not lag (one key process is one frame in x ms. You have N cores, so you have N frames very quickly, then you need to wait x ms again.). I'm not sure about OpenCV, but as far as I know, Gaussian blur and Hough transform do not use multi-core processors.

2) Intel TBB is not only for Intel chips, but also for ARM, as well as for AMD chips. See Configuring OpenCV with TBB for ARM (Ubuntu, 3.0.63)

3-4). You use fairly simple algorithms, everything can be implemented independently, without OpenCV. And the OpenCV Hough conversion or Gaussian blur is pretty fast. C ++ is faster than Python, but from the point of view of "all program execution time." Python OpenCV is just wrappers over C ++ libraries, so their performance is “the same”.

Answer 2

First, as already mentioned, the profile of your code. The Android SDK profiler is awesome, perhaps the best of the few I've tried.

Here are a few things that will easily let you see some of the improvements:

• Do not declare (create instances) these data structures (Mat, Scalar, Point) inside your processing code (the code that is called for each captured image). Try reusing them.

• You do not need to use the full image scale to track objects, you can resize (scale) each image frame or use the image ROI: process a smaller area of ​​the image.

• Nexus 7 supports OpenCV NEON optimization, it's an optimization supported by NVIDIA Tegra hardware, and look at that. Basically you will need OpenCV compiled with NEON support, you will find documentation if you are looking for it.

Edit:

Since you mentioned that the problem with GaussianBlur is a problem, you can try other types of blur (median, normalized box) , which are faster, and you can also increase the sliding window (aka Kernel) size (3rd parameter), the larger the core. the faster it goes through the image.

http://docs.opencv.org/doc/tutorials/imgproc/gausian_median_blur_bilateral_filter/gausian_median_blur_bilateral_filter.html