Face Recognition on Android

Question

I’m trying to develop a Face Recognition application on Android, and since I don’t want to use NDK in the project (I just don’t have time to switch), I adhere to developing the whole application using Java and therefore I am having some problems:

• It seems that the Contrib module is not included in OpenCV 2.4.2. Is there anyway to use it in the project?

• I tried using JavaCV to use the Contrib Module class "FaceRecognizer". There are two classes available: FaceRecognizer and FaceRecognizerPtr. Does anyone know what the difference is between the two?

• The classes mentioned above have a Train method that (In C ++) gets two vectors of the Mat and Integer types ( model->train(images,labels) & train(Vector<mat> theImages, Vector<int> theLabels ) I tried passing them to ArrayList<mat> & ArrayList<integer> and Vectors in Java, but it seems that the method explicitly accepts the data type "CvArr", which I'm not sure how to get ... Here is the error:

The train method (opencv_core.CvArr, opencv_core.CvArr) in type opencv_contrib.FaceRecognizer is not applicable for arguments (ArrayList, ArrayList)

Does anyone know how to change my ArrayList to CvArr ?!

This is my first post, and I was not sure whether to ask all three questions in one message or in three messages, so sorry for any inconvenience ... If you need other information about the project, do not hesitate to ask.

Answer 1

Update

The next article was written by Petter Christian Bjelland , so all his merits. I post it here because his blog seems to be in maintenance mode at the moment, but I think it's worth sharing.

Performing face recognition using JavaCV (from http://pcbje.com )

I could not find a tutorial on how to perform face recognition using OpenCV and Java, so I decided to share a viable solution with them. The solution is very inefficient in its current form, since the training model is built on each run, however, it shows what is needed to make it work.

The following class takes two arguments: the path to the directory containing the learning faces, and the path to the image you want to classify. Not all images should be the same size and that faces should already be cropped from their original images (see here if you have not already done face detection).

For simplicity of this message, the class also requires that the training images have the file format: <label>-rest_of_filename.png . For example:

 1-jon_doe_1.png
 1-jon_doe_2.png
 2-jane_doe_1.png
 2-jane_doe_2.png

... etc.

The code:

 import com.googlecode.javacv.cpp.opencv_core; import static com.googlecode.javacv.cpp.opencv_highgui.*; import static com.googlecode.javacv.cpp.opencv_core.*; import static com.googlecode.javacv.cpp.opencv_imgproc.*; import static com.googlecode.javacv.cpp.opencv_contrib.*; import java.io.File; import java.io.FilenameFilter; public class OpenCVFaceRecognizer { public static void main(String[] args) { String trainingDir = args[0]; IplImage testImage = cvLoadImage(args[1]); File root = new File(trainingDir); FilenameFilter pngFilter = new FilenameFilter() { public boolean accept(File dir, String name) { return name.toLowerCase().endsWith(".png"); } }; File[] imageFiles = root.listFiles(pngFilter); MatVector images = new MatVector(imageFiles.length); int[] labels = new int[imageFiles.length]; int counter = 0; int label; IplImage img; IplImage grayImg; for (File image : imageFiles) { // Get image and label: img = cvLoadImage(image.getAbsolutePath()); label = Integer.parseInt(image.getName().split("\\-")[0]); // Convert image to grayscale: grayImg = IplImage.create(img.width(), img.height(), IPL_DEPTH_8U, 1); cvCvtColor(img, grayImg, CV_BGR2GRAY); // Append it in the image list: images.put(counter, grayImg); // And in the labels list: labels[counter] = label; // Increase counter for next image: counter++; } FaceRecognizer faceRecognizer = createFisherFaceRecognizer(); // FaceRecognizer faceRecognizer = createEigenFaceRecognizer(); // FaceRecognizer faceRecognizer = createLBPHFaceRecognizer() faceRecognizer.train(images, labels); // Load the test image: IplImage greyTestImage = IplImage.create(testImage.width(), testImage.height(), IPL_DEPTH_8U, 1); cvCvtColor(testImage, greyTestImage, CV_BGR2GRAY); // And get a prediction: int predictedLabel = faceRecognizer.predict(greyTestImage); System.out.println("Predicted label: " + predictedLabel); } } 

The class requires the OpenCV Java interface. If you use Maven, you can get the necessary libraries with the following pom.xml:

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.pcbje</groupId> <artifactId>opencvfacerecognizer</artifactId> <version>0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>opencvfacerecognizer</name> <url>http://pcbje.com</url> <dependencies> <dependency> <groupId>com.googlecode.javacv</groupId> <artifactId>javacv</artifactId> <version>0.3</version> </dependency> <!-- For Linux x64 environments --> <dependency> <groupId>com.googlecode.javacv</groupId> <artifactId>javacv</artifactId> <classifier>linux-x86_64</classifier> <version>0.3</version> </dependency> <!-- For OSX environments --> <dependency> <groupId>com.googlecode.javacv</groupId> <artifactId>javacv</artifactId> <classifier>macosx-x86_64</classifier> <version>0.3</version> </dependency> </dependencies> <repositories> <repository> <id>javacv</id> <name>JavaCV</name> <url>http://maven2.javacv.googlecode.com/git/</url> </repository> </repositories> </project> 

Original post

Quote from my answer at http://answers.opencv.org/question/865/the-contrib-module-problem .

Without using javacv, let's see how far we can get just by looking at the interfaces! The project is located on googlecode, which makes it easier to view the code: http://code.google.com/p/javacv .

First, consider how cv::FaceRecognizer was wrapped ( opencv_contrib.java, line 845 at the time of this writing ):

 @Namespace("cv") public static class FaceRecognizer extends Algorithm { static { Loader.load(); } public FaceRecognizer() { } public FaceRecognizer(Pointer p) { super(p); } public /*abstract*/ native void train(@ByRef MatVector src, @Adapter("ArrayAdapter") CvArr labels); public /*abstract*/ native int predict(@Adapter("ArrayAdapter") CvArr src); public /*abstract*/ native void predict(@Adapter("ArrayAdapter") CvArr src, @ByRef int[] label, @ByRef double[] dist); public native void save(String filename); public native void load(String filename); public native void save(@Adapter("FileStorageAdapter") CvFileStorage fs); public native void load(@Adapter("FileStorageAdapter") CvFileStorage fs); } 

Aha, so you need to pass MatVector for images! You can pass tags to CvArr (single row or single column). MatVector defined in opencv_core, line 4629 (as of this writing) and looks like this:

 public static class MatVector extends Pointer { static { load(); } public MatVector() { allocate(); } public MatVector(long n) { allocate(n); } public MatVector(Pointer p) { super(p); } private native void allocate(); private native void allocate(@Cast("size_t") long n); public native long size(); public native void resize(@Cast("size_t") long n); @Index @ValueGetter public native @Adapter("MatAdapter") CvMat getCvMat(@Cast("size_t") long i); @Index @ValueGetter public native @Adapter("MatAdapter") CvMatND getCvMatND(@Cast("size_t") long i); @Index @ValueGetter public native @Adapter("MatAdapter") IplImage getIplImage(@Cast("size_t") long i); @Index @ValueSetter public native MatVector put(@Cast("size_t") long i, @Adapter("MatAdapter") CvArr value); } 

Just by looking at the code again, I think it can be used as follows:

 int numberOfImages = 10; // Allocate some memory: MatVector images = new MatVector(numberOfImages); // Then fill the MatVector, you probably want to do something useful instead: for(int idx = 0; idx < numberOfImages; idx++){ // Load an image: CvArr image = cvLoadImage("/path/to/your/image"); // And put it into the MatVector: images.put(idx, image); } 

You probably want to write yourself a method that performs a conversion from Java ArrayList to MatVector (if such a function does not already exist in javacv).

Now to your second question. FaceRecognizer is the equivalent of cv::FaceRecognizer . OpenCV C ++ native classes return cv::Ptr<cv::FaceRecognizer> , which is a (smart) pointer to cv::FaceRecognizer . It should also be wrapped. See the template here?

The FaceRecognizerPtr interface now looks like this:

 @Name("cv::Ptr<cv::FaceRecognizer>") public static class FaceRecognizerPtr extends Pointer { static { load(); } public FaceRecognizerPtr() { allocate(); } public FaceRecognizerPtr(Pointer p) { super(p); } private native void allocate(); public native FaceRecognizer get(); public native FaceRecognizerPtr put(FaceRecognizer value); } 

So, you can either get a FaceRecognizer from this class, or put a FaceRecognizer in. You only need to worry about get() , as the pointer is populated with a method that creates a specific FaceRecognizer algorithm:

 @Namespace("cv") public static native @ByVal FaceRecognizerPtr createEigenFaceRecognizer(int num_components/*=0*/, double threshold/*=DBL_MAX*/); @Namespace("cv") public static native @ByVal FaceRecognizerPtr createFisherFaceRecognizer(int num_components/*=0*/, double threshold/*=DBL_MAX*/); @Namespace("cv") public static native @ByVal FaceRecognizerPtr createLBPHFaceRecognizer(int radius/*=1*/, int neighbors/*=8*/, int grid_x/*=8*/, int grid_y/*=8*/, double threshold/*=DBL_MAX*/); 

So, once you get FaceRecognizerPtr, you can do things like:

 // Holds your training data and labels: MatVector images; CvArr labels; // Do something with the images and labels... Probably fill them? // ... // Then get a Pointer to a FaceRecognizer (FaceRecognizerPtr). // Java doesn't have default parameters, so you have to add some yourself, // if you pass 0 as num_components to the EigenFaceRecognizer, the number of // components is determined by the data, for the threshold use the maximum possible // value if you don't want one. I don't know the constant in Java: FaceRecognizerPtr model = createEigenFaceRecognizer(0, 10000); // Then train it. See how I call get(), to get the FaceRecognizer inside the FaceRecognizerPtr: model.get().train(images, labels); 

Here you will find the Eigenfaces model. What is it!

Answer 2

I made an Android face recognition application using opencv. For good recognition, you need better detection, you can check it out: https://github.com/yaylas/AndroidFaceRecognizer Hope this helps.