How to find Wally with Python?

Question

Shamelessly jumping on the bandwagon :-)

Inspired by How to find Waldo with Mathematica and the following description How to find Waldo with R as a new python user, I would love to see how this can be done. It seems that python is better suited for this than R, and we don’t have to worry about licenses like with Mathematica or Matlab.

In an example like the one below, it is obvious that just using stripes will not work. It would be interesting if a simple rule-based approach could be used for difficult examples such as this.

I added the [machine-learning] tag, because, in my opinion, the application of ML methods, such as the Limited Boltzmann (RBM) approach proposed by Gregory Klopper in the original topic, will have the correct answer. There is some RBM code available in python that might be a good place to start, but obviously training data is needed for this approach.

At the 2009 IEEE International Workshop on Signal Processing Machines (MLSP 2009), they held a Data Analysis Competition: where is Wally? . Training data is provided in Matlab format. Please note that the links on this website are dead, but the data (along with the source approach taken by Sean McLoone and his colleagues here (see SCM link). It seems that you need to start from one place.

Answer 1

Here's an implementation with mahotas

from pylab import imshow import numpy as np import mahotas wally = mahotas.imread('DepartmentStore.jpg') wfloat = wally.astype(float) r,g,b = wfloat.transpose((2,0,1)) 

Divided into red, green and blue channels. Better to use floating point arithmetic, so we convert at the top.

 w = wfloat.mean(2) 

w is the white channel.

 pattern = np.ones((24,16), float) for i in xrange(2): pattern[i::4] = -1 

Create a template + 1, + 1, -1, -1 on the vertical axis. This is a shirt.

 v = mahotas.convolve(rw, pattern) 

Convert with red minus white. This will give a strong response when the shirt.

 mask = (v == v.max()) mask = mahotas.dilate(mask, np.ones((48,24))) 

Look for the maximum value and expand it to make it visible. Now we are reducing the whole image, with the exception of the region or interest:

 wally -= .8*wally * ~mask[:,:,None] imshow(wally) 

And we get !

Answer 2

You can try matching patterns and then uncheck, which caused the most similarities, and then use machine learning to narrow it down more. It is also very complicated, and with the exact match of patterns, it can simply return every face or face image. I think you will need more than machine learning if you are hoping to do it consistently.

Answer 3

Perhaps you should start by breaking down the problem into two smaller ones:

• create an algorithm that separates people from the background.
• to train the neural network classifier with the maximum possible number of positive and negative examples.

these are still two very big problems to solve ...

By the way, I would choose C ++ and open CV, it seems much more suitable for this.

Answer 4

This is not impossible, but very difficult, because you really have no example of a successful match. Often there are several states (in this case, more examples of find images for books), then you can upload several images to the image conversion program and consider it as a hidden Markov model and use something like the viterbi algorithm for output ( http: // en.wikipedia.org/wiki/Viterbi_algorithm ).

The way I approach him, but assuming you have some images that you can give him examples of the correct answer so that he can learn. If you have only one photograph, then I'm sorry that you might need a different approach.

Answer 5

I realized that there are two main functions that are almost always visible:

• red and white striped shirt.
• dark brown hair under a fancy cap

So, I would do it as follows:

striped shirt search:

• filter red and white (with thresholds in the converted to HSV image). This gives you two mask images.
• add them together → what is the main mask for finding striped shirts.
• create a new image with all filtered red converted to pure red (# FF0000) and all filtered white will be converted to pure white (#FFFFFF).
• now this pure white and white image correlates with the pattern of the strip pattern (I think that all waldo have perfectly perfect horizontal stripes, so rotation of the pattern should not be necessary). Make correlation only inside the specified main mask.
• try to bring together clusters that could be obtained from one shirt.

If there are several shirts, say, more than one cluster of positive correlation, find other functions, such as dark brown hair:

brown hair search

• filter out a specific brown hair color using a converted HSV image and some threshold values.
• find a specific area in this masked image - not too large and not too small.
• Now find the “hair area” that is just above (before) the striped shirt found and has a certain distance to the middle of the shirt.

Answer 6

Here is a solution using neural networks that work great.

The neural network learns a few resolved examples, which are marked by bounding boxes indicating where Wally appears in the picture. The goal of the network is to minimize the error between the predicted field and the actual field from the training / validation data.

The above network uses the Tensorflow Object Detection API for training and forecasting.