Part Recognition / Classification with OpenCV

Question

I'm just starting to learn OpenCV and EmguCV.NET wrappers for it, and I need general guidance from people who understand the big picture of its capabilities, and maybe from those who have solved a task similar to the one I have to complete.

I will have a series of photos, and an object or nothing will appear on each image. The objects are pieces of metal fasteners (bolts) and will lie on their sides so that their length is parallel to the top / bottom of the image (i.e. the picture is taken from above). If there is an object, it will be one of about 100 individual types of bolts, some with very similar, but not identical, characteristics and sizes. For example, all of them will be mostly rectangular in the profile, but will vary in length and width (diameter) and may have hexagonal or round heads (which will be visible in the profile as rectangular or secondary segments of the circle, respectively). ) or will have conical heads for recessed applications. Illustration of the types of parts I'm talking about (this is just to demonstrate the types of parts - my images are photographs of individual parts):

_{(source: donsnotes.com )}

I need to classify them so that all sizes and types are differentiated. A 1-3 / 8 "bolt should be classified as different from a 1-1 / 2" bolt, even if they have the same diameter and type of head. The minimum length difference between the sizes will be 1/8 ", not the standard 1/16".

If that matters, I have good control over the following:

• Lighting (but backlighting will not be practical)
• Appearance of the background (can it be useful for subtracting the background?)
• The distance from the camera to the object (identical objects will always have the same size in the images)
• As a rule, the position of the bolt - it will lie horizontally on its side, parallel to the upper / lower edges of the frame image. I can not control whether his head is left or right in the image.

Unfortunately, I cannot find any online articles or articles that directly relate to what I need to do - but many of them illustrate simpler tasks, such as finding a colored ball or finding rectangular rectangles. I cannot find anything by identifying and classifying each of a large number of different, but similar figures. I have two of the proposed OpenCV books, and although they are great, they don't seem to solve this problem.

I found quite clean Canny edges on my image samples, but there was a lot of noise due to the lighting inside the part. This makes finding the clean stretches of the Hough line quite spotty.

I'm not sure if I should try to reduce the list of possible matches using absolute measurements calculated by measuring along the edges of the Canny - and then use something more reliable, for example, a cascading classifier ...? Or what.

I'm really just looking for someone's opinion on a common strategy or a point in the right direction ...

Can anyone give me something to start trying? I am really at a loss.

Thanks!

Answer 1

From my experience, I offer you the following. You know better than most of us how to manually classify, right? So think about what happens in your brain while you classify. Suppose you see a circular shape on top, so now your problem is how to identify this shape? Lay out such problems here, it will be much easier for other people. Selecting objects is nothing but the brain thinks when you see this object.

On this issue, I suggest that you first preserve the dark background so that it becomes necessary to binarize this image. Then, if its view is from above, you can see a circle, a hexagon, etc. Then get only the ribs. Then we get the minimum bounding circle and get its diameter. The algorithm and code to find the minimum bounding circle can be obtained here .

As for the shape, I suggest you take the gradient of this binary image, and then calculate the angle of this gradient only at the boundary points (which you got in the top view). The histogram of these gradients will be your vector function. See Graph of strokes of various shapes. If your brain can tell, then you might think about which classifier to use. I would rather not comment on it now, because it depends on a lot of things, such as the distribution of functions, their separability and, most importantly, the need for speed. But now do not worry about the classifier.

Now let's try to cope with the height. I assume that you can get a front view and then just calculate the bounding box (you can get this from the regionprops function in MATLAB).

Please note that I just imagine and say all this. You must first do what I said in the first paragraph, and then see if the other part of the answer makes sense. I assume that you have a certain mathematical background for understanding certain terms in this answer. If not, please feel free to ask.

PS +1 for a good question.

Answer 2

Your problem is a typical pattern recognition task. In your script, I would use a form handle and then use it for recognition. An interesting approach is described in the article "Matching the shape and recognition of objects using contexts Shape".

One of the authors of Jitendra Malik is the teacher of an interesting coursera computer vision course, which I attend a couple of months ago.