Chess detection moves out of successive image differences with OpenCV tools

Question

Hey, I am coding a simple robot vision system for chess, I am trying to improve some previous studies to use the camera and the standard chess set, and they are allowed to move during the game. For now, I can find the board in the image taken with the webcam, and I want to detect movements by taking the difference in successive images to determine what has changed, and use the previous board placement information to detect the moves.

My problem is that I can’t confidently detect the changes at the moment, my current pipeline looks as follows: Two images are subtracted → The histogram aligns the difference image → erases and expands the diff image to remove minor changes → make a binary copy and make a remote transform → get the biggest blob (corresponding to the highest value after DT and fill fill that blob) → repeat again until DT returns a value small enough to ignore and Change.

I code all this in OpenCV and C ++. but my fill flow does not always seem to fill the drops, so in most cases I get one detected change. I also tried using cv::inpaint , but that didn't help either. So my question is: am I just using the wrong approach or somehow turing can make change detection more reliable. In the first case, can people offer alternative routes, preferred encoded in C ++ / Python and / or OpenCV in a reasonable amount of time?

thanks

Answer 1

The problem of getting corrections on the board and detecting the movement of pieces can be solved independently, assuming that no one is moving the board, but also moving parts around.

Some thoughts on how I approach him:

Board Orientation Detection

You should be able to handle the rotation of the board in place, as well as move as long as a certain angle is maintained that allows you to see the shapes. This would help if there was something on the board that you could easily identify (for example, a marker on each corner), so that if you lose orientation (for example, someone completely removes the panel from the camera), you can easily find her again.

To track the board, you need to simulate the position of the camera relative to the board in three-dimensional space. This is the same problem as locating a camera moving around a fixed board. Egomotion problem. Once you decide, you can proceed to the next step, which detects movements and tracking objects.

Pieces Motion Detection

This is probably the simpler part of the problem. There are many algorithms for detecting objects in a video. I would add that you can use "key" frames. What I mean is to determine the framework in which you see only the boards before and after one move. for example, you don’t see the hand moving it, hiding the pieces, etc. After you have the frame before / after, you can find out what is moving and where it is located relative to the board.

You may be able to avoid the inability to recognize the shape of each part if you assume continuity (i.e. you have been tracking all the movements since the original location of the board, which is well known).