How to identify all line segments from a list of points created using mouse gestures?

Question

I currently work for a software company, and one of my tasks was to implement mouse gesture recognition. One of the senior developers helped me get started and provided code / projects that use Unistroke Recognizer $ 1 http://depts.washington.edu/aimgroup/proj/dollar/ . I get in broad what Unistroke Recognizer $ 1 does and how it works, but I'm a little overloaded with trying to figure out all the internal details / finer details.

My problem is that I am trying to recognize the gesture of moving the mouse, then up. The Unistroke Recognizer $ 1 recognizes that the gesture I created was a down gesture, and that is what it should do. What I really would like to do is say: "I acknowledge the fingerprint gesture AND THEN the upward gesture."

I don’t know if it’s completely incomprehensible how to recognize Unistroke Recognizer by making me scratch my head, but does anyone have any ideas on how to recognize two different gestures when moving the mouse down and then up?

Here is my idea that I thought might help me, but I would like someone to be an expert or even know a little more than me to tell me what you think. Any help or resources that you know will be greatly appreciated.

How does my application work:

The way my current application works is that I take the points from which the mouse pointer is located when the user holds the left mouse button. Then the list of points receives a feed to the gesture recognizer, and then splashes out what it considers to be the best shape / gesture that coordinates the captured points.

My idea:

What I wanted to do was before I feed the points in the gesture recognizer in order to somehow go through all the points and split them into separate lines or curves. Thus, I could feed each line / curve one at a time and from the main movements down, up, left, right, diagonals and curves, which I could determine the final shape / gesture.

One way that I thought would be good at determining if there are separate lines in my list of items is by selecting groups of points and viewing their slope. If the slope of the sample group of points was X% different from some other group of sample points, then it would be safe to assume that there is indeed a separate line.

What I think about possible problems in my thinking:

• Where can I determine the end of a line and the beginning of a single line? If I used the idea of ​​checking the slope of a group of points, then I decided that there is a separate line, which does not mean that I definitely found the slope of a single line. For example, if you were to draw a straight left "L" at a right angle and try to tilt the points around the "L" corner, you will see that the tilt will give a clear indication of the existence of a separate line, but these points do not correspond to the beginning of a separate line.

• How to deal with the ever-changing slope of the curve? The initial sign that I use, processes the curves already the way I want it. But I don’t want my method, which I used to define individual lines, to continue to search for these so-called individual lines on the curve, because its slope changes all the time when I select groups of points. Would I just stop the sample points as soon as the slope changes by more than X% so many times in a row?

• I do not use the correct “type” of mathematics to define single lines. Math is not my strongest topic, but I did some research. I tried to look at Dot Products and see if it would point me in a certain direction, but I don't know if that would be so. Has anyone used Dot Prodcuts to do something similar or some other method?

Final thoughts, comments and thanks:

Part of my problem that I feel about is that I don’t know how to arrange my question. I would not be surprised if this problem has already been asked (one way or another), and there is a solution that could be Googled. But my Google search results didn’t provide any solutions, since I don’t know exactly how to ask my question. If you feel this is confusing, please let me know where and why, and I will help clarify this. In this case, perhaps my searches on Google will become more accurate, and I can find a solution.

I just want to thank you again for reading my post. I know him for a long time, but I did not know where else to ask about it. Imma talks to some other people around the office, but all of my best solutions that I used at school came from the StackOverflow community, so I am very grateful to you.

Editing this entry:

(7/6 4:00 PM) Another idea that I was thinking about was to compare all the points in front of the Min / Max point. For example, if I move the mouse down, then up, the starting point would be the current Max point, and the point at which I start moving the mouse back will be my minute. I could go further and see if there are any points after the minimum point, and if they say so, a new potential line may arise. I don’t know how well this will work on other forms such as stars, but this is another thing that I am going to learn. Has anyone done something like this?

Answer 1

If your problem can be narrowed down to break the general curve into straight or smoothly curved partial lines, you can try this.

Comparing the slope of the segments and determining the breakpoints, where it is larger, a certain threshold will work in a very simplified case. Imagine a perfectly formed L-shape where you have a right angle between two straight lines. Obviously, the corner point would be the only one where the slope difference is above the threshold, as long as the threshold is in the range of 0 to 90 degrees and therefore an identifiable fault point.

However, the vertical and horizontal lines may be slightly curved, so the threshold should be large enough so that these slight differences in slope are ignored as fault points. You will also need to decide how sharp the algorithm should fit like a break. 90 degrees or higher required, or even 30 degrees enough? This is an important issue.

Finally, to make this reliable, I would not be satisfied with comparing the slopes of two adjacent segments. Hands can tremble, corners can be smoothed, and ideal conditions for finding straight lines and sharp angles are likely to never happen. For each point examined for a break, I would take the average slope N of the previous segments and compare it with the average slope N of the next segments. This can be effectively implemented using an average value. By choosing a good sample number N (depending on the accuracy of the input, the total number of points, etc.), the algorithm can avoid noise and make better detections.

Basically the algorithm:

• For each point studied (the beginning of N indicates a sequence and completes N points to the end).
  • Calculate the average slope of N previous segments.
  • Calculate the average slope N of the following segments.
  • If the average difference exceeds a threshold, mark the current point with a breakpoint.

This is not at all in my head. You should try this in your application.

Answer 2

if you work with absolute angles, for example up and down, you can simply take the absolute slope between two points (not necessarily adjacent) to determine if it has RIGHT, LEFT, UP, DOWN (if this is enough to distinguish)

the art is to find the distance between the points, so that the angle is not random (with 1px the angle will be a multiple of 45 °)

There is a firefox plugin for navigation using mouse gestures, which works very well. I think this is FireGestures, but I'm not sure. I think you can get inspiration from this.

An additional thought: if you draw a figure by connecting g consecutive points, and then return to the first point, the ratio between the area and the final length of the line segment will also become an indicator of the “vigor” of gestures

Answer 3

If you are just interested in up / down / left / right, the first approximation is to check the 45-degree segments of the circle. This is easy to do by checking the horizontal difference between the (consecutive) points against the vertical difference between the points.

Say that you have a greater positive horizontal difference than the vertical difference, then it will be “RIGHT”.

The only difficulty, for example, arises, for example, when differentiating UP / DOWN from UP / RIGHT / DOWN. But this can be done by the distance between the points. If you determine that the mouse moved the RIGHT by less than 20 pixels, let's say you can ignore this movement.