Natural language processing for smart homes

Question

I am writing Smart Home software for my undergraduate degree that will only simulate a real home, but I am stuck in part of the NLP project. The idea is that the client listens to voice inputs (already completed), converts them to text (executes) and sends them to the server, which makes all the difficult lifting / decision-making.

So, all my inputs will be quite short (for example, “please turn on the porch light”). Based on this, I want to make a decision about which object to act and how to act. So I came up with a few things to do something somewhat effective.

• Get rid of unnecessary words (in the previous example, “please” and “the” are words that do not change the meaning of what needs to be done, but if I say “turn off the light”, it is quite important).
• Work with synonyms (“turn on the lights” should do the same thing as “turn on the lights” - I know, this is a stupid example). I suppose the only option is to have some kind of dictionary (possibly XML) and just have a list of possible words for one specific object in the house.
• Verb and subject detection. "turn on" is a verb, and "lights" is an object. I need a good way to detect this.
• General implementation. How do these things usually evolve in terms of algorithms? I managed to find only one article on NLP in Smart Homes, which was very vague (and had poor English). Any links are welcome.

I hope that the question is quite straightforward (I saw NLP questions on SO, no one helped), that it will not be closed.

Answer 1

I am not a pioneer in NLP (I love it, though), but let me try it out. For your project, I suggest you go through Stanford Parser

• From your definition of the problem, I think you do not need anything other than verbs and nouns. SP generates POS (part of speech tags), which you can use to trim words that you don't need.

• For this, I can’t come up with a better option than what you now have in mind.

• To do this, again you can use the grammatical dependency structure from SP, and I am sure that it is good enough to solve this problem.

• Here lies your research part. I think you can find enough patterns using the GD and POS tags to come up with an algorithm for your problem. I have little doubt that any algorithm would be efficient enough to process each set of input sentences (structured + unstructured), but what is 85% more important should be enough for you.

Answer 2

If you do not have much time to solve the problem with NLP, you can use the Wit API ( http://wit.ai ), which maps natural language sentences to JSON:

This is based on computer training, so you need to provide sample sentences + JSON to customize it according to your needs. It should be much more reliable than grammar-based approaches, especially because the voice-to-speech mechanism can make mistakes that violate your grammar (but the machine learning module can still get the meaning of the sentence).

Answer 3

First, I would build a list of all possible commands (not every possible way to say a command, but the actual function itself: “kitchen light” and “turn on the light in the kitchen” is the same command) based on the actual functions available to the smart home . I assume that there is a discrete number of them in the amount of not more than a hundred. Assign each code to your identifier.

Then your work will display the input:

• English sentence
• speaker location
• time of day, day of the week
• any other input

to display the confidence level (from 0.0 to 1.0) for each team.

Then the system will execute the best match command if the confidence exceeds a certain custom threshold (say, more than 0.70).

From here it becomes a machine learning application. There are several different approaches (and, in addition, approaches can be combined together if they compete based on input characteristics).

To begin with, I will be working on the NLP book by Jurafsky / Manning from Stanford . This is a good overview of current NLP algorithms.

From there, you will get some ideas on how imaging can be learned by the machine. More importantly, how natural language can be broken down into the mathematical structure of machine learning.

After the text has been semantically analyzed, the simplest ML algorithm, to try first, will be controlled. To generate training data, there is a regular graphical interface, say a command, and then press the corresponding command manually. This creates a single supervised training case. Make some of them. Set aside a little for testing. It is also an unskilled job to help other people. You can then use them as your set for your ML algorithm.