How to make a career confidence system smart

Question

Well, finally, I’m working on my project last year, which is an intelligent web-based career trust system, the main functionality of my system is

Recommendation system

Basically, our recommendation system will carefully study user preferences, taking part in user interest tests and academic records of users, and based on this verified information, it will provide the user with the best career options, for example, BS Computer Science course, etc.

• Entering a recommendation system is a student’s credentials and a percent test, and in the percent test, questions will be given in accordance with the user's academic history and the answers that he gives in the test, so basically the test will not ask everyone the same questions, he will decide in real time what to ask the user according to the rules defined by the system.
  
• Its output will be a variant of the fields that will be determined based on the Interest test.

Problem

When I defended my sphere of activity of the committee, they said: "It's simple, if-yet," this system is not smart.
My question is which AI method or Algorithm can be used to make this system intelligent. I searched a lot, but the documents related to my system are much more superficial, they simply emphasize the idea not according to the methodology. I want to do all my work in Java. It's great if the answer is technology.
You people can submit my question to any other stackexchange site if it is not related to SO Q & A criteria.

Edit

After getting some idea from the answers, I want to implement an expert system with a rule-based and inference mechanism. Now I want to more clearly understand the technological aspect in order to implement a rule-based mechanism. After searching, I found Drools the best, but is it compatible with web applications? And I also found Tohu the best dynamic form generator (since it is also necessary for my project). can i use tohu with drools to build my web application? Is it easy to implement this type of system or not?

Answer 1

In my last project, I had some experience with the Jena RDF engine . Basically, what you do with it, create a kind of knowledge base with rules such as "if the user selected this answer, he has this quality" and "if the user has these qualities, he can be good for this job." Adding answers to the system will allow you to request its current status and adjust the questions accordingly. It's pretty easy to create a proof of concept with it, it's easier to do than a bunch of if-else, and if your professors worship prologue-ish-style, they will like it.

Answer 2

• If you have a big question, each of them can represent a function. Assuming you have many functions, finding a series of if-else statements that satisfy the criteria is difficult (recall that a complete tree with n questions will have 2^n "leaves" - presenting 2^n possible answers to these questions, assuming that each question is yes / no question).

• Since hard programming above is impossible for a sufficiently large (and probably realistic size n ), there is a place for heuristic solutions, one of which is Machine Learning , namely classification . people, responding to your survey, with an “expert” saying that they have a better career and let the algorithm find a classifier for the common problem (if you want to automatically convert it to a series of “yes-no” questions, it can be with a decision tree and an algorithm like C4.5 to create a tree).

• It may also be important to determine which issues are really relevant. Is gender relevant? Does the height fit? These questions can also be answered using ML algorithms by selecting functions , for example (one of them is PCA )

• Regarding the “technology” aspect, java has a good library called Weka that implements many of the classification algorithms there.

• One question you might ask (and try to figure out in your project) , which classification algorithm would be best suited for this problem? Some features: The aforementioned C4.5, Naive Bayes , Linear Regression , Neural Networks , KNN or SVM (which, as a rule, turned out to be the best for me). You can try and decide your decision which algorithm to use with statistical research and statistical proof , which is better. Wilcoxon test is the standard for this.

---

EDIT: more about item 2:

• Here, an “expert” can be a classifier of a person from the field of HR who reads functions and classifies answers. Obtaining this data (usually called “learning data”) is complex and expansive sometimes, if your university has an IE or HR faculty, they may be willing to help.
• Idea: gather a group of people who will first answer your survey. Then give it to the human classifier (“expert”), who would prefer the best career for that person based on his answers. The data with the classification provided by the expert is the introduction of a learning algorithm, its output will be a classifier.
• The classifier - this is the function itself, which provides answers to the polls - predicts what is the "classification" (proposed career) for the person who made this survey.
• Please note: if you have a classifier, you no longer need to maintain training data, just one classifier is enough. However, you should have a list of questions, and the answers to these questions will be the functions provided to the classifier.

Answer 3

All you have to do to satisfy them is to create a simple learning system:

• Change your thesis terminology so that it is described as “learning a better career” instead of using the word “smart.” Learning is a form of artificial intelligence.
• Create a training mode. Do this by submitting a questionnaire to people who already have careers, as well as asking questions to find out how satisfied they are with their careers. Thus, your system can teach what makes a good career and what makes a bad one.
• Select a training system to absorb data from (2). For example, one recent source of ideas might be this recent document: http://journals.cluteonline.com/index.php/RBIS/article/download/4405/4493. Product summarization networks are at the forefront of AI and are well-suited to problems like an expert system.

Lastly, try turning the way your technology makes it specific to your problem.

Answer 4

A program is never smarter than the person who wrote it. So, I would first use a collective intelligence that has already been built and opened.

Pass your set of known data points as an input to Apache Mahout PearsonCorrelationSimilarity and use the result to predict which course is best. In addition to the open source code and scalability, you can also record the result and return it to the system to improve accuracy over time . It is very difficult to compare this level of performance, because it is much easier to configure the algorithm from the box or replace it with your own than to deal with a bunch of if else conditions.

I would suggest reading this book . It contains an example of how to use PearsonCorrelationSimilarity.

Mahout also created built-in algorithms such as NearestNeighborClusterSimilarity which can simplify your decision.

The book has a good start code. You can build on it.

The student credentials, interest survey questions, and answers are the baseline. Choosing a career is a result that you can associate with entry. Now that is a very simplified approach, but you can start with this. In the end, you will have to apply the classifier methods proposed by Amit, and Mahout can also help you with this.

Answer 5

As @amit suggested, Bayesian analysis can give you recommendations on the following issue. Another mistake of dynamic tests is artificial thresholds ("if your rating is 28, you are in this category, if your score is 27, you are not"), a problem that fuzzy logic can solve. Another advantage of fuzzy logic is that adding a new category is relatively simple because the domain expert is only asked to make qualitative estimates, not quantitative thresholds.

Answer 6

Drools can be used over the Internet, but watch out; it may be a little beast to set up and probably a bust for your application. This is a "corporate" type of solution that focuses on rule management, not rule enforcement.

Drools is an IF-THEN system, and almost all rule engines use the Rete algorithm. http://en.wikipedia.org/wiki/Rete_algorithm - so if your initial question is about how not to use the IF-THEN system, Drools is not the right choice. Now there is a part of Solver and Planner from Drools that are not IF-THEN algorithms, but it is not the main Drools algorithm.

However, this seems like a smart choice for your application. Just don't expect this to be considered an “intellectual” system by those who consider themselves experts. Typically, rule engines are used to codify (that is, create software) rules and regulations of a business, such as "if you are approved for a mortgage" or "how much your car insurance costs," etc. “what kind of work you should do” is a wise application of the same.

If you want to add more AI, like intelligence, here are a few ideas

• Use machine learning to get user feedback on earlier recommendations. So, if someone loves or hates a sentence, add it as a person’s function. Now you are doing basic feedback / reinforcement training (bikes, neural networks) to try to better classify a person into a career.

• Consider the questions you ask the person. Need to ask all the questions? If you can change the flow of questions based on their answers (evaluating which person), you are trying to study a series of questions that provide the most useful knowledge for the recommendation.

If you want special software, look at Weka http://www.cs.waikato.ac.nz/ml/weka/ - it has a lot of great classification algorithms. And it is a Java library, so you can easily use it in a web application.

Good luck.