How to evaluate functions by their value in the Weka classifier?

Question

I use Weka to successfully create a classifier. Now I would like to evaluate how effective or important my functions are. For this I use AssributeSelection. But I do not know how to derive different functions with their corresponding value. I just want to list the functions in descending order of their indicators of obtaining information!

Answer 1

There are many ways in Weka to scoring functions called attributes. These methods are available as subclasses of weka.attributeSelection.ASEvaluation .

Any of these rating classes will give you a rating for each attribute. If you use information gain for scoring, for example, you will use its InfoGainAttributeEval class. Useful methods are

• InfoGainAttributeEval.html#buildEvaluator() and
• InfoGainAttributeEval.html#evaluateAttribute()

Other types of scoring (gain, correlation, etc.) have the same methods for scoring. Using any of them, you can evaluate all your functions.

The ranking itself is independent of Weka. Of the many ways to do this, this is one thing:

 Map<Attribute, Double> infogainscores = new HashMap<Attribute, Double>(); for (int i = 0; i < instances.numAttributes(); i++) { Attribute t_attr = instaces.attribute(i); double infogain = evaluation.evaluateAttribute(i); infogainscores.put(t_attr, infogain); } 

Now you have a map that needs to be sorted by value. Here is the general code for this:

  /** * Provides a {@code SortedSet} of {@code Map.Entry} objects. The sorting is in ascending order if {@param order} > 0 * and descending order if {@param order} <= 0. * @param map The map to be sorted. * @param order The sorting order (positive means ascending, non-positive means descending). * @param <K> Keys. * @param <V> Values need to be {@code Comparable}. * @return A sorted set of {@code Map.Entry} objects. */ static <K,V extends Comparable<? super V>> SortedSet<Map.Entry<K,V>> entriesSortedByValues(Map<K,V> map, final int order) { SortedSet<Map.Entry<K,V>> sortedEntries = new TreeSet<>( new Comparator<Map.Entry<K,V>>() { public int compare(Map.Entry<K,V> e1, Map.Entry<K,V> e2) { return (order > 0) ? compareToRetainDuplicates(e1.getValue(), e2.getValue()) : compareToRetainDuplicates(e2.getValue(), e1.getValue()); } } ); sortedEntries.addAll(map.entrySet()); return sortedEntries; } 

and finally

 private static <V extends Comparable<? super V>> int compareToRetainDuplicates(V v1, V v2) { return (v1.compareTo(v2) == -1) ? -1 : 1; } 

Now you have a list of records sorted by value (ascending or descending, as you wish). Go crazy!

Note that you should handle the case when more than one attribute has the same information gain . That's why I went through the process of sorting by values, keeping duplicates.