First, let's find out what the βbestβ attribute means in the light of decision trees β the attribute that βbestβ classifies the available case studies. To determine the "best" entropy and obtain information, you need to know two terms that you need to know. Entropy is a term in information theory β a number representing how a heterogeneous set of examples is based on their target class. The playerβs view of entropy is the number of bits needed to encode a random example class from a set of examples. The information gain, on the other hand, shows how much the entropy of the set of examples will decrease if a particular attribute is selected. Alternative perspective - this shows the reduction in the number of bits that will be needed to represent a random example class if a specific attribute is selected.
, "" ? , - , , . , .
, , "" . , , , . , , , .
ββββββββ¦ββββββββββ¦ββββββββββ¦βββββββββββ
β Exam β Friends β Weather β Activity β
β βββββββ¬ββββββββββ¬ββββββββββ¬βββββββββββ£
β yes β yes β sunny β study β
β no β yes β sunny β picnic β
β yes β no β rain β study β
β yes β yes β rain β study β
β no β yes β rain β play β
β no β no β rain β play β
ββββββββ©ββββββββββ©ββββββββββ©βββββββββββ
, :
IG(D, Exam) ~ 1
IG(D, Friends) ~ 0.13
IG(D, Weather) ~ 0.46
"" - . - , , , . , , . , :
IG(D-No-Exam, Friends) ~ 0.25
IG(D-No-Exam, Weather) ~ 0.92
, , , , , :
Exam?
/ \
yes no
/ \
STUDY Weather?
/ \
sunny rain
/ \
PICNIC PLAY
, , - , . :
Friends?
/ \
yes no
/ \
Exam? Exam?
/ \ / \
yes no yes no
/ \ | |
STUDY Weather? STUDY PLAY
/ \
sunny rain
/ \
PICNIC PLAY
. , .
, "" , , , .