17 Mar 2004
CS 3243 - Learning
34
Entropy curve
nFor p/t between 0 & 1, the 2-class entropy is
¡0 when p/(p+n) is
0
¡1 when p/(p+n) is
0.5
¡0 when p/(p+n) is
1
¡monotonically
increasing between 0 and
0.5
¡monotonically
decreasing between 0.5 and 1
¡When the data is
pure, only need to send 1
bit
1
0.5