Society of Actuaries (SOA) PA Practice Exam

What does information gain signify in decision tree analysis?

1. The total number of nodes in a tree

2. The change in purity of a dataset before and after a transformation

3. How often classes are evenly split

4. The randomness of predictions

The correct answer is: The change in purity of a dataset before and after a transformation

Information gain in decision tree analysis reflects the effectiveness of a feature in classifying the data. It measures the change in purity of a dataset before and after a particular transformation, specifically when that feature is used to split the data into subsets. When a dataset is split based on a feature, the information gain quantifies how much the uncertainty about the class label is reduced. A higher information gain indicates that the feature contributes significantly to distinguishing between classes, leading to more homogeneous subsets in terms of class composition. This ultimately helps in creating a more accurate classification model. The other options do not accurately capture the concept of information gain. The total number of nodes in a tree relates to the complexity of the tree rather than its classification power. The notion of classes being evenly split does not reflect the idea of measuring uncertainty reduction. Lastly, the randomness of predictions does not align with the fundamental purpose of information gain, which is to enhance predictability by maximizing purity in the resulting subsets.