Society of Actuaries (SOA) PA Practice Exam

What characteristic of Lasso Regression differentiates it from Ridge Regression?

1. Lasso uses the absolute value in its penalty

2. Ridge allows for eliminating variables completely

3. Lasso retains all variables without modification

4. Ridge only performs stepwise selection

The correct answer is: Lasso uses the absolute value in its penalty

Lasso Regression is distinguished from Ridge Regression primarily by its use of the L1 penalty, which involves the absolute values of the coefficients. This characteristic leads to different behaviors in how each method handles variable selection and shrinks coefficients. Specifically, Lasso applies a penalty equal to the absolute value of the magnitude of coefficients (L1 norm), which not only shrinks coefficients but can also set some of them to zero during the model fitting process. This effectively performs variable selection by excluding less important features from the model entirely. In contrast, Ridge Regression employs an L2 penalty, which is based on the square of the coefficient values. This penalty tends to shrink coefficients without actually eliminating any of them, hence maintaining all variables in the final model, albeit with smaller coefficients. Understanding this distinction is crucial when deciding which regression method to use based on the goals of the analysis, particularly in datasets with many features, where variable selection can play a significant role in improving model interpretability and reducing overfitting.