Society of Actuaries (SOA) PA Practice Exam

What is one similarity between Ridge and Lasso Regression?

1. Both provide variable elimination techniques

2. Both have a hyperparameter that controls reduction

3. Both use a sum of squared coefficients

4. Both yield the same model accuracy

The correct answer is: Both have a hyperparameter that controls reduction

The correct answer highlights a crucial aspect of both Ridge and Lasso Regression concerning their regularization methods. Both techniques incorporate a hyperparameter that governs the strength of the penalty applied during the model training process. In Ridge Regression, this hyperparameter (often denoted as λ or alpha) determines the extent to which the coefficients are shrunk towards zero, thus controlling the amount of regularization imposed. Similarly, in Lasso Regression, there is also a hyperparameter that influences how much shrinkage is applied to the coefficients. This results in varying degrees of regularization depending on the value set for this hyperparameter in both methods, making it a fundamental characteristic they share. Other options may relate to the properties or outcomes of these regression techniques, but they do not accurately capture this particular shared feature. For instance, variable elimination is more applicable to Lasso than Ridge, which tends to retain all variables but reduces their impact. Moreover, while both approaches aim to minimize some loss function, they do so through different means, which affects model accuracy differently. Lastly, since the performance of models developed through these methods can vary significantly across different datasets, stating that they yield the same model accuracy is incorrect. The unique characteristics of each regression technique, including their hyperparameter dynamics,