Society of Actuaries (SOA) PA Practice Exam

In Lasso Regression, what is the value of lambda typically set to for a shrinkage effect?

1. 0

2. 1

3. 0.01

4. 0.5

The correct answer is: 0.01

In Lasso Regression, the value of lambda plays a critical role in determining the strength of the penalty applied to the coefficients of the regression model. The purpose of using a non-zero lambda is to apply a penalty that encourages the model to reduce the magnitude of coefficients, effectively shrinking them towards zero. This shrinkage effect helps to prevent overfitting, especially in scenarios with a large number of predictors relative to the number of observations. Typically, a small positive value for lambda is desired to achieve a balance between fitting the data well and maintaining a model that is generalizable. Setting lambda to a very low value, such as 0.01, enables the regularization effect of Lasso without overly constraining the coefficients. This allows some variables to retain influence while still benefiting from the shrinkage that minimizes the risk of overfitting. Setting lambda to inappropriate values, such as 0 (which would remove any penalty and lead to ordinary least squares regression) or excessively high values (which could lead to too much shrinkage and potentially eliminate important predictors), would not provide the desired balance. Therefore, a lambda value of 0.01 is commonly used in practice to ensure effective regularization while allowing for meaningful contributions from significant variables.