Bruce E. Hansen
The Risk of James-Stein and Lasso Shrinkage
Econometric Reviews (2016)
Abstract:
This paper compares the mean-squared error (or l2 risk) of Ordinary Least-Squares, James-Stein, and Lasso shrinkage estimators in simple linear regression where the number of regressors is smaller than the sample size. We compare and contrast the known risk bounds for these estimators, which shows that neither James-Stein nor Lasso uniformly dominates the other. We investigate the finite sample risk using a simple simulation experiment. We find that the risk of Lasso estimation is particularly sensitive to coefficient parameterization, and for a significant portion of the parameter space Lasso has higher mean-squared error than OLS. This investigation suggests that there are potential pitfalls arising with Lasso estimation, and simulation studies need to be more attentive to careful exploration of the parameter space.
Download PDF file
Link to Programs
Some of the above material is based upon work supported by the National Science Foundation under Grants No. SES-9022176, SES-9120576, SBR-9412339, and SBR-9807111.
Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the author(s), and do not necessarily reflect the views of the NSF.