Bruce E. Hansen
Nonparametric Sieve Regression: Least Squares, Averaging Least Squares, and Cross-Validation
Oxford Handbook of Applied Nonparametric and Semiparametric Econometrics and Statistics (2014)
Abstract:
This chapter concerns selection and combination of nonparametric sieve regression estimators. We review the concepts of series and sieve approximations, introduce least-squares estimates of sieve approximations, and measure the accuracy of the estimators by integrated mean-squared error (IMSE). We show that the critical issue in applications is selection of the order of the sieve, as the IMSE greatly varies across the choice. We introduce the cross-validation criterion as an estimator of mean-squared forecast error (MSFE) and IMSE. We extend the current optimality theory, by showing that cross-validation selection is asymptotically IMSE equivalent to the infeasible best sieve approximation.
We also introduce weighted averages of sieve regression estimators. Averaging estimators have lower IMSE than selection estimators. Following Hansen and Racine (2012) we introduce a cross-validation (or jackknife) criterion for the weight vector, and recommend selection of the weights by minimizing this criterion. The resulting jackknife model averaging (JMA) estimator is a feasible averaging sieve estimator. We show that the JMA estimator is optimal in the sense that it is asymptotically IMSE equivalent to the infeasible optimal weighted average sieve estimator. While computation of the JMA weights is a simple application of quadratic programming, we also introduce a simple algorithm which closely approximates the JMA solution without the need for quadratic programming.
Download PDF file
Link to Programs
Some of the above material is based upon work supported by the National Science Foundation under Grants No. SES-9022176, SES-9120576, SBR-9412339, and SBR-9807111.
Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the author(s), and do not necessarily reflect the views of the NSF.