Partial Least Squares in Constructing Candidates Model Averaging

Abstract

Model averaging has been developed as an alternative method in regression analysis when number of observations is smaller than number of explanatory variables (also known as high-dimensional regression). Main concept about this method is weighted average of several candidate models, in order to improve prediction accuracy. There are two steps in model averaging: construct several candidate models and determine weights for candidate models. Our research proposed partial least squares model averaging (PLSMA) as an approach to construct candidate models, while partial least squares (PLS) method was applied during that process to reduce and transform original explanatory variables become new variables that called components. The evaluation of PLSMA is conducted by measured Root Mean Squared Error of Prediction (RMSEP) with simulation data. Compared to other methods, PLSMA has given the smallest RMSEP, so this result indicates that this method had yielded more accurate prediction than other existing methods.