Projection pursuit regression

In statistics, projection pursuit regression (PPR) is a statistical model developed by Jerome H. Friedman and Werner Stuetzle which is an extension of additive models. This model adapts the additive models in that it first projects the data matrix of explanatory variables in the optimal direction before applying smoothing functions to these explanatory variables.

The model consists of linear combinations of non-linear transformations of linear combinations of explanatory variables. The basic model takes the form

where x is a column vector containing a particular row of the design matrix X which contains p explanatory variables (columns) and n observations (row). Here Y is a particular observation variable (identifying the row being considered) to be predicted, {β_j} is a collection of r vectors (each a unit vector of length p) which contain the unknown parameters. Finally r is the number of modelled smoothed non-parametric functions to be used as constructed explanatory variables. The value of r is found through cross-validation or a forward stage-wise strategy which stops when the model fit cannot be significantly improved. For large values of r and an appropriate set of functions f_j, the PPR model is considered a universal estimator as it can estimate any continuous function in R^p.

Thus this model takes the form of the basic additive model but with the additional β_j component; making it fit $\beta _{j}'x$ $\beta _{j}'x$ rather than the actual inputs x. The vector $\beta _{j}'X$ $\beta _{j}'X$ is the projection of X onto the unit vector β_j, where the directions β_j are chosen to optimize model fit. The functions f_j are unspecified by the model and estimated using some flexible smoothing method; preferably one with well defined second derivatives to simplify computation. This allows the PPR to be very general as it fits non-linear functions f_j of any class of linear combinations in X. Due to the flexibility and generality of this model, it is difficult to interpret the fitted model because each input variable has been entered into the model in a complex and multifaceted way. Thus the model is far more useful for prediction than creating a model to understand the data.

...
Wikipedia