Partial least squares regression

PLS1 is a widely used algorithm appropriate for the vector Y case. It estimates T as an orthonormal matrix. (Caution: the t vectors in the code below may not be normalized appropriately; see talk.) In pseudocode it is expressed below (capital letters are matrices, lower case letters are vectors if they are superscripted and scalars if they are subscripted).

 1 function PLS1(X, y, l)
 2     ${\displaystyle X^{(0)}\gets X}$
 3     ${\displaystyle w^{(0)}\gets X^{\mathrm {T} }y/\|X^{\mathrm {T} }y\|}$, an initial estimate of w.
 4     for ${\displaystyle k=0}$ to ${\displaystyle l-1}$
 5         ${\displaystyle t^{(k)}\gets X^{(k)}w^{(k)}}$
 6         ${\displaystyle t_{k}\gets {t^{(k)}}^{\mathrm {T} }t^{(k)}}$ (note this is a scalar)
 7         ${\displaystyle t^{(k)}\gets t^{(k)}/t_{k}}$
 8         ${\displaystyle p^{(k)}\gets {X^{(k)}}^{\mathrm {T} }t^{(k)}}$
 9         ${\displaystyle q_{k}\gets {y}^{\mathrm {T} }t^{(k)}}$ (note this is a scalar)
10         if ${\displaystyle q_{k}=0}$
11             ${\displaystyle l\gets k}$, break the for loop
12         if ${\displaystyle k<(l-1)}$
13             ${\displaystyle X^{(k+1)}\gets X^{(k)}-t_{k}t^{(k)}{p^{(k)}}^{\mathrm {T} }}$
14             ${\displaystyle w^{(k+1)}\gets {X^{(k+1)}}^{\mathrm {T} }y}$
15     end for
16     define W to be the matrix with columns ${\displaystyle w^{(0)},w^{(1)},...,w^{(l-1)}}$.
       Do the same to form the P matrix and q vector.
17     ${\displaystyle B\gets W{(P^{\mathrm {T} }W)}^{-1}q}$
18     ${\displaystyle B_{0}\gets q_{0}-{P^{(0)}}^{\mathrm {T} }B}$
19     return ${\displaystyle B,B_{0}}$

This form of the algorithm does not require centering of the input X and Y, as this is performed implicitly by the algorithm. This algorithm features 'deflation' of the matrix X (subtraction of ${\displaystyle t_{k}t^{(k)}{p^{(k)}}^{\mathrm {T} }}$), but deflation of the vector y is not performed, as it is not necessary (it can be proved that deflating y yields the same results as not deflating[10]). The user-supplied variable l is the limit on the number of latent factors in the regression; if it equals the rank of the matrix X, the algorithm will yield the least squares regression estimates for B and ${\displaystyle B_{0}}$

Geometric interpretation of the deflation step in the input space

In 2002 a new method was published called orthogonal projections to latent structures (OPLS). In OPLS, continuous variable data is separated into predictive and uncorrelated (orthogonal) information. This leads to improved diagnostics, as well as more easily interpreted visualization. However, these changes only improve the interpretability, not the predictivity, of the PLS models.[11] Similarly, OPLS-DA (Discriminant Analysis) may be applied when working with discrete variables, as in classification and biomarker studies.

The general underlying model of OPLS is

${\displaystyle X=TP^{\mathrm {T} }+T_{\text{Y-orth}}P_{\text{Y-orth}}^{\mathrm {T} }+E}$

${\displaystyle Y=UQ^{\mathrm {T} }+F}$

or in O2-PLS[12]

${\displaystyle X=TP^{\mathrm {T} }+T_{\text{Y-orth}}P_{\text{Y-orth}}^{\mathrm {T} }+E}$

${\displaystyle Y=UQ^{\mathrm {T} }+U_{\text{X-orth}}Q_{\text{X-orth}}^{\mathrm {T} }+F}$

Another extension of PLS regression, named L-PLS for its L-shaped matrices, connects 3 related data blocks to improve predictability.[13] In brief, a new Z matrix, with the same number of columns as the X matrix, is added to the PLS regression analysis and may be suitable for including additional background information on the interdependence of the predictor variables.

In 2015 partial least squares was related to a procedure called the three-pass regression filter (3PRF).[14] Supposing the number of observations and variables are large, the 3PRF (and hence PLS) is asymptotically normal for the "best" forecast implied by a linear latent factor model. In stock market data, PLS has been shown to provide accurate out-of-sample forecasts of returns and cash-flow growth.[15]

A PLS version based on singular value decomposition (SVD) provides a memory efficient implementation that can be used to address high-dimensional problems, such as relating millions of genetic markers to thousands of imaging features in imaging genetics, on consumer-grade hardware.[16]

PLS correlation (PLSC) is another methodology related to PLS regression,[17] which has been used in neuroimaging [17][18][19] and sport science,[20] to quantify the strength of the relationship between data sets. Typically, PLSC divides the data into two blocks (sub-groups) each containing one or more variables, and then uses singular value decomposition (SVD) to establish the strength of any relationship (i.e. the amount of shared information) that might exist between the two component sub-groups.[21] It does this by using SVD to determine the inertia (i.e. the sum of the singular values) of the covariance matrix of the sub-groups under consideration.[21][17]