Correspondence analysis

Before proceeding to the central computational step of the algorithm, the values in matrix C have to be transformed.[10] First compute a set of weights for the columns and the rows (sometimes called masses),[11][12] where row and column weights are given by the row and column vectors, respectively:

${\displaystyle w_{m}={\frac {1}{n_{C}}}C\mathbf {1} ,\quad w_{n}={\frac {1}{n_{C}}}\mathbf {1} ^{T}C.}$

Here ${\displaystyle n_{C}=\sum _{i=1}^{n}\sum _{j=1}^{m}C_{ij}}$ is the sum of all cell values in matrix C, or short the sum of C, and ${\displaystyle \mathbf {1} }$ is a column vector of ones with the appropriate dimension.

Put in simple words, ${\displaystyle w_{m}}$ is just a vector whose elements are the row sums of C divided by the sum of C, and ${\displaystyle w_{n}}$ is a vector whose elements are the column sums of C divided by the sum of C.

The weights are transformed into diagonal matrices

${\displaystyle W_{m}=\operatorname {diag} (1/{\sqrt {w_{m}}})}$

and

${\displaystyle W_{n}=\operatorname {diag} (1/{\sqrt {w_{n}}})}$

where the diagonal elements of ${\displaystyle W_{n}}$ are ${\displaystyle 1/{\sqrt {w_{n}}}}$ and those of ${\displaystyle W_{m}}$ are ${\displaystyle 1/{\sqrt {w_{m}}}}$ respectively i.e. the vector elements are the inverses of the square roots of the masses. The off-diagonal elements are all 0.

Next, compute matrix ${\displaystyle P}$ by dividing ${\displaystyle C}$ by its sum

${\displaystyle P={\frac {1}{n_{C}}}C.}$

In simple words, Matrix ${\displaystyle P}$ is just the data matrix (contingency table or binary table) transformed into portions i.e. each cell value is just the cell portion of the sum of the whole table.

Finally, compute matrix ${\displaystyle S}$, sometimes called the matrix of standardized residuals,[13] by matrix multiplication as

${\displaystyle S=W_{m}(P-w_{m}w_{n})W_{n}}$

Note, the vectors ${\displaystyle w_{m}}$ and ${\displaystyle w_{n}}$ are combined in an outer product resulting in a matrix of the same dimensions as ${\displaystyle P}$. In words the formula reads: matrix ${\displaystyle \operatorname {outer} (w_{m},w_{n})}$ is subtracted from matrix ${\displaystyle P}$ and the resulting matrix is scaled (weighted) by the diagonal matrices ${\displaystyle W_{m}}$ and ${\displaystyle W_{n}}$. Multiplying the resulting matrix by the diagonal matrices is equivalent to multiply the i-th row (or column) of it by the i-th element of the diagonal of ${\displaystyle W_{m}}$ or ${\displaystyle W_{n}}$, respectively[14].

The vectors ${\displaystyle w_{m}}$ and ${\displaystyle w_{n}}$ are the row and column masses or the marginal probabilities for the rows and columns, respectively. Subtracting matrix ${\displaystyle \operatorname {outer} (w_{m},w_{n})}$ from matrix ${\displaystyle P}$ is the matrix algebra version of double centering the data. Multiplying this difference by the diagonal weighting matrices results in a matrix containing weighted deviations from the origin of a vector space. This origin is defined by matrix ${\displaystyle \operatorname {outer} (w_{m},w_{n})}$.

In fact matrix ${\displaystyle \operatorname {outer} (w_{m},w_{n})}$ is identical with the matrix of expected frequencies in the chi-squared test. Therefore ${\displaystyle S}$ is computationally related to the independence model used in that test. But since CA is not an inferential method the term independence model is inappropriate here.

The table ${\displaystyle S}$ is then decomposed[10] by a singular value decomposition as

${\displaystyle S=U\Sigma V^{*}\,}$

where ${\displaystyle U}$ and ${\displaystyle V}$ are the left and right singular vectors of ${\displaystyle S}$ and ${\displaystyle \Sigma }$ is a square diagonal matrix with the singular values ${\displaystyle \sigma _{i}}$ of ${\displaystyle S}$ on the diagonal. ${\displaystyle \Sigma }$ is of dimension ${\displaystyle p\leq (\min(m,n)-1)}$ hence ${\displaystyle U}$ is of dimension m×p and ${\displaystyle V}$ is of n×p. As orthonormal vectors ${\displaystyle U}$ and ${\displaystyle V}$ fulfill

${\displaystyle U^{*}U=V^{*}V=I}$.

In other words, the multivariate information that is contained in ${\displaystyle C}$ as well as in ${\displaystyle S}$ is now distributed across two (coordinate) matrices ${\displaystyle U}$ and ${\displaystyle V}$ and a diagonal (scaling) matrix ${\displaystyle \Sigma }$. The vector space defined by them has as number of dimensions p, that is the smaller of the two values, number of rows and number of columns, minus 1.

While a principal component analysis may be said to decompose the (co)variance, and hence its measure of success is the amount of (co-)variance covered by the first few PCA axes - measured in eigenvalue -, a CA works with a weighted (co-)variance which is called inertia.[15] The sum of the squared singular values is the total inertia ${\displaystyle \mathrm {I} }$ of the data table, computed as

${\displaystyle \mathrm {I} =\sum _{i=1}^{p}\sigma _{i}^{2}.}$

The total inertia ${\displaystyle \mathrm {I} }$ of the data table can also computed directly from ${\displaystyle S}$ as

${\displaystyle \mathrm {I} =\sum _{i=1}^{n}\sum _{j=1}^{m}s_{ij}^{2}.}$

The amount of inertia covered by the i-th set of singular vectors is ${\displaystyle \iota _{i}}$, the principal inertia. The higher the portion of inertia covered by the first few singular vectors i.e. the larger the sum of the principal inertiae in comparison to the total inertia, the more successful a CA is.[15] Therefore all principal inertia values are expressed as portion ${\displaystyle \epsilon _{i}}$ of the total inertia

${\displaystyle \epsilon _{i}=\sigma _{i}^{2}/\sum _{i=1}^{p}\sigma _{i}^{2}}$

and are presented in the form of a scree plot. In fact a scree plot is just a bar plot of all principal inertia portions ${\displaystyle \epsilon _{i}}$.

To transform the singular vectors to coordinates which preserve the chisquare distances between rows or columns an additional weighting step is necessary. The resulting coordinates are called principal coordinates[10] in CA text books. If principal coordinates are used for rows their visualization is called a row isometric[16] scaling in econometrics and scaling 1[17] in ecology. Since the weighting includes the singular values ${\displaystyle \Sigma }$ of the matrix of standardized residuals ${\displaystyle S}$ these coordinates are sometimes referred to as singular value scaled singular vectors, or, a little bit misleading, as eigenvalue scaled eigenvectors. In fact the non-trivial eigenvectors of ${\displaystyle SS^{*}}$are the left singular vectors ${\displaystyle U}$ of ${\displaystyle S}$ and those of ${\displaystyle S^{*}S}$ are the right singular vectors ${\displaystyle V}$ of ${\displaystyle S}$ while the eigenvalues of either of these matrices are the squares of the singular values ${\displaystyle \Sigma }$. But since all modern algorithms for CA are based on a singular value decomposition this terminology should be avoided. In the french tradition of CA the coordinates are sometimes called (factor) scores.

Factor scores or principal coordinates for the rows of matrix C are computed by

${\displaystyle F_{m}=W_{m}U\Sigma }$

i.e. the left singular vectors are scaled by the inverse of the square roots of the row masses and by the singular values. Because principal coordinates are computed using singular values they contain the information about the spread between the rows (or columns) in the original table. Computing the euclidean distances between the entities in principal coordinates results in values that equal their chisquare distances which is the reason why CA is said to "preserve chisquare distances".

Compute principal coordinates for the columns by

${\displaystyle F_{n}=W_{n}V\Sigma .}$

To represent the result of CA in a proper biplot, those categories which are not plotted in principal coordinates, i.e. in chisquare distance preserving coordinates, should be plotted in so called standard coordinates.[10] They are called standard coordinates because each vector of standard coordinates has been standardized to exhibit mean 0 and variance 1.[18] When computing standard coordinates the singular values are omitted which is a direct result of applying the biplot rule by which one of the two sets of singular vector matrices must be scaled by singular values raised to the power of zero i.e. multiplied by one i.e. be computed by omitting the singular values if the other set of singular vectors have been scaled by the singular values. This reassures the existence of a inner product between the two sets of coordinates i.e. it leads to meaningful interpretations of their spatial relations in a biplot.

In practical terms one can think of the standard coordinates as the vertices of the vector space in which the set of principal coordinates (i.e. the respective points) "exists".[19] The standard coordinates for the rows are

${\displaystyle G_{m}=W_{m}U}$

and those for the columns are

${\displaystyle G_{n}=W_{n}V}$

Note that a scaling 1[17] biplot in ecology implies the rows to be in principal and the columns to be in standard coordinates while scaling 2 implies the rows to be in standard and the columns to be in principal coordinates. I.e. scaling 1 implies a biplot of ${\displaystyle F_{m}}$together with ${\displaystyle G_{n}}$ while scaling 2 implies a biplot of ${\displaystyle F_{n}}$together with ${\displaystyle G_{m}}$.