Multiple factor analysis

The third analysis of the introductory example implicitly assumes a balance between flora and soil. However, in this example, the mere fact that the flora is represented by 50 variables and the soil by 11 variables implies that the PCA with 61 active variables will be influenced mainly by the flora at least on the first axis). This is not desirable: there is no reason to wish one group play a more important role in the analysis.

The core of MFA is based on a factorial analysis (PCA in the case of quantitative variables, MCA in the case of qualitative variables) in which the variables are weighted. These weights are identical for the variables of the same group (and vary from one group to another). They are such that the maximum axial inertia of a group is equal to 1: in other words, by applying the PCA (or, where applicable, the MCA) to one group with this weighting, we obtain a first eigenvalue equal to 1. To get this property, MFA assigns to each variable of group ${\displaystyle j}$ a weight equal to the inverse of the first eigenvalue of the analysis (PCA or MCA according to the type of variable) of the group ${\displaystyle j}$.

Formally, noting ${\displaystyle \lambda _{1}^{j}}$ the first eigenvalue of the factorial analysis of one group ${\displaystyle j}$, the MFA assigns weight ${\displaystyle 1/\lambda _{1}^{j}}$ for each variable of the group ${\displaystyle j}$.

Balancing maximum axial inertia rather than the total inertia (= the number of variables in standard PCA) gives the MFA several important properties for the user. More directly, its interest appears in the following example.

Let two groups of variables defined on the same set of individuals.

1. Group 1 is composed of two uncorrelated variables A and B.
2. Group 2 is composed of two variables {C1, C2} identical to the same variable C uncorrelated with the first two.

This example is not completely unrealistic. It is often necessary to simultaneously analyse multi-dimensional and (quite) one-dimensional groups.

Each group having the same number of variables has the same total inertia.

In this example the first axis of the PCA is almost coincident with C. Indeed, in the space of variables, there are two variables in the direction of C: group 2, with all its inertia concentrated in one direction, influences predominantly the first axis. For its part, group 1, consisting of two orthogonal variables (= uncorrelated), has its inertia uniformly distributed in a plane (the plane generated by the two variables) and hardly weighs on the first axis.

Numerical Example

Table 1. MFA. Test data. A and B (group 1) are uncorrelated. C1 and C2 (group 2) are identical. ${\displaystyle A}$ ${\displaystyle B}$ ${\displaystyle C_{1}}$ ${\displaystyle C_{2}}$ ${\displaystyle 1}$ 1 1 1 1 ${\displaystyle 2}$ 2 3 4 4 ${\displaystyle 3}$ 3 5 2 2 ${\displaystyle 4}$ 4 5 2 2 ${\displaystyle 5}$ 5 3 4 4 ${\displaystyle 6}$ 6 1 2 2

Table 2. Test data. Decomposition of the inertia in the PCA and in the MFA applied to data in Table 1. ${\displaystyle F_{1}}$ ${\displaystyle F_{2}}$ PCA Inertia 2.14 (100%) 1 group 1 0.24(11%) 1 group 2 1.91(89%) 0 MFA Inertia 1.28(100%) 1 group 1 0.64(50%) 1 group 2 0.64(50%) 0

Table 2 summarizes the inertia of the first two axes of the PCA and of the MFA applied to Table 1.

Group 2 variables contribute to 88.95% of the inertia of the axis 1 of the PCA. The first axis (${\displaystyle F_{1}}$) is almost coincident with C: the correlation between C and ${\displaystyle F_{1}}$ is .976;

The first axis of the MFA (on Table 1 data) shows the balance between the two groups of variables: the contribution of each group to the inertia of this axis is strictly equal to 50%.

The second axis, meanwhile, depends only on group 1. This is natural since this group is two-dimensional while the second group, being one-dimensional, can be highly related to only one axis (here the first axis).

Introducing several active groups of variables in a factorial analysis implicitly assumes a balance between these groups.

This balance must take into account that a multidimensional group influences naturally more axes than a one-dimensional group does (which may not be closely related to one axis).

The weighting of the MFA, which makes the maximum axial inertia of each group equal to 1, plays this role.

The core of MFA is a weighted factorial analysis: MFA firstly provides the classical results of the factorial analyses.

1. Representations of individuals in which two individuals are close to each other if they exhibit similar values for many variables in the different variable groups; in practice the user particularly studies the first factorial plane.

2.Representations of quantitative variables as in PCA (correlation circle).

Figure1. MFA. Test data. Representation of individuals on the first plane.

Figure2. MFA. Test data. Representation of variables on the first plane.

In the example:

• The first axis mainly opposes individuals 1 and 5 (Figure 1).
• The four variables have a positive coordinate (Figure 2): the first axis is a size effect. Thus, individual 1 has low values for all the variables and individual 5 has high values for all the variables.

3. Indicators aiding interpretation: projected inertia, contributions and quality of representation. In the example, the contribution of individuals 1 and 5 to the inertia of the first axis is 45.7% + 31.5% = 77.2% which justifies the interpretation focussed on these two points.

4. Representations of categories of qualitative variables as in MCA (a category lies at the centroid of the individuals who possess it). No qualitative variables in the example.

5. Superimposed representations of individuals « seen » by each group. An individual considered from the point of view of a single group is called partial individual (in parallel, an individual considered from the point of view of all variables is said mean individual because it lies at the center of gravity of its partial points). Partial cloud ${\displaystyle N_{i}^{j}}$ gathers the ${\displaystyle I}$ individuals from the perspective of the single group ${\displaystyle j}$ (ie ${\displaystyle {i^{j},j=1,J}}$): that is the cloud analysed in the separate factorial analysis (PCA or MCA) of the group ${\displaystyle j}$. The superimposed representation of the ${\displaystyle N_{i}^{j}}$ provided by the MFA is similar in its purpose to that provided by the Procrustes analysis.

Figure 3. MFA. Test data. Superimposed representation of mean and partial clouds.

In the example (figure 3), individual 1 is characterized by a small size (i.e. small values) both in terms of group 1 and group 2 (partial points of the individual 1 have a negative coordinate and are close one another). On the contrary, the individual 5 is more characterized by high values for the variables of group 2 than for the variables of group 1 (for the individual 5, group 2 partial point lies further from the origin than group 1 partial point). This reading of the graph can be checked directly in the data.

6. Representations of groups of variables as such. In these graphs, each group of variables is represented by a single point. Two groups of variables are close one another when they define the same structure on individuals. Extreme case: two groups of variables that define homothetic clouds of individuals ${\displaystyle N_{i}^{j}}$ coincide. The coordinate of group ${\displaystyle j}$ along the axis ${\displaystyle s}$ is equal to the contribution of the group ${\displaystyle j}$ to the inertia of MFA dimension of rank ${\displaystyle s}$. This contribution can be interpreted as an indicator of relationship (between the group ${\displaystyle j}$ and the axis ${\displaystyle s}$, hence the name relationship square given to this type of representation). This representation also exists in other factorial methods (MCA and FAMD in particular) in which case the groups of variable are each reduced to a single variable.

Figure4. MFA. Test data. Representation of groups of variables.

In the example (Figure 4), this representation shows that the first axis is related to the two groups of variables, while the second axis is related to the first group. This agrees with the representation of the variables (figure 2). In practice, this representation is especially precious when the groups are numerous and include many variables.

Other reading grid. The two groups of variables have in common the size effect (first axis) and differ according to axis 2 since this axis is specific to group 1 (he opposes the variables A and B).

7. Representations of factors of separate analyses of the different groups. These factors are represented as supplementary quantitative variables (correlation circle).

Figure 5. MFA. Test data. Representation of the principal components of separate PCA of each group.

In the example (figure 5), the first axis of the MFA is relatively strongly correlated (r = .80) to the first component of the group 2. This group, consisting of two identical variables, possesses only one principal component (confounded with the variable). The group 1 consists of two orthogonal variables: any direction of the subspace generated by these two variables has the same inertia (equal to 1). So there is uncertainty in the choice of principal components and there is no reason to be interested in one of them in particular. However, the two components provided by the program are well represented: the plane of the MFA is close to the plane spanned by the two variables of group 1.