Fréchet inception distance

For any two probability distributions ${\displaystyle \mu ,\nu }$ over ${\displaystyle \mathbb {R} ^{n}}$ having finite mean and variances, their Fréchet distance is[4]

$${\displaystyle d_{F}(\mu ,\nu ):=\left(\inf _{\gamma \in \Gamma (\mu ,\nu )}\int _{\mathbb {R} ^{n}\times \mathbb {R} ^{n}}\|x-y\|^{2}\,\mathrm {d} \gamma (x,y)\right)^{1/2},}$$

where ${\displaystyle \Gamma (\mu ,\nu )}$ is the set of all measures on ${\displaystyle \mathbb {R} ^{n}\times \mathbb {R} ^{n}}$ with marginals ${\displaystyle \mu }$ and ${\displaystyle \nu }$ on the first and second factors respectively. (The set ${\displaystyle \Gamma (\mu ,\nu )}$ is also called the set of all couplings of ${\displaystyle \mu }$ and ${\displaystyle \nu }$.). In other words, it is the 2-Wasserstein distance on ${\displaystyle \mathbb {R} ^{n}}$. For two multidimensional Gaussian distributions ${\displaystyle {\mathcal {N}}(\mu ,\Sigma )}$ and ${\displaystyle {\mathcal {N}}(\mu ',\Sigma ')}$, it is explicitly solvable as[5]

$${\displaystyle d_{F}({\mathcal {N}}(\mu ,\Sigma ),{\mathcal {N}}(\mu ',\Sigma '))^{2}=\lVert \mu -\mu '\rVert _{2}^{2}+\operatorname {tr} \left(\Sigma +\Sigma '-2\left(\Sigma ^{\frac {1}{2}}\cdot \Sigma '\cdot \Sigma ^{\frac {1}{2}}\right)^{\frac {1}{2}}\right)}$$

This allows us to define the FID in pseudocode form:

INPUT a function ${\displaystyle f:\Omega _{X}\to \mathbb {R} ^{n}}$.

INPUT two datasets ${\displaystyle S,S'\subset \Omega _{X}}$.

Compute ${\displaystyle f(S),f(S')\subset \mathbb {R} ^{n}}$.

Fit two gaussian distributions ${\displaystyle {\mathcal {N}}(\mu ,\Sigma ),{\mathcal {N}}(\mu ',\Sigma ')}$, respectively for ${\displaystyle f(S),f(S')}$.

RETURN ${\displaystyle d_{F}({\mathcal {N}}(\mu ,\Sigma ),{\mathcal {N}}(\mu ',\Sigma '))^{2}}$.

In most practical uses of the FID, ${\displaystyle \Omega _{X}}$ is the space of images, and ${\displaystyle f}$ is an Inception v3 model trained on the ImageNet, but without its final classification layer. Technically, it is the 2048-dimensional activation vector of its last pooling layer. Of the two datasets ${\displaystyle S,S'}$, one of them is a reference dataset, which could be the ImageNet itself, and the other is a set of images generated by a generative model, such as GAN, or diffusion model.[1]

Specialized variants of FID have been suggested as evaluation metric for music enhancement algorithms as Fréchet Audio Distance (FAD),[6] for generative models of video as Fréchet Video Distance (FVD),[7] and for AI-generated molecules as Fréchet ChemNet Distance (FCD).[8]

Chong and Forsyth[9] showed FID to be statistically biased, in the sense that their expected value over a finite data is not their true value. Also, because FID measured the Wasserstein distance towards the ground-truth distribution, it is inadequate for evaluating the quality of generators in domain adaptation setups, or in zero-shot generation. Finally, while FID is more consistent with human judgement than previously used inception score, there are cases where FID is inconsistent with human judgment (e.g. Figure 3,5 in Liu et al.).[10]

• Fréchet distance