Qualitative variation

The formula for the variation around the mode (ModVR) is derived as follows:

${\displaystyle M=\sum _{i=1}^{K}(f_{m}-f_{i})}$

where f_m is the modal frequency, K is the number of categories and f_i is the frequency of the i^th group.

This can be simplified to

${\displaystyle M=Kf_{m}-N}$

where N is the total size of the sample.

Freeman's index (or variation ratio) is[2]

${\displaystyle v=1-{\frac {f_{m}}{N}}}$

This is related to M as follows:

${\displaystyle {\frac {({\frac {f_{m}}{N}})-{\frac {1}{K}}}{{\frac {N}{K}}{\frac {(K-1)}{N}}}}={\frac {M}{N(K-1)}}}$

The ModVR is defined as

${\displaystyle \operatorname {ModVR} =1-{\frac {Kf_{m}-N}{N(K-1)}}={\frac {K(N-f_{m})}{N(K-1)}}={\frac {Kv}{K-1}}}$

where v is Freeman's index.

Low values of ModVR correspond to small amount of variation and high values to larger amounts of variation.

When K is large, ModVR is approximately equal to Freeman's index v.

This is based on the range around the mode. It is defined to be

${\displaystyle \operatorname {RanVR} =1-{\frac {f_{m}-f_{l}}{f_{m}}}={\frac {f_{l}}{f_{m}}}}$

where f_m is the modal frequency and f_l is the lowest frequency.

This is an analog of the mean deviation. It is defined as the arithmetic mean of the absolute differences of each value from the mean.

${\displaystyle \operatorname {AvDev} =1-{\frac {1}{2N}}{\frac {K}{K-1}}\sum _{i=1}^{K}\left|f_{i}-{\frac {N}{K}}\right|}$

This is an analog of the mean difference - the average of the differences of all the possible pairs of variate values, taken regardless of sign. The mean difference differs from the mean and standard deviation because it is dependent on the spread of the variate values among themselves and not on the deviations from some central value.[3]

${\displaystyle \operatorname {MNDif} =1-{\frac {1}{N(K-1)}}\sum _{i=1}^{K-1}\sum _{j=i+1}^{K}|f_{i}-f_{j}|}$

where f_i and f_j are the i^th and j^th frequencies respectively.

The MNDif is the Gini coefficient applied to qualitative data.

This is an analog of the variance.

${\displaystyle \operatorname {VarNC} =1-{\frac {1}{N^{2}}}{\frac {K}{K-1}}\sum \left(f_{i}-{\frac {N}{K}}\right)^{2}}$

It is the same index as Mueller and Schussler's Index of Qualitative Variation[4] and Gibbs' M2 index.

It is distributed as a chi square variable with K – 1 degrees of freedom.[5]

Wilson has suggested two versions of this statistic.

The first is based on AvDev.

${\displaystyle \operatorname {StDev} _{1}=1-{\sqrt {\frac {\sum _{i=1}^{K}\left(f_{i}-{\frac {N}{K}}\right)^{2}}{\left(N-{\frac {N}{K}}\right)^{2}+(K-1)\left({\frac {N}{K}}\right)^{2}}}}}$

The second is based on MNDif

${\displaystyle \operatorname {StDev} _{2}=1-{\sqrt {\frac {\sum _{i=1}^{K-1}\sum _{j=i+1}^{K}(f_{i}-f_{j})^{2}}{N^{2}(K-1)}}}}$

This index was originally developed by Claude Shannon for use in specifying the properties of communication channels.

${\displaystyle \operatorname {HRel} ={\frac {-\sum p_{i}\log _{2}p_{i}}{\log _{2}K}}}$

where p_i = f_i / N.

This is equivalent to information entropy divided by the ${\displaystyle \log _{2}(K)}$ and is useful for comparing relative variation between frequency tables of multiple sizes.

Wilcox adapted a proposal of Kaiser[6] based on the geometric mean and created the B' index. The B index is defined as

${\displaystyle B=1-{\sqrt {1-\left[{\sqrt[{k}]{\prod _{i=1}^{k}{\frac {f_{i}K}{N}}}}\,\right]^{2}}}}$

Several of these indices have been implemented in the R language.[7]

The unstandardized index (M1) (Gibbs & Poston Jr 1975, p. 471) is

${\displaystyle M1=1-\sum _{i=1}^{K}p_{i}^{2}}$

where K is the number of categories and ${\displaystyle p_{i}=f_{i}/N}$ is the proportion of observations that fall in a given category i.

M1 can be interpreted as one minus the likelihood that a random pair of samples will belong to the same category,[9] so this formula for IQV is a standardized likelihood of a random pair falling in the same category. This index has also referred to as the index of differentiation, the index of sustenance differentiation and the geographical differentiation index depending on the context it has been used in.

A second index is the M2[10] (Gibbs & Poston Jr 1975, p. 472) is:

${\displaystyle M2={\frac {K}{K-1}}\left(1-\sum _{i=1}^{K}p_{i}^{2}\right)}$

where K is the number of categories and ${\displaystyle p_{i}=f_{i}/N}$ is the proportion of observations that fall in a given category i. The factor of ${\displaystyle {\frac {K}{K-1}}}$ is for standardization.

M1 and M2 can be interpreted in terms of variance of a multinomial distribution (Swanson 1976) (there called an "expanded binomial model"). M1 is the variance of the multinomial distribution and M2 is the ratio of the variance of the multinomial distribution to the variance of a binomial distribution.

The M4 index is

${\displaystyle M4={\frac {\sum _{i=1}^{K}|X_{i}-m|}{2\sum _{i=1}^{K}X_{i}}}}$

where m is the mean.

The formula for M6 is

${\displaystyle M6=K\left[1-{\frac {\sum _{i=1}^{K}|X_{i}-m|}{2N}}\right]}$

· where K is the number of categories, X_i is the number of data points in the i^th category, N is the total number of data points, || is the absolute value (modulus) and

${\displaystyle m={\frac {\sum _{i=1}^{K}X_{i}}{N}}}$

This formula can be simplified

${\displaystyle M6=K\left[1-{\frac {\sum _{i=1}^{K}\left|p_{i}-{\frac {1}{N}}\right|}{2}}\right]}$

where p_i is the proportion of the sample in the i^th category.

In practice M1 and M6 tend to be highly correlated which militates against their combined use.

The sum

${\displaystyle \sum _{i=1}^{K}p_{i}^{2}}$

has also found application. This is known as the Simpson index in ecology and as the Herfindahl index or the Herfindahl-Hirschman index (HHI) in economics. A variant of this is known as the Hunter–Gaston index in microbiology[11]

In linguistics and cryptanalysis this sum is known as the repeat rate. The incidence of coincidence (IC) is an unbiased estimator of this statistic[12]

${\displaystyle \operatorname {IC} =\sum {\frac {f_{i}(f_{i}-1)}{n(n-1)}}}$

where f_i is the count of the i^th grapheme in the text and n is the total number of graphemes in the text.

M1

The M1 statistic defined above has been proposed several times in a number of different settings under a variety of names. These include Gini's index of mutability,[13] Simpson's measure of diversity,[14] Bachi's index of linguistic homogeneity,[15] Mueller and Schuessler's index of qualitative variation,[16] Gibbs and Martin's index of industry diversification,[17] Lieberson's index.[18] and Blau's index in sociology, psychology and management studies.[19] The formulation of all these indices are identical.

Simpson's D is defined as

${\displaystyle D=1-\sum _{i=1}^{K}{\frac {n_{i}(n_{i}-1)}{n(n-1)}}}$

where n is the total sample size and n_i is the number of items in the i^th category.

For large n we have

${\displaystyle u\sim 1-\sum _{i=1}^{K}p_{i}^{2}}$

Another statistic that has been proposed is the coefficient of unalikeability which ranges between 0 and 1.[20]

${\displaystyle u={\frac {c(x,y)}{n^{2}-n}}}$

where n is the sample size and c(x,y) = 1 if x and y are alike and 0 otherwise.

For large n we have

${\displaystyle u\sim 1-\sum _{i=1}^{K}p_{i}^{2}}$

where K is the number of categories.

Another related statistic is the quadratic entropy

${\displaystyle H^{2}=2\left(1-\sum _{i=1}^{K}p_{i}^{2}\right)}$

which is itself related to the Gini index.

M2

Greenberg's monolingual non weighted index of linguistic diversity[21] is the M2 statistic defined above.

M7

Another index – the M7 – was created based on the M4 index of Gibbs & Poston Jr (1975)[22]

${\displaystyle M7={\frac {\sum _{i=1}^{K}\sum _{j=1}^{L}|R_{i}-R|}{2\sum R_{i}}}}$

where

${\displaystyle R_{ij}={\frac {O_{ij}}{E_{ij}}}={\frac {O_{ij}}{n_{i}p_{j}}}}$

and

${\displaystyle R={\frac {\sum _{i=1}^{K}\sum _{j=1}^{L}R_{ij}}{\sum _{i=1}^{K}n_{i}}}}$

where K is the number of categories, L is the number of subtypes, O_ij and E_ij are the number observed and expected respectively of subtype j in the i^th category, n_i is the number in the i^th category and p_j is the proportion of subtype j in the complete sample.

Note: This index was designed to measure women's participation in the work place: the two subtypes it was developed for were male and female.

The Berger–Parker index equals the maximum ${\displaystyle p_{i}}$ value in the dataset, i.e. the proportional abundance of the most abundant type.[23] This corresponds to the weighted generalized mean of the ${\displaystyle p_{i}}$ values when q approaches infinity, and hence equals the inverse of true diversity of order infinity (1/^∞D).

This index is strictly applicable only to entire populations rather than to finite samples. It is defined as

${\displaystyle I_{B}={\frac {\log(N!)-\sum _{i=1}^{K}(\log(n_{i}!))}{N}}}$

where N is total number of individuals in the population, n_i is the number of individuals in the i^th category and N! is the factorial of N. Brillouin's index of evenness is defined as

${\displaystyle E_{B}=I_{B}/I_{B(\max )}}$

where I_B(max) is the maximum value of I_B.

Hill suggested a family of diversity numbers[24]

${\displaystyle N_{a}={\frac {1}{\left[\sum _{i=1}^{K}p_{i}^{a}\right]^{a-1}}}}$

For given values of a, several of the other indices can be computed

• a = 0: N_a = species richness
• a = 1: N_a = Shannon's index
• a = 2: N_a = 1/Simpson's index (without the small sample correction)
• a = 3: N_a = 1/Berger–Parker index

Hill also suggested a family of evenness measures

${\displaystyle E_{a,b}={\frac {N_{a}}{N_{b}}}}$

where a > b.

Hill's E₄ is

${\displaystyle E_{4}={\frac {N_{2}}{N_{1}}}}$

Hill's E₅ is

${\displaystyle E_{5}={\frac {N_{2}-1}{N_{1}-1}}}$

${\displaystyle I_{\text{Marg}}={\frac {S-1}{\log _{e}N}}}$

where S is the number of data types in the sample and N is the total size of the sample.[25]

${\displaystyle I_{\mathrm {Men} }={\frac {S}{\sqrt {N}}}}$

where S is the number of data types in the sample and N is the total size of the sample.[26]

In linguistics this index is the identical with the Kuraszkiewicz index (Guiard index) where S is the number of distinct words (types) and N is the total number of words (tokens) in the text being examined.[27][28] This index can be derived as a special case of the Generalised Torquist function.[29]

This is a statistic invented by Kempton and Taylor.[30] and involves the quartiles of the sample. It is defined as

${\displaystyle Q={\frac {{\frac {1}{2}}(n_{R1}+n_{R2})+\sum _{j=R_{1}+1}^{R_{2}-1}n_{j}}{\log(R_{2}/R_{1})}}}$

where R₁ and R₁ are the 25% and 75% quartiles respectively on the cumulative species curve, n_j is the number of species in the j_th category, n_Ri is the number of species in the class where R_i falls (i = 1 or 2).

This is taken from information theory

${\displaystyle H=\log _{e}N-{\frac {1}{N}}\sum n_{i}p_{i}\log(p_{i})}$

where N is the total number in the sample and p_i is the proportion in the i^th category.

In ecology where this index is commonly used, H usually lies between 1.5 and 3.5 and only rarely exceeds 4.0.

An approximate formula for the standard deviation (SD) of H is

${\displaystyle \operatorname {SD} (H)={\frac {1}{N}}\left[\sum p_{i}[\log _{e}(p_{i})]^{2}-H^{2}\right]}$

where p_i is the proportion made up by the i^th category and N is the total in the sample.

A more accurate approximate value of the variance of H(var(H)) is given by[31]

${\displaystyle \operatorname {var} (H)={\frac {\sum p_{i}[\log(p_{i})]^{2}-\left[\sum p_{i}\log(p_{i})\right]^{2}}{N}}+{\frac {K-1}{2N^{2}}}+{\frac {-1+\sum p_{i}^{2}-\sum p_{i}^{-1}\log(p_{i})+\sum p_{i}^{-1}\sum p_{i}\log(p_{i})}{6N^{3}}}}$

where N is the sample size and K is the number of categories.

A related index is the Pielou J defined as

${\displaystyle J={\frac {H}{\log _{e}(S)}}}$

One difficulty with this index is that S is unknown for a finite sample. In practice S is usually set to the maximum present in any category in the sample.

The Rényi entropy is a generalization of the Shannon entropy to other values of q than unity. It can be expressed:

${\displaystyle {}^{q}H={\frac {1}{1-q}}\;\ln \left(\sum _{i=1}^{K}p_{i}^{q}\right)}$

which equals

${\displaystyle {}^{q}H=\ln \left({1 \over {\sqrt[{q-1}]{\sum _{i=1}^{K}p_{i}p_{i}^{q-1}}}}\right)=\ln({}^{q}\!D)}$

This means that taking the logarithm of true diversity based on any value of q gives the Rényi entropy corresponding to the same value of q.

The value of ${\displaystyle {}^{q}\!D}$ is also known as the Hill number.[24]

${\displaystyle D={\frac {N-{\sqrt {\sum _{i=1}^{K}n_{i}}}}{N-{\sqrt {N}}}}}$

where N is the total sample size and n_i is the number in the i^th category.

${\displaystyle E={\frac {N-{\sqrt {\sum _{i=1}^{K}n_{i}}}}{N-{\frac {N}{\sqrt {K}}}}}}$

where K is the number of categories.

This was the first index to be derived for diversity.[32]

${\displaystyle K=\alpha \ln(1+{\frac {N}{\alpha }})}$

where K is the number of categories and N is the number of data points in the sample. Fisher's α has to be estimated numerically from the data.

The expected number of individuals in the r^th category where the categories have been placed in increasing size is

${\displaystyle \operatorname {E} (n_{r})=\alpha {\frac {X^{r}}{r}}}$

where X is an empirical parameter lying between 0 and 1. While X is best estimated numerically an approximate value can be obtained by solving the following two equations

${\displaystyle N={\frac {\alpha X}{1-X}}}$

${\displaystyle K=-\alpha \ln(1-X)}$

where K is the number of categories and N is the total sample size.

The variance of α is approximately[33]

${\displaystyle \operatorname {var} (\alpha )={\frac {\alpha }{\ln(X)(1-X)}}}$

This index (D_w) is the distance between the Lorenz curve of species distribution and the 45 degree line. It is closely related to the Gini coefficient.[34]

In symbols it is

${\displaystyle D_{w}=max[{\frac {c_{i}}{K}}-{\frac {i}{N}}]}$

where max() is the maximum value taken over the N data points, K is the number of categories (or species) in the data set and c_i is the cumulative total up and including the i_th category.

This is related to Simpson's D and is defined as

${\displaystyle E={\frac {1/D}{K}}}$

where D is Simpson's D and K is the number of categories in the sample.

Smith and Wilson suggested a number of indices based on Simpson's D.

${\displaystyle E_{1}={\frac {1-D}{1-{\frac {1}{K}}}}}$

${\displaystyle E_{2}={\frac {\log _{e}(D)}{\log _{e}(K)}}}$

where D is Simpson's D and K is the number of categories.

${\displaystyle E={\frac {e^{H}-1}{K-1}}}$

where H is the Shannon entropy and K is the number of categories.

This index is closely related to Sheldon's index which is

${\displaystyle E={\frac {e^{H}}{K}}}$

where H is the Shannon entropy and K is the number of categories.

This index was created by Camargo in 1993.[35]

${\displaystyle E=1-\sum _{i=1}^{K}\sum _{j=i+1}^{K}{\frac {p_{i}-p_{j}}{K}}}$

where K is the number of categories and p_i is the proportion in the i^th category.

This index was proposed by Smith and Wilson in 1996.[36]

${\displaystyle B=1-{\frac {2}{\pi }}\arctan(\theta )}$

where θ is the slope of the log(abundance)-rank curve.

This is the slope of the log(abundance)-rank curve.

There are two versions of this index - one for continuous distributions (E_c) and the other for discrete (E_d).[37]

${\displaystyle E_{c}={\frac {O-{\frac {1}{K}}}{1-{\frac {1}{K}}}}}$

${\displaystyle E_{d}={\frac {O-{\frac {1}{K}}-{\frac {K-1}{N}}}{1-{\frac {1}{K}}-{\frac {K-1}{N}}}}}$

where

${\displaystyle O=1-{\frac {1}{2}}\left|p_{i}-{\frac {1}{K}}\right|}$

is the Schoener–Czekanoski index, K is the number of categories and N is the sample size.

This index (R_ik) is based on Shannon's entropy.[38] It is defined as

${\displaystyle R_{ik}={\frac {H_{\max }-H_{\mathrm {obs} }}{H_{\max }-H_{\min }}}}$

where

${\displaystyle X=\sum x_{ij}}$

${\displaystyle X=\sum x_{kj}}$

${\displaystyle H(X)=\sum {\frac {x_{ij}}{X}}\log {\frac {X}{x_{ij}}}}$

${\displaystyle H(Y)=\sum {\frac {x_{kj}}{Y}}\log {\frac {Y}{x_{kj}}}}$

${\displaystyle H_{\min }={\frac {X}{X+Y}}H(X)+{\frac {Y}{X+Y}}H(Y)}$

${\displaystyle H_{\max }=\sum \left({\frac {x_{ij}}{X+Y}}\log {\frac {X+Y}{x_{ij}}}+{\frac {x_{kj}}{X+Y}}\log {\frac {X+Y}{x_{kj}}}\right)}$

${\displaystyle H_{\mathrm {obs} }=\sum {\frac {x_{ij}+x_{kj}}{X+Y}}\log {\frac {X+Y}{x_{ij}+x_{kj}}}}$

In these equations x_ij and x_kj are the number of times the j^th data type appears in the i^th or k^th sample respectively.

In a rarefied sample a random subsample n in chosen from the total N items. In this sample some groups may be necessarily absent from this subsample. Let ${\displaystyle X_{n}}$ be the number of groups still present in the subsample of n items. ${\displaystyle X_{n}}$ is less than K the number of categories whenever at least one group is missing from this subsample.

The rarefaction curve, ${\displaystyle f_{n}}$ is defined as:

${\displaystyle f_{n}=\operatorname {E} [X_{n}]=K-{\binom {N}{n}}^{-1}\sum _{i=1}^{K}{\binom {N-N_{i}}{n}}}$

Note that 0 ≤ f(n) ≤ K.

Furthermore,

${\displaystyle f(0)=0,\ f(1)=1,\ f(N)=K.}$

Despite being defined at discrete values of n, these curves are most frequently displayed as continuous functions.[39]

This index is discussed further in Rarefaction (ecology).

This is a z type statistic based on Shannon's entropy.[40]

${\displaystyle V={\frac {H-\operatorname {E} (H)}{\operatorname {SD} (H)}}}$

where H is the Shannon entropy, E(H) is the expected Shannon entropy for a neutral model of distribution and SD(H) is the standard deviation of the entropy. The standard deviation is estimated from the formula derived by Pielou

${\displaystyle SD(H)={\frac {1}{N}}\left[\sum p_{i}[\log _{e}(p_{i})]^{2}-H^{2}\right]}$

where p_i is the proportion made up by the i^th category and N is the total in the sample.

This is

${\displaystyle I_{LG}={\frac {K}{K'}}}$

where K is the number of categories and K' is the number of categories according to MacArthur's broken stick model yielding the observed diversity.

This index is used to compare the relationship between hosts and their parasites.[41] It incorporates information about the phylogenetic relationship amongst the host species.

${\displaystyle S_{TD}=2{\frac {\sum \sum _{i<j}\omega _{ij}}{s(s-1)}}}$

where s is the number of host species used by a parasite and ω_ij is the taxonomic distinctness between host species i and j.

Several indices with this name have been proposed.

One of these is

${\displaystyle IQV={\frac {K(100^{2}-\sum _{i=1}^{K}p_{i}^{2})}{100^{2}(K-1)}}={\frac {K}{K-1}}(1-\sum _{i=1}^{K}(p_{i}/100)^{2})}$

where K is the number of categories and p_i is the proportion of the sample that lies in the i^th category.

This index is also known as the multigroup entropy index or the information theory index. It was proposed by Theil in 1972.[42] The index is a weighted average of the samples entropy.

Let

${\displaystyle E_{a}=\sum _{i=1}^{a}p_{i}log(p_{i})}$

and

${\displaystyle H=\sum _{i=1}^{r}{\frac {n_{i}(E-E_{i})}{NE}}}$

where p_i is the proportion of type i in the a^th sample, r is the total number of samples, n_i is the size of the i^th sample, N is the size of the population from which the samples were obtained and E is the entropy of the population.

Let A and B be two types of data item. Then the index of dissimilarity is

${\displaystyle D={\frac {1}{2}}\sum _{i=1}^{K}\left|{\frac {A_{i}}{A}}-{\frac {B_{i}}{B}}\right|}$

where

${\displaystyle A=\sum _{i=1}^{K}A_{i}}$

${\displaystyle B=\sum _{i=1}^{K}B_{i}}$

A_i is the number of data type A at sample site i, B_i is the number of data type B at sample site i, K is the number of sites sampled and || is the absolute value.

This index is probably better known as the index of dissimilarity (D).[43] It is closely related to the Gini index.

This index is biased as its expectation under a uniform distribution is > 0.

A modification of this index has been proposed by Gorard and Taylor.[44] Their index (GT) is

${\displaystyle GT=D\left(1-{\frac {A}{A+B}}\right)}$

The index of segregation (IS)[45] is

${\displaystyle SI={\frac {1}{2}}\sum _{i=1}^{K}\left|{\frac {A_{i}}{A}}-{\frac {t_{i}-A_{i}}{T-A}}\right|}$

where

${\displaystyle A=\sum _{i=1}^{K}A_{i}}$

${\displaystyle T=\sum _{i=1}^{K}t_{i}}$

and K is the number of units, A_i and t_i is the number of data type A in unit i and the total number of all data types in unit i.

This index (H) is defined as[46]

${\displaystyle H=1-\sum _{i=1}^{K}\sum _{j=1}^{i}{\sqrt {p_{i}p_{j}}}}$

where p_i is the proportion of the sample composed of the i^th variate.

This index ( L_xy ) was invented by Lieberson in 1981.[47]

${\displaystyle L_{xy}={\frac {1}{N}}\sum _{i=1}^{K}{\frac {X_{i}Y_{i}}{X_{\mathrm {tot} }}}}$

where X_i and Y_i are the variables of interest at the i^th site, K is the number of sites examined and X_tot is the total number of variate of type X in the study.

This index is defined as[48]

${\displaystyle I_{R}={\frac {p_{xx}-p_{x}}{1-p_{x}}}}$

where p_x is the proportion of the sample made up of variates of type X and

${\displaystyle p_{xx}={\frac {\sum _{i=1}^{K}x_{i}p_{i}}{N_{x}}}}$

where N_x is the total number of variates of type X in the study, K is the number of samples in the study and x_i and p_i are the number of variates and the proportion of variates of type X respectively in the i^th sample.

The index of isolation is

${\displaystyle II=\sum _{i=1}^{K}{\frac {A_{i}}{A}}{\frac {A_{i}}{t_{i}}}}$

where K is the number of units in the study, A_i and t_i is the number of units of type A and the number of all units in i_th sample.

A modified index of isolation has also been proposed

${\displaystyle MII={\frac {II-{\frac {A}{T}}}{1-{\frac {A}{T}}}}}$

The MII lies between 0 and 1.

This index (GS) is defined as

${\displaystyle GS={\frac {1}{2}}\sum _{i=1}^{K}\left|{\frac {A_{i}}{A}}-{\frac {t_{i}}{T}}\right|}$

where

${\displaystyle A=\sum _{i=1}^{K}A_{i}}$

${\displaystyle T=\sum _{i=1}^{K}t_{i}}$

and A_i and t_i are the number of data items of type A and the total number of items in the i^th sample.

This index is defined as

${\displaystyle IE=\sum _{i=1}^{K}{\frac {A_{i}}{A}}{\frac {B_{i}}{t_{i}}}}$

where

${\displaystyle A=\sum _{i=1}^{K}A_{i}}$

and A_i and B_i are the number of types A and B in the i^th category and t_i is the total number of data points in the i^th category.

This is a binary form of the cosine index.[49] It is used to compare presence/absence data of two data types (here A and B). It is defined as

${\displaystyle O={\frac {a}{\sqrt {(a+b)(a+c)}}}}$

where a is the number of sample units where both A and B are found, b is number of sample units where A but not B occurs and c is the number of sample units where type B is present but not type A.

This coefficient was invented by Stanisław Kulczyński in 1927[50] and is an index of association between two types (here A and B). It varies in value between 0 and 1. It is defined as

${\displaystyle K={\frac {a}{2}}\left({\frac {1}{a+b}}+{\frac {1}{a+c}}\right)}$

where a is the number of sample units where type A and type B are present, b is the number of sample units where type A but not type B is present and c is the number of sample units where type B is present but not type A.

This index was invented by Yule in 1900.[51] It concerns the association of two different types (here A and B). It is defined as

${\displaystyle Q={\frac {ad-bc}{ad+bc}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. Q varies in value between -1 and +1. In the ordinal case Q is known as the Goodman-Kruskal γ.

Because the denominator potentially may be zero, Leinhert and Sporer have recommended adding +1 to a, b, c and d.[52]

This index is defined as

${\displaystyle Y={\frac {{\sqrt {ad}}-{\sqrt {bc}}}{{\sqrt {ad}}+{\sqrt {bc}}}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present.

This index was invented by Baroni-Urbani and Buser in 1976.[53] It varies between 0 and 1 in value. It is defined as

${\displaystyle BUB={\frac {{\sqrt {ad}}+a}{{\sqrt {ad}}+a+b+c}}={\frac {{\sqrt {ad}}+a}{N+{\sqrt {ad}}-d}}=1-{\frac {N-(a-d)}{N+{\sqrt {ad}}-d}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

When d = 0, this index is identical to the Jaccard index.

This coefficient is defined as

${\displaystyle H={\frac {(a+d)-(b+c)}{a+b+c+d}}={\frac {(a+d)-(b+c)}{N}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle RT={\frac {a+d}{a+2(b+c)+d}}={\frac {a+d}{N+b+c}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size

This coefficient is defined as

${\displaystyle SS={\frac {2(a+d)}{2(a+d)+b+c}}={\frac {2(a+d)}{N+a+d}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle SBD={\sqrt {\frac {b+c}{a+b+c+d}}}={\sqrt {\frac {b+c}{N}}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle RR={\frac {a}{a+b+c+d}}={\frac {a}{N}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle \varphi ={\frac {ad-bc}{\sqrt {(a+b)(a+c)(b+c)(c+d)}}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present.

This coefficient is defined as

${\displaystyle S={\frac {b+c}{b+c+d}}={\frac {b+c}{N-a}}}$

where b is the number of samples where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle S={\frac {a}{a+\min(b,c)}}}$

where b is the number of samples where type A is present but not type B, c is the number of samples where type B is present but not type A.

This coefficient is defined as

${\displaystyle D={\frac {ad-bc}{\sqrt {(a+b+c+d)(a+b)(a+c)}}}={\frac {ad-bc}{\sqrt {N(a+b)(a+c)}}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient was proposed by Stephen Alfred Forbes in 1907.[54] It is defined as

${\displaystyle F={\frac {aN}{(a+b)(a+c)}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size (N = a + b + c + d).

A modification of this coefficient which does not require the knowledge of d has been proposed by Alroy[55]

${\displaystyle F_{A}={\frac {a(n+{\sqrt {n}})}{a(n+{\sqrt {n}})+{\frac {3}{2}}bc}}=1-{\frac {3bc}{2a(n+{\sqrt {n}})+3bc}}}$

Where n = a + b + c.

This coefficient is defined as

${\displaystyle SM={\frac {a+d}{a+b+c+d}}={\frac {a+d}{N}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle F={\frac {(a+b+c+d)(a-0.5)^{2}}{(a+b)(a+c)}}={\frac {N(a-0.5)^{2}}{(a+b)(a+c)}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

This coefficient is defined as

${\displaystyle S=\log \left[{\frac {n(|ad-bc|-{\frac {n}{2}})^{2}}{(a+b)(a+c)(b+d)(c+d)}}\right]}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A, d is the sample count where neither type A nor type B are present, n equals a + b + c + d and || is the modulus (absolute value) of the difference.

This coefficient is defined as

${\displaystyle M={\frac {4(ad-bc)}{(a+d)^{2}+(b+c)^{2}}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present.

In 1884 Charles Peirce suggested[56] the following coefficient

${\displaystyle P={\frac {ab+bc}{ab+2bc+cd}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present.

In 1975 Hawkin and Dotson proposed the following coefficient

${\displaystyle HD={\frac {1}{2}}\left({\frac {a}{a+b+c}}+{\frac {d}{b+c+d}}\right)={\frac {1}{2}}\left({\frac {a}{N-d}}+{\frac {d}{N-a}}\right)}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

In 1901 Benini proposed the following coefficient

${\displaystyle B={\frac {a-(a+b)(a+c)}{a+\min(b,c)-(a+b)(a+c)}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A. Min(b, c) is the minimum of b and c.

Gilbert proposed the following coefficient

${\displaystyle G={\frac {a-(a+b)(a+c)}{a+b+c-(a+b)(a+c)}}={\frac {a-(a+b)(a+c)}{N-(a+b)(a+c)-d}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

The Gini index is

${\displaystyle G={\frac {a-(a+b)(a+c)}{\sqrt {(1-(a+b)^{2})(1-(a+c)^{2})}}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A.

The modified Gini index is

${\displaystyle G_{M}={\frac {a-(a+b)(a+c)}{1-{\frac {|b-c|}{2}}-(a+b)(a+c)}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A.

Kuhn proposed the following coefficient in 1965

${\displaystyle I={\frac {2(ad-bc)}{K(2a+b+c)}}={\frac {2(ad-bc)}{K(N+a-d)}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A. K is a normalizing parameter. N is the sample size.

This index is also known as the coefficient of arithmetic means.

Eyraud proposed the following coefficient in 1936

${\displaystyle I={\frac {a-(a+b)(a+c)}{(a+c)(a+d)(b+d)(c+d)}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the number of samples where both A and B are not present.

This is defined as

${\displaystyle \operatorname {SD} ={\frac {b+c}{b+c+d}}={\frac {b+c}{N-a}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the number of samples where both A and B are not present. N is the sample size.

This is defined as

${\displaystyle TI=1-{\frac {a}{b+c+d}}=1-{\frac {a}{N-a}}={\frac {N-2a}{N-a}}}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the number of samples where both A and B are not present. N is the sample size.

This is defined as

${\displaystyle PSI=a-bc}$

where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A.

This is also known as the Bray–Curtis index, Schoener's index, least common percentage index, index of affinity or proportional similarity. It is related to the Sørensen similarity index.

${\displaystyle CZI={\frac {\sum \min(x_{i},x_{j})}{\sum (x_{i}+x_{j})}}}$

where x_i and x_j are the number of species in sites i and j respectively and the minimum is taken over the number of species in common between the two sites.

The Canberra distance is a weighted version of the L₁ metric. It was introduced by introduced in 1966[57] and refined in 1967[58] by G. N. Lance and W. T. Williams. It is used to define a distance between two vectors – here two sites with K categories within each site.

The Canberra distance d between vectors p and q in an K-dimensional real vector space is

${\displaystyle d(\mathbf {p} ,\mathbf {q} )=\sum _{i=1}^{n}{\frac {|p_{i}-q_{i}|}{|p_{i}|+|q_{i}|}}}$

where p_i and q_i are the values of the i^th category of the two vectors.

This is used to measure similarities between communities.

${\displaystyle CC={\frac {2c}{s_{1}+s_{2}}}}$

where s₁ and s₂ are the number of species in community 1 and 2 respectively and c is the number of species common to both areas.

This is a measure of the similarity between two samples:

${\displaystyle J={\frac {A}{A+B+C}}}$

where A is the number of data points shared between the two samples and B and C are the data points found only in the first and second samples respectively.

This index was invented in 1902 by the Swiss botanist Paul Jaccard.[59]

Under a random distribution the expected value of J is[60]

${\displaystyle J={\frac {1}{A}}\left({\frac {1}{A+B+C}}\right)}$

The standard error of this index with the assumption of a random distribution is

${\displaystyle SE(J)={\sqrt {\frac {A(B+C)}{N(A+B+C)^{3}}}}}$

where N is the total size of the sample.

This is a measure of the similarity between two samples:

${\displaystyle D={\frac {2A}{2A+B+C}}}$

where A is the number of data points shared between the two samples and B and C are the data points found only in the first and second samples respectively.

This is a measure of the similarity between two samples:

${\displaystyle M={\frac {N-B-C}{N}}=1-{\frac {B+C}{N}}}$

where N is the number of data points in the two samples and B and C are the data points found only in the first and second samples respectively.

Morisita’s index of dispersion ( I_m ) is the scaled probability that two points chosen at random from the whole population are in the same sample.[61] Higher values indicate a more clumped distribution.

${\displaystyle I_{m}={\frac {\sum x(x-1)}{nm(m-1)}}}$

An alternative formulation is

${\displaystyle I_{m}=n{\frac {\sum x^{2}-\sum x}{\left(\sum x\right)^{2}-\sum x}}}$

where n is the total sample size, m is the sample mean and x are the individual values with the sum taken over the whole sample. It is also equal to

${\displaystyle I_{m}={\frac {n\ IMC}{nm-1}}}$

where IMC is Lloyd's index of crowding.[62]

This index is relatively independent of the population density but is affected by the sample size.

Morisita showed that the statistic[61]

${\displaystyle I_{m}\left(\sum x-1\right)+n-\sum x}$

is distributed as a chi-squared variable with n − 1 degrees of freedom.

An alternative significance test for this index has been developed for large samples.[63]

${\displaystyle z={\frac {I_{m}-1}{2/nm^{2}}}}$

where m is the overall sample mean, n is the number of sample units and z is the normal distribution abscissa. Significance is tested by comparing the value of z against the values of the normal distribution.

Morisita's overlap index is used to compare overlap among samples.[64] The index is based on the assumption that increasing the size of the samples will increase the diversity because it will include different habitats

${\displaystyle C_{D}={\frac {2\sum _{i=1}^{S}x_{i}y_{i}}{(D_{x}+D_{y})XY}}}$

x_i is the number of times species i is represented in the total X from one sample.

y_i is the number of times species i is represented in the total Y from another sample.

D_x and D_y are the Simpson's index values for the x and y samples respectively.

S is the number of unique species

C_D = 0 if the two samples do not overlap in terms of species, and C_D = 1 if the species occur in the same proportions in both samples.

Horn's introduced a modification of the index[65]

${\displaystyle C_{H}={\frac {2\sum _{i=1}^{S}x_{i}y_{i}}{\left({\sum _{i=1}^{S}x_{i}^{2} \over X^{2}}+{\sum _{i=1}^{S}y_{i}^{2} \over Y^{2}}\right)XY}}}$

Smith-Gill developed a statistic based on Morisita’s index which is independent of both sample size and population density and bounded by −1 and +1. This statistic is calculated as follows[66]

First determine Morisita's index ( I_d ) in the usual fashion. Then let k be the number of units the population was sampled from. Calculate the two critical values

${\displaystyle M_{u}={\frac {\chi _{0.975}^{2}-k+\sum x}{\sum x-1}}}$

${\displaystyle M_{c}={\frac {\chi _{0.025}^{2}-k+\sum x}{\sum x-1}}}$

where χ² is the chi square value for n − 1 degrees of freedom at the 97.5% and 2.5% levels of confidence.

The standardised index ( I_p ) is then calculated from one of the formulae below

When I_d ≥ M_c > 1

${\displaystyle I_{p}=0.5+0.5\left({\frac {I_{d}-M_{c}}{k-M_{c}}}\right)}$

When M_c > I_d ≥ 1

${\displaystyle I_{p}=0.5\left({\frac {I_{d}-1}{M_{u}-1}}\right)}$

When 1 > I_d ≥ M_u

${\displaystyle I_{p}=-0.5\left({\frac {I_{d}-1}{M_{u}-1}}\right)}$

When 1 > M_u > I_d

${\displaystyle I_{p}=-0.5+0.5\left({\frac {I_{d}-M_{u}}{M_{u}}}\right)}$

I_p ranges between +1 and −1 with 95% confidence intervals of ±0.5. I_p has the value of 0 if the pattern is random; if the pattern is uniform, I_p < 0 and if the pattern shows aggregation, I_p > 0.

These indices are a measure of evenness between samples.[67]

${\displaystyle E_{1}={\frac {I-I_{\min }}{I_{\max }-I_{\min }}}}$

${\displaystyle E_{2}={\frac {I}{I_{\max }}}}$

where I is an index of diversity, I_max and I_min are the maximum and minimum values of I between the samples being compared.

Loevinger has suggested a coefficient H defined as follows:

${\displaystyle H={\sqrt {\frac {p_{\max }(1-p_{\min })}{p_{\min }(1-p_{\max })}}}}$

where p_max and p_min are the maximum and minimum proportions in the sample.

The Tversky index [68] is an asymmetric measure that lies between 0 and 1.

For samples A and B the Tversky index (S) is

${\displaystyle S={\frac {|A\cap B|}{|A\cap B|+\alpha |A-B|+\beta |B-A|}}}$

The values of α and β are arbitrary. Setting both α and β to 0.5 gives Dice's coefficient. Setting both to 1 gives Tanimoto's coefficient.

A symmetrical variant of this index has also been proposed.[69]

${\displaystyle S_{1}={\frac {|A\cap B|}{|A\cap B|+\beta \left(\alpha a+(1-\alpha )b\right)}}}$