Distribution-Free Tests

Non-Parametric Statistics

The term "non-parametric statistics" has, at least, two different meanings.

1. The first meaning of non-parametric covers techniques that do not rely on data belonging to any particular distribution. These include, among others:

• distribution free methods, which do not rely on assumptions that the data are drawn from a given probability distribution. ( As such, it is the opposite of parametric statistics. It includes non-parametric descriptive statistics, statistical models, inference, and statistical tests).
• non-parametric statistics (in the sense of a statistic over data, which is defined to be a function on a sample that has no dependency on a parameter), whose interpretation does not depend on the population fitting any parameterized distributions. Order statistics, which are based on the ranks of observations, are one example of such statistics. These play a central role in many non-parametric approaches.

2. The second meaning of non-parametric covers techniques that do not assume that the structure of a model is fixed. Typically, the model grows in size to accommodate the complexity of the data. In these techniques, individual variables are typically assumed to belong to parametric distributions. Assumptions are also made about the types of connections among variables.

Non-parametric methods are widely used for studying populations that take on a ranked order (such as movie reviews receiving one to four stars). The use of non-parametric methods may be necessary when data have a ranking but no clear numerical interpretation, such as assessing preferences. In terms of levels of measurement, non-parametric methods result in "ordinal" data.

Distribution-Free Tests

Distribution-free statistical methods are mathematical procedures for testing statistical hypotheses which, unlike parametric statistics, make no assumptions about the probability distributions of the variables being assessed. The most frequently used tests include the following:

Anderson–Darling test: tests whether a sample is drawn from a given distribution.

Statistical Bootstrap Methods: estimates the accuracy/sampling distribution of a statistic.

Cochran's $Q$: tests whether $k$ treatments in randomized block designs with $0/1$ outcomes have identical effects.

Cohen's kappa: measures inter-rater agreement for categorical items.

Friedman two-way analysis of variance by ranks: tests whether $k$ treatments in randomized block designs have identical effects.

Kaplan–Meier: estimates the survival function from lifetime data, modeling censoring.

Kendall's tau: measures statistical dependence between two variables.

Kendall's W: a measure between $0$ and $1$ of inter-rater agreement.

Kolmogorov–Smirnov test: tests whether a sample is drawn from a given distribution, or whether two samples are drawn from the same distribution.

Kruskal-Wallis one-way analysis of variance by ranks: tests whether more than 2 independent samples are drawn from the same distribution.

Kuiper's test: tests whether a sample is drawn from a given distribution that is sensitive to cyclic variations such as day of the week.

Logrank Test: compares survival distributions of two right-skewed, censored samples.

Mann–Whitney $U$ or Wilcoxon rank sum test: tests whether two samples are drawn from the same distribution, as compared to a given alternative hypothesis.

McNemar's test: tests whether, in $2 \times 2$ contingency tables with a dichotomous trait and matched pairs of subjects, row and column marginal frequencies are equal.

Median test: tests whether two samples are drawn from distributions with equal medians.

Pitman's permutation test: a statistical significance test that yields exact $p$ values by examining all possible rearrangements of labels.

Rank products: differentially detects expressed genes in replicated microarray experiments.

Siegel–Tukey test: tests for differences in scale between two groups.

Sign test: tests whether matched pair samples are drawn from distributions with equal medians.

Spearman's rank correlation coefficient: measures statistical dependence between two variables using a monotonic function.

Squared ranks test: tests equality of variances in two or more samples.

Wald–Wolfowitz runs test: tests whether the elements of a sequence are mutually independent/random.

Wilcoxon signed-rank test: tests whether matched pair samples are drawn from populations with different mean ranks.

Best Cars of 2010

This image shows a graphical representation of a ranked list of the highest rated cars in 2010. Non-parametric statistics is widely used for studying populations that take on a ranked order.