Law of the unconscious statistician

Suppose that X is a random variable which takes on only finitely or countably many different values x₁, x₂, ..., with probabilities p₁, p₂, .... Then for any function g of these values, the random variable g(X) has values g(x₁), g(x₂), ..., although some of these may coincide with each other. For example, this is the case if X can take on both values 1 and −1 and g(x) = x².

Let y₁, y₂, ... enumerate the possible distinct values of g(X), and for each i let I_i denote the collection of all j with g(x_j) = y_i. Then according to the definition of expected value of a random variable as a weighted average of possible outputs, there is

${\displaystyle \operatorname {E} [g(X)]=\sum _{i}y_{i}{\text{P}}(g(X)=y_{i}).}$

But this can be rewritten as

${\displaystyle \sum _{i}y_{i}{\text{P}}(g(X)=y_{i})=\sum _{i}y_{i}{\text{P}}(X=x_{j}{\text{ for some }}j\in I_{i})=\sum _{i}y_{i}\sum _{j\in I_{i}}p_{j}=\sum _{j}g(x_{j})p_{j}.}$

This equality relates the average of the outputs of g(X) as weighted by the probabilities of the outputs themselves to the average of the outputs of g(X) as weighted by the probabilities of the outputs of X.

If X takes on only finitely many possible values, the above is fully rigorous. However, if X takes on countably many values, the last equality given does not always hold, as seen by the Riemann series theorem. Because of this, it is necessary to assume the absolute convergence of the sums in question.[5]

Suppose that X is a random variable whose distribution has a continuous density f. If g is a general function, then the probability that g(X) is valued in a set of real numbers K equals the probability that X is valued in g⁻¹(K), which is given by

${\displaystyle \int _{g^{-1}(K)}f(x)\,\mathrm {d} x.}$

Under various conditions on g, the change-of-variables formula for integration can be applied to relate this to an integral over K, and hence to identify the density of g(X) in terms of the density of X. In the simplest case, if g is differentiable with nowhere-vanishing derivative, then the above integral can be written as

${\displaystyle \int _{K}f(g^{-1}(y))(g^{-1})'(y)\,\mathrm {d} y,}$

thereby identifying g(X) as possessing the density f (g⁻¹(y))(g⁻¹)′(y). The expected value of g(X) is then identified as

${\displaystyle \int _{-\infty }^{\infty }yf(g^{-1}(y))(g^{-1})'(y)\,\mathrm {d} y=\int _{-\infty }^{\infty }g(x)f(x)\,\mathrm {d} x,}$

where the equality follows by another use of the change-of-variables formula for integration. This shows that the expected value of g(X) is encoded entirely by the function g and the density f of X.[6]

The assumption that g is differentiable with nonvanishing derivative, which is necessary for applying the usual change-of-variables formula, excludes many typical cases, such as g(x) = x². The result still holds true in these broader settings, although the proof requires more sophisticated results from mathematical analysis such as Sard's theorem and the coarea formula. In even greater generality, using the Lebesgue theory as below, it can be found that the identity

${\displaystyle \operatorname {E} [g(X)]=\int _{-\infty }^{\infty }g(x)f(x)\,\mathrm {d} x}$

holds true whenever X has a density f (which does not have to be continuous) and whenever g is a measurable function for which g(X) has finite expected value. (Every continuous function is measurable.) Furthermore, without modification to the proof, this holds even if X is a random vector (with density) and g is a multivariable function; the integral is then taken over the multi-dimensional range of values of X.

An abstract and general form of the result is available using the framework of measure theory and the Lebesgue integral. Here, the setting is that of a measure space (Ω, μ) and a measurable map X from Ω to a measurable space Ω'. The theorem then says that for any measurable function g on Ω' which is valued in real numbers (or even the extended real number line), there is

${\displaystyle \int _{\Omega }g\circ X\,\mathrm {d} \mu =\int _{\Omega '}g\,\mathrm {d} (X_{\sharp }\mu ),}$

(interpreted as saying, in particular, that either side of the equality exists if the other side exists). Here X_♯ μ denotes the pushforward measure on Ω'. The 'discrete case' given above is the special case arising when X takes on only countably many values and μ is a probability measure. In fact, the discrete case (although without the restriction to probability measures) is the first step in proving the general measure-theoretic formulation, as the general version follows therefrom by an application of the monotone convergence theorem.[7] Without any major changes, the result can also be formulated in the setting of outer measures.[8]

If μ is a σ-finite measure, the theory of the Radon–Nikodym derivative is applicable. In the special case that the measure X_♯ μ is absolutely continuous relative to some background σ-finite measure ν on Ω', there is a real-valued function f_X on Ω' representing the Radon–Nikodym derivative of the two measures, and then

${\displaystyle \int _{\Omega '}g\,\mathrm {d} (X_{\sharp }\mu )=\int _{\Omega '}gf_{X}\,\mathrm {d} \nu .}$

In the further special case that Ω' is the real number line, as in the contexts discussed above, it is natural to take ν to be the Lebesgue measure, and this then recovers the 'continuous case' given above whenever μ is a probability measure. (In this special case, the condition of σ-finiteness is vacuous, since Lebesgue measure and every probability measure are trivially σ-finite.)[9]