Power law

One attribute of power laws is their scale invariance. Given a relation ${\displaystyle f(x)=ax^{-k}}$, scaling the argument ${\displaystyle x}$ by a constant factor ${\displaystyle c}$ causes only a proportionate scaling of the function itself. That is,

${\displaystyle f(cx)=a(cx)^{-k}=c^{-k}f(x)\propto f(x),\!}$

where ${\displaystyle \propto }$ denotes direct proportionality. That is, scaling by a constant ${\displaystyle c}$ simply multiplies the original power-law relation by the constant ${\displaystyle c^{-k}}$. Thus, it follows that all power laws with a particular scaling exponent are equivalent up to constant factors, since each is simply a scaled version of the others. This behavior is what produces the linear relationship when logarithms are taken of both ${\displaystyle f(x)}$ and ${\displaystyle x}$, and the straight-line on the log–log plot is often called the signature of a power law. With real data, such straightness is a necessary, but not sufficient, condition for the data following a power-law relation. In fact, there are many ways to generate finite amounts of data that mimic this signature behavior, but, in their asymptotic limit, are not true power laws (e.g., if the generating process of some data follows a Log-normal distribution). Thus, accurately fitting and validating power-law models is an active area of research in statistics; see below.

A power-law ${\displaystyle x^{-k}}$ has a well-defined mean over ${\displaystyle x\in [1,\infty )}$ only if ${\displaystyle k>2}$, and it has a finite variance only if ${\displaystyle k>3}$; most identified power laws in nature have exponents such that the mean is well-defined but the variance is not, implying they are capable of black swan behavior.[2] This can be seen in the following thought experiment:[10] imagine a room with your friends and estimate the average monthly income in the room. Now imagine the world's richest person entering the room, with a monthly income of about 1 billion US$. What happens to the average income in the room? Income is distributed according to a power-law known as the Pareto distribution (for example, the net worth of Americans is distributed according to a power law with an exponent of 2).

On the one hand, this makes it incorrect to apply traditional statistics that are based on variance and standard deviation (such as regression analysis).[11] On the other hand, this also allows for cost-efficient interventions.[10] For example, given that car exhaust is distributed according to a power-law among cars (very few cars contribute to most contamination) it would be sufficient to eliminate those very few cars from the road to reduce total exhaust substantially.[12]

The median does exist, however: for a power law x ^–k, with exponent ${\displaystyle k>1}$, it takes the value 2^{1/(k – 1)}x_min, where x_min is the minimum value for which the power law holds.[2]

The equivalence of power laws with a particular scaling exponent can have a deeper origin in the dynamical processes that generate the power-law relation. In physics, for example, phase transitions in thermodynamic systems are associated with the emergence of power-law distributions of certain quantities, whose exponents are referred to as the critical exponents of the system. Diverse systems with the same critical exponents—that is, which display identical scaling behaviour as they approach criticality—can be shown, via renormalization group theory, to share the same fundamental dynamics. For instance, the behavior of water and CO₂ at their boiling points fall in the same universality class because they have identical critical exponents. In fact, almost all material phase transitions are described by a small set of universality classes. Similar observations have been made, though not as comprehensively, for various self-organized critical systems, where the critical point of the system is an attractor. Formally, this sharing of dynamics is referred to as universality, and systems with precisely the same critical exponents are said to belong to the same universality class.

More than a hundred power-law distributions have been identified in physics (e.g. sandpile avalanches), biology (e.g. species extinction and body mass), and the social sciences (e.g. city sizes and income).[15] Among them are:

• Kepler's third law
• The initial mass function of stars
• The differential energy spectrum of cosmic-ray nuclei
• The M–sigma relation

• The Angstrom exponent in aerosol optics
• The frequency-dependency of acoustic attenuation in complex media
• The Stefan–Boltzmann law
• The input-voltage–output-current curves of field-effect transistors and vacuum tubes approximate a square-law relationship, a factor in "tube sound".
• Square–cube law (ratio of surface area to volume)
• A 3/2-power law can be found in the plate characteristic curves of triodes.
• The inverse-square laws of Newtonian gravity and electrostatics, as evidenced by the gravitational potential and Electrostatic potential, respectively.
• Self-organized criticality with a critical point as an attractor
• Model of van der Waals force
• Force and potential in simple harmonic motion
• Gamma correction relating light intensity with voltage
• Behaviour near second-order phase transitions involving critical exponents
• The safe operating area relating to maximum simultaneous current and voltage in power semiconductors.
• Supercritical state of matter and supercritical fluids, such as supercritical exponents of heat capacity and viscosity.[16]
• The Curie–von Schweidler law in dielectric responses to step DC voltage input.
• The damping force over speed relation in antiseismic dampers calculus
• Folded solvent-exposed surface areas of centered amino acids in protein structure segments[17]

• Stevens's power law of psychophysics (challenged with demonstrations that it may be logarithmic[18][19])
• The power law of forgetting[20]

• Kleiber's law relating animal metabolism to size, and allometric laws in general
• The two-thirds power law, relating speed to curvature in the human motor system.[21]
• The Taylor's law relating mean population size and variance of populations sizes in ecology
• Neuronal avalanches[4]
• The species richness (number of species) in clades of freshwater fishes[22]
• The Harlow Knapp effect, where a subset of the kinases found in the human body compose a majority of published research[23]
• The size of forest patches globally follows a power law [24]
• The species-area relationship relating the number of species found in an area as a function of the size of the area

• The size of rain-shower cells,[25] energy dissipation in cyclones,[26] and the diameters of dust devils on Earth and Mars [27]

• Exponential growth and random observation (or killing)[28]
• Progress through exponential growth and exponential diffusion of innovations[29]
• Highly optimized tolerance
• Proposed form of experience curve effects
• Pink noise
• The law of stream numbers, and the law of stream lengths (Horton's laws describing river systems)[30]
• Populations of cities (Gibrat's law)[31]
• Bibliograms, and frequencies of words in a text (Zipf's law)[32]
• 90–9–1 principle on wikis (also referred to as the 1% rule)[33][34]
• Richardson's Law for the severity of violent conflicts (wars and terrorism)[35][36]
• The relationship between a CPU's cache size and the number of cache misses follows the power law of cache misses.
• The spectral density of the weight matrices of deep neural networks[37]

• Fractals
• Pareto distribution and the Pareto principle also called the "80–20 rule"
• Zipf's law in corpus analysis and population distributions amongst others, where frequency of an item or event is inversely proportional to its frequency rank (i.e. the second most frequent item/event occurs half as often as the most frequent item, the third most frequent item/event occurs one third as often as the most frequent item, and so on).
• Zeta distribution (discrete)
• Yule–Simon distribution (discrete)
• Student's t-distribution (continuous), of which the Cauchy distribution is a special case
• Lotka's law
• The scale-free network model

• Population sizes of cities in a region or urban network, Zipf's law.
• Distribution of artists by the average price of their artworks.[38]
• Distribution of income in a market economy.
• Distribution of degrees in banking networks.[39]

• The mean absolute change of the logarithmic mid-prices[40]
• Number of tick counts over time
• Size of the maximum price move
• Average waiting time of a directional change[41]
• Average waiting time of an overshoot

Some models of the initial mass function use a broken power law; here Kroupa (2001) in red.

A broken power law is a piecewise function, consisting of two or more power laws, combined with a threshold. For example, with two power laws:[42]

${\displaystyle f(x)\propto x^{\alpha _{1}}}$ for ${\displaystyle x<x_{\text{th}},}$

${\displaystyle f(x)\propto x_{\text{th}}^{\alpha _{1}-\alpha _{2}}x^{\alpha _{2}}{\text{ for }}x>x_{\text{th}}}$.

A power law with an exponential cutoff is simply a power law multiplied by an exponential function:[9]

${\displaystyle f(x)\propto x^{-\alpha }e^{-\beta x}.}$

${\displaystyle f(x)\propto x^{\alpha +\beta x}}$[43]

Although more sophisticated and robust methods have been proposed, the most frequently used graphical methods of identifying power-law probability distributions using random samples are Pareto quantile-quantile plots (or Pareto Q–Q plots), mean residual life plots[47][48] and log–log plots. Another, more robust graphical method uses bundles of residual quantile functions.[49] (Please keep in mind that power-law distributions are also called Pareto-type distributions.) It is assumed here that a random sample is obtained from a probability distribution, and that we want to know if the tail of the distribution follows a power law (in other words, we want to know if the distribution has a "Pareto tail"). Here, the random sample is called "the data".

Pareto Q–Q plots compare the quantiles of the log-transformed data to the corresponding quantiles of an exponential distribution with mean 1 (or to the quantiles of a standard Pareto distribution) by plotting the former versus the latter. If the resultant scatterplot suggests that the plotted points " asymptotically converge" to a straight line, then a power-law distribution should be suspected. A limitation of Pareto Q–Q plots is that they behave poorly when the tail index ${\displaystyle \alpha }$ (also called Pareto index) is close to 0, because Pareto Q–Q plots are not designed to identify distributions with slowly varying tails.[49]

On the other hand, in its version for identifying power-law probability distributions, the mean residual life plot consists of first log-transforming the data, and then plotting the average of those log-transformed data that are higher than the i-th order statistic versus the i-th order statistic, for i = 1, ..., n, where n is the size of the random sample. If the resultant scatterplot suggests that the plotted points tend to "stabilize" about a horizontal straight line, then a power-law distribution should be suspected. Since the mean residual life plot is very sensitive to outliers (it is not robust), it usually produces plots that are difficult to interpret; for this reason, such plots are usually called Hill horror plots [50]

A straight line on a log–log plot is necessary but insufficient evidence for power-laws, the slope of the straight line corresponds to the power law exponent.

Log–log plots are an alternative way of graphically examining the tail of a distribution using a random sample. Caution has to be exercised however as a log–log plot is necessary but insufficient evidence for a power law relationship, as many non power-law distributions will appear as straight lines on a log–log plot.[9][51] This method consists of plotting the logarithm of an estimator of the probability that a particular number of the distribution occurs versus the logarithm of that particular number. Usually, this estimator is the proportion of times that the number occurs in the data set. If the points in the plot tend to "converge" to a straight line for large numbers in the x axis, then the researcher concludes that the distribution has a power-law tail. Examples of the application of these types of plot have been published.[52] A disadvantage of these plots is that, in order for them to provide reliable results, they require huge amounts of data. In addition, they are appropriate only for discrete (or grouped) data.

Another graphical method for the identification of power-law probability distributions using random samples has been proposed.[49] This methodology consists of plotting a bundle for the log-transformed sample. Originally proposed as a tool to explore the existence of moments and the moment generation function using random samples, the bundle methodology is based on residual quantile functions (RQFs), also called residual percentile functions,[53][54][55][56][57][58][59] which provide a full characterization of the tail behavior of many well-known probability distributions, including power-law distributions, distributions with other types of heavy tails, and even non-heavy-tailed distributions. Bundle plots do not have the disadvantages of Pareto Q–Q plots, mean residual life plots and log–log plots mentioned above (they are robust to outliers, allow visually identifying power laws with small values of ${\displaystyle \alpha }$, and do not demand the collection of much data). In addition, other types of tail behavior can be identified using bundle plots.

In general, power-law distributions are plotted on doubly logarithmic axes, which emphasizes the upper tail region. The most convenient way to do this is via the (complementary) cumulative distribution (ccdf) that is, the survival function, ${\displaystyle P(x)=\mathrm {Pr} (X>x)}$,

${\displaystyle P(x)=\Pr(X>x)=C\int _{x}^{\infty }p(X)\,\mathrm {d} X={\frac {\alpha -1}{x_{\min }^{-\alpha +1}}}\int _{x}^{\infty }X^{-\alpha }\,\mathrm {d} X=\left({\frac {x}{x_{\min }}}\right)^{1-\alpha }.}$

The cdf is also a power-law function, but with a smaller scaling exponent. For data, an equivalent form of the cdf is the rank-frequency approach, in which we first sort the ${\displaystyle n}$ observed values in ascending order, and plot them against the vector ${\displaystyle \left[1,{\frac {n-1}{n}},{\frac {n-2}{n}},\dots ,{\frac {1}{n}}\right]}$.

Although it can be convenient to log-bin the data, or otherwise smooth the probability density (mass) function directly, these methods introduce an implicit bias in the representation of the data, and thus should be avoided.[9][60] The survival function, on the other hand, is more robust to (but not without) such biases in the data and preserves the linear signature on doubly logarithmic axes. Though a survival function representation is favored over that of the pdf while fitting a power law to the data with the linear least square method, it is not devoid of mathematical inaccuracy. Thus, while estimating exponents of a power law distribution, maximum likelihood estimator is recommended.

There are many ways of estimating the value of the scaling exponent for a power-law tail, however not all of them yield unbiased and consistent answers. Some of the most reliable techniques are often based on the method of maximum likelihood. Alternative methods are often based on making a linear regression on either the log–log probability, the log–log cumulative distribution function, or on log-binned data, but these approaches should be avoided as they can all lead to highly biased estimates of the scaling exponent.[9]

For real-valued, independent and identically distributed data, we fit a power-law distribution of the form

${\displaystyle p(x)={\frac {\alpha -1}{x_{\min }}}\left({\frac {x}{x_{\min }}}\right)^{-\alpha }}$

to the data ${\displaystyle x\geq x_{\min }}$, where the coefficient ${\displaystyle {\frac {\alpha -1}{x_{\min }}}}$ is included to ensure that the distribution is normalized. Given a choice for ${\displaystyle x_{\min }}$, the log likelihood function becomes:

${\displaystyle {\mathcal {L}}(\alpha )=\log \prod _{i=1}^{n}{\frac {\alpha -1}{x_{\min }}}\left({\frac {x_{i}}{x_{\min }}}\right)^{-\alpha }}$

The maximum of this likelihood is found by differentiating with respect to parameter ${\displaystyle \alpha }$, setting the result equal to zero. Upon rearrangement, this yields the estimator equation:

${\displaystyle {\hat {\alpha }}=1+n\left[\sum _{i=1}^{n}\ln {\frac {x_{i}}{x_{\min }}}\right]^{-1}}$

where ${\displaystyle \{x_{i}\}}$ are the ${\displaystyle n}$ data points ${\displaystyle x_{i}\geq x_{\min }}$.[2][61] This estimator exhibits a small finite sample-size bias of order ${\displaystyle O(n^{-1})}$, which is small when n > 100. Further, the standard error of the estimate is ${\displaystyle \sigma ={\frac {{\hat {\alpha }}-1}{\sqrt {n}}}+O(n^{-1})}$. This estimator is equivalent to the popular Hill estimator from quantitative finance and extreme value theory.

For a set of n integer-valued data points ${\displaystyle \{x_{i}\}}$, again where each ${\displaystyle x_{i}\geq x_{\min }}$, the maximum likelihood exponent is the solution to the transcendental equation

${\displaystyle {\frac {\zeta '({\hat {\alpha }},x_{\min })}{\zeta ({\hat {\alpha }},x_{\min })}}=-{\frac {1}{n}}\sum _{i=1}^{n}\ln {\frac {x_{i}}{x_{\min }}}}$

where ${\displaystyle \zeta (\alpha ,x_{\mathrm {min} })}$ is the incomplete zeta function. The uncertainty in this estimate follows the same formula as for the continuous equation. However, the two equations for ${\displaystyle {\hat {\alpha }}}$ are not equivalent, and the continuous version should not be applied to discrete data, nor vice versa.

Further, both of these estimators require the choice of ${\displaystyle x_{\min }}$. For functions with a non-trivial ${\displaystyle L(x)}$ function, choosing ${\displaystyle x_{\min }}$ too small produces a significant bias in ${\displaystyle {\hat {\alpha }}}$, while choosing it too large increases the uncertainty in ${\displaystyle {\hat {\alpha }}}$, and reduces the statistical power of our model. In general, the best choice of ${\displaystyle x_{\min }}$ depends strongly on the particular form of the lower tail, represented by ${\displaystyle L(x)}$ above.

More about these methods, and the conditions under which they can be used, can be found in .[9] Further, this comprehensive review article provides usable code (Matlab, Python, R and C++) for estimation and testing routines for power-law distributions.

Another method for the estimation of the power-law exponent, which does not assume independent and identically distributed (iid) data, uses the minimization of the Kolmogorov–Smirnov statistic, ${\displaystyle D}$, between the cumulative distribution functions of the data and the power law:

${\displaystyle {\hat {\alpha }}={\underset {\alpha }{\operatorname {arg\,min} }}\,D_{\alpha }}$

with

${\displaystyle D_{\alpha }=\max _{x}|P_{\mathrm {emp} }(x)-P_{\alpha }(x)|}$

where ${\displaystyle P_{\mathrm {emp} }(x)}$ and ${\displaystyle P_{\alpha }(x)}$ denote the cdfs of the data and the power law with exponent ${\displaystyle \alpha }$, respectively. As this method does not assume iid data, it provides an alternative way to determine the power-law exponent for data sets in which the temporal correlation can not be ignored.[4]

This criterion can be applied for the estimation of power-law exponent in the case of scale free distributions and provides a more convergent estimate than the maximum likelihood method. It has been applied to study probability distributions of fracture apertures. In some contexts the probability distribution is described, not by the cumulative distribution function, by the cumulative frequency of a property X, defined as the number of elements per meter (or area unit, second etc.) for which X > x applies, where x is a variable real number. As an example, the cumulative distribution of the fracture aperture, X, for a sample of N elements is defined as 'the number of fractures per meter having aperture greater than x . Use of cumulative frequency has some advantages, e.g. it allows one to put on the same diagram data gathered from sample lines of different lengths at different scales (e.g. from outcrop and from microscope).