Lutz–Kelker bias

Mathematically, the Lutz-Kelker Bias originates from the dependence of the number density on the observed parallax that is translated into the conditional probability of parallax measurements. Assuming a Gaussian distribution of the observed parallax about the true parallax due to errors in measurement, we can write the conditional probability distribution function of measuring a parallax of ${\displaystyle p_{o}}$ given that the true parallax is ${\displaystyle p}$ as

${\displaystyle g(p_{o}|p)={\dfrac {1}{\sqrt {2\pi \sigma }}}\exp {{\Big (}{\dfrac {-(p_{o}-p)^{2}}{2\sigma ^{2}}}{\Big )}}}$

since the estimation is of a true parallax based on the measured parallax, the conditional probability of the true parallax being ${\displaystyle p}$, given that the observed parallax is ${\displaystyle p_{o}}$is of interest. In the original treatment of the phenomenon by Lutz & Kelker, this probability, using Bayes theorem, is given as

${\displaystyle g(p|p_{o})={\dfrac {g(p_{o}|p)\ g(p)}{g(p_{o})}}}$

where ${\displaystyle g(p)dp}$ and ${\displaystyle g(p_{o})dp_{o}}$ are the prior probabilities of the true and observed parallaxes respectively.

The probability density of finding a star with apparent magnitude ${\displaystyle m}$ at a distance ${\displaystyle s}$ can be similarly written as

${\displaystyle h(s|m)={\dfrac {h(m|s)\ h(s)}{h(m)}}}$

where ${\displaystyle h(m|s)}$ is the probability density of finding a star with apparent magnitude m with a given distance ${\displaystyle s}$. Here, ${\displaystyle h(m|s)}$ will be dependent on the luminosity function of the star, which depends on its absolute magnitude. ${\displaystyle h(m)}$ is the probability density function of the apparent magnitude independent of distance. The probability of a star being at distance ${\displaystyle s}$ will be proportional to ${\displaystyle s^{2}}$ such that

${\displaystyle h(s)\propto n(s)\ s^{2}}$

Assuming a uniform distribution of stars in space, the number density ${\displaystyle n(s)}$ becomes a constant and we can write

${\displaystyle g(p)\ dp=h(s|m)\ {\Bigg |}{\dfrac {\partial s}{\partial p}}{\Bigg |}_{m}\ dp\propto s^{2}{\Bigg |}{\dfrac {\partial s}{\partial p}}{\Bigg |}_{m}dp}$, where ${\displaystyle s=1/p}$.

Since we deal with the probability distribution of the true parallax based on a fixed observed parallax, the probability density ${\displaystyle g(p_{o})}$ becomes irrelevant and we can conclude that the distribution will have the proportionality[2]

${\displaystyle g(p|p_{o})\propto g(p|p_{o})\ p^{-4}}$ and thus,

${\displaystyle g(p|p_{o})\propto {\dfrac {1}{p^{4}}}\exp {\Big (}{-{\dfrac {(p-p_{o})^{2}}{2\ \sigma ^{2}}}}{\Big )}}$

The conditional probability of the true parallax based on the observed parallax is divergent around zero for the true parallax. Therefore, it is not possible to normalize this probability. Following the original description of the bias,[1] we can define a normalization by including the observed parallax as

${\displaystyle g(p|p_{o})\propto {\Big (}{\dfrac {p_{o}}{p}}{\Big )}^{4}\ \exp {\Big (}{-{\dfrac {(p-p_{o})^{2}}{2\ \sigma ^{2}}}}{\Big )}}$

The inclusion of ${\displaystyle p_{o}}$does not affect proportionality since it is a fixed constant. Moreover, in this defined "normalization", we will get a probability of 1 when the true parallax is equal to the observed parallax, regardless of the errors in measurement. Therefore, we can define a dimensionless parallax ${\displaystyle Z:=p/p_{o}}$ and get the dimensionless distribution of the true parallax as

${\displaystyle G(Z)\propto Z^{-4}\ \exp {\Big (}{-{\dfrac {(Z-1)^{2}}{2\ (\sigma /p_{o})^{2}}}}{\Big )}}$

Here, ${\displaystyle Z=1}$ represents the point where the measurement in parallax is equal to its true value, where the probability distribution should be centered. However, this distribution, due to ${\displaystyle Z^{-4}}$ factor will deviate from the point ${\displaystyle Z=1}$ to smaller values. This presents the systematic Lutz-Kelker Bias. The value of this bias will be based on the value of ${\displaystyle \sigma /p_{o}}$, the marginal uncertainty in parallax measurement.

In the original treatment of the Lutz–Kelker bias as it was first proposed[1] the uncertainty in parallax measurement is considered to be the sole source of bias. As a result of the parallax dependence of stellar distributions, smaller uncertainty in the observed parallax will result in only a slight bias from the true parallax value. Larger uncertainties in contrast would yield higher systematic deviations of the observed parallax from its true value. Large errors in parallax measurement become apparent in luminosity calculations and are therefore easy to detect. Consequently, the original treatment of the phenomenon considered the bias to be effective when the uncertainty in the observed parallax, ${\displaystyle \sigma }$, is close to about 15% of the measured value, ${\displaystyle p_{o}}$.[1] This was a very strong statement indicating that if the uncertainty in the parallax in about 15–20%, the bias is so effective that we lose most of the parallax and distance information. Several subsequent work on the phenomenon refuted this argument and it was shown that the scope is actually very sample based and may be dependent on other sources of bias. Therefore, more recently it is argued that the scope for most stellar samples is not as drastic as first proposed.

Following the original statement, the scope of the effects of the bias, as well as its existence and relative methods of correction have been discussed in many works in recent literature, including subsequent work of Lutz himself.[5][6][7][8] Several subsequent work state that the assumption of uniform stellar distribution may not be applicable depending on the choice of stellar sample. Moreover, the effects of different distributions of stars in space as well as that of measurement errors would yield different forms of bias.[6] This suggests the bias is largely dependent on the specific choice of sample and measurement error distributions, although the term Lutz–Kelker bias is commonly used generically for the phenomenon on all stellar samples. It is also questioned whether other sources of error and bias such as the Malmquist Bias actually counter-effect or even cancel the Lutz–Kelker bias, so that the effects are not as drastic as initially described by Lutz and Kelker.[9] Overall, such differences are discussed to result in effects of the bias to be largely overestimated in the original treatment.

More recently, the effects of the Lutz–Kelker bias became relevant in the context of the high-precision measurements of Gaia mission. The scope of effects of Lutz–Kelker bias on certain samples is discussed in the recent Gaia data releases, including the original assumptions and the possibility of different distributions.[10] It remains important to take bias effects with caution regarding sample selection as stellar distribution is expected to be non-uniform at large distance scales. As a result, it is questioned whether correction methods, including the Lutz-Kelker correction proposed in the original work, are applicable for a given stellar sample, since effects are expected to depend on the stellar distribution. Moreover, following the original description and the dependence of the bias on the measurement errors, the effects are expected to be lower due to the higher precision of current instruments such as Gaia.