Nyquist–Shannon sampling theorem

Sampling is a process of converting a signal (for example, a function of continuous time or space) into a sequence of values (a function of discrete time or space). Shannon's version of the theorem states:[2]

If a function ${\displaystyle x(t)}$ contains no frequencies higher than B hertz, it is completely determined by giving its ordinates at a series of points spaced ${\displaystyle 1/(2B)}$ seconds apart.

A sufficient sample-rate is therefore anything larger than ${\displaystyle 2B}$ samples per second. Equivalently, for a given sample rate ${\displaystyle f_{s}}$, perfect reconstruction is guaranteed possible for a bandlimit ${\displaystyle B<f_{s}/2}$.

When the bandlimit is too high (or there is no bandlimit), the reconstruction exhibits imperfections known as aliasing. Modern statements of the theorem are sometimes careful to explicitly state that ${\displaystyle x(t)}$ must contain no sinusoidal component at exactly frequency ${\displaystyle B,}$ or that ${\displaystyle B}$ must be strictly less than ½ the sample rate. The threshold ${\displaystyle 2B}$ is called the Nyquist rate and is an attribute of the continuous-time input ${\displaystyle x(t)}$ to be sampled. The sample rate must exceed the Nyquist rate for the samples to suffice to represent ${\displaystyle x(t).}$  The threshold ${\displaystyle f_{s}/2}$ is called the Nyquist frequency and is an attribute of the sampling equipment. All meaningful frequency components of the properly sampled ${\displaystyle x(t)}$ exist below the Nyquist frequency. The condition described by these inequalities is called the Nyquist criterion, or sometimes the Raabe condition. The theorem is also applicable to functions of other domains, such as space, in the case of a digitized image. The only change, in the case of other domains, is the units of measure attributed to ${\displaystyle t,}$ ${\displaystyle f_{s},}$ and ${\displaystyle B.}$

The normalized sinc function: sin(πx) / (πx) ... showing the central peak at x = 0, and zero-crossings at the other integer values of x.

The symbol ${\displaystyle T\triangleq 1/f_{s}}$ is customarily used to represent the interval between samples and is called the sample period or sampling interval. The samples of function ${\displaystyle x(t)}$ are commonly denoted by ${\displaystyle x[n]\triangleq x(nT)}$ (alternatively ${\displaystyle x_{n}}$ in older signal processing literature), for all integer values of ${\displaystyle n.}$  Another convenient definition is ${\displaystyle x[n]\triangleq T\cdot x(nT),}$ which preserves the energy of the signal as ${\displaystyle T}$ varies.[3]

A mathematically ideal way to interpolate the sequence involves the use of sinc functions. Each sample in the sequence is replaced by a sinc function, centered on the time axis at the original location of the sample ${\displaystyle nT,}$ with the amplitude of the sinc function scaled to the sample value, ${\displaystyle x[n].}$ Subsequently, the sinc functions are summed into a continuous function. A mathematically equivalent method is to convolve one sinc function with a series of Dirac delta pulses, weighted by the sample values. Neither method is numerically practical. Instead, some type of approximation of the sinc functions, finite in length, is used. The imperfections attributable to the approximation are known as interpolation error.

Practical digital-to-analog converters produce neither scaled and delayed sinc functions, nor ideal Dirac pulses. Instead they produce a piecewise-constant sequence of scaled and delayed rectangular pulses (the zero-order hold), usually followed by a lowpass filter (called an "anti-imaging filter") to remove spurious high-frequency replicas (images) of the original baseband signal.

The samples of two sine waves can be identical when at least one of them is at a frequency above half the sample rate.

When ${\displaystyle x(t)}$ is a function with a Fourier transform ${\displaystyle X(f)}$:

${\displaystyle X(f)\ \triangleq \ \int _{-\infty }^{\infty }x(t)\ e^{-i2\pi ft}\ {\rm {d}}t,}$

the Poisson summation formula indicates that the samples, ${\displaystyle x(nT)}$, of ${\displaystyle x(t)}$ are sufficient to create a periodic summation of ${\displaystyle X(f)}$. The result is:

${\displaystyle X_{s}(f)\ \triangleq \sum _{k=-\infty }^{\infty }X\left(f-kf_{s}\right)=\sum _{n=-\infty }^{\infty }T\cdot x(nT)\ e^{-i2\pi nTf}}$ (Eq.1)

X(f) (top blue) and X_A(f) (bottom blue) are continuous Fourier transforms of two different functions, ${\displaystyle x(t)}$ and ${\displaystyle x_{A}(t)}$ (not shown). When the functions are sampled at rate ${\displaystyle f_{s}}$, the images (green) are added to the original transforms (blue) when one examines the discrete-time Fourier transforms (DTFT) of the sequences. In this hypothetical example, the DTFTs are identical, which means the sampled sequences are identical, even though the original continuous pre-sampled functions are not. If these were audio signals, ${\displaystyle x(t)}$ and ${\displaystyle x_{A}(t)}$ might not sound the same. But their samples (taken at rate f_s) are identical and would lead to identical reproduced sounds; thus x_A(t) is an alias of x(t) at this sample rate.

which is a periodic function and its equivalent representation as a Fourier series, whose coefficients are ${\displaystyle T\cdot x(nT).}$ This function is also known as the discrete-time Fourier transform (DTFT) of the sample sequence.

As depicted, copies of ${\displaystyle X(f)}$ are shifted by multiples of the sampling rate ${\displaystyle f_{s}}$ and combined by addition. For a band-limited function  ${\displaystyle (X(f)=0,{\text{ for all }}|f|\geq B)}$  and sufficiently large ${\displaystyle f_{s},}$ it is possible for the copies to remain distinct from each other. But if the Nyquist criterion is not satisfied, adjacent copies overlap, and it is not possible in general to discern an unambiguous ${\displaystyle X(f).}$ Any frequency component above ${\displaystyle f_{s}/2}$ is indistinguishable from a lower-frequency component, called an alias, associated with one of the copies. In such cases, the customary interpolation techniques produce the alias, rather than the original component. When the sample-rate is pre-determined by other considerations (such as an industry standard), ${\displaystyle x(t)}$ is usually filtered to reduce its high frequencies to acceptable levels before it is sampled. The type of filter required is a lowpass filter, and in this application it is called an anti-aliasing filter.

Spectrum, X_s(f), of a properly sampled bandlimited signal (blue) and the adjacent DTFT images (green) that do not overlap. A brick-wall low-pass filter, H(f), removes the images, leaves the original spectrum, X(f), and recovers the original signal from its samples.

The figure on the left shows a function (in gray/black) being sampled and reconstructed (in gold) at steadily increasing sample-densities, while the figure on the right shows the frequency spectrum of the gray/black function, which does not change. The highest frequency in the spectrum is ½ the width of the entire spectrum. The width of the steadily-increasing pink shading is equal to the sample-rate. When it encompasses the entire frequency spectrum it is twice as large as the highest frequency, and that is when the reconstructed waveform matches the sampled one.

When there is no overlap of the copies (also known as "images") of ${\displaystyle X(f)}$, the ${\displaystyle k=0}$ term of Eq.1 can be recovered by the product:

${\displaystyle X(f)=H(f)\cdot X_{s}(f),}$      where:

${\displaystyle H(f)\ \triangleq \ {\begin{cases}1&|f|<B\\0&|f|>f_{s}-B.\end{cases}}}$

The sampling theorem is proved since ${\displaystyle X(f)}$ uniquely determines ${\displaystyle x(t).}$

All that remains is to derive the formula for reconstruction. ${\displaystyle H(f)}$ need not be precisely defined in the region ${\displaystyle [B,\ f_{s}-B]}$ because ${\displaystyle X_{s}(f)}$ is zero in that region. However, the worst case is when ${\displaystyle B=f_{s}/2,}$ the Nyquist frequency. A function that is sufficient for that and all less severe cases is:

${\displaystyle H(f)=\mathrm {rect} \left({\frac {f}{f_{s}}}\right)={\begin{cases}1&|f|<{\frac {f_{s}}{2}}\\0&|f|>{\frac {f_{s}}{2}},\end{cases}}}$

where rect(•) is the rectangular function.  Therefore:

${\displaystyle X(f)=\mathrm {rect} \left({\frac {f}{f_{s}}}\right)\cdot X_{s}(f)}$

${\displaystyle =\mathrm {rect} (Tf)\cdot \sum _{n=-\infty }^{\infty }T\cdot x(nT)\ e^{-i2\pi nTf}}$      (from  Eq.1, above).

${\displaystyle =\sum _{n=-\infty }^{\infty }x(nT)\cdot \underbrace {T\cdot \mathrm {rect} (Tf)\cdot e^{-i2\pi nTf}} _{{\mathcal {F}}\left\{\mathrm {sinc} \left({\frac {t-nT}{T}}\right)\right\}}.}$     [upper-alpha 1]

The inverse transform of both sides produces the Whittaker–Shannon interpolation formula:

${\displaystyle x(t)=\sum _{n=-\infty }^{\infty }x(nT)\cdot \mathrm {sinc} \left({\frac {t-nT}{T}}\right),}$

which shows how the samples, ${\displaystyle x(nT),}$ can be combined to reconstruct ${\displaystyle x(t).}$

• Larger-than-necessary values of f_s (smaller values of T), called oversampling, have no effect on the outcome of the reconstruction and have the benefit of leaving room for a transition band in which H(f) is free to take intermediate values. Undersampling, which causes aliasing, is not in general a reversible operation.
• Theoretically, the interpolation formula can be implemented as a low-pass filter, whose impulse response is sinc(t/T) and whose input is ${\displaystyle \textstyle \sum _{n=-\infty }^{\infty }x(nT)\cdot \delta (t-nT),}$ which is a Dirac comb function modulated by the signal samples. Practical digital-to-analog converters (DAC) implement an approximation like the zero-order hold. In that case, oversampling can reduce the approximation error.

Poisson shows that the Fourier series in Eq.1 produces the periodic summation of ${\displaystyle X(f)}$, regardless of ${\displaystyle f_{s}}$ and ${\displaystyle B}$. Shannon, however, only derives the series coefficients for the case ${\displaystyle f_{s}=2B}$. Virtually quoting Shannon's original paper:

Let ${\displaystyle X(\omega )}$ be the spectrum of ${\displaystyle x(t).}$  Then

${\displaystyle x(t)={1 \over 2\pi }\int _{-\infty }^{\infty }X(\omega )e^{i\omega t}\;{\rm {d}}\omega ={1 \over 2\pi }\int _{-2\pi B}^{2\pi B}X(\omega )e^{i\omega t}\;{\rm {d}}\omega ,}$

because ${\displaystyle X(\omega )}$ is assumed to be zero outside the band ${\displaystyle \left|{\tfrac {\omega }{2\pi }}\right|<B.}$  If we let ${\displaystyle t={\tfrac {n}{2B}},}$ where ${\displaystyle n}$ is any positive or negative integer, we obtain:

${\displaystyle x\left({\tfrac {n}{2B}}\right)={1 \over 2\pi }\int _{-2\pi B}^{2\pi B}X(\omega )e^{i\omega {n \over {2B}}}\;{\rm {d}}\omega .}$

(Eq.2)

On the left are values of ${\displaystyle x(t)}$ at the sampling points. The integral on the right will be recognized as essentially[lower-alpha 1] the n^th coefficient in a Fourier-series expansion of the function ${\displaystyle X(\omega ),}$ taking the interval ${\displaystyle -B}$ to ${\displaystyle B}$ as a fundamental period. This means that the values of the samples ${\displaystyle x(n/2B)}$ determine the Fourier coefficients in the series expansion of ${\displaystyle X(\omega ).}$  Thus they determine ${\displaystyle X(\omega ),}$ since ${\displaystyle X(\omega )}$ is zero for frequencies greater than B, and for lower frequencies ${\displaystyle X(\omega )}$ is determined if its Fourier coefficients are determined. But ${\displaystyle X(\omega )}$ determines the original function ${\displaystyle x(t)}$ completely, since a function is determined if its spectrum is known. Therefore the original samples determine the function ${\displaystyle x(t)}$ completely.

Shannon's proof of the theorem is complete at that point, but he goes on to discuss reconstruction via sinc functions, what we now call the Whittaker–Shannon interpolation formula as discussed above. He does not derive or prove the properties of the sinc function, but these would have been familiar to engineers reading his works at the time, since the Fourier pair relationship between rect (the rectangular function) and sinc was well known.

Let ${\displaystyle x_{n}}$ be the n^th sample. Then the function ${\displaystyle x(t)}$ is represented by:

${\displaystyle x(t)=\sum _{n=-\infty }^{\infty }x_{n}{\sin(\pi (2Bt-n)) \over \pi (2Bt-n)}.}$

As in the other proof, the existence of the Fourier transform of the original signal is assumed, so the proof does not say whether the sampling theorem extends to bandlimited stationary random processes.

Subsampled image showing a Moiré pattern

Properly sampled image

The sampling theorem is usually formulated for functions of a single variable. Consequently, the theorem is directly applicable to time-dependent signals and is normally formulated in that context. However, the sampling theorem can be extended in a straightforward way to functions of arbitrarily many variables. Grayscale images, for example, are often represented as two-dimensional arrays (or matrices) of real numbers representing the relative intensities of pixels (picture elements) located at the intersections of row and column sample locations. As a result, images require two independent variables, or indices, to specify each pixel uniquely—one for the row, and one for the column.

Color images typically consist of a composite of three separate grayscale images, one to represent each of the three primary colors—red, green, and blue, or RGB for short. Other colorspaces using 3-vectors for colors include HSV, CIELAB, XYZ, etc. Some colorspaces such as cyan, magenta, yellow, and black (CMYK) may represent color by four dimensions. All of these are treated as vector-valued functions over a two-dimensional sampled domain.

Similar to one-dimensional discrete-time signals, images can also suffer from aliasing if the sampling resolution, or pixel density, is inadequate. For example, a digital photograph of a striped shirt with high frequencies (in other words, the distance between the stripes is small), can cause aliasing of the shirt when it is sampled by the camera's image sensor. The aliasing appears as a moiré pattern. The "solution" to higher sampling in the spatial domain for this case would be to move closer to the shirt, use a higher resolution sensor, or to optically blur the image before acquiring it with the sensor using an optical low-pass filter.

Another example is shown to the right in the brick patterns. The top image shows the effects when the sampling theorem's condition is not satisfied. When software rescales an image (the same process that creates the thumbnail shown in the lower image) it, in effect, runs the image through a low-pass filter first and then downsamples the image to result in a smaller image that does not exhibit the moiré pattern. The top image is what happens when the image is downsampled without low-pass filtering: aliasing results.

The sampling theorem applies to camera systems, where the scene and lens constitute an analog spatial signal source, and the image sensor is a spatial sampling device. Each of these components is characterized by a modulation transfer function (MTF), representing the precise resolution (spatial bandwidth) available in that component. Effects of aliasing or blurring can occur when the lens MTF and sensor MTF are mismatched. When the optical image which is sampled by the sensor device contains higher spatial frequencies than the sensor, the under sampling acts as a low-pass filter to reduce or eliminate aliasing. When the area of the sampling spot (the size of the pixel sensor) is not large enough to provide sufficient spatial anti-aliasing, a separate anti-aliasing filter (optical low-pass filter) may be included in a camera system to reduce the MTF of the optical image. Instead of requiring an optical filter, the graphics processing unit of smartphone cameras performs digital signal processing to remove aliasing with a digital filter. Digital filters also apply sharpening to amplify the contrast from the lens at high spatial frequencies, which otherwise falls off rapidly at diffraction limits.

The sampling theorem also applies to post-processing digital images, such as to up or down sampling. Effects of aliasing, blurring, and sharpening may be adjusted with digital filtering implemented in software, which necessarily follows the theoretical principles.

To illustrate the necessity of ${\displaystyle f_{s}>2B}$, consider the family of sinusoids generated by different values of ${\displaystyle \theta }$ in this formula:

${\displaystyle x(t)={\frac {\cos(2\pi Bt+\theta )}{\cos(\theta )}}\ =\ \cos(2\pi Bt)-\sin(2\pi Bt)\tan(\theta ),\quad -\pi /2<\theta <\pi /2.}$

A family of sinusoids at the critical frequency, all having the same sample sequences of alternating +1 and –1. That is, they all are aliases of each other, even though their frequency is not above half the sample rate.

With ${\displaystyle f_{s}=2B}$ or equivalently ${\displaystyle T=1/2B}$, the samples are given by:

${\displaystyle x(nT)=\cos(\pi n)-\underbrace {\sin(\pi n)} _{0}\tan(\theta )=(-1)^{n}}$

regardless of the value of ${\displaystyle \theta }$. That sort of ambiguity is the reason for the strict inequality of the sampling theorem's condition.

As discussed by Shannon:[2]

A similar result is true if the band does not start at zero frequency but at some higher value, and can be proved by a linear translation (corresponding physically to single-sideband modulation) of the zero-frequency case. In this case the elementary pulse is obtained from sin(x)/x by single-side-band modulation.

That is, a sufficient no-loss condition for sampling signals that do not have baseband components exists that involves the width of the non-zero frequency interval as opposed to its highest frequency component. See Sampling (signal processing) for more details and examples.

For example, in order to sample the FM radio signals in the frequency range of 100–102 MHz, it is not necessary to sample at 204 MHz (twice the upper frequency), but rather it is sufficient to sample at 4 MHz (twice the width of the frequency interval).

A bandpass condition is that X(f) = 0, for all nonnegative f outside the open band of frequencies:

$${\displaystyle \left({\frac {N}{2}}f_{\mathrm {s} },{\frac {N+1}{2}}f_{\mathrm {s} }\right),}$$

for some nonnegative integer N. This formulation includes the normal baseband condition as the case N=0.

The corresponding interpolation function is the impulse response of an ideal brick-wall bandpass filter (as opposed to the ideal brick-wall lowpass filter used above) with cutoffs at the upper and lower edges of the specified band, which is the difference between a pair of lowpass impulse responses:

$${\displaystyle (N+1)\,\operatorname {sinc} \left({\frac {(N+1)t}{T}}\right)-N\,\operatorname {sinc} \left({\frac {Nt}{T}}\right).}$$

Other generalizations, for example to signals occupying multiple non-contiguous bands, are possible as well. Even the most generalized form of the sampling theorem does not have a provably true converse. That is, one cannot conclude that information is necessarily lost just because the conditions of the sampling theorem are not satisfied; from an engineering perspective, however, it is generally safe to assume that if the sampling theorem is not satisfied then information will most likely be lost.

The sampling theory of Shannon can be generalized for the case of nonuniform sampling, that is, samples not taken equally spaced in time. The Shannon sampling theory for non-uniform sampling states that a band-limited signal can be perfectly reconstructed from its samples if the average sampling rate satisfies the Nyquist condition.[4] Therefore, although uniformly spaced samples may result in easier reconstruction algorithms, it is not a necessary condition for perfect reconstruction.

The general theory for non-baseband and nonuniform samples was developed in 1967 by Henry Landau.[5] He proved that the average sampling rate (uniform or otherwise) must be twice the occupied bandwidth of the signal, assuming it is a priori known what portion of the spectrum was occupied. In the late 1990s, this work was partially extended to cover signals whose amount of occupied bandwidth was known, but the actual occupied portion of the spectrum was unknown.[6] In the 2000s, a complete theory was developed (see the section Sampling below the Nyquist rate under additional restrictions below) using compressed sensing. In particular, the theory, using signal processing language, is described in this 2009 paper.[7] They show, among other things, that if the frequency locations are unknown, then it is necessary to sample at least at twice the Nyquist criteria; in other words, you must pay at least a factor of 2 for not knowing the location of the spectrum. Note that minimum sampling requirements do not necessarily guarantee stability.

The Nyquist–Shannon sampling theorem provides a sufficient condition for the sampling and reconstruction of a band-limited signal. When reconstruction is done via the Whittaker–Shannon interpolation formula, the Nyquist criterion is also a necessary condition to avoid aliasing, in the sense that if samples are taken at a slower rate than twice the band limit, then there are some signals that will not be correctly reconstructed. However, if further restrictions are imposed on the signal, then the Nyquist criterion may no longer be a necessary condition.

A non-trivial example of exploiting extra assumptions about the signal is given by the recent field of compressed sensing, which allows for full reconstruction with a sub-Nyquist sampling rate. Specifically, this applies to signals that are sparse (or compressible) in some domain. As an example, compressed sensing deals with signals that may have a low over-all bandwidth (say, the effective bandwidth EB), but the frequency locations are unknown, rather than all together in a single band, so that the passband technique does not apply. In other words, the frequency spectrum is sparse. Traditionally, the necessary sampling rate is thus 2B. Using compressed sensing techniques, the signal could be perfectly reconstructed if it is sampled at a rate slightly lower than 2EB. With this approach, reconstruction is no longer given by a formula, but instead by the solution to a linear optimization program.

Another example where sub-Nyquist sampling is optimal arises under the additional constraint that the samples are quantized in an optimal manner, as in a combined system of sampling and optimal lossy compression.[8] This setting is relevant in cases where the joint effect of sampling and quantization is to be considered, and can provide a lower bound for the minimal reconstruction error that can be attained in sampling and quantizing a random signal. For stationary Gaussian random signals, this lower bound is usually attained at a sub-Nyquist sampling rate, indicating that sub-Nyquist sampling is optimal for this signal model under optimal quantization.[9]

The sampling theorem was implied by the work of Harry Nyquist in 1928,[10] in which he showed that up to 2B independent pulse samples could be sent through a system of bandwidth B; but he did not explicitly consider the problem of sampling and reconstruction of continuous signals. About the same time, Karl Küpfmüller showed a similar result[11] and discussed the sinc-function impulse response of a band-limiting filter, via its integral, the step-response sine integral; this bandlimiting and reconstruction filter that is so central to the sampling theorem is sometimes referred to as a Küpfmüller filter (but seldom so in English).

The sampling theorem, essentially a dual of Nyquist's result, was proved by Claude E. Shannon.[2] V. A. Kotelnikov published similar results in 1933,[12] as did the mathematician E. T. Whittaker in 1915,[13] J. M. Whittaker in 1935,[14] and Gabor in 1946 ("Theory of communication"). In 1999, the Eduard Rhein Foundation awarded Kotelnikov their Basic Research Award "for the first theoretically exact formulation of the sampling theorem".

In 1948 and 1949, Claude E. Shannon published – 16 years after Vladimir Kotelnikov – the two revolutionary articles in which he founded the information theory.[15][16][2] In Shannon 1948 the sampling theorem is formulated as “Theorem 13”: Let f(t) contain no frequencies over W. Then

${\displaystyle f(t)=\sum _{n=-\infty }^{\infty }X_{n}{\frac {\sin \pi (2Wt-n)}{\pi (2Wt-n)}},}$ where ${\displaystyle X_{n}=f\left({\frac {n}{2W}}\right)}$.

It was not until these articles were published that the theorem known as “Shannon’s sampling theorem” became common property among communication engineers, although Shannon himself writes that this is a fact which is common knowledge in the communication art.[upper-alpha 2] A few lines further on, however, he adds: "but in spite of its evident importance, [it] seems not to have appeared explicitly in the literature of communication theory".

• 44,100 Hz, a customary rate used to sample audible frequencies is based on the limits of human hearing and the sampling theorem
• Balian–Low theorem, a similar theoretical lower bound on sampling rates, but which applies to time–frequency transforms
• Cheung–Marks theorem, which specifies conditions where restoration of a signal by the sampling theorem can become ill-posed
• Shannon–Hartley theorem
• Nyquist ISI criterion
• Reconstruction from zero crossings
• Zero-order hold

Wikimedia Commons has media related to Nyquist Shannon theorem.

• Learning by Simulations Interactive simulation of the effects of inadequate sampling
• Interactive presentation of the sampling and reconstruction in a web-demo Institute of Telecommunications, University of Stuttgart
• Undersampling and an application of it
• Sampling Theory For Digital Audio
• Journal devoted to Sampling Theory
• Huang, J.; Padmanabhan, K.; Collins, O.M. (June 2011). "The Sampling Theorem With Constant Amplitude Variable Width Pulses". IEEE Transactions on Circuits and Systems I: Regular Papers. 58 (6): 1178–90. doi:10.1109/TCSI.2010.2094350.
• Lüke, Hans Dieter (April 1999). "The Origins of the Sampling Theorem" (PDF). IEEE Communications Magazine. 37 (4): 106–108. CiteSeerX 10.1.1.163.2887. doi:10.1109/35.755459.