Bussgang theorem

In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation of a Gaussian signal before and after it has passed through a nonlinear operation are equal up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.[1]

Statement

Let $\left\{X(t)\right\}$ be a zero-mean stationary Gaussian random process and $\left\{Y(t)\right\}=g(X(t))$ where $g(\cdot )$ is a nonlinear amplitude distortion.

If $R_{X}(\tau )$ is the autocorrelation function of $\left\{X(t)\right\}$ , then the cross-correlation function of $\left\{X(t)\right\}$ and $\left\{Y(t)\right\}$ is

R_{XY}(\tau )=CR_{X}(\tau ),

where $C$ is a constant that depends only on $g(\cdot )$ .

It can be further shown that

C={\frac {1}{\sigma ^{3}{\sqrt {2\pi }}}}\int _{-\infty }^{\infty }ug(u)e^{-{\frac {u^{2}}{2\sigma ^{2}}}}\,du.

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of $y_{1}$ and $y_{2}$ depends only on their covariance and is given explicitly by the expression

p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}

where $y_{1}$ and $y_{2}$ are standard Gaussian random variables with correlation $\phi _{y_{1}y_{2}}=\rho$ .

Assume that $r_{2}=Q(y_{2})$ , the correlation between $y_{1}$ and $r_{2}$ is,

\phi _{y_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }y_{1}Q(y_{2})e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}\,dy_{1}dy_{2}

.

Since

\int _{-\infty }^{\infty }y_{1}e^{-{\frac {1}{2(1-\rho ^{2})}}y_{1}^{2}+{\frac {\rho y_{2}}{1-\rho ^{2}}}y_{1}}\,dy_{1}=\rho {\sqrt {2\pi (1-\rho ^{2})}}y_{2}e^{\frac {\rho ^{2}y_{2}^{2}}{2(1-\rho ^{2})}}

,

the correlation $\phi _{y_{1}r_{2}}$ may be simplified as

\phi _{y_{1}r_{2}}={\frac {\rho }{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}

.

The integral above is seen to depend only on the distortion characteristic $Q()$ and is independent of $\rho$ .

Remembering that $\rho =\phi _{y_{1}y_{2}}$ , we observe that for a given distortion characteristic $Q()$ , the ratio ${\frac {\phi _{y_{1}r_{2}}}{\phi _{y_{1}y_{2}}}}$ is $K_{Q}={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{\infty }y_{2}Q(y_{2})e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}$ .

Therefore, the correlation can be rewritten in the form

$\phi _{y_{1}r_{2}}=K_{Q}\phi _{y_{1}y_{2}}$ .

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If $Q(x)={\text{sign}}(x)$ , or called one-bit quantization, then $K_{Q}={\frac {2}{\sqrt {2\pi }}}\int _{0}^{\infty }y_{2}e^{-{\frac {y_{2}^{2}}{2}}}\,dy_{2}={\sqrt {\frac {2}{\pi }}}$ .

[2][3][1][4]

Arcsine law

If the two random variables are both distorted, i.e., $r_{1}=Q(y_{1}),r_{2}=Q(y_{2})$ , the correlation of $r_{1}$ and $r_{2}$ is

$\phi _{r_{1}r_{2}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }Q(y_{1})Q(y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}$ .

When $Q(x)={\text{sign}}(x)$ , the expression becomes,

$\phi _{r_{1}r_{2}}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}-\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]$

where $\alpha ={\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}$ .

Noticing that

$\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}\left[\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}+\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}\right]=1$ ,

and $\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}$ , $\int _{0}^{\infty }\int _{-\infty }^{0}e^{-\alpha }\,dy_{1}dy_{2}=\int _{-\infty }^{0}\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}$ ,

we can simplify the expression of $\phi _{r_{1}r_{2}}$ as

$\phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\infty }\int _{0}^{\infty }e^{-\alpha }\,dy_{1}dy_{2}-1$

Also, it is convenient to introduce the polar coordinate $y_{1}=R\cos \theta ,y_{2}=R\sin \theta$ . It is thus found that

$\phi _{r_{1}r_{2}}={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}-2R^{2}\rho \cos \theta \sin \theta \ }{2(1-\rho ^{2})}}}R\,dRd\theta -1={\frac {4}{2\pi {\sqrt {1-\rho ^{2}}}}}\int _{0}^{\pi /2}\int _{0}^{\infty }e^{-{\frac {R^{2}(1-\rho \sin 2\theta )}{2(1-\rho ^{2})}}}R\,dRd\theta -1$ .

Integration gives

$\phi _{r_{1}r_{2}}={\frac {2{\sqrt {1-\rho ^{2}}}}{\pi }}\int _{0}^{\pi /2}{\frac {d\theta }{1-\rho \sin 2\theta }}-1=-{\frac {2}{\pi }}\arctan \left({\frac {\rho -\tan \theta }{\sqrt {1-\rho ^{2}}}}\right){\Bigg |}_{0}^{\pi /2}-1={\frac {2}{\pi }}\arcsin(\rho )$ ，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966.[2][3] The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.[4][5]

The function $f(x)={\frac {2}{\pi }}\arcsin x$ can be approximated as $f(x)\approx {\frac {2}{\pi }}x$ when $x$ is small.

Price's Theorem

Given two jointly normal random variables $y_{1}$ and $y_{2}$ with joint probability function

${\displaystyle p(y_{1},y_{2})={\frac {1}{2\pi {\sqrt {1-\rho ^{2}}}}}e^{-{\frac {y_{1}^{2}+y_{2}^{2}-2\rho y_{1}y_{2}}{2(1-\rho ^{2})}}}}$ ,

we form the mean

$I(\rho )=E(g(y_{1},y_{2}))=\int _{-\infty }^{+\infty }\int _{-\infty }^{+\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}$

of some function $g(y_{1},y_{2})$ of $(y_{1},y_{2})$ . If $g(y_{1},y_{2})p(y_{1},y_{2})\rightarrow 0$ as $(y_{1},y_{2})\rightarrow 0$ , then

${\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}=E\left({\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\right)$ .

Proof. The joint characteristic function of the random variables $y_{1}$ and $y_{2}$ is by definition the integral

$\Phi (\omega _{1},\omega _{2})=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }p(y_{1},y_{2})e^{j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,dy_{1}dy_{2}=\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}$ .

From the two-dimensional inversion formula of Fourier transform, it follows that

$p(y_{1},y_{2})={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}={\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\exp \left\{-{\frac {\omega _{1}^{2}+\omega _{2}^{2}+2\rho \omega _{1}\omega _{2}}{2}}\right\}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}$ .

Therefore, plugging the expression of $p(y_{1},y_{2})$ into $I(\rho )$ , and differentiating with respect to $\rho$ , we obtain

${\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{n}\Phi (\omega _{1},\omega _{2})}{\partial \rho ^{n}}}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {(-1)^{n}}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\omega _{1}^{n}\omega _{2}^{n}\Phi (\omega _{1},\omega _{2})e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2})\left({\frac {1}{4\pi ^{2}}}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\Phi (\omega _{1},\omega _{2}){\frac {\partial ^{2n}e^{-j(\omega _{1}y_{1}+\omega _{2}y_{2})}}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,d\omega _{1}d\omega _{2}\right)\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\\end{aligned}}$

After repeated integration by parts and using the condition at $\infty$ , we obtain the Price's theorem.

${\begin{aligned}{\frac {\partial ^{n}I(\rho )}{\partial \rho ^{n}}}&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }g(y_{1},y_{2}){\frac {\partial ^{2n}p(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}\,dy_{1}dy_{2}\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}{\frac {\partial ^{2n-2}p(y_{1},y_{2})}{\partial y_{1}^{n-1}\partial y_{2}^{n-1}}}\,dy_{1}dy_{2}\\&=\cdots \\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }{\frac {\partial ^{2n}g(y_{1},y_{2})}{\partial y_{1}^{n}\partial y_{2}^{n}}}p(y_{1},y_{2})\,dy_{1}dy_{2}\end{aligned}}$

[4][5]

Proof of Arcsine law by Price's Theorem

If $g(y_{1},y_{2})={\text{sign}}(y_{1}){\text{sign}}(y_{2})$ , then ${\frac {\partial ^{2}g(y_{1},y_{2})}{\partial y_{1}\partial y_{2}}}=4\delta (y_{1})\delta (y_{2})$ where $\delta ()$ is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

${\frac {\partial E({\text{sign}}(y_{1}){\text{sign}}(y_{2}))}{\partial \rho }}={\frac {\partial I(\rho )}{\partial \rho }}=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }4\delta (y_{1})\delta (y_{2})p(y_{1},y_{2})\,dy_{1}dy_{2}={\frac {2}{\pi {\sqrt {1-\rho ^{2}}}}}$ .

When $\rho =0$ , $I(\rho )=0$ . Thus

$E\left({\text{sign}}(y_{1}){\text{sign}}(y_{2})\right)=I(\rho )={\frac {2}{\pi }}\int _{0}^{\rho }{\frac {1}{\sqrt {1-\rho ^{2}}}}\,d\rho ={\frac {2}{\pi }}\arcsin(\rho )$ ,

which is Van Vleck's well-known result of "Arcsine law".

[2][3]

Application

This theorem implies that a simplified correlator can be designed. Instead of having to multiply two signals, the cross-correlation problem reduces to the gating of one signal with another.

References

J.J. Bussgang,"Cross-correlation function of amplitude-distorted Gaussian signals", Res. Lab. Elec., Mas. Inst. Technol., Cambridge MA, Tech. Rep. 216, March 1952.
Vleck, J. H. Van. "The Spectrum of Clipped Noise". Radio Research Laboratory Report of Harvard University. No. 51.
Vleck, J. H. Van; Middleton, D. (January 1966). "The spectrum of clipped noise". Proceedings of the IEEE. 54 (1): 2–19. doi:10.1109/PROC.1966.4567. ISSN 1558-2256.
Price, R. (June 1958). "A useful theorem for nonlinear devices having Gaussian inputs". IRE Transactions on Information Theory. 4 (2): 69–72. doi:10.1109/TIT.1958.1057444. ISSN 2168-2712.
Papoulis, Athanasios (2002). Probability, Random Variables, and Stochastic Processes. McGraw-Hill. p. 396. ISBN 0-07-366011-6.