Quantum indeterminacy

An adequate account of quantum indeterminacy requires a theory of measurement. Many theories have been proposed since the beginning of quantum mechanics and quantum measurement continues to be an active research area in both theoretical and experimental physics.[1] Possibly the first systematic attempt at a mathematical theory was developed by John von Neumann. The kinds of measurements he investigated are now called projective measurements. That theory was based in turn on the theory of projection-valued measures for self-adjoint operators which had been recently developed (by von Neumann and independently by Marshall Stone) and the Hilbert space formulation of quantum mechanics (attributed by von Neumann to Paul Dirac).

In this formulation, the state of a physical system corresponds to a vector of length 1 in a Hilbert space H over the complex numbers. An observable is represented by a self-adjoint (i.e. Hermitian) operator A on H. If H is finite dimensional, by the spectral theorem, A has an orthonormal basis of eigenvectors. If the system is in state ψ, then immediately after measurement the system will occupy a state which is an eigenvector e of A and the observed value λ will be the corresponding eigenvalue of the equation A e = λ e. It is immediate from this that measurement in general will be non-deterministic. Quantum mechanics, moreover, gives a recipe for computing a probability distribution Pr on the possible outcomes given the initial system state is ψ. The probability is

$${\displaystyle \operatorname {Pr} (\lambda )=\langle \operatorname {E} (\lambda )\psi \mid \psi \rangle }$$

where E(λ) is the projection onto the space of eigenvectors of A with eigenvalue λ.

Example

Bloch sphere showing eigenvectors for Pauli Spin matrices. The Bloch sphere is a two-dimensional surface the points of which correspond to the state space of a spin 1/2 particle. At the state ψ the values of σ₁ are +1 whereas the values of σ₂ and σ₃ take the values +1, −1 with probability 1/2.

In this example, we consider a single spin 1/2 particle (such as an electron) in which we only consider the spin degree of freedom. The corresponding Hilbert space is the two-dimensional complex Hilbert space C², with each quantum state corresponding to a unit vector in C² (unique up to phase). In this case, the state space can be geometrically represented as the surface of a sphere, as shown in the figure on the right.

The Pauli spin matrices

$${\displaystyle \sigma _{1}={\begin{pmatrix}0&1\\1&0\end{pmatrix}},\quad \sigma _{2}={\begin{pmatrix}0&-i\\i&0\end{pmatrix}},\quad \sigma _{3}={\begin{pmatrix}1&0\\0&-1\end{pmatrix}}}$$

are self-adjoint and correspond to spin-measurements along the 3 coordinate axes.

The Pauli matrices all have the eigenvalues +1, −1.

• For σ₁, these eigenvalues correspond to the eigenvectors
  $${\displaystyle {\frac {1}{\sqrt {2}}}(1,1),{\frac {1}{\sqrt {2}}}(1,-1)}$$

  
• For σ₃, they correspond to the eigenvectors
  $${\displaystyle (1,0),(0,1)}$$

  

Thus in the state

$${\displaystyle \psi ={\frac {1}{\sqrt {2}}}(1,1),}$$

σ₁ has the determinate value +1, while measurement of σ₃ can produce either +1, −1 each with probability 1/2. In fact, there is no state in which measurement of both σ₁ and σ₃ have determinate values.

There are various questions that can be asked about the above indeterminacy assertion.

1. Can the apparent indeterminacy be construed as in fact deterministic, but dependent upon quantities not modeled in the current theory, which would therefore be incomplete? More precisely, are there hidden variables that could account for the statistical indeterminacy in a completely classical way?
2. Can the indeterminacy be understood as a disturbance of the system being measured?

Von Neumann formulated the question 1) and provided an argument why the answer had to be no, if one accepted the formalism he was proposing. However, according to Bell, von Neumann's formal proof did not justify his informal conclusion.[2] A definitive but partial negative answer to 1) has been established by experiment: because Bell's inequalities are violated, any such hidden variable(s) cannot be local (see Bell test experiments).

The answer to 2) depends on how disturbance is understood, particularly since measurement entails disturbance (however note that this is the observer effect, which is distinct from the uncertainty principle). Still, in the most natural interpretation the answer is also no. To see this, consider two sequences of measurements: (A) which measures exclusively σ₁ and (B) which measures only σ₃ of a spin system in the state ψ. The measurement outcomes of (A) are all +1, while the statistical distribution of the measurements (B) is still divided between +1, −1 with equal probability.

Quantum indeterminacy is the assertion that the state of a system does not determine a unique collection of values for all its measurable properties. Indeed, according to the Kochen–Specker theorem, in the quantum mechanical formalism it is impossible that, for a given quantum state, each one of these measurable properties (observables) has a determinate (sharp) value. The values of an observable will be obtained non-deterministically in accordance with a probability distribution which is uniquely determined by the system state. Note that the state is destroyed by measurement, so when we refer to a collection of values, each measured value in this collection must be obtained using a freshly prepared state.

This indeterminacy might be regarded as a kind of essential incompleteness in our description of a physical system. Notice however, that the indeterminacy as stated above only applies to values of measurements not to the quantum state. For example, in the spin 1/2 example discussed above, the system can be prepared in the state ψ by using measurement of σ₁ as a filter which retains only those particles such that σ₁ yields +1. By the von Neumann (so-called) postulates, immediately after the measurement the system is assuredly in the state ψ.

However, Einstein believed that quantum state cannot be a complete description of a physical system and, it is commonly thought, never came to terms with quantum mechanics. In fact, Einstein, Boris Podolsky and Nathan Rosen showed that if quantum mechanics is correct, then the classical view of how the real world works (at least after special relativity) is no longer tenable. This view included the following two ideas:

1. A measurable property of a physical system whose value can be predicted with certainty is actually an element of (local) reality (this was the terminology used by EPR).
2. Effects of local actions have a finite propagation speed.

This failure of the classical view was one of the conclusions of the EPR thought experiment in which two remotely located observers, now commonly referred to as Alice and Bob, perform independent measurements of spin on a pair of electrons, prepared at a source in a special state called a spin singlet state. It was a conclusion of EPR, using the formal apparatus of quantum theory, that once Alice measured spin in the x direction, Bob's measurement in the x direction was determined with certainty, whereas immediately before Alice's measurement Bob's outcome was only statistically determined. From this it follows that either value of spin in the x direction is not an element of reality or that the effect of Alice's measurement has infinite speed of propagation.

We have described indeterminacy for a quantum system which is in a pure state. Mixed states are a more general kind of state obtained by a statistical mixture of pure states. For mixed states the "quantum recipe" for determining the probability distribution of a measurement is determined as follows:

Let A be an observable of a quantum mechanical system. A is given by a densely defined self-adjoint operator on H. The spectral measure of A is a projection-valued measure defined by the condition

${\displaystyle \operatorname {E} _{A}(U)=\int _{U}\lambda \,d\operatorname {E} (\lambda ),}$

for every Borel subset U of R. Given a mixed state S, we introduce the distribution of A under S as follows:

${\displaystyle \operatorname {D} _{A}(U)=\operatorname {Tr} (\operatorname {E} _{A}(U)S).}$

This is a probability measure defined on the Borel subsets of R which is the probability distribution obtained by measuring A in S.

Quantum indeterminacy is often understood as information (or lack of it) whose existence we infer, occurring in individual quantum systems, prior to measurement. Quantum randomness is the statistical manifestation of that indeterminacy, witnessable in results of experiments repeated many times. However, the relationship between quantum indeterminacy and randomness is subtle and can be considered differently.[4]

In classical physics, experiments of chance, such as coin-tossing and dice-throwing, are deterministic, in the sense that, perfect knowledge of the initial conditions would render outcomes perfectly predictable. The ‘randomness’ stems from ignorance of physical information in the initial toss or throw. In diametrical contrast, in the case of quantum physics, the theorems of Kochen and Specker,[5] the inequalities of John Bell,[6] and experimental evidence of Alain Aspect,[7][8] all indicate that quantum randomness does not stem from any such physical information.

In 2008, Tomasz Paterek et al. provided an explanation in mathematical information. They proved that quantum randomness is, exclusively, the output of measurement experiments whose input settings introduce logical independence into quantum systems.[9][10]

Logical independence is a well-known phenomenon in Mathematical Logic. It refers to the null logical connectivity that exists between mathematical propositions (in the same language) that neither prove nor disprove one another.[11]

In the work of Paterek et al., the researchers demonstrate a link connecting quantum randomness and logical independence in a formal system of Boolean propositions. In experiments measuring photon polarisation, Paterek et al. demonstrate statistics correlating predictable outcomes with logically dependent mathematical propositions, and random outcomes with propositions that are logically independent.[12][13]

In 2020, Steve Faulkner reported on work following up on the findings of Tomasz Paterek et al; showing what logical independence in the Paterek Boolean propositions means, in the domain of Matrix Mechanics proper. He showed how indeterminacy's indefiniteness arises in evolved density operators representing mixed states, where measurement processes encounter irreversible 'lost history' and ingression of ambiguity.[14]

• A. Aspect, Bell's inequality test: more ideal than ever, Nature 398 189 (1999).
• G. Bergmann, The Logic of Quanta, American Journal of Physics, 1947. Reprinted in Readings in the Philosophy of Science, Ed. H. Feigl and M. Brodbeck, Appleton-Century-Crofts, 1953. Discusses measurement, accuracy and determinism.
• J.S. Bell, On the Einstein–Poldolsky–Rosen paradox, Physics 1 195 (1964).
• A. Einstein, B. Podolsky, and N. Rosen, Can quantum-mechanical description of physical reality be considered complete? Phys. Rev. 47 777 (1935). Archived 2006-02-08 at the Wayback Machine
• G. Mackey, Mathematical Foundations of Quantum Mechanics, W. A. Benjamin, 1963 (paperback reprint by Dover 2004).
• J. von Neumann, Mathematical Foundations of Quantum Mechanics, Princeton University Press, 1955. Reprinted in paperback form. Originally published in German in 1932.
• R. Omnès, Understanding Quantum Mechanics, Princeton University Press, 1999.

• Common Misconceptions Regarding Quantum Mechanics See especially part III "Misconceptions regarding measurement".