Miller–Rabin primality test

The property is the following. For a given odd integer n > 2, let’s write n − 1 as 2^sd where s is a positive integer and d is an odd positive integer. Let’s consider an integer a, called a base, which is coprime to n. Then, n is said to be a strong probable prime to base a if one of these congruence relations holds:

• ${\displaystyle a^{d}\equiv 1{\pmod {n}}}$;
• ${\displaystyle a^{2^{r}d}\equiv -1{\pmod {n}}}$ for some 0 ≤ r < s.

The idea beneath this test is that when n is an odd prime, it passes the test because of two facts:

• by Fermat's little theorem, ${\displaystyle a^{n-1}\equiv 1{\pmod {n}}}$ (this property alone defines the weaker notion of probable prime to base a, on which the Fermat test is based);
• the only square roots of 1 modulo n are 1 and −1.

Hence, by contraposition, if n is not a strong probable prime to base a, then n is definitely composite, and a is called a witness for the compositeness of n.

However, this property is not an exact characterization of prime numbers. If n is composite, it may nonetheless be a strong probable prime to base a, in which case it is called a strong pseudoprime, and a is a strong liar.

Thankfully, no composite number is a strong pseudoprime to all bases at the same time (contrary to the Fermat primality test for which Fermat pseudoprimes to all bases exist: the Carmichael numbers). However no simple way of finding a witness is known. A naïve solution is to try all possible bases, which yields an inefficient deterministic algorithm. The Miller test is a more efficient variant of this (see section Miller test below).

Another solution is to pick a base at random. This yields a fast probabilistic test. When n is composite, most bases are witnesses, so the test will detect n as composite with a reasonably high probability (see section Accuracy below). We can quickly reduce the probability of a false positive to an arbitrarily small rate, by combining the outcome of as many independently chosen bases as necessary to achieve the said rate. This is the Miller–Rabin test. The number of bases to try does not depend on n. There seems to be diminishing returns in trying many bases, because if n is a pseudoprime to some base, then it seems more likely to be a pseudoprime to another base.[4]^: §8

Note that a^d ≡ 1 (mod n) holds trivially for a ≡ 1 (mod n), because the congruence relation is compatible with exponentiation. And a^d = a^20d ≡ −1 (mod n) holds trivially for a ≡ −1 (mod n) since d is odd, for the same reason. That is why random a are usually chosen in the interval 1 < a < n − 1.

For testing arbitrarily large n, choosing bases at random is essential, as we don't know the distribution of witnesses and strong liars among the numbers 2, 3, …, n − 2.[lower-alpha 2]

However, a pre-selected set of a few small bases guarantees the identification of all composites up to a pre-computed maximum. This maximum is generally quite large compared to the bases. This gives very fast deterministic tests for small enough n (see section Testing against small sets of bases below).

Here is a proof that, if n is a prime, then the only square roots of 1 modulo n are 1 and −1.

Here is a proof that, if n is an odd prime, then it is a strong probable prime to base a.

Using repeated squaring, the running time of this algorithm is O(k log³ n), where n is the number tested for primality, and k is the number of rounds performed; thus this is an efficient, polynomial-time algorithm. FFT-based multiplication (Harvey-Hoeven algorithm) can decrease the running time to O(k log² n log log n) = Õ(k log² n).

The error made by the primality test is measured by the probability that a composite number is declared probably prime. The more bases a are tried, the better the accuracy of the test. It can be shown that if n is composite, then at most 1/4 of the bases a are strong liars for n.[2][7] As a consequence, if n is composite then running k iterations of the Miller–Rabin test will declare n probably prime with a probability at most 4^−k.

This is an improvement over the Solovay–Strassen test, whose worst‐case error bound is 2^−k. Moreover, the Miller–Rabin test is strictly stronger than the Solovay–Strassen test in the sense that for every composite n, the set of strong liars for n is a subset of the set of Euler liars for n, and for many n, the subset is proper.

In addition, for large values of n, the probability for a composite number to be declared probably prime is often significantly smaller than 4^−k. For instance, for most numbers n, this probability is bounded by 8^−k; the proportion of numbers n which invalidate this upper bound vanishes as we consider larger values of n.[8] Hence the average case has a much better accuracy than 4^−k, a fact which can be exploited for generating probable primes (see below). However, such improved error bounds should not be relied upon to verify primes whose probability distribution is not controlled, since a cryptographic adversary might send a carefully chosen pseudoprime in order to defeat the primality test.[lower-alpha 3] In such contexts, only the worst‐case error bound of 4^−k can be relied upon.

The above error measure is the probability for a composite number to be declared as a strong probable prime after k rounds of testing; in mathematical words, it is the conditional probability ${\displaystyle \Pr(M\!R_{k}\mid \lnot P)}$ where P is the event that the number being tested is prime, and MR_k is the event that it passes the Miller–Rabin test with k rounds. We are often interested instead in the inverse conditional probability ${\displaystyle \Pr(\lnot P\mid M\!R_{k})}$: the probability that a number which has been declared as a strong probable prime is in fact composite. These two probabilities are related by Bayes' law:

${\displaystyle {\begin{aligned}\Pr(\lnot P\mid M\!R_{k})&={\frac {\Pr(\lnot P\land M\!R_{k})}{\Pr(\lnot P\land M\!R_{k})+\Pr(P\land M\!R_{k})}}\\&={\frac {1}{1+{\frac {\Pr(M\!R_{k}\mid P)}{\Pr(M\!R_{k}\mid \lnot P)}}{\frac {\Pr(P)}{\Pr(\lnot P)}}}}\\&={\frac {1}{1+{\frac {1}{\Pr(M\!R_{k}\mid \lnot P)}}{\frac {\Pr(P)}{1-\Pr(P)}}}}\end{aligned}}}$

In the last equation, we simplified the expression using the fact that all prime numbers are correctly reported as strong probable primes (the test has no false negative). By dropping the left part of the denominator, we derive a simple upper bound:

${\displaystyle \Pr(\lnot P\mid M\!R_{k})<\Pr(M\!R_{k}\mid \lnot P)\left({\tfrac {1}{\Pr(P)}}-1\right)}$

Hence this conditional probability is related not only to the error measure discussed above — which is bounded by 4^−k — but also to the probability distribution of the input number. In the general case, as said earlier, this distribution is controlled by a cryptographic adversary, thus unknown, so we cannot deduce much about ${\displaystyle \Pr(\lnot P\mid M\!R_{k})}$. However, in the case when we use the Miller–Rabin test to generate primes (see below), the distribution is chosen by the generator itself, so we can exploit this result.

The Miller–Rabin algorithm can be made deterministic by trying all possible a below a certain limit. Taking n as the limit would imply O(n) trials, hence the running time would be exponential with respect to the size log n of the input. To improve the running time, the challenge is then to lower the limit as much as possible while keeping the test reliable.

If the tested number n is composite, the strong liars a coprime to n are contained in a proper subgroup of the group (Z/nZ)*, which means that if we test all a from a set which generates (Z/nZ)*, one of them must lie outside the said subgroup, hence must be a witness for the compositeness of n. Assuming the truth of the generalized Riemann hypothesis (GRH), it is known that the group is generated by its elements smaller than O((ln n)²), which was already noted by Miller.[1] The constant involved in the Big O notation was reduced to 2 by Eric Bach.[10] This leads to the following deterministic primality testing algorithm, known as the Miller test:

Input: n > 2, an odd integer to be tested for primality
Output: “composite” if n is composite, “prime” otherwise

let s > 0 and d odd > 0 such that n − 1 = 2sd  # by factoring out powers of 2 from n − 1
for all a in the range [2, min(n − 2, ⌊2(ln n)2⌋)]:
    x ← ad mod n
    repeat s times:
        y ← x2 mod n
        if y = 1 and x ≠ 1 and x ≠ n − 1 then  # nontrivial square root of 1 modulo n
            return “composite”
        x ← y
    if y ≠ 1 then
        return “composite”
return “prime”

The full power of the generalized Riemann hypothesis is not needed to ensure the correctness of the test: as we deal with subgroups of even index, it suffices to assume the validity of GRH for quadratic Dirichlet characters.[7]

The running time of the algorithm is, in the soft-O notation, Õ((log n)⁴) (using FFT‐based multiplication).

The Miller test is not used in practice. For most purposes, proper use of the probabilistic Miller–Rabin test or the Baillie–PSW primality test gives sufficient confidence while running much faster. It is also slower in practice than commonly used proof methods such as APR-CL and ECPP which give results that do not rely on unproven assumptions. For theoretical purposes requiring a deterministic polynomial time algorithm, it was superseded by the AKS primality test, which also does not rely on unproven assumptions.

When the number n to be tested is small, trying all a < 2(ln n)² is not necessary, as much smaller sets of potential witnesses are known to suffice. For example, Pomerance, Selfridge, Wagstaff[4] and Jaeschke[11] have verified that

• if n < 2,047, it is enough to test a = 2;
• if n < 1,373,653, it is enough to test a = 2 and 3;
• if n < 9,080,191, it is enough to test a = 31 and 73;
• if n < 25,326,001, it is enough to test a = 2, 3, and 5;
• if n < 3,215,031,751, it is enough to test a = 2, 3, 5, and 7;
• if n < 4,759,123,141, it is enough to test a = 2, 7, and 61;
• if n < 1,122,004,669,633, it is enough to test a = 2, 13, 23, and 1662803;
• if n < 2,152,302,898,747, it is enough to test a = 2, 3, 5, 7, and 11;
• if n < 3,474,749,660,383, it is enough to test a = 2, 3, 5, 7, 11, and 13;
• if n < 341,550,071,728,321, it is enough to test a = 2, 3, 5, 7, 11, 13, and 17.

Using the work of Feitsma and Galway enumerating all base 2 pseudoprimes in 2010, this was extended (see OEIS: A014233), with the first result later shown using different methods in Jiang and Deng:[12]

• if n < 3,825,123,056,546,413,051, it is enough to test a = 2, 3, 5, 7, 11, 13, 17, 19, and 23.
• if n < 18,446,744,073,709,551,616 = 2⁶⁴, it is enough to test a = 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, and 37.

Sorenson and Webster[13] verify the above and calculate precise results for these larger than 64‐bit results:

• if n < 318,665,857,834,031,151,167,461, it is enough to test a = 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, and 37.
• if n < 3,317,044,064,679,887,385,961,981, it is enough to test a = 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, and 41.

Other criteria of this sort, often more efficient (fewer bases required) than those shown above, exist.[14][15][16][17] They give very fast deterministic primality tests for numbers in the appropriate range, without any assumptions.

There is a small list of potential witnesses for every possible input size (at most b values for b‐bit numbers). However, no finite set of bases is sufficient for all composite numbers. Alford, Granville, and Pomerance have shown that there exist infinitely many composite numbers n whose smallest compositeness witness is at least (ln n)^{1/(3ln ln ln n)}.[18] They also argue heuristically that the smallest number w such that every composite number below n has a compositeness witness less than w should be of order Θ(log n log log n).

Of course the worst-case running time is infinite, since the outer loop may never terminate, but that happens with probability zero. As per the geometric distribution, the expected number of draws is ${\displaystyle {\tfrac {1}{\Pr(M\!R_{k})}}}$ (reusing notations from earlier).

As any prime number passes the test, the probability of being prime gives a coarse lower bound to the probability of passing the test. If we draw odd integers uniformly in the range [2^b−1, 2^b−1], then we get:

${\displaystyle \Pr(M\!R_{k})>\Pr(P)={\frac {\pi \left(2^{b}\right)-\pi \left(2^{b-1}\right)}{2^{b-2}}}}$

where π is the prime-counting function. Using an asymptotic expansion of π (an extension of the prime number theorem), we can approximate this probability when b grows towards infinity. We find:

${\displaystyle \Pr(P)={\tfrac {2}{\ln 2}}b^{-1}+{\mathcal {O}}\left(b^{-3}\right)}$

${\displaystyle {\tfrac {1}{\Pr(P)}}={\tfrac {\ln 2}{2}}b+{\mathcal {O}}\left(b^{-1}\right)}$

Hence we can expect the generator to run no more Miller–Rabin tests than a number proportional to b. Taking into account the worst-case complexity of each Miller–Rabin test (see earlier), the expected running time of the generator with inputs b and k is then bounded by O(k b⁴) (or Õ(k b³) using FFT-based multiplication).

The error measure of this generator is the probability that it outputs a composite number.

Using the relation between conditional probabilities (shown in an earlier section) and the asymptotic behavior of ${\displaystyle \Pr(P)}$ (shown just before), this error measure can be given a coarse upper bound:

${\displaystyle \Pr(\lnot P\mid M\!R_{k})<\Pr(M\!R_{k}\mid \lnot P)\left({\tfrac {1}{\Pr(P)}}-1\right)\leq 4^{-k}\left({\tfrac {\ln 2}{2}}b-1+{\mathcal {O}}\left(b^{-1}\right)\right).}$

Hence, for large enough b, this error measure is less than ${\displaystyle {\tfrac {\ln 2}{2}}4^{-k}b}$. However, much better bounds exist.

Using the fact that the Miller–Rabin test itself often has an error bound much smaller than 4^−k (see earlier), Damgård, Landrock and Pomerance derived several error bounds for the generator, with various classes of parameters b and k.[8] These error bounds allow an implementor to choose a reasonable k for a desired accuracy.

One of these error bounds is 4^−k, which holds for all b ≥ 2 (the authors only showed it for b ≥ 51, while Ronald Burthe Jr. completed the proof with the remaining values 2 ≤ b ≤ 50[20]). Again this simple bound can be improved for large values of b. For instance, another bound derived by the same authors is:

${\displaystyle \left({\frac {1}{7}}b^{\frac {15}{4}}2^{-{\frac {b}{2}}}\right)4^{-k}}$

which holds for all b ≥ 21 and k ≥ b/4. This bound is smaller than 4^−k as soon as b ≥ 32.