Sequential decoding

Given a partially explored tree (represented by a set of nodes which are limit of exploration), we would like to know the best node from which to explore further. The Fano metric (named after Robert Fano) allows one to calculate from which is the best node to explore further. This metric is optimal given no other constraints (e.g. memory).

For a binary symmetric channel (with error probability ${\displaystyle p}$) the Fano metric can be derived via Bayes theorem. We are interested in following the most likely path ${\displaystyle P_{i}}$ given an explored state of the tree ${\displaystyle X}$ and a received sequence ${\displaystyle {\mathbf {r} }}$. Using the language of probability and Bayes theorem we want to choose the maximum over ${\displaystyle i}$ of:

${\displaystyle \Pr(P_{i}|X,{\mathbf {r} })\propto \Pr({\mathbf {r} }|P_{i},X)\Pr(P_{i}|X)}$

We now introduce the following notation:

• ${\displaystyle N}$ to represent the maximum length of transmission in branches
• ${\displaystyle b}$ to represent the number of bits on a branch of the code (the denominator of the code rate, ${\displaystyle R}$).
• ${\displaystyle d_{i}}$ to represent the number of bit errors on path ${\displaystyle P_{i}}$ (the Hamming distance between the branch labels and the received sequence)
• ${\displaystyle n_{i}}$ to be the length of ${\displaystyle P_{i}}$ in branches.

We express the likelihood ${\displaystyle \Pr({\mathbf {r} }|P_{i},X)}$ as ${\displaystyle p^{d_{i}}(1-p)^{n_{i}b-d_{i}}2^{-(N-n_{i})b}}$ (by using the binary symmetric channel likelihood for the first ${\displaystyle n_{i}b}$ bits followed by a uniform prior over the remaining bits).

We express the prior ${\displaystyle \Pr(P_{i}|X)}$ in terms of the number of branch choices one has made, ${\displaystyle n_{i}}$, and the number of branches from each node, ${\displaystyle 2^{Rb}}$.

Therefore:

${\displaystyle {\begin{aligned}\Pr(P_{i}|X,{\mathbf {r} })&\propto p^{d_{i}}(1-p)^{n_{i}b-d_{i}}2^{-(N-n_{i})b}2^{-n_{i}Rb}\\&\propto p^{d_{i}}(1-p)^{n_{i}b-d_{i}}2^{n_{i}b}2^{-n_{i}Rb}\end{aligned}}}$

We can equivalently maximise the log of this probability, i.e.

${\displaystyle {\begin{aligned}&d_{i}\log _{2}p+(n_{i}b-d_{i})\log _{2}(1-p)+n_{i}b-n_{i}Rb\\=&d_{i}(\log _{2}p+1-R)+(n_{i}b-d_{i})(\log _{2}(1-p)+1-R)\end{aligned}}}$

This last expression is the Fano metric. The important point to see is that we have two terms here: one based on the number of wrong bits and one based on the number of right bits. We can therefore update the Fano metric simply by adding ${\displaystyle \log _{2}p+1-R}$ for each non-matching bit and ${\displaystyle \log _{2}(1-p)+1-R}$ for each matching bit.

For sequential decoding to a good choice of decoding algorithm, the number of states explored wants to remain small (otherwise an algorithm which deliberately explores all states, e.g. the Viterbi algorithm, may be more suitable). For a particular noise level there is a maximum coding rate ${\displaystyle R_{0}}$ called the computational cutoff rate where there is a finite backtracking limit. For the binary symmetric channel:

${\displaystyle R_{0}=1-\log _{2}(1+2{\sqrt {p(1-p)}})}$

• John Wozencraft and B. Reiffen, Sequential decoding, ISBN 0-262-23006-2
• Rolf Johannesson and Kamil Sh. Zigangirov, Fundamentals of convolutional coding (chapter 6), ISBN 0-470-27683-5

• "Correction trees" - simulator of correction process using priority queue to choose maximum metric node (called weight)