Misra–Gries heavy hitters algorithm

A bag is like a set in which the same value may occur multiple times. Assume that a bag is available as an array b[0:n – 1] of n elements. In the abstract description of the algorithm, we treat b and its segments also as bags. Henceforth, a heavy hitter of bag b is a value that occurs more than n ÷ k times in it, for some integer k, k≥2.

A k-reduced bag for bag b is derived from b by repeating the following operation until no longer possible: Delete k distinct elements from b. From its definition, a k-reduced bag contains fewer than k different values. The following theorem is easy to prove:

Theorem 1. Each heavy-hitter of b is an element of a k-reduced bag for b.

The first pass of the heavy-hitters computation constructs a k-reduced bag t. The second pass declares an element of t to be a heavy-hitter if it occurs more than n ÷ k times in b. According to Theorem 1, this procedure determines all and only the heavy-hitters. The second pass is easy to program, so we describe only the first pass.

In order to construct t, scan the values in b in arbitrary order, for specificity the following algorithm scans them in the order of increasing indices. Invariant P of the algorithm is that t is a k-reduced bag for the scanned values and d is the number of distinct values in t. Initially, no value has been scanned, t is the empty bag, and d is zero.

P: 0 ≤ i ≤ n ${\displaystyle \wedge }$
     t is a k-reduced bag for b[0:i – 1] ${\displaystyle \wedge }$
     d is the number of distinct values in t ${\displaystyle \wedge }$ 0 ≤ d < k

Whenever element b[i] is scanned, in order to preserve the invariant: (1) if b[i] is not in t, add it to t and increase d by 1, (2) if b[i] is in t, add it to t but don't modify d, and (3) if d becomes equal to k, reduce t by deleting k distinct values from it and update d appropriately.

algorithm Misra–Gries is
    1 = t, d := { }, 0;
    for i from 0 to n-1 do
        if b[i] ${\displaystyle \notin }$ t then
            t, d:= t ∪ {b[i]}, d+1
        else 
            t, d:= t ∪ {b[i]}
        endif
        if d = k then
            Delete k distinct values from t; update d
        endif
    endfor

A possible implementation of t is as a set of pairs of the form (v_i, c_i) where each v_i is a distinct value in t and c_i is the number of occurrences of v_i in t. Then d is the size of this set. The step "Delete k distinct values from t" amounts to reducing each c_i by 1 and then removing any pair (v_i, c_i) from the set if c_i becomes 0.

Using an AVL tree implementation of t, the algorithm has a running time of O(n log k). In order to assess the space requirement, assume that the elements of b can have m possible values, so the storage of a value v_i needs O(log m) bits. Since each counter c_i may have a value as high as n, its storage needs O(log n) bits. Therefore, for O(k) value-counter pairs, the space requirement is O(k (log n + log m)).

In the field of streaming algorithms, the output of the Misra-Gries algorithm in the first pass may be called a summary, and such summaries are used to solve the frequent elements problem in the data stream model. A streaming algorithm makes a small, bounded number of passes over a list of data items called a stream. It processes the elements using at most logarithmic amount of extra space in the size of the list to produce an answer.

The term Misra–Gries summary appears to have been coined by Graham Cormode.[5][6]