Monoculture (computer science)

This section demonstrates the concern of suboptimal monoculture convergence using automated hiring as a case study. Hiring is the process of ranking a group of candidates and hiring the top-valued. In recent years automated hiring (automatically ranking candidates based on their interaction with an AI powered system) became popular.

As shown by Kleinberg,[5] under some assumptions, suboptimal automated hiring monocultures naturally form, namely, choosing the correlated algorithm is a dominant strategy, thus converging to monoculture that leads suboptimal social welfare.

In this scenario we will consider two firms and a group ${\displaystyle S}$ of ${\displaystyle n}$ candidate with hidden utilities of ${\displaystyle x_{i}}$. For hiring process - each firm will produce a noisy-ranking of the candidates, then each firm (in a random order) hires the first available candidate in their ranking. Each firm can choose to use either an independent human rankers or use a common algorithmic ranking.

The ranking algorithm ${\displaystyle {\mathcal {F}}_{\theta }}$ is modeled as a noisy distribution above permutations of ${\displaystyle S}$ parametrized by an accuracy parameter ${\displaystyle \theta >0}$.

In order for ${\displaystyle {\mathcal {F}}_{\theta }}$ to make sense it should satisfy these conditions:

1. Differentiability: The probability of each permutation ${\displaystyle \pi }$ is continues and differentiable in ${\displaystyle \theta }$
2. Asymptotic optimality: For the true ranking ${\displaystyle \pi ^{*}}$: ${\displaystyle \lim _{\theta \to \infty }Pr[\pi ^{*}]=1}$
3. Monotonicity: The expected utility of the top-ranked candidate gets better as ${\displaystyle \theta }$ increases, even if any subset of ${\displaystyle S}$ is removed.

These conditions state that a firm should always prefer higher values of ${\displaystyle \theta }$, even if it is not first in the selection order.

Both the algorithmic and human ranking methods are of the form of ${\displaystyle {\mathcal {F}}_{\theta }}$ and differ by the accuracy parameters ${\displaystyle \theta _{A},\theta _{H}}$. The algorithmic ranking output is corotated - it always outputs the same permutation. In contrast, a human ranked premutation is drawn from ${\displaystyle {\mathcal {F}}_{\theta _{H}}}$ independently for each of firms.

For ${\displaystyle s_{1},s_{2}\in \{A,H\}}$ strategies of the first and second firm, Social welfare ${\displaystyle W_{s_{1},s_{2}}}$ is defied as the sum of utilities of the hired candidates.

The Braess's like paradox in this framework is suboptimal monocultures converges. That is, using the algorithmic ranking is dominant strategy thus converging toward monoculture yet it yields suboptimal welfare ${\displaystyle W_{A,A}<W_{H,H}}$ (welfare in a world without algorithmic ranking is higher).

The main theorem proved by Kleinberg[5] of this model is that for any ${\displaystyle \theta _{H}}$ and any noisy ranking family ${\displaystyle {\mathcal {F}}_{\theta }}$ that satisfy these conditions:

1. Preference for the first position: For all ${\displaystyle \theta >0}$ if ${\displaystyle \pi ,\sigma \sim {\mathcal {F}}_{\theta }}$ then ${\displaystyle \mathbb {E} [\pi _{1}-\pi _{2}|\pi _{1}\neq \sigma _{1}]>0}$.
2. Preference for weaker competition: For all ${\displaystyle \theta _{1}>\theta _{2},\sigma \sim {\mathcal {F}}_{\theta _{1}}and\ \pi ,\tau \sim {\mathcal {F}}_{\theta _{2}}:\mathbb {E} [\pi _{1}^{(-\sigma _{1})}]<\mathbb {E} [\pi _{1}^{(-\tau _{1})}]}$.

there exists a ${\displaystyle \theta _{A}>\theta _{H}}$ such that both firms prefer using the sherd algorithmic ranking even though the social welfare is higher when both use the human evaluators. In other words - regardless of the accuracy of the human rankers there exists a more accurate algorithm whose introduction leads to suboptimal monoculture convergence.

The implications of this theorem is that under these conditions, firms will chose to use the algorithmic ranking even though that the correlated nature of algorithmic monocultures degrades total social welfare. Even though algorithmic rankings are more accurate.

The first condition on ${\displaystyle {\mathcal {F}}_{\theta }}$ (Preference for the first position) is equivalent to a preference of firms to have independent ranking (in our setting - non algorithmic). This means that a firm should prefers independent ranking methods given all else is equal.

The intuition behind preference for weaker competition is that when a candidate is removed (hired by a different firm), the best remaining candidate is better in expectation when the removed candidate is chosen based on a less accurate ranking. Thus, a firm should always prefer that its competitors would be less accurate.

These conditions are met for ${\displaystyle {\mathcal {F}}_{\theta }}$ that is the Mallows Model distributions and some types of random utility models (Gaussian or Laplacian noise).