Human-based computation game

Games with a purpose categorized as output agreement games are microtask games where players are matched into pairs and randomly assigned partners attempt to match output with each other given a shared visible input. ESP is an example of an output agreement game.

Given an image, the ESP Game can be used to determine what objects are in the image, but cannot be used to determine the location of the object in the image. Location information is necessary for training and testing computer vision algorithms, so the data collected by the ESP Game is not sufficient. Thus, to deal with this problem, a new type of microtask game known as inversion problem games were introduced by creator of ESP, von Ahn in 2006. Peekaboom extended upon ESP and had players associate labels with a specific region of an image. In inversion problem games, two players are randomly paired together. One is assigned as the describer and the other is the guesser. The describer is given an input, which the guesser must reproduce given hints from the describer. In Peekaboom, for example, the describer slowly reveals small sections of an image until the guesser correctly guesses the label provided to the describer.[5]

In input-agreement games two randomly paired players are each given an input that is hidden from the other player. Player inputs will either match or be different. The goal of these games is for players to tag their input such that the other player can determine whether or not the two inputs match. In 2008, Edith L. M. Law created the input-agreement game called TagATune. In this game, players label sound clips. In TagATune, players describe sound clips and guess if their partner's sound clip is the same as their own given their partner's tags.[6]

Macrotask games, unlike microtask games, contain complex problems that are usually left to experts to solve. In 2008, a macrotask game called Foldit was created by Seth Cooper. The idea was that players would attempt to fold a three-dimensional representation of a protein. This task was a hard problem for computers to automate completely. Locating the biologically relevant native conformation of a protein is a difficult computational challenge given the very large size of the search space. By gamification and implementation of user friendly versions of algorithms, players are able to perform this complex task without much knowledge of biology.[7][8][9]

The Apetopia game helps determining perceived color differences. Players' choices are used to model better color metrics.[10] The Apetopia game, which was launched by University of Berlin, is designed to help scientists understand perceived color differences. This game is intended to provide data on how the shades of color are perceived by people in order to model the best color parameters.

Artigo[11] is a Web platform currently offering six artwork annotation games as well as an artwork search engine in English, French, and German. Three of Artigo's games, the ARTigo game, ARTigo Taboo, and TagATag, are variations[12] of Luis von Ahn's ESP game (later Google Image Labeler). Three other games of the Artigo platform, Karido,[13] Artigo-Quiz, and Combino, have been conceived so as to complement the data collected by the three aforementioned ESP game variations.[14][15] Artigo's search engine relies on an original tensor latent semantic analysis.[15][16]

As of September 2013, Artigo had over 30,000 (pictures of) artworks mostly of Europe and of the "long 19th century", from the Promotheus Image Archive,[17] the Rijksmuseum, Amsterdam, the Netherlands, the Staatliche Kunsthalle Karlsruhe, Karlsruhe, Germany, the University Museum of Contemporary Art, campus of the University of Massachusetts Amherst, USA. From 2008 through 2013, Artigo has collected over 7 million tags (mostly in German), 180,000 players (about a tenth of whom are registered), and in average 150 players per day.[18]

Artigo is a joint research endeavor of art historians and computer scientists aiming at both developing an art work search engine and data analysis in art history.

The first example was the ESP game, an effort in human computation originally conceived by Luis von Ahn of Carnegie Mellon University, which labels images. To make it an entertaining effort for humans, two players attempt to assign the same labels to an image. The game records the results of matches as image labels and the players enjoy the encounter because of the competitive and timed nature of it. To ensure that people do their best to accurately label the images, the game requires two people (chosen at random and unknown to each other), who have only the image in common, to choose the same word as an image label. This discourages vandalism because it would be self-defeating as a strategy. The ESP game is a human-based computation game developed to address the problem of creating difficult metadata. The idea behind the game is to use the computational power of humans to perform a task that computers cannot (originally, image recognition) by packaging the task as a game. Google bought a licence to create its own version of the game (Google Image Labeler) in 2006 in order to return better search results for its online images.[19] The license of the data acquired by Ahn's ESP Game, or the Google version, is not clear. Google's version was shut down on 16 September 2011 as part of the Google Labs closure in September 2011.

PeekaBoom is a web-based game that helps computers locate objects in images by using human gameplay to collect valuable metadata. Humans understand and are able to analyze everyday images with minimal effort (what objects are in the image, their location, as well as background and foreground information), while computers have trouble with these basic visual tasks.[20] Peekaboom has two main components: "Peek" and "Boom". Two random players from the Web participate by taking different roles in the game. When one player is Peek, the other is Boom. Peek starts out with a blank screen, while Boom starts with an image and a word related to it. The goal of the game is for Boom to reveal parts of the image to Peek. In the meantime, Peek can guess associated words with the revealed parts of the image. When Peek guesses words that are closer to the image, Boom can indicate whether Peek's guesses are hot or cold. When Peek correctly, the players gets points and then switch roles.[5]

EteRNA is a game in which players attempt to design RNA sequences that fold into a given configuration. The widely varied solutions from players, often non-biologists, are evaluated to improve computer models predicting RNA folding. Some designs are actually synthesized to evaluate the actual folding dynamics and directly compare with the computer models.

Eyewire is a game for finding the connectome of the retina.[21]

Gameplay in Foldit

Crowdsourcing has been gamified in games like Foldit, a game designed by the University of Washington, in which players compete to manipulate proteins into more efficient structures. A 2010 paper in science journal Nature credited Foldit's 57,000 players with providing useful results that matched or outperformed algorithmically computed solutions.[22]

Foldit, while also a GWAP, has a different type of method for tapping the collective human brain. This game challenges players to use their human intuition of 3-dimensional space to help with protein folding algorithms. Unlike the ESP game, which focuses on the results that humans are able to provide, Foldit is trying to understand how humans approach complicated 3-dimensional objects. By 'watching' how humans play the game, researchers hope to be able to improve their own computer programs. Instead of simply performing tasks that computers cannot do, this GWAP is asking humans to help make current machine algorithms better.

Guess the Correlation requires the user to estimate the correlation coefficient of scatter plot diagrams

Guess the Correlation is a game with a purpose challenging players to guess the true Pearson correlation coefficient in scatter plots. The collected data is used to study what features in scatter plots skew human perception of the true correlation. The game was developed by Omar Wagih at the European Bioinformatics Institute.[23][24]

JeuxDeMots[25] is a game aiming to build a large semantic network. People are asked to associate terms according to some instructions that are provided for a given word. The French version of the produced network contains so far more than 350 million relations between 5 million lexical items (March 2021). The project was developed by academics of the Laboratoire d'Informatique, de Robotique et de Microélectronique de Montpellier/Montpellier 2 University.

Nanocrafter is a game about assembling pieces of DNA into structures with functional properties, such as logic circuits, to solve problems.[26] Like Foldit, it is developed at the University of Washington.

OnToGalaxy is a game in which players help to acquire common sense knowledge about words. Implemented as a space shooter, OnToGalaxy in its design quite different from other human computation games.[27] The game was developed by Markus Krause at the University of Bremen.

Phrase Detectives is an "annotation game" geared towards lovers of literature, grammar and language. It lets users indicate relationships between words and phrases to create a resource that is rich in linguistic information. Players are awarded with points for their contributions and are featured on a leader board.[28] It was developed by academics Jon Chamberlain, Massimo Poesio and Udo Kruschwitz at the University of Essex.

Phylo[29] allows gamers to contribute to the greater good by trying to decode the code for genetic diseases. While playing the game and aligning the colored squares, one is helping the scientific community get a step closer to solving the age-old problem of multiple sequence alignment. The problem of multiple sequence alignment is too big for computers to handle. The goal is to understand how and where the function of an organism is encoded in the DNA. The game explains that "a sequence alignment is a way of arranging the sequences of DNA, RNA or protein to identify regions of similarity".

Play to Cure: Genes in Space is a mobile game that uses the collective force of players to analyse real genetic data to help with cancer research.[30]

Quantum Moves is a dexterity and spatial problem solving game, where players move slippery particles across quantum space. Players' solutions on various levels are used to program and fine tune a real quantum computer at Aarhus University.[31] The game was first developed as a graphical interface for quantum simulation and education in 2012. In 2013 it was released to the public in a user-friendly form, and has been continually updated throughout 2014.

Reverse The Odds is a mobile based game which helps researchers learn about analyzing cancers. By incorporating data analysis into Reverse The Odds, researchers can get thousands of players to help them learn more about different cancers including head and neck, lung, and bladder cancer.[32]

Robot Trainer is a game with a purpose that aims in gathering Commonsense Knowledge. The player takes the role of a teacher. The goal of the game is to train a robot that will travel in deep space and will carry a significant amount of human knowledge so that it can teach other humans in the future, far away from earth. The game has three levels. At each level, the player gets a specific task, like building knowledge rules to answer questions, resolving conflicts and validating other players’ knowledge rules. Players are rewarded for submitting knowledge rules that help the robot answer a question and match the contribution of their fellow teachers.[33]

Sea Hero Quest is an iOS and Android based game that helps advancing the research in the field of dementia.[34]

In the browser-based game Smorball,[35] players are asked to type the words they see as quickly and accurately as possible to help their team to victory in the fictional sport of Smorball. The game presents players with phrases from scanned pages in the Biodiversity Heritage Library. After verification, the words players type are sent to the libraries that store the corresponding pages, allowing those pages to be searched and data mined and ultimately making historic literature more usable for institutions, scholars, educators, and the public. The game was developed by Tiltfactor Lab.

Train Robots is an annotation game similar to Phrase Detectives. Players are shown pairs of before/after images of a robot arm and blocks on a board, and asked to enter commands to instruct the robot to move from the first configuration to the second. The game collects natural language data for training linguistic and robotic processing systems.[36]

The Wikidata Game represents a gamification approach to let users help resolve questions regarding persons, images etc. and thus automatically edit the corresponding data items in Wikidata, the structured knowledge repository supporting Wikipedia and Wikimedia Commons, the other Wikimedia projects, and more.[37][38]

ZombiLingo is a French game where players are asked to find the right head (a word or expression) to gain brains and become a more and more degraded zombie. While playing, they in fact annotate syntactic relations in French corpora.[39][40] It was designed and developed by researchers from LORIA and Université Paris-Sorbonne.[41]

While there are many games with a purpose that deal with visual data, there are few that attempt to label audio data. Annotating audio data can be used to search and index music and audio databases as well as generate training data for machine learning. However, currently manually labeling data is costly. Thus, one way to lessen the cost is to create a game with a purpose with the intention of labeling audio data.[42] TagATune is an audio based online game that has human players tag and label descriptions of sounds and music. TagATune is played by randomly paired partners. The partners are given three minutes to come up with agreed descriptions for as many sounds as possible. In each round, a sound is randomly selected from the database and presented to the partners. The description then becomes a tag that can be used for search when it is agreed upon by enough people. After the first round, the comparison round presents a tune and asks players to compare it to one of two other tunes of the same type.[6]

MajorMiner is an online game in which players listen to 10 seconds of randomly selected sound and then describe the sound with tags. If one of the tags the players choose matches that of another players, each player gains one point. If that was the first time that tag was used for that specific sound, the player gains two points.[43] The goal is to use player input to research automatic music labelling and recommendation based on the audio itself.

A game of the wikiracing type, where players are given two Wikipedia articles (start and target) and are tasked with finding a path from the start article to the target article, exclusively by clicking hyperlinks encountered along the way. The path data collected via the game sheds light on the ways in which people reason about encyclopedic knowledge and how they interact with complex networks.[44]