DeepMind

As opposed to other AIs, such as IBM's Deep Blue or Watson, which were developed for a pre-defined purpose and only function within its scope, DeepMind claims that its system is not pre-programmed: it learns from experience, using only raw pixels as data input. Technically it uses deep learning on a convolutional neural network, with a novel form of Q-learning, a form of model-free reinforcement learning.[35][52] They test the system on video games, notably early arcade games, such as Space Invaders or Breakout.[52][53] Without altering the code, the AI begins to understand how to play the game, and after some time plays, for a few games (most notably Breakout), a more efficient game than any human ever could.[53]

In 2013, DeepMind published research on an AI system that could surpass human abilities in games such as Pong, Breakout and Enduro, while surpassing state of the art performance on Seaquest, Beamrider, and Q*bert.[54][55] This work reportedly led to the company's acquisition by Google.[7] DeepMind's AI had been applied to video games made in the 1970s and 1980s; work was ongoing for more complex 3D games such as Quake, which first appeared in the 1990s.[53]

In 2020, DeepMind published Agent57,[56][57] an AI Agent which surpasses human level performance on all 57 games of the Atari2600 suite.[58]

In 2014, the company published research on computer systems that are able to play Go.[59]

In October 2015, a computer Go program called AlphaGo, developed by DeepMind, beat the European Go champion Fan Hui, a 2 dan (out of 9 dan possible) professional, five to zero.[60] This was the first time an artificial intelligence (AI) defeated a professional Go player.[61] Previously, computers were only known to have played Go at "amateur" level.[60][62] Go is considered much more difficult for computers to win compared to other games like chess, due to the much larger number of possibilities, making it prohibitively difficult for traditional AI methods such as brute-force.[60][62]

In March 2016 it beat Lee Sedol—a 9th dan Go player and one of the highest ranked players in the world—with a score of 4–1 in a five-game match.

In the 2017 Future of Go Summit, AlphaGo won a three-game match with Ke Jie, who at the time continuously held the world No. 1 ranking for two years.[63][64] It used a supervised learning protocol, studying large numbers of games played by humans against each other.[65]

In 2017, an improved version, AlphaGo Zero, defeated AlphaGo 100 games to 0. AlphaGo Zero's strategies were self-taught. AlphaGo Zero was able to beat its predecessor after just three days with less processing power than AlphaGo; in comparison, the original AlphaGo needed months to learn how to play.[66]

Later that year, AlphaZero, a modified version of AlphaGo Zero but for handling any two-player game of perfect information, gained superhuman abilities at chess and shogi. Like AlphaGo Zero, AlphaZero learned solely through self-play.

DeepMind researchers published a new model named MuZero that mastered the domains of Go, chess, shogi, and Atari 2600 games without human data, domain knowledge, or known rules.[67][68]

Researchers applied MuZero to solve the real world challenge of video compression with a set number of bits with respect to Internet traffic on sites such as YouTube, Twitch, and Google Meet. The goal of MuZero is to optimally compress the video so the quality of the video is maintained with a reduction in data. The final result using MuZero was a 6.28% average reduction in bitrate.[69][70]

In October 2022, DeepMind unveiled a new version of AlphaZero, called AlphaTensor, in a paper published in Nature.[71][72] The version discovered a faster way to perform matrix multiplication – one of the most fundamental tasks in computing – using reinforcement learning.[71][72]

AlphaGo technology was developed based on the deep reinforcement learning approach. This makes AlphaGo different from the rest of AI technologies on the market. With that said, AlphaGo's ‘brain’ was introduced to various moves based on the historical tournament data. The number of moves was increased gradually until it eventually processed over 30 million of them. The aim was to have the system mimic the human player and eventually become better. It played against itself and learned not only from its own defeats but wins as well; thus, it learned to improve itself over the time and increased its winning rate as a result.[73]

AlphaGo used two deep neural networks: a policy network to evaluate move probabilities and a value network to assess positions. The policy network trained via supervised learning, and was subsequently refined by policy-gradient reinforcement learning. The value network learned to predict winners of games played by the policy network against itself. After training, these networks employed a lookahead Monte Carlo tree search (MCTS), using the policy network to identify candidate high-probability moves, while the value network (in conjunction with Monte Carlo rollouts using a fast rollout policy) evaluated tree positions.[74]

AlphaGo Zero was trained using reinforcement learning in which the system played millions of games against itself. Its only guide was to increase its win rate. It did so without learning from games played by humans. Its only input features are the black and white stones from the board. It uses a single neural network, rather than separate policy and value networks. Its simplified tree search relies upon this neural network to evaluate positions and sample moves. A new reinforcement learning algorithm incorporates lookahead search inside the training loop.[74] AlphaGo Zero employed around 15 people and millions in computing resources.[75] Ultimately, it needed much less computing power than AlphaGo, running on four specialized AI processors (Google TPUs), instead of AlphaGo's 48.[76]

In 2016, DeepMind turned its artificial intelligence to protein folding, one of the toughest problems in science. In December 2018, DeepMind's AlphaFold won the 13th Critical Assessment of Techniques for Protein Structure Prediction (CASP) by successfully predicting the most accurate structure for 25 out of 43 proteins. “This is a lighthouse project, our first major investment in terms of people and resources into a fundamental, very important, real-world scientific problem,” Hassabis said to The Guardian.[77] In 2020, in the 14th CASP, AlphaFold's predictions achieved an accuracy score regarded as comparable with lab techniques. Dr Andriy Kryshtafovych, one of the panel of scientific adjudicators, described the achievement as "truly remarkable", and said the problem of predicting how proteins fold had been "largely solved".[78][79][80]

In July 2021, the open-source RoseTTAFold and AlphaFold2 were released to allow scientists to run their own versions of the tools. A week later OpenMind announced that AlphaFold had completed its prediction of nearly all human proteins as well as the entire 'proteomes' of 20 other widely studied organisms.[81] The structures were released on the AlphaFold Protein Structure Database. In July 2022, it was announced that the predictions of over 200 million proteins, representing virtually all known proteins, would be released on the AlphaFold database.[14][15]

In 2016, DeepMind introduced WaveNet, a text-to-speech system. It was originally too computationally intensive for use in consumer products, but in late 2017 it became ready for use in consumer applications such as Google Assistant.[82][83] In 2018 Google launched a commercial text-to-speech product, Cloud Text-to-Speech, based on WaveNet.[84][85]

In 2018, DeepMind introduced a more efficient model called WaveRNN co-developed with Google AI.[86][87] In 2020 WaveNetEQ, a packet loss concealment method based on a WaveRNN architecture, was presented.[88] In 2019, Google started to roll WaveRNN with WavenetEQ out to Google Duo users.[89]

In 2016, Hassabis discussed the game StarCraft as a future challenge, since it requires strategic thinking and handling imperfect information.[90]

In January 2019, DeepMind introduced AlphaStar, a program playing the real-time strategy game StarCraft II. AlphaStar used reinforcement learning based on replays from human players, and then played against itself to enhance its skills. At the time of the presentation, AlphaStar had knowledge equivalent to 200 years of playing time. It won 10 consecutive matches against two professional players, although it had the unfair advantage of being able to see the entire field, unlike a human player who has to move the camera manually. A preliminary version in which that advantage was fixed lost a subsequent match.[91]

In July 2019, AlphaStar began playing against random humans on the public 1v1 European multiplayer ladder. Unlike the first iteration of AlphaStar, which played only Protoss v. Protoss, this one played as all of the game's races, and had earlier unfair advantages fixed.[92][93] By October 2019, AlphaStar reached Grandmaster level on the StarCraft II ladder on all three StarCraft races, becoming the first AI to reach the top league of a widely popular esport without any game restrictions.[94]

In 2022, DeepMind unveiled AlphaCode, an AI-powered coding engine that creates computer programs at a rate comparable to that of an average programmer, with the company testing the system against coding challenges created by Codeforces utilized in human competitive programming competitions.[95] AlphaCode earned a rank equivalent to 54% of the median score on Codeforces after being trained on GitHub data and Codeforce problems and solutions. The program was required to come up with a unique solution and stopped from duplicating answers.

Gato is a "generalist agent" that learns multiple tasks simultaneously.

Google has stated that DeepMind algorithms have greatly increased the efficiency of cooling its data centers.[96] In addition, DeepMind (alongside other Alphabet AI researchers) assists Google Play's personalized app recommendations.[84] DeepMind has also collaborated with the Android team at Google for the creation of two new features which were made available to people with devices running Android Pie, the ninth installment of Google's mobile operating system. These features, Adaptive Battery and Adaptive Brightness, use machine learning to conserve energy and make devices running the operating system easier to use. It is the first time DeepMind has used these techniques on such a small scale, with typical machine learning applications requiring orders of magnitude more computing power.[97]

DeepMind researchers have applied machine learning models to the sport of football, often referred to as soccer in North America, to modelling the behaviour of football players, including the goalkeeper, defenders, and strikers during different scenarios such as penalty kicks. The researchers used heat maps and cluster analysis to organize players based on their tendency to behave a certain way during the game when confronted with a decision on how to score or prevent the other team from scoring. The researchers mention that machine learning models could be used to democratize the football industry by automatically selecting interesting video clips of the game that serve as highlights. This can be done by searching videos for certain events, which is possible because video analysis is an established field of machine learning. This is also possible because of extensive sports analytics based on data including annotated passes or shots, sensors that capture data about the players movements many times over the course of a game, and game theory models.[98][99]

Google has unveiled a new archaeology document program named Ithaca after the home island of mythical hero Odysseys. The deep neural network helps researchers restore the empty text of damaged documents, identify the place they originated from, and give them a definite accurate date. The work builds on another text analysis network named Pythia.[100] The model Ithaca achieves 62% accuracy in restoring damaged texts, 71% location accuracy, and has a dating precision of 30 years. The tool has already been used by historians and ancient Greek archaeologists to make new discoveries in ancient Greek history. The team is working on extending the model to other ancient languages, including Demotic, Akkadian, Hebrew, and Mayan.[101]

In April 2016, New Scientist obtained a copy of a data sharing agreement between DeepMind and the Royal Free London NHS Foundation Trust. The latter operates three London hospitals where an estimated 1.6 million patients are treated annually. The agreement shows DeepMind Health had access to admissions, discharge and transfer data, accident and emergency, pathology and radiology, and critical care at these hospitals. This included personal details such as whether patients had been diagnosed with HIV, suffered from depression or had ever undergone an abortion in order to conduct research to seek better outcomes in various health conditions.[113][114]

A complaint was filed to the Information Commissioner's Office (ICO), arguing that the data should be pseudonymised and encrypted.[115] In May 2016, New Scientist published a further article claiming that the project had failed to secure approval from the Confidentiality Advisory Group of the Medicines and Healthcare products Regulatory Agency.[116]

In 2017, the ICO concluded a year-long investigation that focused on how the Royal Free NHS Foundation Trust tested the app, Streams, in late 2015 and 2016.[117] The ICO found that the Royal Free failed to comply with the Data Protection Act when it provided patient details to DeepMind, and found several shortcomings in how the data was handled, including that patients were not adequately informed that their data would be used as part of the test. DeepMind published its thoughts[118] on the investigation in July 2017, saying “we need to do better” and highlighting several activities and initiatives they had initiated for transparency, oversight and engagement. This included developing a patient and public involvement strategy[119] and being transparent in its partnerships.

In May 2017, Sky News published a leaked letter from the National Data Guardian, Dame Fiona Caldicott, revealing that in her "considered opinion" the data-sharing agreement between DeepMind and the Royal Free took place on an "inappropriate legal basis".[120] The Information Commissioner's Office ruled in July 2017 that the Royal Free hospital failed to comply with the Data Protection Act when it handed over personal data of 1.6 million patients to DeepMind.[121]