Stockfish (chess)

Since 2013, Stockfish has been developed using a distributed testing framework named Fishtest, where volunteers can donate CPU time for testing improvements to the program.[20][21][22]

Changes to game-playing code are accepted or rejected based on results of playing of tens of thousands of games on the framework against an older "reference" version of the program, using sequential probability ratio testing. Tests on the framework are verified using the chi-squared test, and only if the results are statistically significant are they deemed reliable and used to revise the software code.

After the inception of Fishtest, Stockfish experienced an explosive growth of 120 Elo points in just 12 months, propelling it to the top of all major rating lists.[23] In Stockfish 7, Fishtest author Gary Linscott was added to the official list of authors in acknowledgement of his contribution to Stockfish's strength.

As of October 2023, the framework has used a total of more than 11200 years of CPU time to play over 6.6 billion chess games.[24]

Stockfish's NNUE visualized

In June 2020, an efficiently updatable neural network (NNUE) fork introduced by computer shogi programmers called Stockfish NNUE was discussed by developers.[25][26] In July 2020 chess news reported that Stockfish NNUE had "broken new ground in computer chess by incorporating a neural network into the already incredibly powerful Stockfish chess engine."[27] A NNUE merge into Stockfish was then announced and development builds became available.[28][29]

"The NNUE branch maintained by @nodchip has demonstrated strong results and offers great potential, and we will proceed to merge ... This merge will introduce machine learning based coding to the engine, thus enlarging the community of developers, bringing in new skills. We are eager to keep everybody on board, including all developers and users of diverse hardware, aiming to be an inclusive community ...the precise steps needed will become clearer as we proceed, I look forward to working with the community to make this happen!"

— Joost VandeVondele, 25 July 2020[28]

On 2 September 2020, the twelfth version of Stockfish was released, incorporating the aforementioned neural network improvement. According to the blog announcement, this new version "plays significantly stronger than any of its predecessors", typically winning ten times more game pairs than it loses when matched against version eleven.[30][31]

Stockfish is a TCEC multiple-time champion and the current leader in trophy count. Ever since TCEC restarted in 2013, Stockfish has finished first or second in every season except one. In TCEC Season 4 and 5, Stockfish finished runner-up, with Superfinal scores of 23–25 first against Houdini 3 and later against Komodo 1142. Season 5 was notable for the winning Komodo team as they accepted the award posthumously for the program's creator Don Dailey, who succumbed to an illness during the final stage of the event. In his honor, the version of Stockfish that was released shortly after that season was named "Stockfish DD".[32]

On 30 May 2014, Stockfish 170514 (a development version of Stockfish 5 with tablebase support) convincingly won TCEC Season 6, scoring 35.5–28.5 against Komodo 7x in the Superfinal.[33] Stockfish 5 was released the following day.[34] In TCEC Season 7, Stockfish again made the Superfinal, but lost to Komodo with the score of 30.5–33.5.[33] In TCEC Season 8, despite losses on time caused by buggy code, Stockfish nevertheless qualified once more for the Superfinal, but lost the ensuing 100-game match 46.5–53.5 to Komodo.[33] In Season 9, Stockfish defeated Houdini 5 with a score of 54.5 versus 45.5.[33][35]

Stockfish finished third during season 10 of TCEC, the only season since 2013 in which Stockfish had failed to qualify for the superfinal. It did not lose a game but was still eliminated because it was unable to score enough wins against lower-rated engines. After this technical elimination, Stockfish went on a long winning streak, winning seasons 11 (59 vs. 41 against Houdini 6.03),[33][36] 12 (60 vs. 40 against Komodo 12.1.1),[33][37] and 13 (55 vs. 45 against Komodo 2155.00)[33][38] convincingly.[39] In Season 14, Stockfish faced a new challenger in Leela Chess Zero, but managed to eke out a win by one game (50.5–49.5).[33][40] Its winning streak was finally ended in season 15, when Leela qualified again and won 53.5–46.5,[33] but Stockfish promptly won season 16, defeating AllieStein 54.5–45.5, after Leela failed to qualify for the superfinal.[33] In season 17, Stockfish faced Leela again in the superfinal, losing 52.5–47.5. However, Stockfish has won every superfinal since: beating Leela 53.5–46.5 in season 18, 54.5–45.5 in season 19, 53–47 in season 20, and 56–44 in season 21.[33] In Season 22, Komodo Dragon beat out Leela to qualify for the superfinal, but was crushed by Stockfish 59.5-40.5. Stockfish did not lose an opening pair in this match.[41] Leela made the superfinal in Season 23, but was also crushed by Stockfish 58.5-41.5.[42]

Stockfish also took part in the TCEC cup, winning the first edition, but was surprisingly upset by Houdini in the semifinals of the second edition.[33][43] Stockfish recovered to beat Komodo in the third-place playoff.[33] In the third edition, Stockfish made it to the finals, but was defeated by Leela Chess Zero after blundering in a 7-man endgame tablebase draw. It turned this result around in the fourth edition, defeating Leela in the final 4.5–3.5.[33] In TCEC Cup 6, Stockfish finished third after losing to AllieStein in the semifinals, the first time it had failed to make the finals. Since then, Stockfish has consistently made the finals, generally defeating Leela, although Leela won the 11th edition 8.5-7.5, Stockfish managed to win the most recent edition 14.5-13.5

Ever since chess.com hosted its first Chess.com Computer Chess Championship in 2018, Stockfish has been the most successful engine. It dominated the earlier championships, winning six consecutive titles before finishing second in CCC7. Since then, its dominance has come under threat from the neural-network engines Leelenstein and Leela Chess Zero, but it has continued to perform well, reaching at least the superfinal in every edition up to CCC11. CCC12 had for the first time a knockout format, with seeding placing CCC11 finalists Stockfish and Leela in the same half. Leela eliminated Stockfish in the semi-finals. However, a post-tournament match against the loser of the final, Leelenstein, saw Stockfish winning in the same format as the main event. After finishing second again to Leela in CCC13, and an uncharacteristic fourth in CCC14, Stockfish went on a long winning streak, taking first place in every championship since.

Stockfish's strength relative to the best human chess players was most apparent in a handicap match with grandmaster Hikaru Nakamura (2798-rated) in August 2014. In the first two games of the match, Nakamura had the assistance of an older version of Rybka, and in the next two games, he received White with pawn odds but no assistance. Nakamura was the world's fifth-best human chess player at the time of the match, while Stockfish 5 was denied use of its opening book and endgame tablebase. Stockfish won each half of the match 1.5–0.5. Both of Stockfish's wins arose from positions in which Nakamura, as is typical for his playing style, pressed for a win instead of acquiescing to a draw.[159]

In December 2017, Stockfish 8 was used as a benchmark to test Google division DeepMind's AlphaZero, with each engine supported by different hardware. AlphaZero was trained through self-play for a total of nine hours, and reached Stockfish's level after just four.[160][161][162] In 100 games from the normal starting position, AlphaZero won 25 games as White, won 3 as Black, and drew the remaining 72, with 0 losses.[163] AlphaZero also played twelve 100-game matches against Stockfish starting from twelve popular openings for a final score of 290 wins, 886 draws and 24 losses, for a point score of 733:467.[164][note 2]

AlphaZero's victory over Stockfish sparked a flurry of activity in the computer chess community, leading to a new open-source engine aimed at replicating AlphaZero, known as Leela Chess Zero. By January 2019, Leela was able to defeat the version of Stockfish that played AlphaZero (Stockfish 8) in a 100-game match. An updated version of Stockfish narrowly defeated Leela Chess Zero in the superfinal of the 14th TCEC season, 50.5–49.5 (+10 =81 −9),[33] but lost the superfinal of the next season to Leela 53.5–46.5 (+14 =79 -7).[33][166] The two engines remain very close in strength to each other even as they continue to improve: Leela defeated Stockfish in the superfinal of TCEC Season 17, but Stockfish won TCEC Season 18, TCEC Season 19, TCEC Season 20, and TCEC Season 21, each time defeating Leela in the superfinal.