CovidSim

Note that the mentioned model ships with and is marked with a list of warnings and user information, e.g. no support, stochastic nature/kernel, criticality of input parameters etc.[15][16]

American programmer John Carmack said in April 2020 that he worked on the code before it was released to the public, when it was a single 15,000-line C programming language file and "some of the functions looked like they were machine translated from Fortran", but that "it fared a lot better going through the gauntlet of code analysis tools I hit it with than a lot of more modern code".[17][18]

New Scientist reported in March 2020 that one group from the New England Complex Systems Institute reviewing the model suggested that it contained "systematic errors".[19][20] British newspaper The Telegraph reported that some software engineers who reviewed the new code called it "totally unreliable" and a "buggy mess".[21]

In the opinion of University of Oxford computer scientist Michael Wooldridge, the code was "developed without the ceremony and rigor" of professional products, which is not untypical for research software and often intended to be not understood by third parties, or to be reused; and "while the extensive criticism about relaxed software engineering practices is perhaps justified, it was not fundamentally flawed".[22]

An independent review by Codecheck led by Dr Stephen Eglen of the University of Cambridge confirmed that they were able to reproduce the key findings from the response team's report by using the software.[17][23][24] A June 2020 editorial in Nature declared the original CovidSim codebase met the requirements of scientific reproducibility.[25]

Further research exists to identify the following three sources of uncertainty in the simulation:[26][27] parametric uncertainty, model structure uncertainty and scenario uncertainty:[28] The simulation output depends critically on the inputs and can change up to 300% based on 940 parameters, of which 19 are considered most sensitive. Model structure and scenario uncertainty must therefore be understood.[28]

The results obtained by Imperial using the model are consistent with other models that make similar assumptions.[2]

Calibration of the model has been hampered by the lack of testing, especially the poor understanding of the prevalence of asymptomatic infection, however the Imperial College team makes reasonable assumptions. The model's reliance on a simplified picture of social interactions limits its extensibility to counterfactuals. The general nature of conclusions based on such a model can be expected to be similar to those of a simple compartmental model.[2]

• Additional research is based on the model, e.g. for simulation of effect of school closures on mortality.[29]
• Wouter Edeling et al. contributed a FabSim3 plug-in called FabCovidSim,[30][31] which is based on EasyVVUQ, a Python 3 library to facilitate verification, validation and uncertainty quantification (VVUQ) for a wide variety of simulations.[32][33]
• In a recent publication on MedRxiv, which was now accepted by BMJ Open by Laydon et al., the authors utilize the model to "Measure the effects of the Tier system on the COVID-19 pandemic in the UK between the first and second national lockdowns, before the emergence of the B.1.1.7 variant of concern" and conclude that "...interventions at least as stringent as Tier 3 are required to suppress transmission, especially considering more transmissible variants, at least until effective vaccination is widespread or much greater population immunity has amassed."[34]