Agreed, especially the stuff about end users. Of course you want to prevent bugs that will impact users, but many of the other things being discussed have nothing to do with whether there is a non-technical person using the software.
I dislike this philosophy, that a person in one field of expertise can have nothing constructive to say about another, or can't learn it.
Reminds me of "don't roll your own cryptography." Well, all the standard cryptographic algorithms were rolled by someone, and they had to have thrown out their first pancake too.
Especially when there is such a big overlap between software engineering and scientific computing.
Not all software engineers write to-do apps in angular
"They aren’t worried about it — not because this is a disaster they are trying to cover up, but because this is a routine bug that doesn’t really affect anything important."
If you have a bug you can never know the side effects that it can be causing.
So neither the article author nor the guys who wrote the covid sim understand complex systems.
It would do them much good to read a few works by Taleb.
Their problem is that they don't know what they don't know, they think they know what they don't know, but they don't.
Also saying that it's "ok that scientific code is crappy formated/strutured" clearly demonstrates that these people, neither the author of the article, truly understand intuitively how quickly things can (and always do) go wrong with programs because they are complex beasts.
This reads as an acadmic without truly intuitive knowledge of how things can go badly defending other academics.
If a product is meant to put millions into quarantine it must be damn well tested and not a quick demo playground.
A good product can obviously evolve from a quick academic playground, but academics seem to finish a product fully so rarely that they don't understand that unless they test it severly thourougly that it will have bugs.
Most scientific software developers have been developing and working with enormously complex codebases for years. You're clearly starting from the perspective that they're eggheads who have no experience with this area; that assumption is almost certainly wrong.
"Working with and enormous codebase" and "actualing ensuring it works" are two very different things.
Academic "scientific software developers" (lol) never have to prove anything works in production long term in majority of cases. They make a one-off project and never touch it again. So this generlization stands.
And yes, I am saying that they are eggheads.
Because this article and the covid sim clearly shows that "scientific software developers" (lol) must not be trusted with real world-affecting systems without some other real world developer checking it.
The fact is that the covid sim is garbage and anyone who tries to defend it does not understand programming and complex systems well enough.
Scientists have to undergo extremely rigorous peer review from other experts who are massively incentivized to challenge them and show that their results are wrong. The individual code may not be reviewed, but spurious results will be noticed and called out.
The closest thing we have to this in industry is bug reporting and software security evaluation, and this is sporadic at best. Most commercial software developers keep their code closed source and do everything they can to stifle public knowledge of bugs, even after they've been fixed.
Why are you defending this work Matthew? You have your own reputation to defend as well.
This code has never been peer reviewed, ever. That is indisputable because Ferguson has claimed multiple times that the code is/was entirely undocumented, and nobody outside him and his few colleagues understands how it works. The moment it started being actually reviewed, people found a breathtaking number of bugs like out-of-bounds reads, typos in random number generator constants etc. Broken RNGs! Of all things I'd hope an expert in cryptography would be very worried by, a missing digit in an RNG constant should surely be one of them?
Most scientific software developers have been developing and working with enormously complex codebases for years.
I've spent many years of my career working on highly technical software. I've worked in offices full of professional programmers where many also had postgraduate qualifications in STEM subjects. Those teams did indeed work on quite complex codebases for years.
Do you know what else they typically had? Testing infrastructure that was ahead of its time, careful validation of the mathematics being used before it was implemented, a reasonably clean and systematic software design, and good traceability from the mathematical code back to the underlying mathematics.
This is not rocket science. This is not even unusual if you work in this field. It's just what happens when you let professionals with an appropriate skill set do the job: programmers with technical backgrounds implement models constructed by subject matter experts.
Having one set of skills without the other is not a recipe for success.
Then you and your team should take over the epidemic modeling software development, there's a real need. I assume you can deliver on an academic budget and produce high-quality code in time for it to be useful in this pandemic?
Because if you cannot do those things, you're not really contributing anything of any quality.
The total cost of the pandemic to this country could plausibly be approaching the trillions of pounds scale by the time we're through the worst of it. I think the government could have sprung for a few software developers who knew what they were doing back at the start of the year. Apparently they'd already had plenty of time to establish a relatively large group of privileged expert advisors, including some who worked on these kinds of models. Perhaps if they'd been more open about that process, the gap in their capabilities could have been pointed out soon enough to do something useful about it.
You write as if you think what I'm describing is somehow unusual. It is not. I have probably worked with at least 30-40 well-qualified people just in my own career from whom any well-mixed team of say 3-5 developers could have done what we're talking about here within a relevant time frame. We are not talking about rocket science, just good software development practices being employed by experienced programmers with technical backgrounds who are used to implementing the models of subject matter experts. This isn't some magical unicorn skill set.
This is just a list of excuses.
Yes, there are different standards, for scientific code compared to a website. but any number that is used for a decision, should be up to a standard. Including unit tests, readability and cleanliness. If it’s just a model, and ‘doesn’t need these’ then it should remain in the lab and far from policy decisions.
> While these checks could also be handled by unit tests, most scientists generally just end up with their own weird set of ad hoc test outputs and print statements. It’s ugly, and not infallible, but it tends to work well given the intensive nature of our result-testing behaviour and community cross-checking.
The sense in which this statement is plainly wrong is actually addressed in the original article:
> Regressions like that are common when working on a complex piece of software, which is why industrial software-engineering teams write automated regression tests.
This is the whole point. You can only run your "ad hoc" print-statement-based tests once off, which is why such "tests" are useless for finding regressions.
Alarm bells are ringing from their refusal to release the original code. There is no justification whatsoever for that, and the excuses given are totally unrealistic.
Scientific code is no different than prototypes of commercial products. The goal is to iterate quickly, rather than to make a perfect releasable product. Take a look at the early internal sketches of the iPhone and iPad and you'll see they look massively less elegant than commercial products: but nobody writes breathless articles about how Apple Does Not Known How To Design Consumer Products because of it.
I think the problem here is when buggy “science code” / “prototype code” is used to inform life-and-death decisions made in the real world.
It is therefore rather irresponsible of the scientific community to allow such a bad state of code technical debt to live on for so long, when it’s dealing with a field that is inherently tied to the study of life-or-death scenarios (like we are facing right now).
The only way to avoid this inevitable conclusion of irresponsibility would be if the scientists behind the model warned that it cannot be trusted at all due to the potential of a large number of unknown bugs (did they?). Otherwise, either the scientists are irresponsibility incapable of understanding how to write reliable complex code, or were not honest about the trustworthiness of their code.
We're in the middle of a rapidly-evolving epidemic that is killing thousands of people per day. Leaders need information, even imperfect information right now. There is no time to hire a team of production engineers so they can release a slightly more stable multi-core version 1.0 in 2023.
If there is ever an asteroid heading towards the earth, I swear that some segment of HN will write angry articles about how the hastily built nuclear-powered spaceship we sent to intercept it doesn't have proper unit tests. They'll demand that we delay the launch until 300 days after impact.
When that nuclear powered spaceship blows up because we didn’t test it. It’s pointless. Testing Should be part of the r and d process, increasing as the product matures. To go fast you go slow. Would you let a surgeon operate on you without washing their hands, to save time?
We're in the middle of a rapidly-evolving epidemic that is killing thousands of people per day. Leaders need information, even imperfect information right now. There is no time to hire a team of production engineers so they can release a slightly more stable multi-core version 1.0 in 2023.
We knew this was coming for months. The code isn't that large, it's just horribly written. If the scientists had worked in collaboration with an expert team of technical programmers, they could have handed off their mathematical models and had a well implemented, extensively verified program to run those models within a useful timeframe.
That might sound extreme, but once again, I will reiterate that expert advice apparently based in part on the output from this program is being used to inform decisions that will affect whether thousands live or die and that are severely curtailing the normal lives of the entire population with other adverse consequences that nobody fully understands yet. Powerful responses require powerful justifications.
> If the scientists had worked in collaboration with an expert team of technical programmers
If you look at the original article, you'll note that for at least a month the code has been receiving attention and refactoring from a team at Microsoft. Your criticism is that an academic team should have somehow convinced Microsoft to donate these resources before the epidemic became severe.
I'm going to go way out on the thinnest limb and propose that major software companies were not chomping at the bit to donate resources to epidemic modeling teams until very recently, and this isn't the fault of the epidemic modeling teams -- as much as you seem to think it is.
To clarify, I do not believe any of this is the fault of the academics who were operating on shoestring budgets before the pandemic became the big issue that it now is (edit: unless they wilfully misrepresented their capabilities I suppose). I do think however that the government, with several months of advance warning that this issue could become very important and having demonstrably identified key advisors including those academics, failed to provide more than a shoestring budget to upgrade the capabilities of their expert team in such an obvious way at a much earlier stage.
I'm also curious about why they brought in the people they did to do whatever cleaning up was done before publishing this code. There are several businesses in the UK that do this kind of work as a major part of their activity in one way or another. Current or former staff with that sort of background would have been the obvious people to tap for assistance. To the best of my knowledge, the people who have actually been identified as having helped have no such specialist background.
So, I looked at the article they are refuting (or one of them?) and expected a bunch of superficial criticism as suggested in this refutation, but it does not look to me at all that "superficial" criticism is being defended against: https://lockdownsceptics.org/code-review-of-fergusons-model/
^^^Swallow your politics if you can for a moment and read. At a minimum it seems that the rebuttal here was being argued in very bad faith. Given the stage set, I was expecting bikeshedding about the Law if Demeter or JSON vs other format, etc...
> A software developer will care more about maintainability and end-user experience than a scientific coder, who will likely prize flexibility and control instead. Importantly, this means that programming patterns and norms that work for one may not work for the other — exhortations to keep code simple, to remove cruft, to limit the number of parameters and settings, might actually interfere with the intended applications of a scientific code for example.
Granted, but when publishing, these need to be primary concerns. Source code, then, is a communications medium.
Many famous scientists like Einstein and Feynman would do their work in their heads, not by filling blackboards with equations. When they needed to publish, they'd write down equations because that is the medium of communications. You make it as simple as possible, so people can follow you. E=mc^2 is brilliant because it is so simple. Maxwell's equations got a lot more traction once they were simplified down to just 4.
Scientists using computers for research or analysis are welcome to use any messy code they want, but if they wish for others to follow their work, they need to clean it up to the standards of the industry.
It was beat into my head from a young age that research is only part of being a scientist. You also need to be able to communicate your ideas -- hence all of the writing class requirements. Scientists need to take this to heart, and if they wish to publish in terms of source code, to write source code that others can read.
When even professionals in the field have trouble verifying your work, you've got a problem. It's been a while since I worked in scientific computing, but I never saw a single published paper with a program where the program even matched the description in the paper it was attached to.
Yes, I agree with this. Everyone is welcome to write their own messy code while in the "get it working" phase, but it needs to be cleaned up and made somewhat verifiable before it is shared with others. I currently work with and mentor software development interns, some of whom are aspiring data scientists, and we make sure to give them training on how to write clean code.
There's no quality of communication that can magically circumvent a motivated-reasoning sort of critique. That's why this is an issue with the critique and not the other way round.
As a trivial historical example - an entire world superpower regularly and persistently engaged in and acted on large-scale unscientific motivated reasoning against its own self-interest and the problem with that was not that the best scientifically verifiable ideas available were inadequately communicated.
I read the critique in depth. Let me summarize the claims:
1. The developers had a couple of bugs that cause non-deterministic outputs in weird settings. It's a randomized model so this isn't a disaster, but obviously it would be better to have all non-determinism derived from a single seed. However, none of the bugs point to scientific issues. Still, these are real bugs that should be fixed. And indeed: another team pointed out some issues, they triaged the report, and it appears that they've been fixing these issues.
2. The code doesn't run well multi-threaded, and so the scientists run it on a single core. This seems like a feature the developers have de-prioritized. The critics make a HUGE deal about how using a single core is inefficient, but at the end of the day: if scientists can get simulations done in a reasonable time using a single core, this seems like a valid prioritization of limited development effort. The critics know better, obviously.
3. There aren't a bunch of unit tests. But as the authors point out code is in development and being altered continuously to reflect new developments in society and new knowledge about the virus, so it seems hard to develop unit tests that won't become broken as the underlying components change.
4. The critics seem to have a strong grounding in epidemiology. I'm just kidding, they claim to be pure software engineers, but still insist on spending a significant fraction of their critique disputing the calculation of R0 and lecturing the epidemiologists on how they think it should be calculated. This is a giant red flag given that the critics' sole stated expertise is software engineering.
5. The developers are continuing to add features to simulate new aspects of the ongoing pandemic (e.g., new code to simulate the impact of contact tracing apps) instead of fixing every minor issue in the code. Given that we're in the midst of a deadly epidemic that is putting human beings into the ground, this seems like a completely unavoidable resource allocation decision to me.
6. The critics can't even be bothered to pretend their criticism is objective. E.g., the piece concludes with "on a personal level, I’d go further and suggest that all academic epidemiology be defunded." Yeah, sure, you're a totally disinterested software expert here.
I guarantee you if you look at any piece of software -- commercial or scientific, no matter the budget -- with the politically-motivated goal of tearing apart the author, you will succeed. I can say this having spent years doing security and expert witness code reviews for some of the biggest and smallest companies in the software industry. This piece isn't a critique, it's just a hatchet job. And not even a particularly creative one.
That may all be true, but it doesn't change the fact that the code is written in a style that has proven to be almost totally immune to proper verification of any kind. That being the case, and regardless of the soundness or utility of the underlying science or mathematical modelling, you don't know if it works. You can't. Neither can I. Neither can the people who wrote it, or the people relying on their advice.
Any arguments that the developers of this code are somehow above needing to mechanically test, or formally verify, or independently peer review the code to ensure quality are trivially debunked by the history of bug fixes and changes in behaviour that we do know about, even before making any judgements based on the apparent attempts to hide that history and to conceal the original code from before they brought in programming experts to help tidy it up.
If you have indeed been an expert witness doing code reviews, are you seriously contending that you would not annihilate any case made in a court that hinged on the results of code like this?
As an expert witness, I would be expected to present evidence that the code is wrong in a way that produces scientifically-invalid output. If instead I produced a bunch of nonsense about how the code would be in my opinion more efficient if it ran on multiple cores, that would be strong evidence that I failed in my task and was blowing smoke to cover it up.
You're asking a scientific question. To answer it you'd need to do difficult things like read scientific papers, understand epidemiology well enough to determine whether their models made sense, check their past outputs against real epidemic outcomes (accounting for the fact that national policy changes really matter). In short, you'd have to do exactly what the scientific community does all day, not just glance at some software and make loud claims.
This is much more difficult than reading a Git issue tracker, so obviously the critics have nothing to offer us here.
I don't claim any expertise at all, but without documentation and tests (both of which are called out as largely missing for this codebase), how can the expert epidemiologists tell if the software is behaving correctly?
I presume they run it many times and compare the final and intermediate outputs to reality, then adjust parameters and see how it performs under those parameters. And other models do similar things and compare to you. There are hundreds of papers on Google Scholar about epidemic simulation. This isn't an area where your performance is going to be determined by a few unit tests.
So the argument is that determining the correctness of the simulation is so complex that any kind of automated tests won't matter.
The rebuttal makes a couple of statements about how scientists spend their time to verify their work:
> Scientific software tends to be validated in a more ad hoc way. This is a domain-specific norm, not a matter of competence.
> Practically every scientist I know spends most of their time checking their results rather than coding, and I sincerely doubt that the Imperial group is any different.
I wonder if the team could iterate more quickly if some of these ad hoc validations were automated.
But it's not just a science question. That is the point of the criticism.
Assuming you are the Matthew D Green known for his cryptography work, surely you of all people understand that a poor implementation can have profound consequences even if the original mathematics is sound?
I am, and I've written both production and non-production software. I know that every piece of software has limited development resources, and has different demands on those resources. We're in the midst of a pandemic right now and demands are particularly acute for fresh simulation data, with simulation parameters changing constantly based on new information and policy proposals.
So what do you do with your limited resources? Do you tell the government you can't get them any preliminary data because your multithreading system doesn't work, and you need four months to refactor the code (even though you could just run single-core)? Do you tell them that because you have to write a complete unit testing framework, you can't give them any modeling data or update your model parameters, sorry, good luck with everything.
Do you tell them that another 30K people may have to die based on wrong policy choices because you have decided to prioritize "shipping-production-quality" software standards, even though you are not shipping your software?
You can pretend that this is not a trolley problem. But in this case it literally is. You are standing at the switch in the tracks, and on one side is a detailed set of unit tests and multi-core support, and on the other hand might be tens of thousands of living people. And you know your code better than anyone else in the world. What do you choose?
I don't have an answer. I'm saying the answer is hard, and we're not equipped to lay claims on it based on glancing at a Git repo.
I'm not arguing that there is much you can usefully change now. It's probably far too late to make much material difference to public policy now, for the reasons you have set out yourself. But this could surely have been much better handled during a usefully long window back towards the start of the year when attention was starting to be paid to the virus but it hadn't taken hold in the UK on any significant scale yet.
Also, just to be clear, all of the specific issues with the Imperial code that you've mentioned here and in other parts of our discussion today, such as whether to rewrite for better multi-core behaviour or just use single-core, are coming from you not me. My argument is a much simpler one: when it started to become clear that this virus was a significant risk, the government should have provided the necessary resources and the expert advisors should have switched to a model where scientists did the science and programmers did the programming. I don't think anyone can credibly claim that there wasn't enough awareness that this could potentially become a serious issue and so that bringing in broader expertise was appropriate by the time we had things like COBRA meetings being held and secret expert advisory groups being assembled.
You don't appear to know much about the history of this program, as you keep defending it on the basis it was written quickly - and that the alternative is no modelling.
It started being developed in 2007. It's by now over a decade old, with a lot of work done in 2014. This program wasn't written quickly, it was re-parameterised with a few values for COVID and otherwise thinks it's simulating the flu. Some other pieces of code date back multiple decades.
You also appear to think that "limited resources" is a valid excuse. The UK science budget is over 8 billion pounds. Universities there churn out a large number of papers. The funding is there if they wanted to do less work but of higher quality, yet they don't.
And finally, just to make that clear: yes, saying "We don't have a reliable model for you" is an entirely valid thing to say to a government. You seem to think even low quality simulations are better than none. Humanity has been fighting disease for thousands of years without bug-ridden 15,000 LOC C programs to "help" us. Not having an academic model is a completely reasonable position to be in.
In particular there's a lot of evidence this particular model has already killed a lot of people: it predicted a surge in hospital needs that never happened. In anticipation of this surge a lot of elderly patients that had tested positive for COVID were discharged back into care homes, because the hospitals thought they'd need the beds within days. Those elderly people then infected the others in the care homes, causing the death rate to go up. If there had been no model, there'd have been no panic-induced mass discharge, patients would have been kept in wards until they were fully recovered and the virus wouldn't have been flushed straight into the populations of the most vulnerable. Outcomes could have been much better as a result.
> write a complete unit testing framework
Why? Aren't there already frameworks for unit testing? Didn't they have over a decade to write one?
> Do you tell them that another 30K people may have to die based on wrong policy choices
"How many more people should die before...." - where have we heard this, ooh just a couple thousand times before?
Here's a simple fact, if you want to go this route: every extra 1% of unemployment rate (for US) leads to extra ~40,000 death a year due to a whole stew of reasons.
So, which one is better: have a team of professional developers use best practices and existing tools (like yes - testing framework) and minimize what seems to be a result of amateur programming done over the span of 15 years, or release an unreliable yet sensationalist report and cause tens of millions of people loose their jobs and livelihood?
@matthewdgreen , with all due respect you're avoiding answering to concrete questions and keep defending lousy programming on the ground of urgency and appeal to emotions. This is a mere demagoguery.
Let's put it this way: were there any of your peers - who read all the papers, understood epidemiology and all other things you have just suggested - able to reproduce the results and confirm them as a solid ground for the policy recommendations?
Perhaps, you can also share with us the _original_ code of the model, based on which the "Report 9" has been made? So far, as I know, those calls went answered.
The most worrying thing about your reply is probably the premise that the burden of proof would be on you to prove it materially incorrect, and not on those advocating policy based on it to prove that it had any scientific value in the first place. If obfuscating code to the point where it could be doing just about anything but it's so difficult to understand that no-one can positively prove it's doing something wrong is a viable strategy, the system is broken a priori.
What evidence would you accept? The code has multiple memory safety bugs and a broken random number generator. You don't seem to think that makes the results scientifically invalid output, which is bizarre.
The problem is not the lack of unit tests: The problem is that for outside observers there are no tests at all that compare the model outputs to some ground data.
All the criticism would be moot if the model delivered sensible predictions.
I think what you're saying is that you want to run the model on inputs from, say, February and see if it produces results that are similar to today's caseload.* So why don't you go do that? That isn't in any way a software engineering problem, it's a fundamentally difficult scientific one.
* You'll obviously run into the "testing on your training data" problem, but again this isn't much of a criticism. We're in the midst of an epidemic and scientists are rapidly adjusting a model to reflect developing knowledge. The alternative is to have no models.
I don't have any political goals and don't give a damn about politics, however, being a professional software developer who spend his whole life in open-source, I can't help but wonder if you are for real in pp. 1-3?
> It's a randomized model so this isn't a disaster
What you're saying here is that no matter what distribution properties the "randomness" has it's ok to substitute one with another, right?
> There aren't a bunch of unit tests
It isn't a Microsoft Windows, you know, where a lack of unit tests and uncaught bugs could be potentially fixed in the next release (or not). Report 9 is out, the harm is done.
#4 isn't an argument as we aren't arguing about epidemiological characteristics of the code, but about its validity and correctness.
I am loosing the interest in this discussion very quickly, because it is full of straw-man constructions and intellectual dishonesty, starting with the fact that the original code/model has never been shown to anyone besides of a very small group of hand-picked developers to groom it into some presentable form. Pity...
What is point 6 meant to mean? The author concludes that the code is unfit for purpose, because it was written by academics who ultimately have no incentive to write high quality code even when it's being used to make massive safety-critical decisions.
This is hardly an unexpected conclusion given articles like this one and comments like yours, in which people defend buggy code on the grounds that academics shouldn't even be expected to produce work that isn't buggy. Academics are claiming they structurally can't produce correct, well documented code, not even when it's being used to advise governments.
Hence the conclusion that maybe academics shouldn't be writing that sort of code: how is that not an obvious inference to draw? You seem to believe that any criticism of academia is "politically motivated". It's the same attitude that can be seen in Imperial College's non-answer to the problems: you can't state the private sector has higher standards than academia despite overwhelming evidence it does, because then you're partisan and should be automatically ignored?
You're also an academic, aren't you? Surely that means you're the politically biased one here. How could it be otherwise?
The suggestion was not that academics should not write this kind of code. The suggestions was
"suggest that all academic epidemiology be defunded"
Not do not write bad code but all epidemiology be defunded. So who will do epidimioly studies and develop models of deadly diseases? The software engineers? ;-)
Thanks for summary! The claims give me a really strange feeling that culprit tries to attack in defence and tries to become victimized. In regards with the claims,
1. Or do they? Or may be they do in certain cases? How can you guarantee that if the output matched your biased assumption but the output is non-deterministic that it's actually proves the theory?
2. That's may be an excuse to some degree
3. Bad excuse, it's all varies by design of the app. You can write either easy to test and modify code or spaghetti-code. Even if the code very volatile investing in proper design of the app will give you a plenty of room for unit testing
4. May be there are some of them who actually do have a grounding. In the age of internet it's quite easy to learn the basics having some math background. It won't make you an expert in that field but will give you enough to spot obvious mistakes.
5. If they personally can't do that why not to gather community together and combine the effort? Especially when it's a deadly epidemic.
6. Or may be somebody takes the critics too personal. Take it easy, you can always void it by a phrase - "please send a Pull Request". But you need to be ready to collaborate and actually accept PRs.
The whole situation looks very dirty and it's hard to filter out personal attacs and false critics. Although there are definitely some valid ones.
edit: formatting
The sheer arrogance displayed in this article is breathtaking. Scientists are so super-smart that not only are they experts on the science, they are also better at the entirely orthogonal skill of programming than the entire programming industry!
The results from using this software have directly influenced political decisions that have resulted in the greatest mass curtailment of personal liberty in modern history, not to mention the rest of the response to a public health situation that is going to cost hundreds of thousands of lives around the world.
The idea that code that demonstrably contains bugs and exhibits non-deterministic behaviour, written in a style that has proven time and again to be resistant to mechanical testing, formal verification or expert peer review, should be defended because while it's written in such a style that doesn't matter because this code was written by scientists is... I don't even have strong enough words. It's laughable. It's patently absurd.
Sorry for the slightly ranty comment, but I am genuinely furious that people are trying to defend this work in this way. It is insulting to me as a professional software developer who has also worked on many scientific and engineering programs over the years. It is the antithesis of good scientific practice, where methods are transparently disclosed and peer review and attempts to reproduce (or fail to reproduce) important results are actively encouraged. And it is downright offensive to me as someone who can't look after close family members right now because laws made in no small part because of this software prevent it.
In my time in research I've come across this contemptuous attitude scientists have for coders and engineers again and again. It's just arrogance and insecurity mixed together and has no place in today's research environments where results depend more and more on complex models.
This reminds me of those code reviews where you have a colleague that prefers to spend hours trying to refute reasonable minor comments, instead of 5 minutes to address them.
If they are truly minor, they should not block the change.
I am guessing colleague tries to refute reasonability, which totally makes sense, because if in every review they have to fix minor unreasonable comments to progress, that stalls the thing completely.
Whether the Imperial code is good or bad doesn't actually matter: you can derive the headline numbers that motivated the lockdowns analytically, without any simulation at all [1].
(The real problem with the worst case estimate is that it assumes people don't individually change their behavior in the face of a pandemic.)
This debate is sort of in the weeds. The models are not that important. We have plenty of real world evidence to be very concerned about Covid19. The following facts are not contradictory:
1. The code is bad, maybe even unreliable
2. Scientific software should adhere to best practices for software development
3. At a bare minimum, complex simulation software should be able to generate correct answers for well-understood toy models
4. Covid19 is a dangerous disease that can spread exponentially and kill a lot of people if mitigation measures are not taken (look at Wuhan, Lombardy, NYC)
60 comments
[ 3.0 ms ] story [ 110 ms ] threadhttps://cacm.acm.org/magazines/2010/9/98038-science-has-only...
https://guscost.com/2020/05/12/pandemic-woo/
Following good practices have nothing to do whether it is "commercial software" or "front-end software", as the author claims.
Reminds me of "don't roll your own cryptography." Well, all the standard cryptographic algorithms were rolled by someone, and they had to have thrown out their first pancake too.
"They aren’t worried about it — not because this is a disaster they are trying to cover up, but because this is a routine bug that doesn’t really affect anything important."
If you have a bug you can never know the side effects that it can be causing.
So neither the article author nor the guys who wrote the covid sim understand complex systems.
It would do them much good to read a few works by Taleb. Their problem is that they don't know what they don't know, they think they know what they don't know, but they don't.
Also saying that it's "ok that scientific code is crappy formated/strutured" clearly demonstrates that these people, neither the author of the article, truly understand intuitively how quickly things can (and always do) go wrong with programs because they are complex beasts.
This reads as an acadmic without truly intuitive knowledge of how things can go badly defending other academics.
If a product is meant to put millions into quarantine it must be damn well tested and not a quick demo playground.
A good product can obviously evolve from a quick academic playground, but academics seem to finish a product fully so rarely that they don't understand that unless they test it severly thourougly that it will have bugs.
/s
/s
Academic "scientific software developers" (lol) never have to prove anything works in production long term in majority of cases. They make a one-off project and never touch it again. So this generlization stands.
And yes, I am saying that they are eggheads.
Because this article and the covid sim clearly shows that "scientific software developers" (lol) must not be trusted with real world-affecting systems without some other real world developer checking it.
The fact is that the covid sim is garbage and anyone who tries to defend it does not understand programming and complex systems well enough.
The closest thing we have to this in industry is bug reporting and software security evaluation, and this is sporadic at best. Most commercial software developers keep their code closed source and do everything they can to stifle public knowledge of bugs, even after they've been fixed.
This code has never been peer reviewed, ever. That is indisputable because Ferguson has claimed multiple times that the code is/was entirely undocumented, and nobody outside him and his few colleagues understands how it works. The moment it started being actually reviewed, people found a breathtaking number of bugs like out-of-bounds reads, typos in random number generator constants etc. Broken RNGs! Of all things I'd hope an expert in cryptography would be very worried by, a missing digit in an RNG constant should surely be one of them?
I've spent many years of my career working on highly technical software. I've worked in offices full of professional programmers where many also had postgraduate qualifications in STEM subjects. Those teams did indeed work on quite complex codebases for years.
Do you know what else they typically had? Testing infrastructure that was ahead of its time, careful validation of the mathematics being used before it was implemented, a reasonably clean and systematic software design, and good traceability from the mathematical code back to the underlying mathematics.
This is not rocket science. This is not even unusual if you work in this field. It's just what happens when you let professionals with an appropriate skill set do the job: programmers with technical backgrounds implement models constructed by subject matter experts.
Having one set of skills without the other is not a recipe for success.
Because if you cannot do those things, you're not really contributing anything of any quality.
You write as if you think what I'm describing is somehow unusual. It is not. I have probably worked with at least 30-40 well-qualified people just in my own career from whom any well-mixed team of say 3-5 developers could have done what we're talking about here within a relevant time frame. We are not talking about rocket science, just good software development practices being employed by experienced programmers with technical backgrounds who are used to implementing the models of subject matter experts. This isn't some magical unicorn skill set.
> While these checks could also be handled by unit tests, most scientists generally just end up with their own weird set of ad hoc test outputs and print statements. It’s ugly, and not infallible, but it tends to work well given the intensive nature of our result-testing behaviour and community cross-checking.
The sense in which this statement is plainly wrong is actually addressed in the original article:
> Regressions like that are common when working on a complex piece of software, which is why industrial software-engineering teams write automated regression tests.
This is the whole point. You can only run your "ad hoc" print-statement-based tests once off, which is why such "tests" are useless for finding regressions.
It is therefore rather irresponsible of the scientific community to allow such a bad state of code technical debt to live on for so long, when it’s dealing with a field that is inherently tied to the study of life-or-death scenarios (like we are facing right now).
The only way to avoid this inevitable conclusion of irresponsibility would be if the scientists behind the model warned that it cannot be trusted at all due to the potential of a large number of unknown bugs (did they?). Otherwise, either the scientists are irresponsibility incapable of understanding how to write reliable complex code, or were not honest about the trustworthiness of their code.
If there is ever an asteroid heading towards the earth, I swear that some segment of HN will write angry articles about how the hastily built nuclear-powered spaceship we sent to intercept it doesn't have proper unit tests. They'll demand that we delay the launch until 300 days after impact.
We knew this was coming for months. The code isn't that large, it's just horribly written. If the scientists had worked in collaboration with an expert team of technical programmers, they could have handed off their mathematical models and had a well implemented, extensively verified program to run those models within a useful timeframe.
That might sound extreme, but once again, I will reiterate that expert advice apparently based in part on the output from this program is being used to inform decisions that will affect whether thousands live or die and that are severely curtailing the normal lives of the entire population with other adverse consequences that nobody fully understands yet. Powerful responses require powerful justifications.
If you look at the original article, you'll note that for at least a month the code has been receiving attention and refactoring from a team at Microsoft. Your criticism is that an academic team should have somehow convinced Microsoft to donate these resources before the epidemic became severe.
I'm going to go way out on the thinnest limb and propose that major software companies were not chomping at the bit to donate resources to epidemic modeling teams until very recently, and this isn't the fault of the epidemic modeling teams -- as much as you seem to think it is.
I'm also curious about why they brought in the people they did to do whatever cleaning up was done before publishing this code. There are several businesses in the UK that do this kind of work as a major part of their activity in one way or another. Current or former staff with that sort of background would have been the obvious people to tap for assistance. To the best of my knowledge, the people who have actually been identified as having helped have no such specialist background.
^^^Swallow your politics if you can for a moment and read. At a minimum it seems that the rebuttal here was being argued in very bad faith. Given the stage set, I was expecting bikeshedding about the Law if Demeter or JSON vs other format, etc...
Granted, but when publishing, these need to be primary concerns. Source code, then, is a communications medium.
Many famous scientists like Einstein and Feynman would do their work in their heads, not by filling blackboards with equations. When they needed to publish, they'd write down equations because that is the medium of communications. You make it as simple as possible, so people can follow you. E=mc^2 is brilliant because it is so simple. Maxwell's equations got a lot more traction once they were simplified down to just 4.
Scientists using computers for research or analysis are welcome to use any messy code they want, but if they wish for others to follow their work, they need to clean it up to the standards of the industry.
It was beat into my head from a young age that research is only part of being a scientist. You also need to be able to communicate your ideas -- hence all of the writing class requirements. Scientists need to take this to heart, and if they wish to publish in terms of source code, to write source code that others can read.
When even professionals in the field have trouble verifying your work, you've got a problem. It's been a while since I worked in scientific computing, but I never saw a single published paper with a program where the program even matched the description in the paper it was attached to.
As a trivial historical example - an entire world superpower regularly and persistently engaged in and acted on large-scale unscientific motivated reasoning against its own self-interest and the problem with that was not that the best scientifically verifiable ideas available were inadequately communicated.
1. The developers had a couple of bugs that cause non-deterministic outputs in weird settings. It's a randomized model so this isn't a disaster, but obviously it would be better to have all non-determinism derived from a single seed. However, none of the bugs point to scientific issues. Still, these are real bugs that should be fixed. And indeed: another team pointed out some issues, they triaged the report, and it appears that they've been fixing these issues.
2. The code doesn't run well multi-threaded, and so the scientists run it on a single core. This seems like a feature the developers have de-prioritized. The critics make a HUGE deal about how using a single core is inefficient, but at the end of the day: if scientists can get simulations done in a reasonable time using a single core, this seems like a valid prioritization of limited development effort. The critics know better, obviously.
3. There aren't a bunch of unit tests. But as the authors point out code is in development and being altered continuously to reflect new developments in society and new knowledge about the virus, so it seems hard to develop unit tests that won't become broken as the underlying components change.
4. The critics seem to have a strong grounding in epidemiology. I'm just kidding, they claim to be pure software engineers, but still insist on spending a significant fraction of their critique disputing the calculation of R0 and lecturing the epidemiologists on how they think it should be calculated. This is a giant red flag given that the critics' sole stated expertise is software engineering.
5. The developers are continuing to add features to simulate new aspects of the ongoing pandemic (e.g., new code to simulate the impact of contact tracing apps) instead of fixing every minor issue in the code. Given that we're in the midst of a deadly epidemic that is putting human beings into the ground, this seems like a completely unavoidable resource allocation decision to me.
6. The critics can't even be bothered to pretend their criticism is objective. E.g., the piece concludes with "on a personal level, I’d go further and suggest that all academic epidemiology be defunded." Yeah, sure, you're a totally disinterested software expert here.
I guarantee you if you look at any piece of software -- commercial or scientific, no matter the budget -- with the politically-motivated goal of tearing apart the author, you will succeed. I can say this having spent years doing security and expert witness code reviews for some of the biggest and smallest companies in the software industry. This piece isn't a critique, it's just a hatchet job. And not even a particularly creative one.
Any arguments that the developers of this code are somehow above needing to mechanically test, or formally verify, or independently peer review the code to ensure quality are trivially debunked by the history of bug fixes and changes in behaviour that we do know about, even before making any judgements based on the apparent attempts to hide that history and to conceal the original code from before they brought in programming experts to help tidy it up.
If you have indeed been an expert witness doing code reviews, are you seriously contending that you would not annihilate any case made in a court that hinged on the results of code like this?
This is much more difficult than reading a Git issue tracker, so obviously the critics have nothing to offer us here.
The rebuttal makes a couple of statements about how scientists spend their time to verify their work:
> Scientific software tends to be validated in a more ad hoc way. This is a domain-specific norm, not a matter of competence.
> Practically every scientist I know spends most of their time checking their results rather than coding, and I sincerely doubt that the Imperial group is any different.
I wonder if the team could iterate more quickly if some of these ad hoc validations were automated.
But it's not just a science question. That is the point of the criticism.
Assuming you are the Matthew D Green known for his cryptography work, surely you of all people understand that a poor implementation can have profound consequences even if the original mathematics is sound?
So what do you do with your limited resources? Do you tell the government you can't get them any preliminary data because your multithreading system doesn't work, and you need four months to refactor the code (even though you could just run single-core)? Do you tell them that because you have to write a complete unit testing framework, you can't give them any modeling data or update your model parameters, sorry, good luck with everything.
Do you tell them that another 30K people may have to die based on wrong policy choices because you have decided to prioritize "shipping-production-quality" software standards, even though you are not shipping your software?
You can pretend that this is not a trolley problem. But in this case it literally is. You are standing at the switch in the tracks, and on one side is a detailed set of unit tests and multi-core support, and on the other hand might be tens of thousands of living people. And you know your code better than anyone else in the world. What do you choose?
I don't have an answer. I'm saying the answer is hard, and we're not equipped to lay claims on it based on glancing at a Git repo.
Also, just to be clear, all of the specific issues with the Imperial code that you've mentioned here and in other parts of our discussion today, such as whether to rewrite for better multi-core behaviour or just use single-core, are coming from you not me. My argument is a much simpler one: when it started to become clear that this virus was a significant risk, the government should have provided the necessary resources and the expert advisors should have switched to a model where scientists did the science and programmers did the programming. I don't think anyone can credibly claim that there wasn't enough awareness that this could potentially become a serious issue and so that bringing in broader expertise was appropriate by the time we had things like COBRA meetings being held and secret expert advisory groups being assembled.
It started being developed in 2007. It's by now over a decade old, with a lot of work done in 2014. This program wasn't written quickly, it was re-parameterised with a few values for COVID and otherwise thinks it's simulating the flu. Some other pieces of code date back multiple decades.
You also appear to think that "limited resources" is a valid excuse. The UK science budget is over 8 billion pounds. Universities there churn out a large number of papers. The funding is there if they wanted to do less work but of higher quality, yet they don't.
And finally, just to make that clear: yes, saying "We don't have a reliable model for you" is an entirely valid thing to say to a government. You seem to think even low quality simulations are better than none. Humanity has been fighting disease for thousands of years without bug-ridden 15,000 LOC C programs to "help" us. Not having an academic model is a completely reasonable position to be in.
In particular there's a lot of evidence this particular model has already killed a lot of people: it predicted a surge in hospital needs that never happened. In anticipation of this surge a lot of elderly patients that had tested positive for COVID were discharged back into care homes, because the hospitals thought they'd need the beds within days. Those elderly people then infected the others in the care homes, causing the death rate to go up. If there had been no model, there'd have been no panic-induced mass discharge, patients would have been kept in wards until they were fully recovered and the virus wouldn't have been flushed straight into the populations of the most vulnerable. Outcomes could have been much better as a result.
> Do you tell them that another 30K people may have to die based on wrong policy choices "How many more people should die before...." - where have we heard this, ooh just a couple thousand times before?
Here's a simple fact, if you want to go this route: every extra 1% of unemployment rate (for US) leads to extra ~40,000 death a year due to a whole stew of reasons.
So, which one is better: have a team of professional developers use best practices and existing tools (like yes - testing framework) and minimize what seems to be a result of amateur programming done over the span of 15 years, or release an unreliable yet sensationalist report and cause tens of millions of people loose their jobs and livelihood?
@matthewdgreen , with all due respect you're avoiding answering to concrete questions and keep defending lousy programming on the ground of urgency and appeal to emotions. This is a mere demagoguery.
Perhaps, you can also share with us the _original_ code of the model, based on which the "Report 9" has been made? So far, as I know, those calls went answered.
All the criticism would be moot if the model delivered sensible predictions.
* You'll obviously run into the "testing on your training data" problem, but again this isn't much of a criticism. We're in the midst of an epidemic and scientists are rapidly adjusting a model to reflect developing knowledge. The alternative is to have no models.
> It's a randomized model so this isn't a disaster
What you're saying here is that no matter what distribution properties the "randomness" has it's ok to substitute one with another, right?
> There aren't a bunch of unit tests
It isn't a Microsoft Windows, you know, where a lack of unit tests and uncaught bugs could be potentially fixed in the next release (or not). Report 9 is out, the harm is done.
#4 isn't an argument as we aren't arguing about epidemiological characteristics of the code, but about its validity and correctness.
I am loosing the interest in this discussion very quickly, because it is full of straw-man constructions and intellectual dishonesty, starting with the fact that the original code/model has never been shown to anyone besides of a very small group of hand-picked developers to groom it into some presentable form. Pity...
This is hardly an unexpected conclusion given articles like this one and comments like yours, in which people defend buggy code on the grounds that academics shouldn't even be expected to produce work that isn't buggy. Academics are claiming they structurally can't produce correct, well documented code, not even when it's being used to advise governments.
Hence the conclusion that maybe academics shouldn't be writing that sort of code: how is that not an obvious inference to draw? You seem to believe that any criticism of academia is "politically motivated". It's the same attitude that can be seen in Imperial College's non-answer to the problems: you can't state the private sector has higher standards than academia despite overwhelming evidence it does, because then you're partisan and should be automatically ignored?
You're also an academic, aren't you? Surely that means you're the politically biased one here. How could it be otherwise?
Not do not write bad code but all epidemiology be defunded. So who will do epidimioly studies and develop models of deadly diseases? The software engineers? ;-)
1. Or do they? Or may be they do in certain cases? How can you guarantee that if the output matched your biased assumption but the output is non-deterministic that it's actually proves the theory?
2. That's may be an excuse to some degree
3. Bad excuse, it's all varies by design of the app. You can write either easy to test and modify code or spaghetti-code. Even if the code very volatile investing in proper design of the app will give you a plenty of room for unit testing
4. May be there are some of them who actually do have a grounding. In the age of internet it's quite easy to learn the basics having some math background. It won't make you an expert in that field but will give you enough to spot obvious mistakes.
5. If they personally can't do that why not to gather community together and combine the effort? Especially when it's a deadly epidemic.
6. Or may be somebody takes the critics too personal. Take it easy, you can always void it by a phrase - "please send a Pull Request". But you need to be ready to collaborate and actually accept PRs.
The whole situation looks very dirty and it's hard to filter out personal attacs and false critics. Although there are definitely some valid ones. edit: formatting
The results from using this software have directly influenced political decisions that have resulted in the greatest mass curtailment of personal liberty in modern history, not to mention the rest of the response to a public health situation that is going to cost hundreds of thousands of lives around the world.
The idea that code that demonstrably contains bugs and exhibits non-deterministic behaviour, written in a style that has proven time and again to be resistant to mechanical testing, formal verification or expert peer review, should be defended because while it's written in such a style that doesn't matter because this code was written by scientists is... I don't even have strong enough words. It's laughable. It's patently absurd.
Sorry for the slightly ranty comment, but I am genuinely furious that people are trying to defend this work in this way. It is insulting to me as a professional software developer who has also worked on many scientific and engineering programs over the years. It is the antithesis of good scientific practice, where methods are transparently disclosed and peer review and attempts to reproduce (or fail to reproduce) important results are actively encouraged. And it is downright offensive to me as someone who can't look after close family members right now because laws made in no small part because of this software prevent it.
If they are truly minor, they should not block the change.
I am guessing colleague tries to refute reasonability, which totally makes sense, because if in every review they have to fix minor unreasonable comments to progress, that stalls the thing completely.
(The real problem with the worst case estimate is that it assumes people don't individually change their behavior in the face of a pandemic.)
[1] https://twitter.com/trvrb/status/1258879531022082049