I would be really impressed if it were possible to control for the differences that would occur if one state did a very good job tracing potential cases and then testing them (Massachusetts), versus more ordinary blanket efforts elsewhere.
The author needs to re-run the analysis using only deaths, or deaths plus hospitalizations (if those figures are available). I fear that all the model reveals as currently written is how well states are getting a handle on their testing, and paradoxically showing states with better testing as worse off because of the growing number of cases.
It is hard to tell if this will even work since a lot of people all around the world can't even follow the simple rule of not going out and just staying at home.
This site also uses "confirmed cases" instead of deaths. That would be useful many weeks ago, but now in most countries it's no longer a useful number. It should use deaths and a country specific reporting factor for e.g. whether deaths include only hospital deaths or also other deaths, if only deaths with positive tests or includes assumed Covid deaths.
For lack of better charts. I have been calculating by hand the number of days since total cases(or deaths) were half as they are today. Example there are 586k cases today. How many days ago we had 290k ? Almost 10 days. So it is still doubling, but not as fast. A week ago it was doubling every 7 days.
I've been following the the death rate, because I think deaths are probably a more accurate statistic than cases - less likely that a covid19 death goes unreported.
This of course is a significantly lagging indicator, but it has been long enough that we can see social distancing is working.
> less likely that a covid19 death goes unreported
I don't think this is true at all. Many people die at home. Many die without having ever been officially tested. I've read that in some places the death toll is only counted from those who were tested and then died in a hospital.
It's definitely still "less likely" that a death goes unreported versus an infection without a death. Both numbers are shit, but deaths likely less so.
I think deaths might be unreliable too, because 1) we don't know the mortality rate, we're extrapolating it from bad data and then using an extrapolated number to calculate a trend and 2) mortality rate would change between different groups, e.g. sick and old people vs healthy people, depending how infections spread.
That's why we're talking about deaths, not mortality. Mortality requires you to know the number of people who are infected which as stated above can be inaccurate because many people are not getting tested. However, it's less likely that a person who dies of pneumonia in hospital won't be tested. Also, there is likely to be a smaller proportion of deaths happening outside of hospitals than there are infections so you should consider the number of deaths to be more accurate than the number of cases.
> That's why we're talking about deaths, not mortality.
I believe you're confusing fatality and mortality. The correct version of that statement is: "That's why we're talking about mortality, not fatality.
The case fatality rate is the ratio of number of deaths to the number of people who are confirmed to have the disease (which depends a lot on standard of care, reporting, and bureaucratic integrity) whereas mortality rate is the rate of deaths in general.
Deaths are also unreliable because a) COVID-19 deaths are being reported differently as a notifiable disease (in the UK at least) as compared to, say, influenza and b) autopies are not being carried out on COVID-19 cases due to how contagious it is.
Many people die all the time but it's much harder to hide how many people die total in a developed nation for systemic reasons. Except for natural disasters, the mortality rate for a country the size of the United States or Italy is surprisingly stable. The difference between deaths in March 2019 and March 2020 gives a lot more reliable information on the impact of the virus than testing numbers, which are limited by the availability of tests and the testing procedure (contact tracing vs testing only the highest risk patients or the most severe cases). However, this is just a more concrete measure of the extreme impact of the virus. Without a lot more data, it's not enough to even predict how many people have it or how infectious/deadly it is.
That is people who test positive for Covid-19 who then die in hospital.
The other one collected by the office for national statistics is people who die where covid-19 is listed on the death certificate. There's some lag in those numbers.
It's hard to know how many deaths occur out of hospital, but it's possible about half of covid-19 deaths are happening in care homes. Covid-19 rips through care homes because the patients are by definition older and more frail; the staff have less PPE and less training in using PPE; the staff (in the UK) are often on very low pay which means they sometimes work in multiple homes; they carehomes sometimes have terrible sick pay terms.
(Lots of caveats, and care needed with these numbers, but)
> Key findings:
> Official data on the numbers of people affected by COVID-19 is not available in many countries
> Due to differences in testing availabilities and policies, and to different approaches to recording deaths, international comparisons are difficult
> Data from 3 epidemiological studies in the United States show that as many as half of people with COVID-19 infections in care homes were asymptomatic (or pre-symptomatic) at the time of testing
> Data from 5 European countries suggest that care home residents have so far accounted for between 42% and 57% of all deaths related to COVID-19.
As far as I have heard from Norway, not a single person that has been on a ventilator due to Covid-19 has died. The average age of the dead is 84 years old, which is a bit older than the average life expectancy in Norway. What I can gather from this information is that almost all deaths have happened in care homes or with older people who gets care at home.
* deaths are also likely to be under reported (although, I agree, probably still better than the total unknown people who haven't been tested)
* a decreasing number of deaths does nothing to fix the actual problem: the possibility of contagion if we all resume normalish activity
My preferred metric has been 'average increase rate over 7 days', in other words, take the new detected cases any given day, calculate what percentage that is over the total for the last week, then average seven of those.
We'll never get a proper idea of how many people are actually infected until we get proper widespread testing, but at least we can get an idea of how bad cases (those that are likely to get tested) are growing. Another data point to look at is how many ICU units are in an area vs. how many ICU COVID cases there are and how many beds are still available. [1] San Francisco has finally started making those stats available:
Thing with deaths is that, as others have noted, many go unreported.
What I've looked at so far are IC admissions. As that depends the least on policy I think. Moreover, its the thing we most want to control (i.e. you want peak IC admissions not to go over IC capacity).
I've been using this to judge the effectiveness of measures taken in the Netherlands though. Which is quite different from using it to measure R_0 or R_t in other countries with other data.
I agree 100% with Taek, and that's what I've been using - corona deaths instead of testing for months.
US testing is so porous as to be meaningless, even today.
Also, I care about bottlenecks like ventilators and ICU staff count, not the 98% of people who don't require hospitalization.
I find all US media and political coverage to completely miss the point, with anecdotes substituting for public health policy. It's like watching a train wreck that won't stop.
> US testing is so porous as to be meaningless, even today.
We've been testing well over 5000 people per day on average for over a month. Suppose 500 tests per day are randomly administered for the purpose of estimating overall prevalence of the virus. That sample ought to pretty clearly establish the total number of cases within 5%.
I'm sure there are some clever statisticians at the CDC with some fancy Bayesian inference or something who can push the confidence up even higher using all the non-randomly administered tests.
Test capacity is fixed, or growing linearly. The variable you are measuring is growing exponentially. What will happen is that the number of confirmed cases quickly starts coming back as a linarly growing number which is dependent mostly on the number of tests made, not on the number of infections.
How can a confidence interval possibly be calculated without accounting for the number of tests—not just the number of positive tests—and the total population size?
What TFA shows us is the Rt assuming that all cases are known.
The model which knows nothing about testing rate, cannot possibly be used to predict R0 or Rt.
As a simple proof, consider what the model would output I f testing simply stopped. 0 out of 0 tests positive. After several days, the model outputs an Rt=0 with 100% confidence.
If you could account for testing rate, and then attempt to account for missed cases through proxies using hospitalization rates, ILI surveillance data, excess fatality, as well as incubation period, and an asymptomatic rate... maybe then you could draw error bars around an estimate of Rt that would pass the smell test?
If testing was run against 1,000 randomly selected people per geographical area each day, with a positivity rate reported over time, that would provide the perfect basis for making these calculations. The time series of test results that we actually have today require enormous amounts of unpacking before they can be used safely to extrapolate much of anything.
Rt is based on the rate of change, not the absolute value, right? So if testing coverage is constant in time, then doesn’t it drop out?
Of course, it isn’t constant with time, but it is also probably not changing that drastically over short time frames (like the seven days the author uses), so it shouldn’t throw the results off too much.
Even if the number of tests stay the same, but just the testing criteria shift over time, it would invalidate the daily positive test count for use in determining Rt without somehow adjusting for the resulting bias.
At first I was happy to see a confidence interval in the plot. Then I realized the plots are based on the same numbers anybody else uses: new cases. I'm tired of people stretching the available numbers too thin. There is only so much information we can get out of them.
And yes like you say I hope we will see daily sampling soon. As tests are geared up that should be easy enough.
As an aside: The RKI in Germany is collecting weekly samples to track the flu. They have been testing for Corona now for eight weeks. In those tests, they found 11 positive samples out of 1089. I don't know the reliability of the tests though. The numbers are certainly too small to estimate noise after eight weeks.
Interesting article. In late March I built a simple SEIRD model for a few US states with python. It was very difficult to estimate R0. R0 is essentially Beta/Gamma, where Beta^-1 is is time between contacts and Gamma^-1 is the time until recovery. I think what should be added to this analysis is population density. Each state has a different rate that people come into contact with one another. It would make sense that R0 in NYC is much higher than in Iowa.
Also this website is crowdsourcing US COVID-19 mortality forecasts if anyone is into data modeling:
Now ask yourself - do you believe that Louisiana has a more stringent, more accepted and more enforced lockdown than say Massachusetts or Pennsylvania? Yet the epidemic seems to slow more drastically in Lousiana than most other states.
Why is that? Investigating that seems to be more fruitful avenue.
Perhaps there is more validity to the initial theory that the virus is a lot less virulent in high humidity and heat. Many more observations seem to back that. Low number of cases in India or Brazil etc.
You're cherry-picking. Louisiana is on the low side, but Alabama, Texas, and Georgia are on the high side.
In noisy and unreliable data it's always possible to pick a single data point that supports any desired conclusion. I'd say that all we can conclude is that everybody should stay vigilant.
Like from a few weeks back, the conclusion that COVID-19 will kill 10% of people who contract it, based on the testing from one town in northern Italy.
I've been following Georgia's data for a while now, and I actually think they are closer to Louisiana than you realize. Over the last few weeks, it wasn't unusual to see counties stop growing for days at a time only to suddenly jump by 20-30%.
I think there are effects being seen from the lack of early testing rather than an indication of recent spread during the (very recent) higher temperatures.
Or it's just inaccurate data. I wonder how many cases of COVID19 deaths being reported as "pneumonia" [1] (technically correct, but also untrue) as in this story?
Also keep in mind the "excess mortality" [2] - many stories of places where the total death toll has gone up by a lot YoY that the formally reported COVID19 death numbers don't account for.
India's tests/M is 149. Brazil's 296. Compared to 8894 of USA as of today.
Positive % out of tests: USA ~20%, India: ~5%, Brazil: ~38%.
But at least here in India we test when we are like really really really really sure this person possibly has COVID-19 (if not the very symptoms then either extremely close proximity of a COVID +ve or at a cluster). So that's also there.
Louisiana didn't close bars and restaurants until March 16th[1], which seems to be the same day that NY did as well[2]. Anecdotally, people were still gathering and going out in the french quarter right up until the ban took effect.
I also would like to see this broken down by city because the state highly dilutes the signal when trying to rate the responses of major metropolitan governments.
So cool to see Kevin doing a deep dive on this crisis with the goal of making an impact. It says a lot about his character that he chooses to spend his time doing this.
But also I think that it's tiresome for people without background in study of disease (or long experience like Bill Gates) to get involved. Not only this but it can be dangerous with any area like diseases.
This is the same as problem that you see in the White House of America with this son-in-law of president. Systrom has no experience with infectious diseases or study of diseases (or I did not hear about it yet and he does not say about it), and human society needs very much people who are experts right now.
Maybe Systrom could also use tech to help as this is his area of expertise.
If you look at test results you probably won’t learn much. There will be far fewer tests than cases and most cases will never be tested. It seems more likely that you’ll just be measuring some other effect of how testing is done, for example if the number of tests per day ramps up slowly (and non exponentially) then it seems likely that you will observe Rt to be decreasing (as the number of confirmed cases isn’t shooting up), especially if people are mostly only tested when they turn up in hospital and are expected to be positive. I don’t know how much the author’s model accounts for that but I find the confidence intervals very unconvincing. I would expect them to be much larger.
You're spot on. Kevin's idea and graphs are nice but they rely on a flawed hypothesis, namely that all states test the same way, taste the same amount per capita etc. They don't. So our graphs can only be as good as the numbers we put in and testing is still very insufficient.
I saw an interesting alternate statistics where you measure the overall mortality rate and compare to previous years. EU only, but something like this might exist elsewhere as well:
The data lags behind a bit, but since it measures total number of deaths, it doesn't matter if different countries measure coronoa-related deaths differently. Everyone measures actual deaths. The numbers are hard to fudge for politicians who wants to boost their results.
As a bonus, this also measures how your society is handling the pandemic overall. We know that people will die of the lockdowns as a second-order or third-order effect, and this way we can measure that as well.
Yes.. although excess mortality is the thing that we'll look at in retrospect to gauge the overall human cost of this disease, I don't think it is all that useful for measuring "how well is containment working" in real time. Too many confounding factors (both positive and negative) as you mention.
Personally I think the best metric is one of the simplest -- the raw percentage of positive tests. This is because it simultaneously captures two important factors: if the percentage is high it means that either you are finding the virus all over the place OR it means your testing capacity is so low that you only are testing highly probably cases. If either of these are true, you should not think about reopening.
There is also category of people who did not died of corona, but would not died had there been no corona epidemic. Hospital being overflown means that many people are not getting same care as they would had there been ventilators, drugs and doctors available.
Excess mortality strikes me as very poor data. It seems very unlikely that, during a once-in-a-century pandemic with completely unprecedented severe lockdowns, we should expect all-cause mortality to be close to the baseline.
Sorry, what isn't what we see? I don't think there's a reasonable baseline at all; I think we have no idea what mortality "should" look like, in a world where everyone's under house arrest and afraid of hospitals. You can argue it's lower because people are taking less risks, higher because they're under more stress, lower because they're eating out less, higher because they don't get hospital treatment they need.
Belgium has this specifically for our country [0]. It has a 4 week lag, but they've started to include interim data in the weekly updates by the national bureau for epidemics. The last weekly update included the "excess deaths" compared to previous years: [1]
In other words: in the last week of March, we had 38.5% more deaths than usual. Or 7 extra deaths per 100K inhabitants. That's terrifying.
Side note: Belgium is one of the only countries that includes deaths in carehomes and at home. The deaths in carehomes is almost half of all covid-19 deaths here.
How do you interpret a zero negative excess death rate? Sweden has over 1000 Covid attributed deaths, but almost no difference in weekly deaths compared to the same weeks in 2019. Death numbers are extremely noisy and especially in the flu season with the flu one year perhaps twice as deadly as the previous year.
Covid restrictions have also reduced (a lot) some other seasonal diseases like flu and noro/caliciviruses, so you get an effect in the other direction too. When those that die from Covid are in the highest age groups it's not unthinkable that a part of them would have been killed by the flu too, only the flu didn't come this year because of Covid. Perhaps that's why we have a lot Covid deaths but no excess deaths? Who knows. It's too noisy and has too many dependencies to be very useful I think.
Didn't think about that. Looking at figures it's less than 6/week killed in traffic, while all deaths is 1700/week, so it's unlikely to make a dent. Easter is probably one of the worst traffic weeks, but even then it's probably not enough for a difference.
This is a good candidate. Although some of this effect might be countered by people dying because they didn't get elective treatment.
Regardless: when you change pretty much every parameter of society (risk behavior, driving, crime, health care, ...) I don't understand how one can use the overall death statistics and try to attribute them to Covid.
I think the elective treatment theory is a better point than my driving guess. If you have a disease and your options are to get surgery or take medicine, you are probably being pushed towards non-surgical options, even if that would have overall worse results on average. This might ultimately raise the mortality rate, but would temporarily depress it.
Anecdotally, I definitely find myself trying to be safer right now. I have avoided ladder work on my house and been super careful when using a knife. I just do not want to go to a hospital right now.
I think looking at the overall death statistics are super interesting though. If traffic deaths fall, that is in some ways a byproduct of Covid. If more cancer patients die, that is also a byproduct. All of this goes into factoring how much damage and protection both the disease and the quarantine orders have caused.
The idea is that if your country/region doesn't have excess total deaths, then whatever pandemic restrictions and policy you've implemented is good enough.
To make an absurd example: Duterte in the Philippines has allegedly ordered the military to shoot at people who violate curfew. That helps bring corona-related deaths down drastically, but if your population is dying because you're shooting them or they're starving to death because they can't leave their homes, that's obviously not a good whole-society policy.
And that's why staring at only the corona-related numbers is wrong.
I'm also following the debate in Sweden, which has less restrictions de jure than its neighbours, but people are de facto self-isolating and keeping their distance to almost the same degree. There are lots of people in Sweden screaming that government absolutely must impose a stricter lockdown because the corona-related numbers are worse than in Denmark and Norway, who have stricter lockdowns, therefore Sweden should also follow suit.
But if Sweden doesn't have excess mortality, if Sweden's total mortality rate is the same as Denmark and Norway, while Sweden's economy is less restricted, then I would argue that Sweden is doing the right thing. Or at least not doing the wrong thing. :-)
> The idea is that if your country/region doesn't have excess total deaths, then whatever pandemic restrictions and policy you've implemented is good enough
I disagree with this inference. Different countries have different demography, different weather, different cultural habits, different infrastructures, etc. On top of that, we still don't know how many people would only have benign symptoms when they are infected.
I would refrain anyone to compare pandemic restrictions with a correlation in total death of a country. There are simply too many unknowns, both politically and scientifically
Hong Kong government is also measuring the real time effective reproduction number: https://chp-dashboard.geodata.gov.hk/covid-19/en.html (click on the arrows at the bottom to scroll left and right through various dashboards)
Most obviously that report used the reporting date of deaths as the death date, even though actual death date data was available. The results are dramatically different because of it.
It's really too late to manage this. Testing hasn't been made available in time mostly due to total failure to plan and political decisions not to manage any of this.
All we can do now is turn up and down the level of isolation depending on how many ventilators and ICU beds are available. We will have to do that on a sort of feedback, trial and error basis because we lack capacity and what capacity we do have is being wasted.
The only question is whether anyone will be held accountable for this colossal mess and the preventable deaths its caused, or if we will just accept this happening ever 5 or so years...
If active cases can be pushed down to a mange-able level (i.e. a level where contact tracing is a realistic option and outbreaks can be, by and large, contained) a return to relative normalcy could still be possible.
So that’s another one, two months of tight lockdown to push down active cases (to, say, 500 in the NY area). Then it becomes feasible to throw massive amounts of people at those 500 cases and to do aggressive contact tracing and quarantining of all contacts.
Maybe just by people interviewing the infected, calling around, playing detective (a bit slow), maybe also with the help of contact tracing through mobile phones. Also, you obviously would need to be able to test for the virus (ideally) whenever someone shows the slightest hint of related symptoms (and since those are so unspecific and not exactly rare you would have to expand testing capability).
Some measures will obviously stay in place, but many of the most drastic and impactful ones could be reduced since there wouldn’t really be uncontrollable community spread.
That, to me, sounds like a realistic plan (unlike a complete lockdown until then or trying to let the wave wash over us) for the time until we have a vaccine and until that vaccine is availible in sufficient quantities.
Of course, this depends a lot on whether we can push down active cases so low.
South Korea seems to be nearly or already there. Of course we do not know whether China tells the complete truth, but even if they lie quite a bit they are also probably nearly or already there. Outbreaks with exponential growth haven’t happened there at least, of that we can be relatively certain (because that would be hard to hide).
So, yeah, it’s probably possible if we can push effective R a bit below zero for a couple of weeks to get into a position where we can regain control. Maybe. Hopefully.
I don't disagree with the plan. It's our only option. But how will we implement it? We have no idea how many active cases their are in the NY area (I assume you mean NYC). And we don't know how many people require hospitalization. So you cannot estimate back from the hospitalized numbers. Plus, that would only tell you how many people we infected a week ago. Except that even that isn't certain because we don't know how long between infection and hospitalization. And all of these numbers depend on what pre-existing conditions (including age) a patient has.
So are we over the worst of it or is it just starting? Should we increase or decrease our lock down right now?
And before you answer, getting it wrong won't be noticed for at least a week (or more if the 1 week incubation figure is wrong, which it is), and in that time any mistake will kill tens of thousands or burn $100bn dollars.
This is what we get for testing senators and NBA stars when we should have been studying populations. China blind sided us with this once by lying about it and covering it up. But we have blind sided ourselves as well by lacking any leadership and ignoring experts. This whole thing as been a huge lesson in how badly western governments are currently running. The slightest hint of a problem and it's every man for himself.
Tesing is still needed at a high rate for patients and hospital staff. But I agree testing millions when you have uncontrolled spread isn't "managing" the outbreak. It's a charade because people have been screaming "more tests" for weeeks. Test only when you can actually take action with the result, such as giving different treatment, tracing contacts.
When the outbreaks in different places are suppressed, you can go back to testing and tracing again.
Tests have false positive and false negative rates of ~20%. Start testing millions and not only do you waste tons of money but inadvertently put millions of people in danger.
Testing 1 person makes no difference at all to the treatment of that person because like you say it's not accurate enough to be sure and because we have no (edit: Specific) treatment for Covid-19 that isn't just the same as any other similar infection. Whether you're tested positive, negative or not tested we will ventilate you if you need it and send you home otherwise. So why test any individual person?
But if we had tested (and regularly re-tested) large groups, we would know:
* What the range and controlling factors for gestation period are. This is important because you know when and how many people will start arriving in hospitals in the future.
* How many people have the disease. Right now, we have very little idea if lots of people have had it already and not noticed and are quarantining for nothing. These are the asymptomatic.
* How many people are asymptomatic but infectious.
* Whether people can get it twice
* What rate people actually require hospital places at.
* What the R0 values are AND what they change to with different levels of social isolation.
The lack of accuracy would not really matter because at that scale it just becomes error bars on values and that's fine. We can work with that.
Those are the main pieces of information we need to actually "manage" this disease. Without them we are not managing, we are just guessing. Will their be less infections next week than last week? Will more or less people need ventilators? How many of the nurses and doctors in the building today will be available for work next week? Did closing schools actually help?
Right now, the answer is no one knows. This could all be over in a week when infections suddenly drop because 80% of people have had it. Or it could be that only 3% of people have had it and we will need to maintain all this for another 20 months. No one knows because no one is testing large groups and making big data sets available for analysis. No one doctor can get 100k tests and use them on 20k people over 5 weeks. But the government could. But they're too busy testing each other and giving speeches and shorting the stock market. Sorry if that's a bit tin foil hat.
Any test can be accurate if you use it multiple times. If the coronavirus is 80% accurate, testing everyone twice makes it 96% accurate and three times is 99.2%. Maybe that's the answer.
Even then though, testing individuals is pretty much pointless. It makes no difference to you're treatment.
Not sure if anyone is doing wide testing. The US and UK are not. Germany may be, they seem efficient. Chinas numbers have proven to be total nonsense. Italy seems to have been overwhelmed too quickly to do much.
One problem: You test positive in a nursing home, but you are a false positive (20% of the time). You are quarantined with COVID patients, and how contract COVID. This is almost a death sentence if you have a pre-existing condition.
However it's not clear how the proposal calculates Rt. No formulas and no references to raw data. So I am afraid this is not very useful.
This said, I could imagine that an approach which takes the number of confirmed cases and the total number of tests could work to estimate Rt. The idea is that while the number of cases is not a good estimator we can try to get a better estimator using the number of tests as well. It's a bit tricky, however, because a low number of tests could mean that only the probable cases are tested which skews the numbers.
Again, much as these are pretty, without decent test data, they are a sideshow. The emphasis really should now be on proper sampling of a cross section of the population.
That should really be weekly testing of 1000 or more, over a range of backgrounds and wealth. Only then can we really see the impact.
Using mortality is only really useful when we know what the r0 is, or the mortality rate. We don't know either.
Locally (upstate NY) per our pediatrician they're only testing people with symptoms bad enough to put them at risk of hospitalization. We're still very short.
Isn't that what you would expect any clinician to say, even if they were doing sampling? They're not going to use people asking for tests, they're going to select a group of people at random from the general population and invite them to be tested. Everyone else who has no symptoms they're going to politely say they're not testing people with no symptoms.
Bearing in mind that most of that came within the last 4 weeks, its not hard to imagine that it'd be possible to put aside 1000 a week for proper modelling.
Are there similar calculations for European countries? Several countries are easing lockdowns based on the idea that the transmission rate has become manageable. I'd love to see how those numbers stack up to US states.
Your second rank ordering chart, the one by the upper bound of Rt, is deceptive and misleading. North Dakota and Arkansas only have a high upper bound because there are so few cases that the uncertainty is high. That chart and any analysis associated with it are completely without merit.
Does having a popular mobile app make people scientists? This elitist shit is annoying. I would much rather see him write up something about how he is distributing his wealth to help with the coronavirus rather than this stuff.
He sold Instagram years ago, so it's no longer his. Also, instead of blindly launching an ad hominem attack against him, judge the article on its contents. Of course, having success in one area doesn't mean he is an expert in another, but it doesn't exclude the possibility as well.
Good post discussing Rt (rarely brought up), but a few points are lacking:
* in the face of increasing (and volatile) testing capacity, Rt is being significantly over-estimated by looking at case counts. It's improbable CA still had an Rt above 1 the first week of April with covid deaths linear by the second week. More likely Rt was 1 about a week after the SIP (late march).
* Ignoring "herd immunity" effects of Rt. Some of the "under control" states (NY, Louisiana) have had very high infection rates which in their own end has dropped Rt down. NYC is a strong example of this - with > 20% of the city having been infected, well, Rt will drop dramatically just by so many contacts already having immunity.
* Implying this argues for states locking down now. I agree there is heavy evidence favoring lockdowns weeks ago to have avoided many deaths, but at this point, any lockdown is going to take a week to have an impact on even confirmed cases. The most susceptible populations (essential workers) are the very ones exempted from lockdowns (and less susceptible ones have already voluntarily socially distanced), so there's likely not much gain to be had. Sweden is conveniently our low density control group using mostly voluntary measures (WA State is similar through March 23) and its peak passing a week ago is a sign that Rt drops under 1 faster than you think.
97 comments
[ 4.5 ms ] story [ 158 ms ] threadThis of course is a significantly lagging indicator, but it has been long enough that we can see social distancing is working.
I don't think this is true at all. Many people die at home. Many die without having ever been officially tested. I've read that in some places the death toll is only counted from those who were tested and then died in a hospital.
I believe you're confusing fatality and mortality. The correct version of that statement is: "That's why we're talking about mortality, not fatality.
The case fatality rate is the ratio of number of deaths to the number of people who are confirmed to have the disease (which depends a lot on standard of care, reporting, and bureaucratic integrity) whereas mortality rate is the rate of deaths in general.
The one that's used most (the one that appears every day) is here: https://www.arcgis.com/apps/opsdashboard/index.html#/f94c3c9...
That is people who test positive for Covid-19 who then die in hospital.
The other one collected by the office for national statistics is people who die where covid-19 is listed on the death certificate. There's some lag in those numbers.
https://blog.ons.gov.uk/2020/03/31/counting-deaths-involving...
https://www.ons.gov.uk/peoplepopulationandcommunity/healthan...
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsde...
It's hard to know how many deaths occur out of hospital, but it's possible about half of covid-19 deaths are happening in care homes. Covid-19 rips through care homes because the patients are by definition older and more frail; the staff have less PPE and less training in using PPE; the staff (in the UK) are often on very low pay which means they sometimes work in multiple homes; they carehomes sometimes have terrible sick pay terms.
https://twitter.com/AdelinaCoHe/status/1249359297588473858
https://ltccovid.org/2020/04/12/mortality-associated-with-co...
(Lots of caveats, and care needed with these numbers, but)
> Key findings:
> Official data on the numbers of people affected by COVID-19 is not available in many countries
> Due to differences in testing availabilities and policies, and to different approaches to recording deaths, international comparisons are difficult
> Data from 3 epidemiological studies in the United States show that as many as half of people with COVID-19 infections in care homes were asymptomatic (or pre-symptomatic) at the time of testing
> Data from 5 European countries suggest that care home residents have so far accounted for between 42% and 57% of all deaths related to COVID-19.
* deaths are also likely to be under reported (although, I agree, probably still better than the total unknown people who haven't been tested)
* a decreasing number of deaths does nothing to fix the actual problem: the possibility of contagion if we all resume normalish activity
My preferred metric has been 'average increase rate over 7 days', in other words, take the new detected cases any given day, calculate what percentage that is over the total for the last week, then average seven of those.
We'll never get a proper idea of how many people are actually infected until we get proper widespread testing, but at least we can get an idea of how bad cases (those that are likely to get tested) are growing. Another data point to look at is how many ICU units are in an area vs. how many ICU COVID cases there are and how many beds are still available. [1] San Francisco has finally started making those stats available:
[1]https://data.sfgov.org/stories/s/San-Francisco-COVID-19-Data...
What I've looked at so far are IC admissions. As that depends the least on policy I think. Moreover, its the thing we most want to control (i.e. you want peak IC admissions not to go over IC capacity).
I've been using this to judge the effectiveness of measures taken in the Netherlands though. Which is quite different from using it to measure R_0 or R_t in other countries with other data.
Where do you find that data?
US testing is so porous as to be meaningless, even today.
Also, I care about bottlenecks like ventilators and ICU staff count, not the 98% of people who don't require hospitalization.
I find all US media and political coverage to completely miss the point, with anecdotes substituting for public health policy. It's like watching a train wreck that won't stop.
We've been testing well over 5000 people per day on average for over a month. Suppose 500 tests per day are randomly administered for the purpose of estimating overall prevalence of the virus. That sample ought to pretty clearly establish the total number of cases within 5%.
I'm sure there are some clever statisticians at the CDC with some fancy Bayesian inference or something who can push the confidence up even higher using all the non-randomly administered tests.
https://ourworldindata.org/coronavirus#the-growth-rate-of-co...
What TFA shows us is the Rt assuming that all cases are known.
The model which knows nothing about testing rate, cannot possibly be used to predict R0 or Rt.
As a simple proof, consider what the model would output I f testing simply stopped. 0 out of 0 tests positive. After several days, the model outputs an Rt=0 with 100% confidence.
If you could account for testing rate, and then attempt to account for missed cases through proxies using hospitalization rates, ILI surveillance data, excess fatality, as well as incubation period, and an asymptomatic rate... maybe then you could draw error bars around an estimate of Rt that would pass the smell test?
If testing was run against 1,000 randomly selected people per geographical area each day, with a positivity rate reported over time, that would provide the perfect basis for making these calculations. The time series of test results that we actually have today require enormous amounts of unpacking before they can be used safely to extrapolate much of anything.
Of course, it isn’t constant with time, but it is also probably not changing that drastically over short time frames (like the seven days the author uses), so it shouldn’t throw the results off too much.
And yes like you say I hope we will see daily sampling soon. As tests are geared up that should be easy enough.
As an aside: The RKI in Germany is collecting weekly samples to track the flu. They have been testing for Corona now for eight weeks. In those tests, they found 11 positive samples out of 1089. I don't know the reliability of the tests though. The numbers are certainly too small to estimate noise after eight weeks.
Also this website is crowdsourcing US COVID-19 mortality forecasts if anyone is into data modeling:
https://www.unitarity.com/app/challenges/us-coronavirus-outb...
This website
Why is that? Investigating that seems to be more fruitful avenue.
Perhaps there is more validity to the initial theory that the virus is a lot less virulent in high humidity and heat. Many more observations seem to back that. Low number of cases in India or Brazil etc.
In noisy and unreliable data it's always possible to pick a single data point that supports any desired conclusion. I'd say that all we can conclude is that everybody should stay vigilant.
Reminds me of the Principle of Propaganda - You can prove anything if you ignore enough facts or data.
I think there are effects being seen from the lack of early testing rather than an indication of recent spread during the (very recent) higher temperatures.
Also keep in mind the "excess mortality" [2] - many stories of places where the total death toll has gone up by a lot YoY that the formally reported COVID19 death numbers don't account for.
[1] https://www.huffpost.com/entry/not-just-the-flu-coronavirus-...
[2] https://towardsdatascience.com/covid-19-excess-mortality-fig...
Positive % out of tests: USA ~20%, India: ~5%, Brazil: ~38%.
But at least here in India we test when we are like really really really really sure this person possibly has COVID-19 (if not the very symptoms then either extremely close proximity of a COVID +ve or at a cluster). So that's also there.
https://www.worldometers.info/coronavirus/
Source: I live in New Orleans, returned from Los Angeles March 9.
[1]https://nola.eater.com/2020/3/16/21182099/louisiana-closes-b... [2]https://ny.eater.com/2020/3/15/21180713/restaurant-bar-shutd...
To know you have to investigate that you first need the number that indicates that, no?
This is the same as problem that you see in the White House of America with this son-in-law of president. Systrom has no experience with infectious diseases or study of diseases (or I did not hear about it yet and he does not say about it), and human society needs very much people who are experts right now.
Maybe Systrom could also use tech to help as this is his area of expertise.
This study tries to estimate Rt in several European countries from the death rates though there are issues with those statistics too: https://imperialcollegelondon.github.io/covid19estimates/
http://euromomo.eu/
The data lags behind a bit, but since it measures total number of deaths, it doesn't matter if different countries measure coronoa-related deaths differently. Everyone measures actual deaths. The numbers are hard to fudge for politicians who wants to boost their results.
As a bonus, this also measures how your society is handling the pandemic overall. We know that people will die of the lockdowns as a second-order or third-order effect, and this way we can measure that as well.
Personally I think the best metric is one of the simplest -- the raw percentage of positive tests. This is because it simultaneously captures two important factors: if the percentage is high it means that either you are finding the virus all over the place OR it means your testing capacity is so low that you only are testing highly probably cases. If either of these are true, you should not think about reopening.
https://www.nytimes.com/2020/04/06/well/live/coronavirus-doc...
Many deaths won't be attributed to covid19 but still caused by it.
Here an explanation:
https://twitter.com/firefoxx66/status/1249996541424816128
Side note: Belgium is one of the only countries that includes deaths in carehomes and at home. The deaths in carehomes is almost half of all covid-19 deaths here.
[0] https://epistat.wiv-isp.be/momo/
[1] https://epidemio.wiv-isp.be/ID/Documents/Covid19/COVID-19_Da... (NL or FR only, sorry)
Covid restrictions have also reduced (a lot) some other seasonal diseases like flu and noro/caliciviruses, so you get an effect in the other direction too. When those that die from Covid are in the highest age groups it's not unthinkable that a part of them would have been killed by the flu too, only the flu didn't come this year because of Covid. Perhaps that's why we have a lot Covid deaths but no excess deaths? Who knows. It's too noisy and has too many dependencies to be very useful I think.
Regardless: when you change pretty much every parameter of society (risk behavior, driving, crime, health care, ...) I don't understand how one can use the overall death statistics and try to attribute them to Covid.
Anecdotally, I definitely find myself trying to be safer right now. I have avoided ladder work on my house and been super careful when using a knife. I just do not want to go to a hospital right now.
I think looking at the overall death statistics are super interesting though. If traffic deaths fall, that is in some ways a byproduct of Covid. If more cancer patients die, that is also a byproduct. All of this goes into factoring how much damage and protection both the disease and the quarantine orders have caused.
To make an absurd example: Duterte in the Philippines has allegedly ordered the military to shoot at people who violate curfew. That helps bring corona-related deaths down drastically, but if your population is dying because you're shooting them or they're starving to death because they can't leave their homes, that's obviously not a good whole-society policy.
And that's why staring at only the corona-related numbers is wrong.
I'm also following the debate in Sweden, which has less restrictions de jure than its neighbours, but people are de facto self-isolating and keeping their distance to almost the same degree. There are lots of people in Sweden screaming that government absolutely must impose a stricter lockdown because the corona-related numbers are worse than in Denmark and Norway, who have stricter lockdowns, therefore Sweden should also follow suit.
But if Sweden doesn't have excess mortality, if Sweden's total mortality rate is the same as Denmark and Norway, while Sweden's economy is less restricted, then I would argue that Sweden is doing the right thing. Or at least not doing the wrong thing. :-)
I disagree with this inference. Different countries have different demography, different weather, different cultural habits, different infrastructures, etc. On top of that, we still don't know how many people would only have benign symptoms when they are infected.
I would refrain anyone to compare pandemic restrictions with a correlation in total death of a country. There are simply too many unknowns, both politically and scientifically
Hong Kong government is also measuring the real time effective reproduction number: https://chp-dashboard.geodata.gov.hk/covid-19/en.html (click on the arrows at the bottom to scroll left and right through various dashboards)
Most obviously that report used the reporting date of deaths as the death date, even though actual death date data was available. The results are dramatically different because of it.
All we can do now is turn up and down the level of isolation depending on how many ventilators and ICU beds are available. We will have to do that on a sort of feedback, trial and error basis because we lack capacity and what capacity we do have is being wasted.
The only question is whether anyone will be held accountable for this colossal mess and the preventable deaths its caused, or if we will just accept this happening ever 5 or so years...
If active cases can be pushed down to a mange-able level (i.e. a level where contact tracing is a realistic option and outbreaks can be, by and large, contained) a return to relative normalcy could still be possible.
So that’s another one, two months of tight lockdown to push down active cases (to, say, 500 in the NY area). Then it becomes feasible to throw massive amounts of people at those 500 cases and to do aggressive contact tracing and quarantining of all contacts.
Maybe just by people interviewing the infected, calling around, playing detective (a bit slow), maybe also with the help of contact tracing through mobile phones. Also, you obviously would need to be able to test for the virus (ideally) whenever someone shows the slightest hint of related symptoms (and since those are so unspecific and not exactly rare you would have to expand testing capability).
Some measures will obviously stay in place, but many of the most drastic and impactful ones could be reduced since there wouldn’t really be uncontrollable community spread.
That, to me, sounds like a realistic plan (unlike a complete lockdown until then or trying to let the wave wash over us) for the time until we have a vaccine and until that vaccine is availible in sufficient quantities.
Of course, this depends a lot on whether we can push down active cases so low.
South Korea seems to be nearly or already there. Of course we do not know whether China tells the complete truth, but even if they lie quite a bit they are also probably nearly or already there. Outbreaks with exponential growth haven’t happened there at least, of that we can be relatively certain (because that would be hard to hide).
So, yeah, it’s probably possible if we can push effective R a bit below zero for a couple of weeks to get into a position where we can regain control. Maybe. Hopefully.
So are we over the worst of it or is it just starting? Should we increase or decrease our lock down right now?
And before you answer, getting it wrong won't be noticed for at least a week (or more if the 1 week incubation figure is wrong, which it is), and in that time any mistake will kill tens of thousands or burn $100bn dollars.
This is what we get for testing senators and NBA stars when we should have been studying populations. China blind sided us with this once by lying about it and covering it up. But we have blind sided ourselves as well by lacking any leadership and ignoring experts. This whole thing as been a huge lesson in how badly western governments are currently running. The slightest hint of a problem and it's every man for himself.
When the outbreaks in different places are suppressed, you can go back to testing and tracing again.
Testing 1 person makes no difference at all to the treatment of that person because like you say it's not accurate enough to be sure and because we have no (edit: Specific) treatment for Covid-19 that isn't just the same as any other similar infection. Whether you're tested positive, negative or not tested we will ventilate you if you need it and send you home otherwise. So why test any individual person?
But if we had tested (and regularly re-tested) large groups, we would know: * What the range and controlling factors for gestation period are. This is important because you know when and how many people will start arriving in hospitals in the future. * How many people have the disease. Right now, we have very little idea if lots of people have had it already and not noticed and are quarantining for nothing. These are the asymptomatic. * How many people are asymptomatic but infectious. * Whether people can get it twice * What rate people actually require hospital places at. * What the R0 values are AND what they change to with different levels of social isolation.
The lack of accuracy would not really matter because at that scale it just becomes error bars on values and that's fine. We can work with that.
Those are the main pieces of information we need to actually "manage" this disease. Without them we are not managing, we are just guessing. Will their be less infections next week than last week? Will more or less people need ventilators? How many of the nurses and doctors in the building today will be available for work next week? Did closing schools actually help?
Right now, the answer is no one knows. This could all be over in a week when infections suddenly drop because 80% of people have had it. Or it could be that only 3% of people have had it and we will need to maintain all this for another 20 months. No one knows because no one is testing large groups and making big data sets available for analysis. No one doctor can get 100k tests and use them on 20k people over 5 weeks. But the government could. But they're too busy testing each other and giving speeches and shorting the stock market. Sorry if that's a bit tin foil hat.
Q: Is there a reliable test that shows if someone has previously had Covid-19?
Q: If yes, is it in widespread use anywhere?
Any test can be accurate if you use it multiple times. If the coronavirus is 80% accurate, testing everyone twice makes it 96% accurate and three times is 99.2%. Maybe that's the answer.
Even then though, testing individuals is pretty much pointless. It makes no difference to you're treatment.
Not sure if anyone is doing wide testing. The US and UK are not. Germany may be, they seem efficient. Chinas numbers have proven to be total nonsense. Italy seems to have been overwhelmed too quickly to do much.
Trying to measure Rt seems to be a good idea.
However it's not clear how the proposal calculates Rt. No formulas and no references to raw data. So I am afraid this is not very useful.
This said, I could imagine that an approach which takes the number of confirmed cases and the total number of tests could work to estimate Rt. The idea is that while the number of cases is not a good estimator we can try to get a better estimator using the number of tests as well. It's a bit tricky, however, because a low number of tests could mean that only the probable cases are tested which skews the numbers.
That should really be weekly testing of 1000 or more, over a range of backgrounds and wealth. Only then can we really see the impact.
Using mortality is only really useful when we know what the r0 is, or the mortality rate. We don't know either.
I find it difficult to believe with all that testing going on that no one is reserving 3% of the weekly tests to do proper estimation as you suggest.
https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/test...
Bearing in mind that most of that came within the last 4 weeks, its not hard to imagine that it'd be possible to put aside 1000 a week for proper modelling.
After all it would make much more accurate models, which would allow better allocation of resources. [1]https://en.wikipedia.org/wiki/COVID-19_testing
EDIT: Turns out there is... https://epiforecasts.io/covid/posts/global/
Infection rate is not the same as number of cases.
* in the face of increasing (and volatile) testing capacity, Rt is being significantly over-estimated by looking at case counts. It's improbable CA still had an Rt above 1 the first week of April with covid deaths linear by the second week. More likely Rt was 1 about a week after the SIP (late march).
* Ignoring "herd immunity" effects of Rt. Some of the "under control" states (NY, Louisiana) have had very high infection rates which in their own end has dropped Rt down. NYC is a strong example of this - with > 20% of the city having been infected, well, Rt will drop dramatically just by so many contacts already having immunity.
* Implying this argues for states locking down now. I agree there is heavy evidence favoring lockdowns weeks ago to have avoided many deaths, but at this point, any lockdown is going to take a week to have an impact on even confirmed cases. The most susceptible populations (essential workers) are the very ones exempted from lockdowns (and less susceptible ones have already voluntarily socially distanced), so there's likely not much gain to be had. Sweden is conveniently our low density control group using mostly voluntary measures (WA State is similar through March 23) and its peak passing a week ago is a sign that Rt drops under 1 faster than you think.