I'm the co-founder and CTO at BillionToOne. I'm happy to answer any questions here. I've also posted a slightly more technical explanation of how the test works and why it can scale here: https://twitter.com/dtsao/status/1247642005510873088?s=21
Edit: Since our site seems to be overwhelmed at the moment, here's a recap:
We’ve been working hard at BillionToOne on a new COVID-19 test that scales testing to everyone in the US. Our test (1) re-purposes existing infrastructure, (2) eliminates time-consuming RNA extraction, and (3) enables a distributed system for COVID-19 testing.
The first thing we figured out is how to run COVID-19 tests on existing automated Sanger sequencers. One sequencer can process up to 3840 samples per day. There are hundreds of sequencers of excess capacity because they were built for the Human Genome Project over 20 years ago.
It would take only 2 sequencers to surpass the current test capacity for all of California. There are far more than 2 sequencers in California (some individual labs have 10 or more).
We tweaked the protocol so COVID-19 could be detected from sequencing data using linear regression. Basically, we add ~100 copies of a known DNA sequence to help us calculate how much virus nucleic acid is in the specimen. It works just as well as gold-standard RT-qPCR.
Lab workflow for COVID-19 testing is traditionally 1. Specimen accessioning, 2. RNA extraction, 3. RT-qPCR 4. Reporting. RNA extraction, in particular, has been a huge bottleneck in terms of reagent shortages and labor-intensiveness.
We showed that we can skip RNA extraction entirely without affecting test sensitivity and limit of detection.
By skipping RNA extraction and using automated Sanger sequencers, we think we can get to an additional 200,000 samples per day test capacity in existing clinical labs.
A distributed system is often the only way to operate at massive scale. A fully distributed system could have different sites and labs responsible for each process and dynamically re-allocate resources based on availability and capacity.
The Broad institute COVID-19 lab has already started doing this. They are asking for specimens to be submitted in a standardized tube format and pre-barcoded. They have essentially distributed the specimen accessioning work.
Because there is a highly developed service industry for Sanger sequencing with <24 hour turnaround, there is an opportunity to further scale up testing by distributing the work to their (currently) idle sequencers.
Distributed testing could scale from 200k to >1 million tests per day, but would require a change in regulations that currently prohibit it.
Thanks to the BillionToOne team for pulling this work together! Next step is to start manufacturing test kits and obtain Emergency Use Authorization from the FDA. We’re eager to work with clinical Lab Directors and contract kit manufacturers.
"When you’re fundraising, it’s AI
When you’re hiring, it’s ML
When you’re implementing, it’s linear regression"
The core of our machine learning is Ax=b :grins:
More seriously, the main reason why traditional sanger sequencing can't be used for COVID-19 testing is because it would be unclear whether a lack of signal is truly due to lack of virus, or if it is just because the assay failed (happens all the time!)
What we've done is introduce a reference sequencing signal that is biochemically very similar to viral RNA, but produces a distinct vector of electrical signals that is different from the signals emitted by viral RNA. Since we know what both the reference and viral signals look like, we can perform linear regression analysis to fit the linear combination of viral and reference signals that best match our data.
For HN it's better to drop the fundraising language and use the implementing language, so I've s/machine learning/linear regression/'d your text above.
I don't make them sign a contract or anything. It's part of the coaching that we do with YC founders on HN (example here: https://news.ycombinator.com/item?id=22808556). Most of that is at https://news.ycombinator.com/launches. We act as the editors of those posts and work closely with the founders for quite a while beforehand. Today was different, but it's a similar relationship.
I wouldn't make a major edit without letting them know. In the present case, I just let the whole community know, because the edit was a substantial one. Had it been a trivial one, I wouldn't have bothered. The priority is interesting discussion, not punctilious bookkeeping, and you can't necessarily have both.
In the flurry of these threads, the founders are focused on responding to substantive questions. That's what they should be doing: satisfying community curiosity in their area of expertise. It's my job, not theirs, to prevent the discussion from getting derailed by things like the unfortuitous use of a buzzword which they have no clue the community has been tired of hearing for years already.
If the users weren't YC founders that we have an explicit coaching relationship with, I'd always consult with them before offering help. That would happen by email. By the way, if anyone wants this sort of coaching, we're happy to help as much as we can via hn@ycombinator.com. I give this sort of advice to startup founders, project creators, and article authors all the time, and everyone is welcome to it. Just don't worry if you don't hear back right away. The inbox has gotten to the point where it piles up regularly.
I would not love it if he improved my post, so I am glad to have the clarification that this is only done to YC-affiliated commenters and with at a minimum some kind of implicit buy-in from them.
~~Please change it back. I appreciate ~dvdt's humorous yet candid explanation that they're using the simplest of machine learning techniques, but unless they explicitly say that it is indeed simple linear regression, I think it risks inaccuracy to describe it so specifically.~~
ETA: never mind, ~dvdt signed off on the change. Appreciate the collaboration!
> Since we know what both the reference and viral signals look like, we can perform linear regression analysis to fit the linear combination of viral and reference signals that best match our data.
Does this mean you assume a linear relationship between the quantity of viral RNA and the strength of the signal?
I know that when back i used to draw calibration curves from my positive controls, there was usually a sublinear relationship, across all sorts of different assays, at least at the upper end.
They get basically two superimposed chromatograms from Sanger sequencing, one from the control and one from the patient sample. They need to sort out which chromatogram peak belongs to which and calculate the relative abundance of the sequences in the sample from the relative peak intensities.
The machine learning angle isn't the exciting part here, it's all the rest. Great idea!
Your company has filed a US patent application (and it appears also a PCT) on the qSanger method, and this application is reference 8 in your document.
If it is specific to COVID-19 testing, we will not seek anything, as long as the end-user is not financially benefiting from it or importantly, selling qSanger kits.
If they need our bioinformatics automation & help with set-up, we would license the method for COVID-19 testing for $3-$5/sample as part of each sample that is being put through our pipeline.
If they ask for 96-well plates with all reagents that are ready to use (so that they just need to add VTM), we would work with manufacturers to produce the reactions and plates, and the price of kit (~$15 per test) would include limited license to use our automated bioinformatics calling pipeline.
When you say 'as long as the end user is not financially benefitting' - is the end user the lab conducting the test?
You said in an earlier comment that the reimbursement for testing is too low to justify buying expensive equipment. You are also proposing to charge half the reimbursed rate for it to run on someone else's equipment.
Are the current equipment owners expected to donate this crucial equipment, because if they are the bottleneck, shouldn't they be the ones compensated to encourage more equipment to be made available?
This equipment isn't something a hospital has unless they have a serious desire to do top notch genetic disorder testing, and that kind of hospital is going to use the equipment for this cause. The Abbot machine is something practices with much harder financial constraints have to seriously worry about paying for.
$15 is half the price of even the bare minimum qPCR kits (e.g., TaqPath). We need to buy the reagents from NEB, IDT, and others and work with a contract manufacturer to mix it into a reaction. Reagent, manufacturing, quality control, and fulfillment cost already add up to ~$11/reaction. That does not take into account any costs associated with developing the assay, supporting the assay, getting it through EUA, customer service, bioinformatics help. And we have to pre-pay for all of the reagent costs in the anticipation of the volume. I anticipate that we will likely end up net negative with this work, and even if it ends up being slightly net positive, it will not impact our valuation in a positive way.
The current equipment owners are already the clinical laboratories. It is unused capacity for them. Other owners are sequencing service providers. The full cost of running an end-to-end Sanger reaction as a provided service is $2-$6, so at the $50 reimbursement price, the laboratories will still be incentivized.
Thank you for what seems to be a breakthrough with implications beyond even the current outbreak.
I had a few questions though:
How do you compare this test to the Abbott machines? Obviously that test is faster, but how does that impact what we can do with it?
For 1m/day to be sufficient, do we need contact tracing programs to be able to find everybody who needs to be tested? How hard will it be to scale these programs?
The Abbott machines are point-of-care devices that typically sit in doctor's offices. One really interesting use case I've heard of for the Abbott machines is to test all OB patients who are coming in to the clinic for routine care to make sure that they are COVID-19 negative. This allows the clinical staff to conserve PPE and use less burdensome precautions.
I think that where the Abbott machines might hit a wall is that they are one at a time, and they require Abbott's consumable test cartridge and device to run (think printer ink / printer). I don't have any firsthand knowledge, but I would anticipate that it is difficult to scale-up manufacturing of the devices rapidly enough to keep pace with the pandemic growth.
We absolutely need contact tracing to find everybody who needs to be tested. We're not working on scaling up contact tracing, but I think several people in the tech community are working on making that easier to perform at scale.
Abott's test is cartridge based, isothermal and modular. There should be no technical reason why they cannot build a high throughput, random access version.
Whether this is the direction the company wants to spend their resources is another story.
Hey, lowly Bio undergrad here, but how are you able to skip the RNA extraction step? I read the paper and you use the viral transport medium, but wouldn't you have to also purify RNA from that (or is it just much easier to extract RNA from that medium)? I also dived into the paper behind the "skip the RNA extraction step" methodology and it basically seems to swap out one RNA extraction kit for another (Qiagen RNeasy Mini kit and the Qiagen RNeasy Micro kit). Couldn't shifting kits from one provider to another introduce supply chain strain? (or am I just oversimplifying it?)
We have the data both with extraction and without and show that it does not make a difference with qSanger. In Figure 4, we add VTM directly to PCR reactions. Seracare VTM samples has SARS-CoV-2 viral RNA in a different capsid to prevent infections in a research setting, but otherwise it reflects real-world VTM samples (and much more realistic than even what EUA requires).
By the way, this robustness is completely expected, as any impurities in VTM would impact spike-in and endogenous viral amplification equally for end-point PCR (so their ratio stays the. same). This is not necessarily true for qPCR where an impurity (caused by lack of RNA extraction) can potentially cause a positive sample look like a negative when the viral RNA does not RT-PCR.
Thanks for the question! The goal of skipping RNA extraction is to decrease the amount of labor necessary for processing samples and also to eliminate a dependency on RNA extraction reagents that have recently become difficult to find. The FDA is very strict about the specific brand and model of kit you use, so showing that you can swap out one RNA kit for another is actually very useful because you will have alleviated some of the supply chain strain (although I agree at high enough load both supply chains will then become limiting).
The way currently available COVID-19 testing works is by detection of viral RNA. Since the amount of viral RNA in a patient sample is too low to detect directly, we first need to amplify it by PCR. However, this viral RNA is packaged within all sorts of proteins and lipids that could make it inaccessible to amplification unless they are first purified away. Furthermore, the sample is shipped in "viral transport medium", which is essentially a cocktail of chemicals designed to preserve the virus. Unfortunately, these preservatives often have the side effect of interfering with PCR amplification, so these too need to be purified from the sample.
However, since RNA extraction is usually the most laborious part of the assay, there has been a lot of interest in optimizing the amplification so that it is resilient to all of these impurities. The preprint referenced in our manuscript (https://www.biorxiv.org/content/10.1101/2020.03.20.001008v1) gave us the initial idea that this could be possible, and much of it comes down to the choice of amplification method (e.g. choice of enzymes and buffers) that you choose.
However, even when you choose a "good" enzyme and buffer, you will still suffer an amplification penalty, and this will cause you to return a false-negative on some affected samples because there was so little virus in the sample to begin with. The innovation we have is to spike-in a correspondingly low level of DNA to the reaction mixture. That way, if you see the low level of DNA without seeing any viral signal, you can be assured that the amplification still worked and that there truly is no virus in the sample.
In the UK they're saying there's a shortage of swabs and pipettes even, do you not need these too?
Also, in the UK our independent and uni labs have been saying for almost a week they could extract the RNA differently but the NHS have a fixed approved way that they won't change.
- Are the chemicals you're using more common or would there just be a new shortage of different chemicals?
- Is there a risk you'd be creating a test that didn't work very well, and the US would end up with a bunch of useless tests (e.g. Italy had to abandon a bunch of Chinese sourced tests, UK's anti-body tests are ineffective)?
Our technique would still be affected by shortages in specimen collection (like swabs).
Purely speculative, but I think if swabs remain an issue for too long, alternatives could start coming online, such as even using qtips + saline (no idea if it works, it's just an example). The current swab + Universal / Viral Transport Medium combo is optimized for flexibility; it is designed to work across a very broad range of viruses and bacteria that have different viral loads and shedding characteristics. The current pandemic is pretty much COVID-19 only, so I think it's a priori feasible that a specimen collection procedure can be found that uses common materials. We did try early on to see if saline or other buffers affected the performance of the assay, and it worked fine in those conditions.
We use fairly standard chemicals. I haven't heard from our suppliers about shortages for the chemicals we use. Chemicals and enzymes tend to be relatively fast to scale up for bulk manufacturing.
There's always manufacturing risk that a product will not work as expected. In fact, the first COVID-19 test developed by the CDC did not work as expected, and this delayed testing by several weeks. We de-risk this as much as possible by performing experiments as early as possible, akin to the fail fast mentality of checking for the highest risk failure modes first. Since we don't have a national healthcare system in the US, the manufacturer takes on the vast majority of the risk of a defective product.
if the virus is known to live on cardboard or plastic for 48-72 hours, is the viral transport medium even necessary, assuming rapid shipping and processing?
Let’s say it’s 50/50 whether it lives 24h without help. That’s would be a pretty bad false negative rate for your test, but a 50/50 of potentially getting infected by your mail is pretty high.
To be more precise, last info I read modeled the virus with exponential decay, with half-life measured in minutes to hours. After an hour (or 3h or 0.5h), half of the virus is already inactive¹.
Even after ~6 half-lives, remaining 1% of viral load is still potentially dangerous, but it's not a good basis for a test if you want it to be sensitive.
¹ Inactive does not mean destroyed. It may be possible or even easier to detect a partially decomposed virus, even with the current tests. Or not.
There are companies out there working on swabs. e.g. Formlabs designed an autoclavable, 3D-printed nasopharyngeal swab using biocompatible Dental resin, in concert with local hospitals. They received FDA Class 1 Exempt status from the CDC and are printing some 150K per day.
I'm not associated with the company, I just own some of their printers. They've also got some 2000+ volunteers who own printers or have CAD expertise signed up and looking for ways to contribute. Apparently we can't make medically-approved swabs (most of us aren't ISO 13485-certified or FDA-registered), but there's other stuff (e.g. hands-free doorknobs). I'm even contemplating shipping one of my printers back to them to help in the effort.
It's standard practise in lots of kinds of sequencing experiments to use a spike-in. Makes perfect sense to use it here - in fact all the other sequencing based SARS-CoV-2 tests I've seen also use spike-ins.
The RNA extraction can also be done without reagents by using a heat reaction on the sample similar to boiling an egg. It is a 5-minute process.
According to an older scientist, Anders Fomsgaard at the Danish Serum Institute, this is how they did it ”in the old days”. He is the father of one of the authors.
This eliminates supply chain problems for reagents and was shared quickly to help in Spain.
Hi David, what about Rosche Cobas 6600/8800 high-throughput diagnostics machines? Are they using the traditional RT-qPCR assay? How can they are able to test 400k samples per week?
Roche 6600/8800 instruments are state-of-the-art automated RT-qPCR machines. We need them as well as other COVID-19 testing instruments.
That said, it is easier to ship the test kits than scale the instrumentation. Both Roche and Abbott still need to build hundreds of their instruments before the kits that they are shipping out this week can be used on the daily rate that they are trying to get to. I am not sure with Roche, but Abbott estimates the end of June to have enough machines shipped to achieve 50K per day capacity on their instruments.
Another potential problem with new instrumentation is that reimbursement for COVID-19 tests is very low ($30-$50), so it becomes financially difficult for hospitals and laboratories to buy very expensive instruments and also pay for test kits that cost $30-$50 per test, on par with reimbursement.
We try to avoid both issues by utilizing a currently unused Sanger install base and low-cost reagents.
> We try to avoid both issues by utilizing a currently unused Sanger install base and low-cost reagents.
Good thought! Thank you.
Additional question: are Sanger instruments also very sophisticated and manufactured by those limited few pharmaceutical companies/medical equipment manufacturers? Or is there a need or necessary to scale up the production for Sanger instruments too? I am asking that is because lack of testing is a global issue, not only in the U.S. Africa/India together has 2 billion people are the testing hurdles there are even more challenging.
Those systems process up to 384 or 1056 samples in ~8 hours. The proposed method should support a similar volume (even a bit higher) than the Roche 8800, but with a different large installed instrument base. So this should add a lot of capacity to that qPCR.
Hi, I read through your paper. Interesting method.
In figure 1A, the workflow includes a standard PCR step before Sanger. Workflow-wise, wouldn't it still have the same bottleneck as qPCR test, i.e. limited by 96/384-well instrument runs?
Thanks; that is a good question. In our laboratory, we have a ratio of 10:1 for PCR to qPCR instruments. In the new laboratory that we are constructing, the ratio is 50:1. It was similar at Stanford academic laboratories during my PhD. Standard PCR instruments are inexpensive and very common. qPCR instruments are definitely not as common, as they are very specialized instruments for a few use-cases.
Most clinical laboratories would have 10 to 50 PCR instruments that they can use to run the initial amplification reaction in parallel before Sanger sequencing. Also, Sanger sequencing uses a plate feeder, so you can add new plates on top as the second round of PCR reactions finish.
But, more importantly, the qSanger can by-pass RNA extraction, which seems to be an important bottleneck in the RT-qPCR workflow.
All the material on your page talks about throughput but not turnaround time which makes me quite suspicious that it's not very good. Obviously, however, if it takes 2 weeks to do the test then it has very limited utility as the person will have recovered (or had an adverse outcome) anyway. Since it cannot be administered at the point of care I am imagining that you are already up against a day or so of logistics just to get a sample to the testing location.
Can you please comment on the actual realistic turnaround time for the test?
A turn around time of 24-36 hours should be easily achievable for the performing lab, depending on the time of day the specimen arrives (morning vs afternoon). It takes about 1 hour for the initial RT-PCR amplification, and sequencing takes about 12 hours.
I would not have even considered trying to make use of the existing Sanger sequencing machines. This opens up a whole extra set of hardware to increase testing bandwidth. Thank you.
It sounds like your work has amazing potential. How far have you come already, and what are your biggest challenges moving forwards? How many tests did you manage to process today?
Your company is an inspiration and are doing a tremendous service to the world. I am a federal consultant and have a couple of companies in the medical space that I help find and write solicitations for. I'd love to help your company free of charge on anything and everything. I have alot of infrastructure built such as pricing tools, automated solicitation writing, a ton of outreach lists, etc. One such outreach list is of clinical labratory's nationwide with emails to executives and a ton of other information. I also work with a company that sells reagents and contract kits. I've also worked at Tesla on the Supply Chain, Capital Equipment team in the past and can help you procure any reagents or automated Sanger sequencers/ labs who have them. Additionally, I am fairly adept at SQL and Python (Pandas for data) and why I am a regular on Hacker News and how I found the comment. Let me know if there is anything I can do as I would be very proud to contribute anything to your mission.
I read the paper and I'm still a bit unsure about how the process works. How specific is it to COVID-19?
Given that a Sanger sequences does sequence all kinds of mRNA, would you also find other RNA-based viruses like the flu? How much modification would be necessary to diagnose all kinds of RNA viruses?
This is a test to see if someone has a current COVID-19 infection. The antibody tests (serological) tests are also important, but since it is estimated that only ~1% of the US has previously contracted COVID-19, it will be a while before serological testing becomes useful at a population level.
Our initial data show no false-positives and no false-negatives out of all specimens assayed. However, it is early days still and none of the leading tests have real-world data on false-positive and false-negative rates. The crucial parameter here to compare test performance is limit of detection (LOD). We showed we could detect as few as 10 molecules of virus, which is on par with the best RT-qPCR tests.
Cost is definitely an important consideration for roll-out of a widespread test. We anticipate that the cost will be about $15 per test.
They say they have not had false positives or false negatives. If that's not a lie, then that's not a lie. It does not mean the test is perfect, but it likely means the false result rate is low.
That he was simply making a claim about what their initial data show. That is not a claim to perfection. Indeed, the whole point of saying 'initial' is to leave open the possibility that later data show something different.
Personally, I don't think your interpretation is more charitable. If they think that their initial results possibly aren't correct then launching a business and claiming things like "Extremely Accurate" and "on par or better than other COVID-19 tests available" is fraudulent.
Their initial results may be correct, but it is unlikely they’ve had the chance to do enough of them to pin down a failure rate with any precision. They also claim that other available COVID-19 tests haven’t undergone enough verification to establish a solid error rate, and if that’s so “on par or better” on the evaluations that have been run is fair as well.
Dismissing the value of serological testing seems like a self-serving move.
Such tests are immediately of very high value because they allow us to understand immunity to Covid-19, and to actually validate the estimates of cases amongst those who haven’t sought treatment.
These are both of huge value regardless of the percentage of the US population estimate to have previously contracted Covid-19.
By all means market your test which seems like an awesome contribution, but please don’t do so by devaluing other important tools.
[edit: the parent post has been edited to be less dismissive without acknowledgement since I made this comment]
[edit: looks like I’m wrong about the post being edited. Sorry for that. I stand by everything else I say here:
Serological tests are useful and needed right now, not at some future stage. It’s not hard to google to verify this, and it’s irresponsible to downplay the value of a test we need now.]
Part of the problem with serological testing is false positives. I'm no expert, but my understanding is that the tests available may have false positive rates on the order of 0.5% to 2%
If only 1% are infected, and the false positive rate is 1%, it's quite hard to determine what the test result actually means. On the other hand, some places like New York City are already at 0.7% per-capita known positives, and they're not even able to test everyone with symptoms. Their true infection rate could very well be 10%, which serological testing could confirm.
Yes, which is why a better serological test would be extremely useful, the sooner the better. In no way does this undermine the value of BillionToOne’s test, but it is irresponsible for their CEO to be saying that serological tests are not useful yet when they are.
I don't think we are in any way undervaluing the serological tests. A great serological test would be very useful. However, because of the indirect measurement, they tend not to be very sensitive or specific, so it is fair to say that they become more important not at the peak of the pandemic but at the post-pandemic period.
By the way, as a company, we really don't have much to gain from this test. If it does not get adopted, we'll go back to building our core business which has been growing 100% quarter over quarter. We have poured resources into the current development, because it is the right thing to do. No investor in our Series B round will take into account any non-recurring sell-almost-at-cost revenues that we get from a once-in-a-century pandemic.
For what? "Google for answers" isn't very helpful. It sounds like they're saying serological tests are valuable, but not viable for immediately testing a large population.
Serological tests are useful and needed right now, not at some future stage.
It takes about a month before an antibody test is accurate in someone who has had a virus, so using them now would only detect cases where the person had the virus in early March. That limits the value of serological testing considerably.
We have already generated and put into our scientific manuscript the data that FDA requires for EUA. As a high-complexity CLIA-licensed clinical laboratory, we don't see significant risk in getting the authorization in 2 weeks. However, we need to start working with clinical laboratories across the country today if this is going to scale to hundreds of thousands of tests as soon as we get the EUA. Also, international laboratories don't need EUA and can start using the test immediately. We did not think it would be responsible to hold off on making the technology public when every day is so critical in our response to the current pandemic.
3) is almost right--these sanger sequencing instruments are already widely available across the country for research use. Here in the Bay Area, I can choose from at least 4 different Sanger sequencing services that will run 10,000 samples at $2/sample in 24 hours. For example, see: https://www.mclab.com/DNA-Sequencing-Services.html
Thanks for your efforts! Inspiring to see our broader community spring into action.
I was part of a volunteer team that tested 3400 people on Friday/Saturday in Santa Clara country for COVID-19 antibodies [1]. It took a team of 100+ volunteers 10 hours / day just to collect samples.
Stanford, for example, has plenty of automated testing capacity, and even reagents. IMHO, the limiting factors are not that we need new tests, but rather we need (1) lighter regulations (2) funding to buy supplies and (3) massive manpower to scale-up drive through testing
Sample collection and accessioning (accessioning is unpacking test tubes one by one and aliquoting them into plates in the lab) is definitely going to require a lot of manpower. I'm hopeful that patients "self swabbing" can help alleviate some of the manpower needs. (Self-swabs are not allowed currently under FDA guidance).
My guess is that self-swabbing is allowed in the Seattle SCAN study because it is a research study. The SCAN study is super fascinating because it would be crazy unusual under normal times for a research study to return results back to patients; I'm very happy they are able to do that, and it speaks to the severity of this pandemic.
> The FDA has recently made clear that no at-home self-collection tests are allowed
I think rightfully so. I was on a government video conference where a doctor showed the current CDC testing procedure, which involves stick a swab in the nose all the way to the back of the throat. They explained that the further the sample is taken from the lungs the less accurate it is.
For clarity, a 500-patient study in WA (not yet peer-reviewed) showing self-swabbing to be as effective as swabbing by health care workers has prompted FDA to allow patient to self-swab at clinics, just not at home: https://medcitynews.com/2020/03/fda-says-patients-can-self-a...
> The SCAN study is super fascinating because it would be crazy unusual under normal times for a research study to return results back to patients; I'm very happy they are able to do that, and it speaks to the severity of this pandemic.
The Seattle Flu Study wasn't allowed to communicate back to patients but "By Feb. 25, Dr. Chu and her colleagues could not bear to wait any longer. They began performing coronavirus tests, without government approval. What came back confirmed their worst fear. They quickly had a positive test from a local teenager with no recent travel history." https://www.nytimes.com/2020/03/10/us/coronavirus-testing-de...
I'm glad to hear they're allowed to do the SCAN study!
The situation is similar in WA state, where the current bottleneck is specimen collection, as there is a shortage of swabs, vials, PPE and personnel. This results in large amount of existing lab capacity, such as at UW Virology labs, not being used. Skipping RNA extraction definitely helps with expanding lab capacity in places where that's the current bottleneck, but isn't the most pressing thing in WA.
The other bottleneck in WA is that the state's multimillion dollar, IBM-contracted data collection program can't keep up ingesting a few thousand records a day.
I find it incredible that no one has done these studies before issuing the lock down orders. Without knowing what the proportion of people who have been exposed but aren't sick any response is the wrong response.
If it turns out that it's already spread to most everybody lock downs are just a way to create more homelessness. If it turns out that the only people with the antibodies are the ones in the hospitals we need martial law.
Yes, if you google "coronavirus iceland" you'll see they've tested almost their entire population. They found 1586 confirmed cases, of whom only 6 died, which gives a fatality rate of 0.4%.
Turning this around, if we multiply confirmed deaths by 264, that gives us an estimate of how many cases there are. So, for example, with UK's death count of 6159 this means about 2.4% of the population is infected. Furthermore, on the Diamond Princess only 20% of the people onboard caught the virus under poorly quarantined conditions. So, to extrapolate even further, this would imply that over 12% of the UK population has already been exposed to SARS-CoV-2. This means that the UK should peak at about 50k deaths, without any protective measures.
In 2018, the UK had 50k deaths due to flu in excess of normal flu deaths.
Iceland has only tested 28,992 of its residents as April 6th, so that throws the rest of this conjecture out the window. (Although regardless the conjecture was probably not a great idea.)
Iceland may have the best statistic on their death rate from this virus, but these things are not static and depend on a zillion factors (slight exaggeration).
I think the real point is that your original claim was a small sample that you were extrapolating wildly from turns out to be 10x smaller than you claimed.
Furthermore you are focusing on a tiny portion of available data instead of all that is available.
Given that New York State has 4,000+ deaths your "model" would indicate that 1MM residents have COVID-19? So if it rips through the remaining 19MM residents in the course of a few weeks the result will only be 80k deaths? And of course the healthcare system won't break down?
Also you said that the Diamond Princess only had a 20% infection rate but the Greg Mortimer is reporting a 60% infection rate. Seems like you've got a lot of facts wrong on the first pass, IDK.
It seems that 'all available' would diminish the signal of Iceland's good stat. What we really want is 'all good stats'. I think only South Korea is the other country with extensive testing.
According to my model NYC should only have a max of 13.5k deaths, without any mitigation.
Based on the Greg Mortimer stat, this bumps up to 41k for NYC.
It doesn't seem to pass just a basic level of consistency with present experience to believe that with no mitigation New York would only have 13.5k deaths when they already have 5.5k+ (and we know many more that have not and maybe will never be confirmed).
Even the idea of 41k at this point beggars belief, they are already well over capacity with some mitigation and the bodies are literally piling up.
I find it hard to wrap my head around the logic here.
My thought is since this is a new virus it will spread faster, and hit peak sooner. Also, those who are most affected are a small portion of the whole population, which will again imply faster time to peak.
It isn't about proportion of population, but whether they are only testing people already admitted at the hospital, which will drive the estimate up, or testing the wider population, driving the estimate down. SK is doing the latter through contact tracing.
Bergamo had an 0.4% of it's population die in excess of usual in March and no one believes that 100% of the population was infected. Which would put the UK at 240,000.
From your own link:
"The failings contributed to the worst flu season for seven years, with 15,000 deaths from the virus, around twice the average figure, and the worst NHS performance on record."
So my calculation shows cov2 is about twice as bad as a strong flu, with no preventative measures. If we hit the 20k or less with preventative measures that the Imperial model predicts, this will be on par with regular flu season.
At any rate, much less than the 200k+ deaths originally predicted by the Imperial model and that pushed the UK to lockdown.
These studies were done well in advance. Every bit of research we have about the 1918 Influenza points to social isolation being the only effective way to combat a pandemic.
South Korea shows that test and trace works just as good or better than lockdowns. Which is why our failure to get high testing rates early on is such an utter travesty.
Your comment, unfortunately, won’t be appreciated by lemmings until the unemployment rate is 45% and hundreds of thousands are dying from disease and starvation.
The lock-downs are because we don't have the tests and we don't have these studies, and still won't, for weeks or months to come. They're being run as fast as possible but there's a lot we don't know about this disease. In the meanwhile, people are getting sick and dying.
We can disagree on whether a lockdown causes the least amount of societal distress compared to some of the other options, we can argue on when we would personally chose to enact lockdown. We could even try and work out what it would take to prevent people losing their homes and dying. $2000/month UBI seems like it might help.
But seriously, doing anything at all is wrong? And the only right response is to do nothing? No ordering more PPE, no preparing for a surge, no rebalancing shifts so the contagion doesn't take out the police force/navy/healthcare workers/etc?
Saying that any possible response is wrong seems like pretending the problem will go away if we pretend it doesn't exist. Which is really hard to do while people are dying.
No response is response too. It would be incredible to done such massive studies before antibodies tests existed and organizations were able to do them on mass scale.
What they had at the time were models (which predicted asymptomatic cases) and information on how it looks like in countries that did not done measures soon enough. Even as China lied and made their numbers smaller then the were, enough was known publically and even more by secret services.
So, I can't get to your site, but this seems a little disingenuous as a title. You're not seemingly "unlocking" anything more than a protocol for Sanger sequencing that, if it eventually was approved and actually tested against real samples and people elsewhere adapted the protocols to use for testing, you would then... at that point... unlock a thing.
Thanks. Make him explain whether the machine learning is real too, since his linked manuscript doesn't say anything about it. This is a really exploitative moment to be hyping things.
It looks like David answered that here: https://news.ycombinator.com/item?id=22808525. I agree that it's better not to use buzzwords, and have edited the top text to say "linear regression" instead of "machine learning".
(This is actually part of the standard advice we give to YC startups who are launching on HN: don't talk like you would to investors—talk as you do to fellow implementors.)
Both the methods and David's twitter clearly describe the linear regression as the method and how it works with spike-in. In practice, it is a lot more complicated, as the signal-to-noise ratio can interfere with the regression and primer N-1 synthesis errors need to be accounted for. That is why we are willing to help other laboratories with our bioinformatics pipeline.
The GP wasn't necessarily a helpful comment, but please don't respond by being snarky. It just makes the thread worse for everyone, and you can make your substantive point (the distinction between those kinds of scaling) without it.
actually, scaling a website is much easier than scaling a medical service.
If you can't be bothered to take some basic steps for web site scaling (google page speed + a CDN) then the idea that you will carefully and successfully navigate to a 1 million tests per day moderate to high complexity medical test seems like it will be a stretch no?
That said, a 100% support MUCh higher test capacity, both this and IGg IGm (which should allow for home testing). So good luck!
> If you can't be bothered to take some basic steps for web site scaling (google page speed + a CDN) then the idea that you will carefully and successfully navigate to a 1 million tests per day moderate to high complexity medical test seems like it will be a stretch no?
No. :)
Seriously, these are orthogonal skills. I briefly worked for a communications satellite company that had a pretty godawful web site, but I doubt any of their prospective customers visited it and said, "Wow, if these people can't design a good looking web page, how can we trust them to put a satellite in orbit?"
"If you can't be bothered to take some basic steps for web site scaling (google page speed + a CDN) then the idea that you will carefully and successfully navigate to a 1 million tests per day moderate to high complexity medical test seems like it will be a stretch no?
"
Their priorities may be on other things than websites.
From experience, the clients who can't get some things right struggle to get many things right.
Seriously.
Someone has a great idea and is able to attract good people to work with them? Then even though item X is the focus, you see quality in items Y -> Z. It naturally seems to happen.
One reason that argument doesn't work is the effects of focus. When you're intensely focused on something, momentum builds up that is like a high-speed train. You can't do nimble switching, even to easier things. One might imagine that they'd forget to, say, pay a utility bill as well.
(I have no inside information and am simply guessing based on past experience with unrelated things.)
One million a day seems like a lot until you realize it would take about a year to test everyone in the US... How can we increase this to 10m per day? Is that possible?
I think we can scale it to significantly more than 1M a day, but it would require the relaxation of certain regulations that currently prevent collaboration between clinical laboratories and other institutions that have Sanger instruments. FDA and CMS require that clinical laboratories process the sample end-to-end, so it prevents a distributed model of initial processing in the clinical laboratory followed up with send-out for Sanger sequencing.
California CLIA and FDA regulations for COVID-19 testing have been relaxed significantly over the last few weeks, so the above is perhaps not impossible.
This would be HUGE. Not only has the testing capacity in America been maxed out, but the % of positive results has been more than 20% and trending upward. In order to quarantine effectively (read: end the shut down) we need to test the family, coworkers, and contacts of every positive result.
Why do you state that our testing capacity has been maxed out? My understanding is that factors like the availability of swabs and the availability of medical personnel to administer tests are a bigger bottleneck than qPCR machines.
At least anecdotally, it doesn’t seem that we’re hitting our full testing potential. I think adding in a new technique would only exacerbate the mismatch between test supply and test utilization.
It's plateaued and has been hovering around 100k per day for 2 weeks (outside California clearing its backlog on 4/4).
The White House promised 27M by 3/31 and we were at 1M.
It doesn't look like it's ramping up.
Scaling a static website involves far less technical knowledge than scaling a high politics / moderate complexity scientific process.
The website looks terribly designed from a scaling standpoint. Have you run even a basic google pagespeed on it? Why no cache TTL on the custom TrueType font? Why no CDN backing?
This is a bit random, but if you have a "proud" web developer who "knows best" find someone with no pride who can get this sort of thing done.
We believe google can scale in part because they seem to be able return web search / autocomplate / google assistant responses pretty quickly.
Say I had a lab with the sequencers needed for this test. I can't claim to understand the work needed to onboard into this new process, but my question is: how long does the onboarding take before the lab can process these 3840/day ?
Thanks for the question. Last week, I sent half-a-page of instructions in a hastily written e-mail to a scientist working for the Switzerland government response team, and they told me that it worked beautifully in their first try this week.
With the details described in the manuscript, the onboarding can be only a few days (or even self-onboarding based on instructions). The only part that is somewhat tricky is the bioinformatics algorithm for positive vs. negative calls, which we can supply.
Can you promise that this test will not be mass-produced in China? It would be criminal for the Chinese to gobble up all the profits of a virus that they released on the world.
We are seeing example after example of the power of entrepreneurship and private enterprise. I hope we remember this going forward and weave it into our national culture deeper and wider than it already is.
> Private enterprise as whole is every bit as f'd up as the government.
This is wrong. What's the basis for this comment/idea?
I have founded and run multiple companies. I have worked for small, medium and large (8,000 employee) companies in various capacities. In a 30+ year career I could not name even one company that I would say fits your assertion at all. I am sure they exist. Yet, I am also sure they are an insignificant percentage of the total.
There's a key difference between private enterprise and government: Failure kills.
Again, except for corner cases or companies that exist due to government money (I don't include these in "private enterprise") entrepreneurship, private industry, capitalism if you will, is a survival of the fittest contest. Like it or not, this is reality.
Government operates with completely different metrics. Failure, for the most part, has no real consequences. A simple example of this is the "high speed" train project here in CA. I've lost track at this point. I think the last time I looked they were at $100BN and the whole thing is a massive smelly pile of manure. Nobody, except for taxpayers, will pay for the consequences of that failure...sadly, one of many at the hands of government.
This crisis is highlighting just how messed up things are under government control. Last night my son, who is in university pursuing a degree in CS, said "Dad, do you know COBOL?". When I asked why, he said New Jersey is trying to hire a bunch of COBOL programmers because their payment systems are badly broken, the code is done in COBOL and it is in need of fixing so they can pay people the aid funds the federal government is providing. I mean, this is typical, sad and ridiculous.
No, private enterprise is a universe away from almost anything government touches. The only service I can identify in government that lives (or dies) by similar metrics is the military. They have real and non-trivial consequences for incompetence. Death. And that means they can't run like a state payment system that's 40 years old and grossly outdated. I am sure the military have issues as well, yet, for the most part, given their mission, could not survive in the long run if they did not operate at a certain level of competency.
I don't disagree with you about capitalism being survival of the fittest, or that companies should die. I don't think the US is a capitalist country though. We're living with corporate socialism.
I mean, look at the pandemic bailout. Nobody, except for taxpayers and employees, is paying for the failure of the airlines to operate responsibly. This is exactly the time every badly run company in the US should collapse and die, and it's not gonna happy. Restaurants, bars, small businesses, maybe some startups will crash and burn. But even private equity firms are getting bailed out.
This is not a one off. The same thing happened in 2008.
And it's weird to think only governments are running Cobol. What do you think the vast majority of hospitals are built on? Or things at IBM? It's the same problem.
> I don't think the US is a capitalist country though
Pure capitalism has never existed anywhere. So, yeah, agreed. Yet what we have (not just the US but lots of nations around the world) has elevated society to levels not attainable through any other system we know.
> the failure of the airlines to operate responsibly
I don't think this is fair. Nobody could have predicted or prepared for this. Nobody. I mean, how?
I've read people say things like "We could have stockpiled 50,000 ventilators ten years ago". This is, frankly, silly on many fronts.
Technology being what it is, ten year old ventilators would have likely been 5 year old designs at the time of purchase, which would make them around 15 year old designs today.
The first level concern would be if those ventilators would be adequate or acceptable in today's context. I'll assume they could be. That said, professionals are trained to operate and know the equipment they use every day. If a dinosaur shows-up they might not be able to (or want to) use them because of lack of training that could lead to negative outcomes for patients.
People have this idea that a word, like "ventilator" fully defines a product, its usage and the entirety of the dynamics surrounding it. Things are never that simple.
Another point is equipment mortality. Anyone who has dealt with hardware in large quantities and across extended periods of time knows what can happen with 50,000 units of anything manufactured 10 to 15 years ago and in storage for ten years. You can end-up with massive failure and reliability problems. You would have to design and manufacture these devices to failure-tolerant aerospace/military standards to improve outcomes...and even then you have no guarantees.
Things are not as simple as "ventilators", not even close.
> The same thing happened in 2008
No, this is nothing --not even close-- to 2008. I lived through that period. It was an absolute mess precipitated by politicians loosening the rules on lenders and pushing them to let anyone buy homes. Great for votes but horrible for the economy, as history has proven. You had people making minimum wage buying half million dollar homes because the government (both Democrats and Republicans, BTW) removed all sorts of regulatory requirements on making loans. The banks operated under the law. And the consequences of the law ended-up being the opposite of what politicians sold us. Instead of "the American dream of home ownership" the thing turned in to the world nightmare of unintended consequences. Frankly, every single US Senator and Representative who voted for those demented changes should have lost their jobs and maybe even done some jail time. Sadly most of them are still there.
> it's weird to think only governments are running Cobol.
The issue is one anyone who has built non-trivial software for enterprises is familiar with. Once software "works" there's very little appetite to spend more money to continue working on it, much less migrate it to new tool sets, hardware, etc. It's just one of those things that, in the world of politics at least, doesn't earn you votes, so you ignore it until people are screaming for it because the system crashes and they can't get their checks.
Before the massive boost in throughput imposed by this pandemic these payment systems essentially worked as required, issued checks and payments on time, accurately and maybe even efficiently (per government standards). Anyone proposing to re-write the entire system, spend tens of millions of dollars or more in order to migrate to another language and add capacity would likely be laughed at. It is only in hindsight that these kinds of things can be justified.
This also applies to such things as CDC/NIH testing capabilities, manufacturing dependencies and more. Humanity rarely acts proactively. I can't think of a single example of this at scale in the history of ...
Your site states you use a “proprietary machine learning algorithm”.
As someone who has seen a gamut of companies use the machine learning phrase to justify almost any conceivable product, could you share any non-confidential information about what you achieve using ML?
I really think this explanation of how the number of people in the population who have the disease affects the chances of a positive test being incorrect, it is very unintuitive but look at the video as it explains that even very Sensitive and Specific tests can be problematic to see who in the population has a disease: https://www.instagram.com/tv/B-qZLJoAf3V/
am i reading correctly that you packaged synthetic viral RNA into viral particles and then suspended in transport medium? do you plan on studying actual patient samples and comparing to gold standard tests?
Cool technique that uses existing Sanger sequencing equipment that has been optimized for decades. So you are sanger sequencing the loci + a synthetic frame-shifted oligo that you spike in. Then you essentially "demultiplex" the chromatogram knowing the two molecules are frame-shifted and compare the peaks to get a relative measure.
I guess the trick is to find a loci that is specific enough to covid-19, but can still produce a good frame-shifted oligo that allows you to maximize the resulting chromatogram signal.
I am surprised that the detection limit of a no-extraction PCR can be this sensitive. But it looks like the data checks out. Does this work for every type of sample? I would assume differing buffers, collection methods will influence the PCR?
246 comments
[ 4.0 ms ] story [ 245 ms ] threadEdit: Since our site seems to be overwhelmed at the moment, here's a recap:
We’ve been working hard at BillionToOne on a new COVID-19 test that scales testing to everyone in the US. Our test (1) re-purposes existing infrastructure, (2) eliminates time-consuming RNA extraction, and (3) enables a distributed system for COVID-19 testing.
We need 1 million tests per day to end the stay-at-home orders. Schools are still open in Iceland because they test 15x more than the US does, per capita (https://www.washingtonpost.com/world/2020/04/02/free-coronav...).
The first thing we figured out is how to run COVID-19 tests on existing automated Sanger sequencers. One sequencer can process up to 3840 samples per day. There are hundreds of sequencers of excess capacity because they were built for the Human Genome Project over 20 years ago.
It would take only 2 sequencers to surpass the current test capacity for all of California. There are far more than 2 sequencers in California (some individual labs have 10 or more).
We tweaked the protocol so COVID-19 could be detected from sequencing data using linear regression. Basically, we add ~100 copies of a known DNA sequence to help us calculate how much virus nucleic acid is in the specimen. It works just as well as gold-standard RT-qPCR.
Lab workflow for COVID-19 testing is traditionally 1. Specimen accessioning, 2. RNA extraction, 3. RT-qPCR 4. Reporting. RNA extraction, in particular, has been a huge bottleneck in terms of reagent shortages and labor-intensiveness.
We showed that we can skip RNA extraction entirely without affecting test sensitivity and limit of detection.
By skipping RNA extraction and using automated Sanger sequencers, we think we can get to an additional 200,000 samples per day test capacity in existing clinical labs.
A distributed system is often the only way to operate at massive scale. A fully distributed system could have different sites and labs responsible for each process and dynamically re-allocate resources based on availability and capacity.
The Broad institute COVID-19 lab has already started doing this. They are asking for specimens to be submitted in a standardized tube format and pre-barcoded. They have essentially distributed the specimen accessioning work.
Because there is a highly developed service industry for Sanger sequencing with <24 hour turnaround, there is an opportunity to further scale up testing by distributing the work to their (currently) idle sequencers.
Distributed testing could scale from 200k to >1 million tests per day, but would require a change in regulations that currently prohibit it.
Thanks to the BillionToOne team for pulling this work together! Next step is to start manufacturing test kits and obtain Emergency Use Authorization from the FDA. We’re eager to work with clinical Lab Directors and contract kit manufacturers.
Edit 2: Link to scientific manuscript: https://www.dropbox.com/s/07esyehsvfpmllc/A%20Highly%20Scala...
The core of our machine learning is Ax=b :grins:
More seriously, the main reason why traditional sanger sequencing can't be used for COVID-19 testing is because it would be unclear whether a lack of signal is truly due to lack of virus, or if it is just because the assay failed (happens all the time!)
What we've done is introduce a reference sequencing signal that is biochemically very similar to viral RNA, but produces a distinct vector of electrical signals that is different from the signals emitted by viral RNA. Since we know what both the reference and viral signals look like, we can perform linear regression analysis to fit the linear combination of viral and reference signals that best match our data.
I wouldn't make a major edit without letting them know. In the present case, I just let the whole community know, because the edit was a substantial one. Had it been a trivial one, I wouldn't have bothered. The priority is interesting discussion, not punctilious bookkeeping, and you can't necessarily have both.
In the flurry of these threads, the founders are focused on responding to substantive questions. That's what they should be doing: satisfying community curiosity in their area of expertise. It's my job, not theirs, to prevent the discussion from getting derailed by things like the unfortuitous use of a buzzword which they have no clue the community has been tired of hearing for years already.
If the users weren't YC founders that we have an explicit coaching relationship with, I'd always consult with them before offering help. That would happen by email. By the way, if anyone wants this sort of coaching, we're happy to help as much as we can via hn@ycombinator.com. I give this sort of advice to startup founders, project creators, and article authors all the time, and everyone is welcome to it. Just don't worry if you don't hear back right away. The inbox has gotten to the point where it piles up regularly.
Thanks Dang!
ETA: never mind, ~dvdt signed off on the change. Appreciate the collaboration!
Does this mean you assume a linear relationship between the quantity of viral RNA and the strength of the signal?
I know that when back i used to draw calibration curves from my positive controls, there was usually a sublinear relationship, across all sorts of different assays, at least at the upper end.
The machine learning angle isn't the exciting part here, it's all the rest. Great idea!
What are your plans for licensing the technology?
If they need our bioinformatics automation & help with set-up, we would license the method for COVID-19 testing for $3-$5/sample as part of each sample that is being put through our pipeline.
If they ask for 96-well plates with all reagents that are ready to use (so that they just need to add VTM), we would work with manufacturers to produce the reactions and plates, and the price of kit (~$15 per test) would include limited license to use our automated bioinformatics calling pipeline.
You said in an earlier comment that the reimbursement for testing is too low to justify buying expensive equipment. You are also proposing to charge half the reimbursed rate for it to run on someone else's equipment.
Are the current equipment owners expected to donate this crucial equipment, because if they are the bottleneck, shouldn't they be the ones compensated to encourage more equipment to be made available?
The current equipment owners are already the clinical laboratories. It is unused capacity for them. Other owners are sequencing service providers. The full cost of running an end-to-end Sanger reaction as a provided service is $2-$6, so at the $50 reimbursement price, the laboratories will still be incentivized.
I had a few questions though:
How do you compare this test to the Abbott machines? Obviously that test is faster, but how does that impact what we can do with it?
For 1m/day to be sufficient, do we need contact tracing programs to be able to find everybody who needs to be tested? How hard will it be to scale these programs?
I think that where the Abbott machines might hit a wall is that they are one at a time, and they require Abbott's consumable test cartridge and device to run (think printer ink / printer). I don't have any firsthand knowledge, but I would anticipate that it is difficult to scale-up manufacturing of the devices rapidly enough to keep pace with the pandemic growth.
We absolutely need contact tracing to find everybody who needs to be tested. We're not working on scaling up contact tracing, but I think several people in the tech community are working on making that easier to perform at scale.
Whether this is the direction the company wants to spend their resources is another story.
By the way, this robustness is completely expected, as any impurities in VTM would impact spike-in and endogenous viral amplification equally for end-point PCR (so their ratio stays the. same). This is not necessarily true for qPCR where an impurity (caused by lack of RNA extraction) can potentially cause a positive sample look like a negative when the viral RNA does not RT-PCR.
The way currently available COVID-19 testing works is by detection of viral RNA. Since the amount of viral RNA in a patient sample is too low to detect directly, we first need to amplify it by PCR. However, this viral RNA is packaged within all sorts of proteins and lipids that could make it inaccessible to amplification unless they are first purified away. Furthermore, the sample is shipped in "viral transport medium", which is essentially a cocktail of chemicals designed to preserve the virus. Unfortunately, these preservatives often have the side effect of interfering with PCR amplification, so these too need to be purified from the sample.
However, since RNA extraction is usually the most laborious part of the assay, there has been a lot of interest in optimizing the amplification so that it is resilient to all of these impurities. The preprint referenced in our manuscript (https://www.biorxiv.org/content/10.1101/2020.03.20.001008v1) gave us the initial idea that this could be possible, and much of it comes down to the choice of amplification method (e.g. choice of enzymes and buffers) that you choose.
However, even when you choose a "good" enzyme and buffer, you will still suffer an amplification penalty, and this will cause you to return a false-negative on some affected samples because there was so little virus in the sample to begin with. The innovation we have is to spike-in a correspondingly low level of DNA to the reaction mixture. That way, if you see the low level of DNA without seeing any viral signal, you can be assured that the amplification still worked and that there truly is no virus in the sample.
How many tests have you run on patient samples?
Also, in the UK our independent and uni labs have been saying for almost a week they could extract the RNA differently but the NHS have a fixed approved way that they won't change.
- Are the chemicals you're using more common or would there just be a new shortage of different chemicals?
- Is there a risk you'd be creating a test that didn't work very well, and the US would end up with a bunch of useless tests (e.g. Italy had to abandon a bunch of Chinese sourced tests, UK's anti-body tests are ineffective)?
Purely speculative, but I think if swabs remain an issue for too long, alternatives could start coming online, such as even using qtips + saline (no idea if it works, it's just an example). The current swab + Universal / Viral Transport Medium combo is optimized for flexibility; it is designed to work across a very broad range of viruses and bacteria that have different viral loads and shedding characteristics. The current pandemic is pretty much COVID-19 only, so I think it's a priori feasible that a specimen collection procedure can be found that uses common materials. We did try early on to see if saline or other buffers affected the performance of the assay, and it worked fine in those conditions.
We use fairly standard chemicals. I haven't heard from our suppliers about shortages for the chemicals we use. Chemicals and enzymes tend to be relatively fast to scale up for bulk manufacturing.
There's always manufacturing risk that a product will not work as expected. In fact, the first COVID-19 test developed by the CDC did not work as expected, and this delayed testing by several weeks. We de-risk this as much as possible by performing experiments as early as possible, akin to the fail fast mentality of checking for the highest risk failure modes first. Since we don't have a national healthcare system in the US, the manufacturer takes on the vast majority of the risk of a defective product.
Let’s say it’s 50/50 whether it lives 24h without help. That’s would be a pretty bad false negative rate for your test, but a 50/50 of potentially getting infected by your mail is pretty high.
Even after ~6 half-lives, remaining 1% of viral load is still potentially dangerous, but it's not a good basis for a test if you want it to be sensitive.
¹ Inactive does not mean destroyed. It may be possible or even easier to detect a partially decomposed virus, even with the current tests. Or not.
https://formlabs.com/covid-19-response/covid-test-swabs/
https://www.plasticstoday.com/medical/formlabs-3d-printing-m...
https://formlabs.com/covid-19-response/
I'm not associated with the company, I just own some of their printers. They've also got some 2000+ volunteers who own printers or have CAD expertise signed up and looking for ways to contribute. Apparently we can't make medically-approved swabs (most of us aren't ISO 13485-certified or FDA-registered), but there's other stuff (e.g. hands-free doorknobs). I'm even contemplating shipping one of my printers back to them to help in the effort.
According to an older scientist, Anders Fomsgaard at the Danish Serum Institute, this is how they did it ”in the old days”. He is the father of one of the authors.
This eliminates supply chain problems for reagents and was shared quickly to help in Spain.
Preprint by Fomsgaard and Rosenstierne:
https://www.medrxiv.org/content/10.1101/2020.03.27.20044495v...
https://diagnostics.roche.com/us/en/news-listing/2020/roche-...
That said, it is easier to ship the test kits than scale the instrumentation. Both Roche and Abbott still need to build hundreds of their instruments before the kits that they are shipping out this week can be used on the daily rate that they are trying to get to. I am not sure with Roche, but Abbott estimates the end of June to have enough machines shipped to achieve 50K per day capacity on their instruments.
Another potential problem with new instrumentation is that reimbursement for COVID-19 tests is very low ($30-$50), so it becomes financially difficult for hospitals and laboratories to buy very expensive instruments and also pay for test kits that cost $30-$50 per test, on par with reimbursement.
We try to avoid both issues by utilizing a currently unused Sanger install base and low-cost reagents.
Good thought! Thank you.
Additional question: are Sanger instruments also very sophisticated and manufactured by those limited few pharmaceutical companies/medical equipment manufacturers? Or is there a need or necessary to scale up the production for Sanger instruments too? I am asking that is because lack of testing is a global issue, not only in the U.S. Africa/India together has 2 billion people are the testing hurdles there are even more challenging.
Thank you for doing something about the pandemic. It's quite heroic.
In figure 1A, the workflow includes a standard PCR step before Sanger. Workflow-wise, wouldn't it still have the same bottleneck as qPCR test, i.e. limited by 96/384-well instrument runs?
Most clinical laboratories would have 10 to 50 PCR instruments that they can use to run the initial amplification reaction in parallel before Sanger sequencing. Also, Sanger sequencing uses a plate feeder, so you can add new plates on top as the second round of PCR reactions finish.
But, more importantly, the qSanger can by-pass RNA extraction, which seems to be an important bottleneck in the RT-qPCR workflow.
"DIRECT RT-qPCR DETECTION OF SARS-CoV-2 RNA FROM PATIENT NASOPHARYNGEAL SWABS WITHOUT AN RNA EXTRACTION STEP" https://www.biorxiv.org/content/10.1101/2020.03.20.001008v2
Can you please comment on the actual realistic turnaround time for the test?
If people can sort out logistics well enough then it sounds like it could be extremely helpful for implementing broad scale testing.
“Most qPCR instruments have low throughput (they can run 1- 48 samples at a time) and they all compete for the same reagents.”
Is it more accurate to say that FDA-approved qPCR machines can run up to 48 samples at a time?
Most qPCR machines can handle 96 samples at a time, and there are versions that go as high as 384 at a time.
The FDA has only approved qPCR machines that go up to 48, correct? Do you know why they’re not running on the more parallel machines?
Thanks!
Do they detect antibodies?
What is the false negative rate for nasal swabs vs anal swabs?
See: https://www.medrxiv.org/content/10.1101/2020.03.26.20039438v...
We are approaching our peak in 3-4 weeks and with current testing capacity totally fucked.
I know many people in Government and on official Covid response team to get the ball rolling if this is a real possibility.
Given that a Sanger sequences does sequence all kinds of mRNA, would you also find other RNA-based viruses like the flu? How much modification would be necessary to diagnose all kinds of RNA viruses?
- Is this test to see if someone currently has it, or if they have the anti-bodies and are (presumably) immune?
- What are the false-positive/false-negative rates? How does this compare to current leading tests?
- What's the cost per test? How does this compare to current leading tests?
This is a test to see if someone has a current COVID-19 infection. The antibody tests (serological) tests are also important, but since it is estimated that only ~1% of the US has previously contracted COVID-19, it will be a while before serological testing becomes useful at a population level.
Our initial data show no false-positives and no false-negatives out of all specimens assayed. However, it is early days still and none of the leading tests have real-world data on false-positive and false-negative rates. The crucial parameter here to compare test performance is limit of detection (LOD). We showed we could detect as few as 10 molecules of virus, which is on par with the best RT-qPCR tests.
Cost is definitely an important consideration for roll-out of a widespread test. We anticipate that the cost will be about $15 per test.
Your test is perfect? Color me skeptical.
"Have curious conversation; don't cross-examine."
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
https://news.ycombinator.com/newsguidelines.html
> Our initial data show no false-positives and no false-negatives out of all specimens assayed.
Is:
> the false result rate [of the test] is low
While you originally responded to the statement from the OP with:
> [You're saying that] your test is perfect? Color me skeptical.
Such tests are immediately of very high value because they allow us to understand immunity to Covid-19, and to actually validate the estimates of cases amongst those who haven’t sought treatment.
These are both of huge value regardless of the percentage of the US population estimate to have previously contracted Covid-19.
By all means market your test which seems like an awesome contribution, but please don’t do so by devaluing other important tools.
[edit: the parent post has been edited to be less dismissive without acknowledgement since I made this comment]
[edit: looks like I’m wrong about the post being edited. Sorry for that. I stand by everything else I say here:
Serological tests are useful and needed right now, not at some future stage. It’s not hard to google to verify this, and it’s irresponsible to downplay the value of a test we need now.]
If only 1% are infected, and the false positive rate is 1%, it's quite hard to determine what the test result actually means. On the other hand, some places like New York City are already at 0.7% per-capita known positives, and they're not even able to test everyone with symptoms. Their true infection rate could very well be 10%, which serological testing could confirm.
By the way, as a company, we really don't have much to gain from this test. If it does not get adopted, we'll go back to building our core business which has been growing 100% quarter over quarter. We have poured resources into the current development, because it is the right thing to do. No investor in our Series B round will take into account any non-recurring sell-almost-at-cost revenues that we get from a once-in-a-century pandemic.
This is not so. They are valuable right now.
https://www.newscientist.com/article/mg24532754-600-can-you-...
https://www.ksbw.com/article/new-study-investigates-californ...
But I’ll accept that I am mistaken about that.
It takes about a month before an antibody test is accurate in someone who has had a virus, so using them now would only detect cases where the person had the virus in early March. That limits the value of serological testing considerably.
This is only true for the IgG class of antibodies; we typically see the IgM class of antibodies arise within 5-7 days of infection.
You are generally correct, however, in stating that antibody tests have a window in which they cannot detect a new infection.
We have already generated and put into our scientific manuscript the data that FDA requires for EUA. As a high-complexity CLIA-licensed clinical laboratory, we don't see significant risk in getting the authorization in 2 weeks. However, we need to start working with clinical laboratories across the country today if this is going to scale to hundreds of thousands of tests as soon as we get the EUA. Also, international laboratories don't need EUA and can start using the test immediately. We did not think it would be responsible to hold off on making the technology public when every day is so critical in our response to the current pandemic.
Edit, is the idea (for the non-expert):
1) Repurposing idle gene sequencing equipment left over from Human Genome project
2) Reducing / removing the step of having to extract the RNA of the virus as the marker
3) Making these tests / machines available widely across the country so that the delay to getting a result is minimized ?
3) is almost right--these sanger sequencing instruments are already widely available across the country for research use. Here in the Bay Area, I can choose from at least 4 different Sanger sequencing services that will run 10,000 samples at $2/sample in 24 hours. For example, see: https://www.mclab.com/DNA-Sequencing-Services.html
I was part of a volunteer team that tested 3400 people on Friday/Saturday in Santa Clara country for COVID-19 antibodies [1]. It took a team of 100+ volunteers 10 hours / day just to collect samples.
Stanford, for example, has plenty of automated testing capacity, and even reagents. IMHO, the limiting factors are not that we need new tests, but rather we need (1) lighter regulations (2) funding to buy supplies and (3) massive manpower to scale-up drive through testing
[1] https://www.stanforddaily.com/2020/04/04/stanford-researcher...
Sample collection and accessioning (accessioning is unpacking test tubes one by one and aliquoting them into plates in the lab) is definitely going to require a lot of manpower. I'm hopeful that patients "self swabbing" can help alleviate some of the manpower needs. (Self-swabs are not allowed currently under FDA guidance).
Is self-swabbing allowed there because it's for research purpose, not clinical purpose? The program does tell users whether they test positive though.
The FDA has recently made clear that no at-home self-collection tests are allowed (for now): https://www.fda.gov/medical-devices/emergency-situations-med...
I think rightfully so. I was on a government video conference where a doctor showed the current CDC testing procedure, which involves stick a swab in the nose all the way to the back of the throat. They explained that the further the sample is taken from the lungs the less accurate it is.
The Seattle Flu Study wasn't allowed to communicate back to patients but "By Feb. 25, Dr. Chu and her colleagues could not bear to wait any longer. They began performing coronavirus tests, without government approval. What came back confirmed their worst fear. They quickly had a positive test from a local teenager with no recent travel history." https://www.nytimes.com/2020/03/10/us/coronavirus-testing-de...
I'm glad to hear they're allowed to do the SCAN study!
If it turns out that it's already spread to most everybody lock downs are just a way to create more homelessness. If it turns out that the only people with the antibodies are the ones in the hospitals we need martial law.
Turning this around, if we multiply confirmed deaths by 264, that gives us an estimate of how many cases there are. So, for example, with UK's death count of 6159 this means about 2.4% of the population is infected. Furthermore, on the Diamond Princess only 20% of the people onboard caught the virus under poorly quarantined conditions. So, to extrapolate even further, this would imply that over 12% of the UK population has already been exposed to SARS-CoV-2. This means that the UK should peak at about 50k deaths, without any protective measures.
In 2018, the UK had 50k deaths due to flu in excess of normal flu deaths.
https://www.telegraph.co.uk/news/2018/11/30/winter-deaths-hi...
https://covid.is/data
Furthermore you are focusing on a tiny portion of available data instead of all that is available.
Given that New York State has 4,000+ deaths your "model" would indicate that 1MM residents have COVID-19? So if it rips through the remaining 19MM residents in the course of a few weeks the result will only be 80k deaths? And of course the healthcare system won't break down?
Also you said that the Diamond Princess only had a 20% infection rate but the Greg Mortimer is reporting a 60% infection rate. Seems like you've got a lot of facts wrong on the first pass, IDK.
According to my model NYC should only have a max of 13.5k deaths, without any mitigation.
Based on the Greg Mortimer stat, this bumps up to 41k for NYC.
Even the idea of 41k at this point beggars belief, they are already well over capacity with some mitigation and the bodies are literally piling up.
I find it hard to wrap my head around the logic here.
South Korea has tested a tiny fraction of their total population, because the testing is largely targeted via contact tracing.
Not 50,000.
https://assets.publishing.service.gov.uk/government/uploads/...
So my calculation shows cov2 is about twice as bad as a strong flu, with no preventative measures. If we hit the 20k or less with preventative measures that the Imperial model predicts, this will be on par with regular flu season.
At any rate, much less than the 200k+ deaths originally predicted by the Imperial model and that pushed the UK to lockdown.
We can disagree on whether a lockdown causes the least amount of societal distress compared to some of the other options, we can argue on when we would personally chose to enact lockdown. We could even try and work out what it would take to prevent people losing their homes and dying. $2000/month UBI seems like it might help.
But seriously, doing anything at all is wrong? And the only right response is to do nothing? No ordering more PPE, no preparing for a surge, no rebalancing shifts so the contagion doesn't take out the police force/navy/healthcare workers/etc?
Saying that any possible response is wrong seems like pretending the problem will go away if we pretend it doesn't exist. Which is really hard to do while people are dying.
What they had at the time were models (which predicted asymptomatic cases) and information on how it looks like in countries that did not done measures soon enough. Even as China lied and made their numbers smaller then the were, enough was known publically and even more by secret services.
https://web.archive.org/web/20200407224229/https://www.billi...
(This is actually part of the standard advice we give to YC startups who are launching on HN: don't talk like you would to investors—talk as you do to fellow implementors.)
It's not a good first impression (especially promising scale) not be to able to see it.
If you can't be bothered to take some basic steps for web site scaling (google page speed + a CDN) then the idea that you will carefully and successfully navigate to a 1 million tests per day moderate to high complexity medical test seems like it will be a stretch no?
That said, a 100% support MUCh higher test capacity, both this and IGg IGm (which should allow for home testing). So good luck!
No. :)
Seriously, these are orthogonal skills. I briefly worked for a communications satellite company that had a pretty godawful web site, but I doubt any of their prospective customers visited it and said, "Wow, if these people can't design a good looking web page, how can we trust them to put a satellite in orbit?"
Their priorities may be on other things than websites.
Seriously.
Someone has a great idea and is able to attract good people to work with them? Then even though item X is the focus, you see quality in items Y -> Z. It naturally seems to happen.
I'd love to know WHY this is the case.
(I have no inside information and am simply guessing based on past experience with unrelated things.)
One million a day seems like a lot until you realize it would take about a year to test everyone in the US... How can we increase this to 10m per day? Is that possible?
California CLIA and FDA regulations for COVID-19 testing have been relaxed significantly over the last few weeks, so the above is perhaps not impossible.
https://docs.google.com/spreadsheets/d/1Y9_ZrMBhhVLg2xQHtBCc...
https://www.nature.com/articles/d41586-020-01068-3
The website looks terribly designed from a scaling standpoint. Have you run even a basic google pagespeed on it? Why no cache TTL on the custom TrueType font? Why no CDN backing?
This is a bit random, but if you have a "proud" web developer who "knows best" find someone with no pride who can get this sort of thing done.
We believe google can scale in part because they seem to be able return web search / autocomplate / google assistant responses pretty quickly.
With the details described in the manuscript, the onboarding can be only a few days (or even self-onboarding based on instructions). The only part that is somewhat tricky is the bioinformatics algorithm for positive vs. negative calls, which we can supply.
This is wrong. What's the basis for this comment/idea?
I have founded and run multiple companies. I have worked for small, medium and large (8,000 employee) companies in various capacities. In a 30+ year career I could not name even one company that I would say fits your assertion at all. I am sure they exist. Yet, I am also sure they are an insignificant percentage of the total.
There's a key difference between private enterprise and government: Failure kills.
Again, except for corner cases or companies that exist due to government money (I don't include these in "private enterprise") entrepreneurship, private industry, capitalism if you will, is a survival of the fittest contest. Like it or not, this is reality.
Government operates with completely different metrics. Failure, for the most part, has no real consequences. A simple example of this is the "high speed" train project here in CA. I've lost track at this point. I think the last time I looked they were at $100BN and the whole thing is a massive smelly pile of manure. Nobody, except for taxpayers, will pay for the consequences of that failure...sadly, one of many at the hands of government.
This crisis is highlighting just how messed up things are under government control. Last night my son, who is in university pursuing a degree in CS, said "Dad, do you know COBOL?". When I asked why, he said New Jersey is trying to hire a bunch of COBOL programmers because their payment systems are badly broken, the code is done in COBOL and it is in need of fixing so they can pay people the aid funds the federal government is providing. I mean, this is typical, sad and ridiculous.
No, private enterprise is a universe away from almost anything government touches. The only service I can identify in government that lives (or dies) by similar metrics is the military. They have real and non-trivial consequences for incompetence. Death. And that means they can't run like a state payment system that's 40 years old and grossly outdated. I am sure the military have issues as well, yet, for the most part, given their mission, could not survive in the long run if they did not operate at a certain level of competency.
I don't disagree with you about capitalism being survival of the fittest, or that companies should die. I don't think the US is a capitalist country though. We're living with corporate socialism.
I mean, look at the pandemic bailout. Nobody, except for taxpayers and employees, is paying for the failure of the airlines to operate responsibly. This is exactly the time every badly run company in the US should collapse and die, and it's not gonna happy. Restaurants, bars, small businesses, maybe some startups will crash and burn. But even private equity firms are getting bailed out.
This is not a one off. The same thing happened in 2008.
And it's weird to think only governments are running Cobol. What do you think the vast majority of hospitals are built on? Or things at IBM? It's the same problem.
Pure capitalism has never existed anywhere. So, yeah, agreed. Yet what we have (not just the US but lots of nations around the world) has elevated society to levels not attainable through any other system we know.
> the failure of the airlines to operate responsibly
I don't think this is fair. Nobody could have predicted or prepared for this. Nobody. I mean, how?
I've read people say things like "We could have stockpiled 50,000 ventilators ten years ago". This is, frankly, silly on many fronts.
Technology being what it is, ten year old ventilators would have likely been 5 year old designs at the time of purchase, which would make them around 15 year old designs today.
The first level concern would be if those ventilators would be adequate or acceptable in today's context. I'll assume they could be. That said, professionals are trained to operate and know the equipment they use every day. If a dinosaur shows-up they might not be able to (or want to) use them because of lack of training that could lead to negative outcomes for patients.
People have this idea that a word, like "ventilator" fully defines a product, its usage and the entirety of the dynamics surrounding it. Things are never that simple.
Another point is equipment mortality. Anyone who has dealt with hardware in large quantities and across extended periods of time knows what can happen with 50,000 units of anything manufactured 10 to 15 years ago and in storage for ten years. You can end-up with massive failure and reliability problems. You would have to design and manufacture these devices to failure-tolerant aerospace/military standards to improve outcomes...and even then you have no guarantees.
Things are not as simple as "ventilators", not even close.
> The same thing happened in 2008
No, this is nothing --not even close-- to 2008. I lived through that period. It was an absolute mess precipitated by politicians loosening the rules on lenders and pushing them to let anyone buy homes. Great for votes but horrible for the economy, as history has proven. You had people making minimum wage buying half million dollar homes because the government (both Democrats and Republicans, BTW) removed all sorts of regulatory requirements on making loans. The banks operated under the law. And the consequences of the law ended-up being the opposite of what politicians sold us. Instead of "the American dream of home ownership" the thing turned in to the world nightmare of unintended consequences. Frankly, every single US Senator and Representative who voted for those demented changes should have lost their jobs and maybe even done some jail time. Sadly most of them are still there.
> it's weird to think only governments are running Cobol.
The issue is one anyone who has built non-trivial software for enterprises is familiar with. Once software "works" there's very little appetite to spend more money to continue working on it, much less migrate it to new tool sets, hardware, etc. It's just one of those things that, in the world of politics at least, doesn't earn you votes, so you ignore it until people are screaming for it because the system crashes and they can't get their checks.
Before the massive boost in throughput imposed by this pandemic these payment systems essentially worked as required, issued checks and payments on time, accurately and maybe even efficiently (per government standards). Anyone proposing to re-write the entire system, spend tens of millions of dollars or more in order to migrate to another language and add capacity would likely be laughed at. It is only in hindsight that these kinds of things can be justified.
This also applies to such things as CDC/NIH testing capabilities, manufacturing dependencies and more. Humanity rarely acts proactively. I can't think of a single example of this at scale in the history of ...
Don't miss the off-topic collapsed subthread, if you like that kind of thing.
Spreadsheet to play with is here: https://docs.google.com/spreadsheets/d/1IPcX2JYt9mXdaTvj-3mh...
I guess the trick is to find a loci that is specific enough to covid-19, but can still produce a good frame-shifted oligo that allows you to maximize the resulting chromatogram signal.
I am surprised that the detection limit of a no-extraction PCR can be this sensitive. But it looks like the data checks out. Does this work for every type of sample? I would assume differing buffers, collection methods will influence the PCR?