The great thing is that their dnas were shockingly similar. It means the sequncing is valid. Now the ancestry composition thing is known to be a nonstandard procedure , just look at their research publications.
I'm imagining some kind of machine-learning training set based algorithm for determining the ancestry percentages. Even at 99.6% difference (due to mutations over time or sampling error) over 700k samples there's gonna be ~300 sequencing differences between twins. Enough to make their ML system draw different inferences for sure.
23andme has some publication about their method. the thing is , they have a lot more data than what is available in public repositories so they basically established their own system.
it is. and tbh ancestry is the least interesting part for me, since i more or less know my ancestry . however, the very distant cousins that i have found in US and CA are very excited to know where they came from. This must be a mostly american thing.
Part of the old-school American tradition many regions (mostly rural, which is where over 80% of residents lived at one time) was to have a bunch of kids and encourage them to spread out as far & wide as they wanted; this still continues in some places. The logistics of 'keeping in touch with family' have some pretty hard limits past a certain point, and recordkeeping can be haphazard/fragile.
A great many people in North America have significant trouble tracing their ancestry back past their great-grandparents.
Being off by a few percent from nothing more than a cheek swab doesn't seem like a sham. I'm sure if they got a lot more sample to test from it could be made more accurate.
Have you looked at things like thermometers? They don't offer more than +- 2F accuracy for those under $100. Some meters, etc only offer +- 5% accuracy or worse. Those aren't shams.
In the article, one service reports 61% Balkan and another reports 23%. That one algorithm is mostly accurate within itself is hardly notable. That they're not close with one another means that one of them is way off or the industry is very inaccurate. If the latter is correct yet they're purported to be reasonably accurate, it's a sham.
Sure, but does it account for a large percentage change? If their reference data is so limited that adding to it can result in monumental changes, then it is currently inaccurate and a sham. If it isn't a large percentage change then lack of refinement doesn't adequately explain the significant variance across companies.
That reminds me I read somewhere that ovens in residential kitchens are within 20 degrees F of the stated temperature, either because of the position in the duty cycle for the heating element or noise or bias in the temperature measurement. But no one decries ovens as shams.
In my opinion, yes. It's a relatively cheap entertainment genetic game. For me, it's akin to playing the lottery or getting an astrologer to read your aura.
Yeah , i mean they have advertising, and people are interested in it. My wife wanted to get one and i was like WIFE, we know where you come from. Your dad was born in Mexico and your mom has some spanish roots. I aint payin any money for this garbage. For all we know they just have some empirical machine learning model they came up with. Or for the public, groundbreaking AI techniques to trace your roots with pinpoint accuracy all for one low fee
> i was like WIFE, we know where you come from. Your dad was born in Mexico and your mom has some spanish roots. I aint payin any money for this garbage
Could have been worse- you could have said "female, we know..."
Besides the needlessly dismissive tone, you are neglecting how know where the family came from is important to some people
I have a rule of thumb about text on the internet: "Always assume the best possible tone (or interpretation) of text you read, and the worst possible for text you write."
You don't need twins for this test. A single person can just sign up twice for these services and get different results, even if they make a single sample and split it between the two vials.
I thought it was pretty well known that these services are estimates only to a large degree? These services only take samples, they don't sequence the entire genome. Of course there will be errors when you have to extrapolate from a sample.
Even for me personally, my results changed when my parents did it and linked up to me, because their results were able to flow into mine.
If you sign up twice, won't they match your account and merge the results specifically to avoid this problem?
I think part of why they used twins is to say its two different people so they'd have to treat it different.
I'm actually curious how repeatable the results are. If they are using sampling of portions of the DNA (are they?) how much variability is there in a single dna sampling enun through the algorithm 10 times? Similarly for how much variability in 10 dna samples from the same person?
> If you sign up twice, won't they match your account and merge the results specifically to avoid this problem?
Not if you sign-up with someone else's identity and pay with a different card. I very much doubt they go to any effort to verify the human the sample came from is the same human who paid and who the results are set to be sent to.
Or you could have a friend buy it then reimburse them. Or buy and swap a few times to see if it affects the results. Would be easier to test the theory if it wasn’t so expensive.
Yeah, I think most people would be surprised to learn that the same blood tests done by different labs can sometimes show 10% difference in values or even more. This could be the difference between a doctor declaring someone "within the normal range" or having a health problem.
"Not perfect" is the hand-waving way of excusing the imperfections and proceeding as though it were nearly perfect
"doesn't provide the same results twice" is the honest acknowledgement that the concept of perfection is itself what is flawed here because it creates false expectations of what science can give us in this arena
No no no, "science" can definitely deliver repeatable and accurate DNA sequencing, this much is beyond question. But that's expensive, and these companies sacrifice accuracy/repeatability to make it cheap.
The accuracy of these tests at these prices is a falsehood that science cannot support
The accuracy of the broad conclusions drawn from population wide samples about any single individual’s ethno-historical origins is a falsehood that science cannot ethically or logically support
I tend to agree, and since people are convicted and incarcerated on "DNA evidence," this seems an appalling and horrific indictment on the fairness of our court systems.
Comparing two DNA samples for identification of a culprit is totally different from determining the percentage of differing heritages that make up your DNA.
Frankly I am skeptical that they could even ascertain any semblance of your heritage but I'm definitely a layperson.
Really? Taking a pregnancy test more than once and getting a second opinion about a medical diagnosis are certainly both well established in public consciousness.
Maybe so but the kind of genotyping test that we are talking about is vastly more accurate than an over-the-counter pregnancy test, both in the mind of the public and in actual fact.
It doesn't misunderstand them. It knows exactly what it's doing. It relies on judges and juries misunderstanding science and statistics in order to get results favorable to the prosecutors.
Basically all forensic science has turned out to be crap or misapplication of good science except for DNA which a) exonerates a heck of a lot of people and b) was a concept that matured in biology before being adopted by forensic scientists.
That will reduce the random error, yes. But there's still the concern of systematic error, which has to be dealt with separately. If you're taking a biased sample to begin with, it doesn't matter how many times you repeat the test; the bias will still be there.
But wouldn't systematic error affect everyone's results in the same way, meaning it couldn't result in any particular pair of people becoming more or less likely to be matched as the same person?
DNA fingerprinting is so different from whole-genome characterization that it's hardly worth comparing them.
DNA fingerprinting involves:
* Sequence amplification of polymorphic sites by a panel of well-characterized primers, many of which cross-check each other by amplifying over the same locus.
* These amplicons are then characterized by size via restriction digest and simple electrophoresis. No sequencing is performed, only the length of the DNA fragments is characterized.
* Due to population-level variation of short-tandem repeats, analysis of ~20 loci can uniquely identify an individual with a desired level of statistical confidence. The kind of data used to confirm a match in e.g. a murder trial will be absurdly robust.
DNA fingerprinting is limited by the quality of sample collection and handling (i.e. don't mix up or contaminate samples), but not the accuracy of the method.
Your confusion is because you think there is one "DNA test". That isn't the case. You can sequence a genome, but that costs on the order of $1k so companies like 23andme don't offer that. They use simpler methods.
Even sequencing a genome is not perfect - the current most common method works by cutting the genome into pieces, sequencing those and then reassembling them. A pretty obvious limitation is that you can't detect repeated sections.
I'm not sure what method the police use but it definitely isn't the same as what 23andme do.
I had a one off blood pressure tests a few years ago that was outrageous 140 over something and completely unusual for me (which previously were like 110). The nurse was like, lets see what happens with a bigger cuff, and the reading from the automated machine was suddenly like 120 over something, and she was like, shrug you should watch it.
So I picked up a mid-range automatic bp cuff, and started to measure it more regularly. The numbers were all over the place and completely unrepeatable unless I maintained exactly the same position/breathing/etc. I can literally swing my BP by 30+ mmHg, by re-positioning my body, drinking a cup of coffee, etc, all while officially still maintaining proper technique for measuring it. And yes, moving up a cuff size also tends to remove a ~5mmHg from my measurement.
I could write a book about how to get any BP you want out of the machine. Its the same with weight, I can shift my weight +-10% by eating a bowl of salty popcorn drinking a few 64oz cups of ice tea, then eating a large meal. Then I can go for a 3 hour bike ride, use the toilet and drop it all back off.
So, its sort of surprises me the fine grained aspect of DR's always measuring your BP/Weight, neither of which really mean anything until they get into the extreem ranges, and in the weight case you can get a better feel for it just by looking at someone...
Where I trained, this is why we don’t diagnose someone as hypertensive unless there are three separate elevated blood pressure measurements on three different clinic visits.
We would schedule people for Nurse only BP checks to help establish the pattern.
This statistical anbiguity is why the medical community as a whole is often sceptical of the Direct to consumer DNA testing and SNP analysis. They report data in a convincing way which is statistically misleading and in no way plainly reveals what is within measurement error.
But it's comparing it to the corpus of known DNA in different location, which is changing and being updated even by the fact of people submitting more DNA samples from different locations.
Unlike what you might think, this is not a sequencing based method, but something based on dna sticking to small parts of a chip coated with complimentary sequences. This method is super error prone, more importantly the error rate is dependent on who does it and with what level of care. So yes it's not going to be repeatable!
You do not offer any citation as to why this, the chip based genotyping, leads to misinterpretation and the article does not suggest this is the reason either. Without getting into it, the article says
>An entire DNA sample is made up of about three billion parts, but companies that provide ancestry tests look at about 700,000 of those to spot genetic differences.
>According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.
>Still, none of the five companies provided the same ancestry breakdown for the twins.
So these people getting results and finding out they aren't related to their parents, are the results suspect?
Ditto the results about genes associated with dementia or heart disease?
I'm aware of the companies having disclaimers, I thought that was on the basis of them not really know what the genes did, not that the test itself was rubbish.
It's true that these companies are not offering "sequencing" per se but rather are genotyping commonly polymorphic SNPs using an array or other genotyping method.
These SNP arrays, however, are exceedingly accurate and reproducible in their calls. A common medium-density SNP array used for human genetics is the Illumina CytoSNP-850K [1], which routinely gets >99.5% accuracy on calls. This is at least as good, or better than, whole genome sequencing.
The challenge in ancestry assignment is not determining what is in the sample, but rather in the interpretation and assignment of ancestral haplotypes.
Since the test results for the twins in the article were 99.6% identical, that seems to match what you say is the usual accuracy of SNP arrays. However, I don't think that can be called exceedingly accurate and reproducible if you consider the intended application.
>99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers.
">99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers."
Or potentially put people in jail based on these results...
If you feel that way about 99.5% accuracy, I'd advise that there are a number of consumer products you absolutely shouldn't buy, including: kitchen scales (small ones), measuring cups, store brand spirit levels, etc.
You can get considerably better accuracy, you just have to be willing to fork over $1500 rather than $60. How much are you willing to pay to know if your 31% German vs 30%?
The difference is that it's not 99.5% accuracy for the entire experiment but for each of the millions of spots, which means there will be hundreds of thousands of spots that are not accurate in each chip. Not an apples to apples comparison.
I am inferring that the problem of matching a particular individual's results with haplotypes is a problem in optimization (finding a best fit) with many variables that is actually solved by maximization (finding a good fit) without doing a comprehensive search of all possible solutions. The space of potential results may be very large -- very many haplotypes from which to select and a percentage to be assigned to each, from 0 to 100, and a perfect result cannot be expected, as ancestors within a given haplotype are nowhere near as similar as identical twins. It is very common, even typical, for algorithms that attempt to find good solutions to such problems to include some randomization (making some guesses based on pseudo-random numbers) followed by more directed attempts to successively improve the guesses previously made. Because some steps of the search are a random walk, different results from the same inputs are common for several iterations of the same algorithm.
DNA microarrays do have limitations vs. RNAseq, but they are accurate and repeatable when used properly. https://en.wikipedia.org/wiki/DNA_microarray Calling them 'super error prone' is simply inaccurate.
There was a good 5-10 years where microarrays were the go-to method for genomic investigation. Since RNA-seq has supplanted it, there has been no revolution in overturning those old results. At best, RNA-seq reveals what microarrays cannot: novel sequences, specific sequences that have difficulty hybridizing, etc.
The variation between these services is due primarily to how the data is interpreted and presented to customers, not the reads themselves.
Note that these companies are not doing DNA microarrays in the sense you're describing (which is for quantifying gene expression by mRNA level). RNA-seq is (now) arguably better for this purpose as well (better dynamic range, primarily), but again this is not what any of the ancestry or personalized genetics companies are doing.
In this case "SNP microarray" is a "digital" (or binary) readout of the genomic DNA sequence; specifically the array has attached to it sequences that match both the "reference allele" (e.g., a "G") and the "alternate allele" (e.g., "A" or some other base). Note that across the world's population, most sites in the human genome are "bi-allelic" meaning that there's really only two variants at each site, though of course it is possible to have tri-allelic or even fully polymorphic (A, T, C, or G) sites. For the purposes of ancestry, bi-allelic identification of commonly polymorphic sites is adequate, though the more sites and the rarer those sites are in world populations, the more specific you can be re: ancestry. This is why the gold standard for ancestry and haplotyping is whole genome sequencing-- you simply get far more informative sites to map to other populations.
It's fair that a cheap test might not be super accurate, but then why are they stating figures like "13%" if the margin of error is at least 10 percentage points? That's just plain misleading.
When my doctor asks how tall I am, if I'm 6'1", then 6'1" is a good answer. 6' (or even better: about 6') can also be an acceptable answer, as it's an approximation. I wouldn't say I'm 6'8", and then try to rationalize it by saying my tape measure is cheap, and anyway it's still right to about 10%.
Reporting what you don't know can be just as important as reporting what you do know. It doesn't matter how good your laboratory method is if you can't report results in a way that doesn't mislead customers.
Depends what you're getting and how much you're spending on it.
You're not going to get a very high degree of accuracy from a ~$100 test. They're not sequencing your entire genome for that amount of money. It is still good enough to determine the broad strokes of someone's ancestry, to within a few percentage points as you see in the linked article.
If you're willing to pay for full sequencing then you're going to get a lot more accuracy and repeatability. You're also going to pay a lot more for it than would be warranted just to satisfy your curiosity as to where your ancestors came from. These kinds of tests would really only be warranted if you have some rare, difficult-to-treat genetic disease. That might not even help you directly, but rather, be done to contribute to the body of scientific knowledge of the disease.
Why? What makes you think your DNA is identical everywhere or that the test is perfectly accurate every time, or that even the concept of DNA-by-region is all that precise?
If you do it 10 times, you will always get the same buckets, but the percentages will differ back and forth for a bit. If you're goal it so find out your heritage, which these services are offering, then you are getting the right heritage repeatedly.
If your goal is to find out how many 0.1% of you is of an exact descendance, then you're going in with the wrong expectations.
Can you explain the difference between sampling and sequencing the entire genome? I had assumed they did the latter but I guess the particular markers for diseases they are interested in are at known location that they can "jump to" somehow?
Think of a genome as 3 billion character sequence. "Sequencing" a genome gives you the actual content of the sequence. Think: "....GAAAAGATACCCACAGAGATTACAAAAC...." (I'll call this string FOO)
Until recently, sequencing an entire genome was really, really expensive. I'm not sure what the price is now, but I know that it has dropped considerably.
When you sample a genome, you're checking for one specific string. Traditionally this is done by mixing a sample of the DNA with an enzyme that breaks apart a specific DNA sequence and then measuring whether the size of the DNA molecules have changed. Take the string that I mentioned above (FOO). Suppose that I mix it with an enzyme that cleaves along "GATTACA" such that "GATTACA" becomes "GAT| |TACA". If I mix a DNA sample containing the string FOO with the enzyme I just mentioned, then the molecule would be split into:
"....GAAAAGATACCCACAGAGAT| |TACAAAAC...."
If I mix a DNA sample that does _not_ contain FOO with the enzyme, no change occurs.
You know when you watch CSI shows* and the DNA evidence is presented as a bunch of light and dark bands? Those are produced by mixing the suspect's DNA sample with a standard cocktail of enzymes, treating them so that the DNA molecules are electrically charged (DNA might already have a charge, not sure), placing them at one end of a container of a standard gelatin with known pore size, and applying an electric field for a standard amount of time. The cocktail of enzymes cleaves the DNA at multiple different substrings and breaks down the DNA strands into multiple smaller substrings. Different size molecules will progress through the gelatin at different rates, so you can effectively compare two people's DNA to see whether the substrings match. You _don't_ know the person's genetic sequence, just whether or not his or her's DNA contains the same substrings as the DNA of the person that you're comparing it to.
* I'm assuming those TV shows present the DNA evidence this way. I don't watch them.
Disclaimer: I'm not a biochemist by any stretch. This is based on my memory from school, so don't take my explanation as absolute truth :-)
I started typing an answer, but realized Wikipedia could probably do it better. In short, sequencing and checking for specific mutations are very, very different processes.
* Full DNA sequencing. This is a multi-million dollar effort taken on by large collaborations, usually to generate a reference genome sequence for a species or particular line. Typically a variety of methods are used, from 'shotgun' style illumina, 454 longer reads, and now some newer even-longer-read methods. Long reads are important for generating accurate assemblies. Generally, some tricky portions will need to be meticulously amplified and sequenced with traditional Sanger sequencing to clean up. This gives you a complete sequence, including all inter-gene sequences, repetitive portions, etc.
* RNA sequencing. This method is the core of many molecular biology labs; you take expressed sequence (i.e. RNA that has been transcribed from DNA), reverse transcribe it to DNA, and then sequence the DNA. This gives you sequence information about the expressed part of genes (this is limited, though, because during the assembly process you'll reject many reads that don't map onto the reference sequence in standard analysis) but mostly gives you an idea of the relative expression of different genes. When people talk about gene expression going up or down in certain mutants, conditions, etc., they're usually talking about RNAseq (or qPCR if only a few loci are involved).
* 'Sampling' in this case refers to chips with various DNA fragments on it. You pass the sample DNA over it, and compatible sequences hybridize together, which is then detected via florescence. These chips are designed to carry sequence fragments that vary in the population. Together, these variants form a DNA profile. The quality of this profile depends on the design of the chip. The sequences included are generally loci that are known to affect particular diseases or to be especially variable in the population for assessing ancestry.
In cost magnitude, this goes from millions to thousands to tens.
Your comment gives the impression that all DNA sequencing is a multi-million dollar effort. So, just to clarify: a modern, accurate (= 30x coverage) whole-genome DNA sequence costs less than 1000 USD. Adding long range information from single-molecule sequencing makes it only slightly more expensive.
I”m glad you added this and want to add further that GP’s impression is less than a decade out of date.
The price of a whole genome sequence has fallen dizzyingly fast, faster than any technology of which I’m aware. We’re close to the point where it’s priced like any basic blood test. Exciting times.
its like taking a low resolution photography of 23 chromosomes and X and Y plus the circular mitochondrial genome. The higher the resolution the more expensive it is and more data and more difficult to interpret. If the feature you interested in is large and unique then a blurry picture may be good enough for example recognize a human body from few yards away, as the features become more refined and more diverse like a face, you can still have little blurr but you need to recognize the facial dimensions extensively to be useful. And if you want to see skin lesions then you want a high resolution pictures to distinguish which type it is...
sampling is a high resolution photo with large holes in it, a full sequencing is a full photograph with varying resolution and quality etc. There are still some areas of the genome that are inaccessible even with full sequencing. Improvements are being made with long read sequencing.
I'm not a biologist so I'm basically making this up, but my guess is no. I think only false negative errors are possible because of the way they do their sampling.
If two samples had enough similarity to say someone is a child of someone else, my guess is that's pretty accurate.
It would take so many errors adding up that I think it would be very impossible. Has anyone ever had a real sibling not come out as around 50% related?
Not necessarily. The results may only be as good as the training set that was used to produce the calibration (probably butchering the terminology). And the results that the twins got for their tests may have been "equally good" in terms of the reliability of the test.
The whole concept of geographic ancestry may have fundamental limitations at the level where people are quoting percentages.
Mine also changed after I complained that the results were obviously wrong. It indicated that several of my chromosome pairs came entirely from my father. Since that's generally impossible, it put the rest of the results in doubt.
how long have you been using an ad-blocker? perhaps the fine print says the expected accuracy is only guaranteed if they can source an adequate profile from data mining resellers?
I haven’t seen a sample report, but doesn’t this mean the outcome could be, “you may or may-not be predisposed to xxx disease”. We need to spend $99 to know that?
>I thought it was pretty well known that these services are estimates only to a large degree? These services only take samples, they don't sequence the entire genome. Of course there will be errors when you have to extrapolate from a sample.
The credit given to the intelligence of society at large here falls somewhere between "cute" and "astounding".
I would recommend flipping through a tabloid and then considering such content sells millions of copies each print. And then reconsidering your thought that it's "pretty well known that these services are estimates only to a large degree".
I disagree. He didn't ask that "don't most people know you need to fill up your car with gas once the tank gets low?" He made it seem like the fact that there is a high degree of uncertainty/estimation in genetic testing is common knowledge. We're talking about a society where half the members do not believe in anthropogenic climate change (note that I do not state the _degree_ of the change) and sizeable numbers of people (millions) don't believe in vaccination.
He said "Of course there will be errors when you have to extrapolate from a sample." The common man doesn't know what extrapolate from a sample even means!
I'm sorry but suggesting society as a whole understands genetic testing methods and techniques is just so far absolutely removed from reality (you're perhaps justified if you work for Illumina or are a faculty member of a University) that the point needed to be made even if harshly. Maybe I just interact more with "non hacker news" types than the average poster here, but that post quite frankly seemed so absurdly out of touch as to remind me of the Lucille Bluth quote from Arrested Development - "I mean, it's one banana Michael. How much could it possibly cost? 10 dollars?"
Your post provides a great data point supporting your thesis, but the real problem with the intelligence of society is rational ignorance. People don't expend a great deal of resources trying to understand things that don't matter to them.
For example, you're happy to fire off about how dumb everyone is for reading tabloids because the finer points of how people unlike you consume media are not particularly important. And, likewise, the typical person buying a DNA test as largely a conversation starter isn't going to invest a great deal of effort to understand how DNA tests work.
Your post provides a great data point supporting your thesis, but the real problem with the intelligence of society is rational ignorance. People don't expend a great deal of resources trying to understand things that don't matter to them.
Tragically people are notoriously bad at long term predictions of what will matter to them and how that relates to a broad corpus of knowledge or habit of skepticism. The result is that people are so ignorant they’re under an illusion of being knowledgeable, and at that point they’re more or less a lost cause.
It might help to know enough about DNA to wish to avoid these services, especially given how they cooperate with law enforcement. Mostly though, it’s just really hard to calibrate your corpus of useful knowledge from a position of profound ignorance. Plus a lot of people really are quite stupid, and ignoring that in favor of some superficially positive message about mutual respect is potentially dangerous.
On top of that even if they were able to sequence things perfectly the thing they are actually measuring is a very fuzzy thing. Like it's no surprise that one of the test had one twin from England but the other from Scotland/Ireland because people having been moving between those 3 countries for millennia so there isn't really a platonic idea of 'English DNA' it's just markers that tend to show up in England, but also show up near England but somewhat less and may also randomly popup other places, but usually not all of them in one other place.
It always surprised me to see them reporting results to a 10th of a percent precision (e.g. "2.3%"). I'd assumed that the best they could do on an individual was a couple of orders of magnitude less precise (e.g. 10%, 20%)
By day, I work with scientific equipment. Most people understand significant digits. But they also have conditioned expectations related to the number of digits that they see, and the quality of the equipment or measurement. People just like to see a digit after the decimal.
Yeah, "You're 76% German" makes my monkey brain think, "Come on, you're making that number up." But "76.2%" makes me think, "Ah, the miracles of science, time for a weizen of pilsner!"
I wonder if there may be two issues here, one related to consistency and one related to twins. That is, you may not need twins to get differing results, but being twins may still be one of the reasons for differing results.
I'm not a biologist, but apparently it's not the case that twins have exactly identical DNA:
I did exactly this back when 23andMe switched to a newer chip. I signed up for a second account and matched with myself as a full brother. The old chip account had less data points (health items, etc) but most of the data was about the same.
My wife did the same with the same result, as did a friend who had purchased the original 23andMe kit and wanted to get the newer data from the new chip.
So while I don't doubt the accuracy of the linked article, my anecdotal experience is that the 23andMe DNA tests have repeatability at least.
same here. I really want to learn more about my past and also learn about my DNA (and all there is to learn with that data), but the invasion of privacy by such companies is just unacceptable (hence my refusal to take such tests).
The only reasonable way of achieving those goals would be to allow totally anonymous submissions. Mail in a sample and simply have a code that you keep to check on results once they are processed.
The fact that none of these services allow for anonymous submission should be a huge tell about their real motivations.
Anonymous submissions are in theory somewhat feasible, albeit cumbersome. You can by the 23andme kits with the full fee paid from certain pharmacies (I think Target had them) using cash. This means you don't have to put down a credit card when signing up. I leave the rest to the reader to work out wrt anonymizing themselves while signing up to the website.
Err, isn't the first thing they do is link up the new submission with every other submission ever received in the past or future, almost none of which are anonymous?
Sure, you can be anonymous, but you're the son of John and Jane Tompkins, the brother of Emily Tompkins, the cousin of Mary Smith, and the father of Dave and Buster Tompkins.... huh, I wonder who you are.
Unless things have changed, this is exactly how 23AndMe works. You get a kit and send it in, then you just have an ID number that ties that kit back to you for setting up your account.
That might work until you have relatives who did not do it anonymously...then you get outed by the DNA istelf.
A service that destroyed your sample and record after processing would help in this regard.
> none of these services allow for anonymous submission
You're incorrect. At the company I work for, which is one of them mentioned in the article, a customer pays for DNA sequencing and analysis. Who pays for it is often separate from who's being analysed. We often have customers who pay for products for their family, friends, or otherwise. The name on the DNA sample is not verified in any way shape or form (and how could we verify it beyond just billing information?) You can opt-in or opt-out of public matching. You can download your raw data anyway (GDPR requires this if you're an EU citizen).
> should be a huge tell about their real motivations
I don't have an opinion about this, but they're probably implying that the business model is the same as Google, Facebook, et. al., which is selling user data to third parties.
That would also help against the (hypothetical?) possibility that data mining sector creates "DNA testing" front companies, which do no testing whatsoever, and simply sell you a pretty printed conclusion from your cyber profile... without oversight anything goes
obviously, but a law alone can only protect you when the deception is detected. the proposal to somehow fully anonymously (so somehow also not through the usual mail system) submit the material and code would be a preventive measure
Don't most services provide (2)? You can download the raw data. It's not your full DNA sequenced or anything, they only do genotyping, i.e. only test for genes that are known to differ between individuals, I don't think there's a consumer service that does full gene sequencing for a reasonable price.
It's not advertised as such but yes, several companies allow you to obtain the "raw data" from the sequencer. The raw data is what gets fed into the analysis software which produces relationship and heritage estimates.
It's also worth noting that if the company sells analysis products to EU residents then GDPR may require those companies to make the raw data available to the customer anyway.
I think you underestimate the quality and power of short read sequencing. Most well-designed sequence-based genotyping assays take into account the length of the read and are easy to read out. (You are thinking of issues with copy number variation and perhaps assembly of longer genes, which do have issues). The sticky part is the analysis--how realiable are your databases and your algorithm for representing the mix of SNPs and INDELs as %European...that is the question.
> Techniques used to read DNA only read short segments, from which larger segments are inferred.
I think you’re describing short-read sequencing here but this isn’t what any of these services does. They do microarray genotyping rather than sequencing. And in neither case is there “inference”. True there’s something called imputation but it works quite differently and I don’t think (though I don’t know) that 23andme uses it for ancestry. I’m also not sure what you mean by “bijection” in this context.
Their results seem well within the margin of error I'd expect out of a $100 DNA sequencing service. It's incredible that this kind of thing is now available at this price point.
Carly's results have a larger 'Broadly European' percentage, but that is a catchall that includes: Italian, Eastern European, Balkan, French & German, Iberian (Others category), and Broadly Southern European (Others category). Therefore Carly's 'Broadly European' traits could be used to match Charlsie's results.
It is unfair to call the 23andMe results 'different'. They are getting the same results, however one twin is getting more specificity.
Why do people assume ancestry results from these DNA tests are "accurate"? Human ethnicity is a continuum, not distinct pools of genes. We are applying labels conceived by modern nationalism to several thousands of years of migration and intermarriage. It would be great if people stopped treating ancestry results from these tests as authoritative reference.
There was a controversy about Korean & Japanese ancestry results from 23andMe where ethnic Koreans are sometimes classified as >30% Japanese. These two groups shared common ancestors since ancient humans migrated via the Korean Peninsula to the Japanese archipelago centuries before the modern nations of Korea and Japan came into existence. Slapping either one of these specific labels on shared ancestry is wildly inaccurate and short sighted.
> Why do people assume ancestry results from these DNA tests are "accurate"?
Years of highly visible advertising by these companies which implies they're accurate? All of those “discovery your ancestry” posters they put up around here had specific countries and locations.
Agreed. Just a quick look at the 23andme site doesn't show me anything that says they are "estimates" or "not accurate".
The landing page shows me this headline "This new year, commit to a healthier you - inspired by your DNA." I'm sure they have it legally covered somewhere in their TOS but it's still not good if these are inaccurate or estimates.
99.6% is still awfully dissimilar, depending on what that number means. Thinking about single polymorphisms, of ca. 3 billion bases, that is still 120,000,000 SNPs between twins. Of course, they aren't doing full genome sequencing, and they are probably targeting loci that are know to be more divergent. Still... that similarity seems awfully low for twins. I wonder what they typically find for similarity between any two individuals. I also wonder if their lab is a little disorganized....
It's totally reasonable (and in line with my knowledge of a similar technology, DNA sequencing). Often the stochasticity comes from biomolecular processes rather than from the optics of the instrument.
No, that’s also incorrect. The real number is way higher. They share 50% of alleles, and hence 50% of variants. But a lot of the DNA within two distinct alleles will be identical, except for (say) a single nucleotide difference.
Yes, that's actually what I meant to say.. couldn't remember the right terminology at the time. The point stands though, I think.. the number "99.6%" for identical twins isn't measuring the same as "98.5%" (or so) for humans vs chimps.
They don’t, this number is extremely misleading. Roughly 97% (I think is the current estimate) of their protein-coding DNA is equivalent, but not 1:1 identical.
Current estimates are that there’s a roughly 1 in 1000 difference between two individuals purely due to random mutations in the germline (or, put differently, the DNA replication error rate in human cells is about 1 in 1000). This would imply 99.9% expected similarity between two identical twins. However, this is in the germline. Somatic cells accumulate mutations over their life time (since they, too, divide and therefore need to replicate their DNA, and are furthermore exposed to environmental stress). In addition, the tests work off saliva samples which are mixed with all kinds of stuff in the mouth, and degrade slightly during transport. Lastly, there’s an inherent replication variability in the genotyping process (though this is very small).
This back of the envelope estimate suggests that 99.6% similarity for somatic samples of identical twins is not unexpected.
Something tells me that the 1/1000 error rate you cite isn't truly accurate, or at least doesn't apply to all portions of DNA equally, because in almost every individual that would result in many SNPs with bad consequences since we have so many subsequences coding essential proteins
This is not accurate. Coarsely, there are roughly 3 million SNPs (out of 3 billion total haploid bases) per individual. When we talk about individuals "sharing" DNA we mean these variants and not the 99.X% of DNA that we all have in common. When we refer to random mutations in the germline (i.e. de novo variants) current estimates of de novo variants per individual are in the low hundreds.
The other factors you discuss are more likely to cause issues. One major one is the error rate of the microarray because of the high number of genotypes being assayed. If the chip has an error rate of 0.1% and 500k tests are performed you would expect 500 mis-calls. I used to work for Illumina, who makes these chips, but I worked on the DNA sequencing side of things so I don't know the actual error rate off the top of my head.
Right, de novo mutation rate is much lower (even lower than what you wrote, just slightly above 1e-8 according to deCODE). The 1/1000 rate is the average difference between two random individuals (not twins), which corresponds to your 3M SNPs per 3B bases.
Yes, but you referred to them as random mutations, which is not the case. They are inherited variants. The DNA replication error rate is nowhere near 1 in 1000.
Hence my correction. Without looking it up now, the actual replication error rate is probably close to the deCODE rate for de novo mutations (modulo gamete selection, so probably higher than that).
Article said identical twins. So I'd expect around a couple thousand SNP differences. Each cell devision has about 30 mutations. Embroyo bifurcated within a dozen or so cell divisions. Dr. Hayflict found human soma cells live aboutb50 divisions.
Semi-related: I was thinking about getting one of the newer iWatches, the one with the heart-rate monitoring and all that jazz. To me, those features are really cool and future-y. So, I waltz on down to the store and try one on. It reads my heart-rate and it seems to be fairly accurate. Ok, well, let's test it a bit. I start running in place. The heart rate goes up. I then just stand there, after a bit, the heart-rate goes down. Nice, the thing actually works! Well, they had a few of the iWatches sitting there, without anyone else taking a poke at them. So, I strap all the free ones on the arm.
Well, they all gave different readings. I repeated the running in place test. Some went up and down much more than the others; maybe 10 bpm more. I tried re-arranging them on my arm and doing the running in place test again, you know, maybe it only works on the wrist since the blood is closer to the surface there? Nope, the readings were all over the place still. It was not super scientific of me. I mean, I was some strange person running in place in a store with a bunch of iWatches strapped to my arm.
Maybe the watches have some sort of learning algo in them that was getting messed up with all the folks trying them on. Maybe I was wearing them 'wrong'. Maybe I should have waited 10 minutes between jogging sessions. I don't know.
But, to me, these things have a long way to go. A lot of the bio-tech and bioengineering out there seems to be a bit of snake-oil right now. I really do want it all to work, I think that would be a great boon to us all [0]. If something like the iWatch can't give repeatable readings in nearly any way, then it's all just a gimmick and not useful.
[0] I mean, can you imagine if Theranos' tech actually did work?! That would change medicine for the better in incalculable ways. I can see why that company's plan was so intoxicating.
That's surprising, since the heart rate monitors on many phones are quite accurate. Many will actually give an ECG-like trace indicating brief changes in blood pressure. I don't see any good reason they shouldn't be extremely accurate.
Well unless you're wearing a cuff you're not getting any blood pressure readings on your phone, and any "ECG-like trace" is a doodle at best as there's no electrical monitoring. Even chest straps that measure electrical output don't give you an ECG display.
I'm not suggesting that absolute blood pressure is being measured by a phone PPG, only relative changes.
Far from being just a doodle, the PPG readout ("ECG-like trace") comes from changes in skin reflectivity that indicate relative changes in blood pressure. [1]
Obviously this isn't as good as an actual ECG (and can be disrupted by movement), but it's far more information than is needed to accurately track heart rate. These devices can be as accurate as a pulse oximeter [2], though they don't account for the movement of the user.
It's not an interesting story, and his method of testing was severely flawed.
There are tons of people that test these devices against chest straps online, and almost all of these bands give results that are surprisingly accurate.. or at very least about what you'd expect out of a wrist based monitor.
I can't say that I know exactly how these things work, but I wouldn't be surprised if they were built to work with the blood vessels and skin thickness of a wrist, where watches are worn, rather than the skin thickness+hair+different vessel locations of a forearm.
Different parts of the body are used for different things. It's why blood oxygen monitors that work on fingers don't work on toes.
Yeah, I tried to move them about over my arm, thinking exactly the same thing as you just did. It gave me all kinds of results. Another commentor points out that the data is 'demo only' data on the in-store watches. The reason for demo only data is that the data might be considered 'medical' information and, I assume, therefore covered under HIPAA. Exposing that to just about anyone that can come in the store and see the record of the bpm is, rightly, very illegal and unethical.
Some of those things stop working correctly when your skin gets damp. Think about that for a second.
An iWatch is not the pinnacle of wrist biometrics, however. I'd be a lot more concerned if a high end Garmin got it wrong, that's their whole business.
Garmin and Apple wearable devices use basically the same optical heart rate sensors. They don't make their own. Every company in the industry buys from the same couple of suppliers which have developed the technology.
Part of how you're supposed to use the dedicated higher-end models (I have a Garmin Forerunner 645) is that it's supposed to be pretty tight against your wrist. Even if you "wear your watch tight" it's probably not as tight as you're supposed to wear it for fitness purposes. If at any point you can't feel it or you forget you're wearing it, it's definitely not tight enough. This helps limit optical issues with movement but also largely eliminates the impact of sweat on the reading.
Still not nearly as accurate as the chest-based models as mentioned elsewhere, but for 95% of people it's plenty.
Wrist worn optical heart rate monitors are actually very accurate, under normal circumstances when worn properly. It's easy to verify accuracy yourself by just counting your own pulse rate with your finger. Try it.
Wrist devices will give inaccurate pulse counts if the band is too loose, skin is dirty/wet, or your arm is moving around quickly. If you need accurate numbers in all circumstances then a chest strap which detects electrical signals is a much better option. I usually wear a Garmin HRM-Tri strap while exercising and it transmits a wireless signal to my fitness tracker.
Anecdotal, but I got an Apple Watch because I never liked the squeezing feeling of the chest strap of my Suunto. I put them to test for the first couple weeks and both at rest and while running the difference in the HR reported by both was less than 1bps.
So I ride bikes and always monitor my heart rate while working out.
I've always used a chest-based heart rate sensor until I started trying wrist-based about 4-5 years ago. I've used countless brands and models (ranging in price from about $49.99 up to about $399) on my wrist - and literally none are as accurate as the chest-based strap. The chest strap responds immediately to increases and decreases in heart rate. Whereas the wrist sensors always seem to lag, sometimes by up to a minute or so.
The wrist-based sensors always seem to differ depending on where on the wrist they are worn - high, low, inside of the wrist, outside of the wrist, etc - by maybe 5 bpm or so.
In either event, if you really want to test the accuracy of the wrist-based sensor, you should really compare it to the output of a chest-based strap.
In fact, the results will largely be personal - because the electric signals may differ per person greatly in different areas of the arm.
Needless to say, my apple watch is worn daily, but when riding, I have gone back to a chest-based strap.
That all being said, there is a guy that reviews these gadgets and will often wear 2-4 sensors while working out, and post an overlay of the results from his workouts. The baseline should generally always assume that the chest-based straps are accurate. His site is called DCRainmaker, if anyone wanted to look up his comparisons/reviews.
The wrist lag is the biggest problem I have with my Fitbit wrist band. It is OK for looking backwards of how my overall work out went, but it is useless for trying to hit a target heart rate.
I wonder if the difference in results is partially due to a difference in measurement strategy: a naive way is to wait one minute while counting the pulses which will necessarily be slower in response (it could still update the screen at a faster rate of once per second by having 60 different integers but each integer gives the count displayed on the screen when it's last second has passed, but that would still have a slow response in the sense that given a hypothetical heart rate change from a fixed constant initial rate to a new fixed higher rate, like when you got scared by say unexpected fireworks, it would still take a minute for the final rate to show up).
A faster response could be got by measuring the time between heartbeats (probably a low but even number of beats).
When I tried on an Apple Watch at an Apple Store, I noticed that the heart rate it displayed was definitely different from mine.
I mentioned it to the employee who was manning that table, and she told me the demo watches had some pre-programmed data for demo purposes.
I was puzzled by this since heart rate monitoring is a key feature users care about. It occurred to me that heart rate data of everyone who has tried on the watch might count as the kind of medical information you shouldn’t publish on demo watches.
Imagine ridiculous stories like “CEO of X’s heart rate was 130bpm when he tried on the new Apple Watch, says journalist who tried it on right after.”
Well, I'll be damned! Thanks for the explanation, I really appreciate it. I'm still wanting to buy one, but now I don't know what to think. I wonder how I am supposed to try one out before I buy one and to make sure it works.
> It occurred to me that heart rate data of everyone who has tried on the watch might count as the kind of medical information you shouldn’t publish on demo watches.
That's an overly generous explanation. Fitbit-like devices are notoriously inaccurate[1], and the simplest explanation seems to be that their inaccuracy is unintentional.
The Apple Watch is in a different regulatory space than the devices listed in that source, since it's cleared as a class II medical device by the FDA. HIPAA laws will apply in some use situations, so their lawyers will have thought about it and made recommendations. (Lawyers aside, the feelings of both the medical community and Apple are generally that you shouldn't leak customer information. This seems like the kind of data that people would accidentally leave behind at the store without thinking about it, which doesn't feel right.)
so we need to find someone with access to 2 identical watches, perhaps someone who bought some for the whole family? instead of strapping all to the same arm, just strap one on each wrist (assume the "left" heart's rate is approximately equal to "the heart on the right").
Are we claiming that every product needs 2 FDA applications: one for the end user and one for a special run of demo watches? I believe (but did not check) that demo watches would be either exempt or go unnoticed considering the amount of data it could gather...
Heart rate data gathered from a store demo device without any personally identifiable information isn't subject to HIPAA rules. If it was reading inaccurately then most likely the band was loose or the sensor was dirty.
Disclaimed: I used to work in this field, but not on the Apple Watch.
The wearing a bunch of them at the same time is going to be wrong, yeah. Wrist mounted fitness devices are calibrated to read on the top side of the wrist. Even reading on the bottom side of the wrist would be a bit different, way up the arm? Very different.
It is possible to make accurate devices that read there, they just need to be constructed to do such!
Wearables are also meant to give an accurate rating over the long term. If you go running for 30 minutes, are the overall calorie burn, average HR, max HR, and min HR, all correct? Tracking in perfect sync with the heart is nice, and of course everyone tries to do that, but if the number is off for a second here or there, well so long as the overall result is the same the usefulness hasn't really changed.
Finally, due to the way optical HR works, random unexpected, movements are really hard to deal with. It can take some time for an HR sensor to lock on to the heart rate. This isn't too long (measured in seconds, and each generation of devices is getting better and better), but it relies on regular patterned movement, such as arms pumping while running or walking, the regular motion of a bench press, etc.
Stopping and checking the HR value every few seconds to see if it is correct is literally the worst case scenario for a lot of wrist mounted sensors!
Ground truth is strapping a heart rate strap to your chest, putting on the wearable, going for a run as you normally would, and comparing the two at the end.
Multiple sources have done this, and found the Apple Watch to be best in class when compared to directly measuring HR.
tl;dr: Stopping and checking wrist mounted devices can confuse them, Apple Watch, and everyone else in this field who is serious, are tested against HR straps and they come within spitting distance of accuracy.
The research I've read [1] says that heart rate monitor in general and Apple Watch in particular should be pretty accurate. If you have darker skin or wider wrists that accuracy may be diminished.
I have the first apple watch. (I bought the newest but haven't used it yet still boxed).
What I found was that the heart rate correlated well with taking it manually and/or a finger heart monitor (pulse ox cheap from Amazon).
But with both devices you could actually tell when it was clearly wrong independent of comparison. The point is you more or less (at least I found) could figure out when it is right and wrong just based on past use and what you would expect (feeling your pulse). So if it was off it was grossly off and obvious. That is my experience with the watch that I have had for years. Either 'correct' or 'vastly off and easy to tell that is the case'.
What are the chances that these two people are not identical twins? i.e. fraternal. That could explain the differences in results from the same company. But they should also be quite trivially declared siblings in that case.
Mistaken zygosity is quite rare, in the single percent range, after doing some tests like blood groups or hair whorls. In this case, the DNA overlap is vastly higher than would be expected for siblings:
> According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.
If they were siblings, ~50% of the (several hundred thousand) SNP calls would be the same, not 99.6%. (Which is about what you would expect from identical twins, since that's where either one disagrees with the other, and errors usually don't strike twice, so a 99.6% overlap roughly implies a per-SNP accuracy rate of sqrt(0.996) = 99.8%, which sounds reasonable given the difficulty of sequencing and the inherent noise & randomness.)
On a fun side note, identical twins are not exactly 100% genetically identical, which is how rapist/murderer identical twins are being convicted these days based on DNA evidence; but there are so few unique mutations to each twin, and the new mutations wouldn't be on the common-SNP tests, that you have to do the equivalent of like 4 whole-genome sequences (IIRC, you need coverage of ~120x vs the more usual WGS of 30x) to get enough evidence to override the prior of any difference being just a repeated sequencing error, and it's very expensive.
DNA in Europe is all mixed up. A lot of this is due to the Romans. There were also a lot of other migrations and displacements.
I was watching a BBC documentary about how Rome influenced the history of Scotland. The announcer, who has a very pronounced Scottish accent, took a DNA ancestry test. The results (IIRC) indicated Germany, Italy, and eastern europe. (The announcer interpreted this last one as Dacia) Come to think of it, he doesn't look so different from Metatron.
The twin results not matching is kinda disturbing, though.
This is a great reason why I won't get this test done. The math is bad and (genetically identical) twins getting different results shows how bad the science is right now.
This reminds me of when economics try to predict what the GDP will be next year. It's almost always wrong, so why would ever listen to anyone who is always wrong?
Actually, the results are not so bad. The technique is rough, but even some basic MDS (multi dimensional scaling) on DNA gives things that look like the actual maps, so there is some scientific base.
Also, you should not underestimate how not knowing where you ancestors came from is disturbing for many people.
Why should that be disturbing to some people? I think the vast majority of people, at least in America, have no idea where their ancestors came from. Is the usual "I'm part Irish part German part Cherokee" meaningfully more satisfying than "I'm west African?"
I don't think it's a good thing for people to attempt to derive any aspect of their identity from their genealogical ancestry, it has no bearing on who you are as a person, and has the potential for great harm. I've seen it used to give people a false sense of culture and belonging to a community that they previously had nothing to do with. If you felt a connection to a particular culture or identity, should we really be telling people that their genome is responsible for determining whether or not they can associate parts of those cultures with themselves?
Racism results from a failure to separate identity from genome. If you listen to the dogma of a white supremacist, it's heavily predicated on associating the accomplishments of their "European" forefathers with themselves. If you listen to the dogma of a black supremacist, it's heavily predicated on assuming the identity of the Egyptians or Israelites and associating their accomplishments with themselves.
I really think that if an individual is truly "disturbed" by not knowing the precise nature of their lineage, that is an individual in search of an identity on a plate, and therefore highly corruptible. The motivation to learn more about one's ancestry should ideally not surpass the level of an intellectual curiosity.
I wholeheartedly agree with your analysis on the risks.
I still think it can have a positive role, if only to help the person emotional investment.
> Is the usual "I'm part Irish part German part Cherokee" meaningfully more satisfying than "I'm west African?"
For that, "I'm west African" is less satisfying than "I did trace my roots to Mali" which can lead the person to study the history of the country, Timbuktu, etc.
These tests aren't a measure of ancestors, it's a measure of a person's DNA and where that DNA possibly comes from.
Theoretically, they are similar, but since you inherit a random 50% of DNA from both parents, on an individual scale, you might inherit a misleading percent from your ancestors.
Let's say 3 of your grandparents are all full-blooded Italian. Like, their families have been in the same village for centuries. And one of your grandparents is from England (and is similarly from an old family in a small village).
So, theoretically your DNA results should 75% from that region of Italy and 25% from English region.
However, lets say from your mom, the parent who is 50% English and 50% Italian, you only inherit a mix of 15% Italian and 35% English (15% + 35% = 50% mom's DNA). It's random, so this is possible, especially on a small sample of ONE person.
And from your Dad, who is 100% Italian, you obviously inherit 50% of your DNA as from that Italian region.
Now, you theoretically should be 75% Italian and 25% English, but since it's random, your true DNA reads as only 65% Italian and 35% English.
This is why it's mostly bullshit to me. It's impossible to know what percents your DNA randomly passed on to your ancestors.
Sure, in our example, the two places where correct, Italy and England, but the percents are way off.
And that's honestly WHY we get these tests. We generally know where our families come from, we just have no idea what the percents are.
So, the whole point of the service is stupid since they equate DNA with ancestry.
Why though? By ancestry, I'm 25% Italian and a whole lot of other stuff. By percentage, I think I'm probably either mostly French or mostly English by ancestry, but that doesn't matter to me at all. What matters to me is that I grew up learning to cook from my Italian grandmother. As a result, in so far as I have any cultural identity other than "American", that identity is Italian, regardless of my genetic origins.
It's hard for me to imagine why anyone would feel any significant connection to an ancestry that they needed a DNA test to know they had.
Some people feel no connection whatsoever towards the culture they supposedly come from. Now it is just the rare person in the family who still feels part of the culture of the birth country (at this culture often gets swallowed by the host country).
Some people have not even such luck, and do not know their ancestors at all. Records are incomplete and some people lie. So instead, they try to grasp to what they can, using anything they can. DNA is but a tool there.
For me, DNA offers answers to questions about my family history, things that otherwise I would be unable to ever know.
I find it psychologically helpful. It helps me relate to ancestors I didn't know I have: who they were, what they were doing. When did they immigrate, what were their hopes, their lives, their challenges.
For me it is part of a healing process.
I don't think it will turn me extremist. And the data is not perfect. But it just feels nice to be able to relate to others when they tell you they are from xx country, their father from yy, their great grandfather from zz, etc. It lets you joint the common story.
Some family have had complex paths, with records lost. Dead ancestors can't tell you much about where they came from. I think it's just sad. I would prefer anyone to have access to the possibility if they want
I sent my grandparents raw dna file to a few different sites and got wildly different results, including 1% nigerian and 10% north african from myheritage.com (something that didn't show up at all on other sites.)
As I understood it he expected the different results were due to contamination or other errors resulting in compromizing the raw data to some degree. The shocking part was that the raw data was ok while the results weren't ok to the same degree.
Disclaimer: I currently work for a genetics genealogy company mentioned in this article. My comments are my own. Some relevant previous comment threads of mine about other DNA related articles: [0], [1], [2], [3]
Personally, I would love to help create legislation aimed to protect consumers from perceived and actual problems related to DNA sequencing and analysis. I don't know how though and I fear it'd fall to the back burner in the current US political climate. Our customers are not only US but international though and that presents yet another dimension to the challenges therein.
This article states:
> Whatever your ancestry results, don't get too attached to them. They could change.
That is absolutely correct. I've stated why in a previous comment [1]; Some DNA analysis software employ stochastic algorithms. That means that the answer they provide can be different if run more than once.
The article also states:
> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.
That is also correct. I've heard peers state that the industry isn't regulated outside of the field of medicine during product discussions. I think that's something which needs to be addressed: leaving it unaddressed can encourage predatory business behavior.
> there is absolutely no government or professional oversight
They are not allowed to sell health-related reports anymore in all of europe and other countries. While i think that some regulation and standards would be good, banning it overall was a dumb measure.
Well 23andme was specifically reprimanded by the FDA for their health related information and blocked from publishing future health advice, until they got limited approval after finally complying with regulation. So it’s inaccurate to say that there is no oversight, although I agree that the current oversight is clearly insufficient, in addition to being post-hoc.
Well, computer-related topics tend to be ok around here... not always, but at least there's a critical mass of folks who work in the area so the probability of coming across an informed opinion is much higher.
> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.
> It's a situation Gravel finds troubling.
What is up with this incessant clamoring for a nanny to supervise every little thing?
This is published on a page where, thanks to "oversight", I have to click a little spam box warning that this page, like every other damn website on the internet, uses cookies. No thank you. I really don't want to pay more taxes for the Central Bureau of Making Sure You Know You Might Be 7% Less Eastern European than Your DNA Test Indicates.
That's overly reductive. There are people making lifestyle changes and healthcare decisions in reaction to the results of their DNA test, so the accuracy of these tests may be more important than "woops, we placed you in the wrong European ethnic group".
And I guess we remove all oversight now because of a bad example? Pack it in, FDA. We're done monitoring food safety thanks to cookie banners.
You're being equally reductive. Obviously all of this exists on a spectrum. Most reasonable people would agree that the FDA is a net positive. Most reasonable people would agree that the government monitoring every private conversation happening in the country and jailing everyone who tells a lie would be a net negative. The happy medium for government checks on behavior is somewhere in the middle.
Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside. People make lifestyle choices based on things they hear from palm readers - we don't use the government to regulate and test the validity of psychics. I think this falls into the same category.
> The happy medium for government checks on behavior is somewhere in the middle.
That was my point.
> Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside.
I think we read different comments; all I got was a snarky condemnation of government oversight.
> Most reasonable people would agree that the FDA is a net positive.
A 1974 study by University of Chicago economist Sam Peltzman concluded that since 1962 the new rules had reduced the rate of introduction of effective new drugs significantly—from an average of forty-three annually in the decade before the amendments to just sixteen annually in the ten years afterward.
... The severest criticism leveled at the drug lag is that without access to a drug available elsewhere, seriously ill patients will suffer or even die. Peltzman raised the subject in his 1974 study. He noted pharmacologist William Wardell's estimate that because the relatively safe hypnotic drug nitrazepam was not cleared for use in the United States until 1971, five years after it was available in Britain, more than 3,700 Americans may have died from less safe sedatives and hypnotics. After earning the Nobel Prize for chemistry in 1988, U.S. drug researcher George Hitchings of Burroughs Wellcome Company said of an antileukemia drug he helped develop before the 1962 amendments: "We went from synthesis to the commercial drug in three years. That is absolutely impossible today."
At this rate, the benefits of FDA regulation relative to that in foreign countries could reasonably be put at some 5,000 casualties per decade, or 10,000 per decade for worst-case scenarios. In comparison, it has been argued above that the cost of FDA delay can be estimated at anywhere from 21,000 to 120,000 lives per decade. These figures would seem to support the conclusion that the costs of post—1962 regulation outweigh benefits by a wide margin, similar to Peltzman's results of a 4:1 cost-benefit ratio for the 1962 amendments.
This is the product they are offering, they're telling you which ethnic groups you match, this is what is being discussed, this is what is being criticized as inaccurate. What is being proposed here here is someone from the government to step in and look over their shoulder and make sure they approve of the methodology being used to generate the results.
If a company is making health claims there's already abundant regulation in place to prevent you from making unfounded statements. If you are making health decisions based on an Ancestry DNA test without consulting a doctor, you're clearly on your own.
they can't offer health results at all in most of europe and other countries. I don't think that's fair either, why is my government trying to protect me from learning anything about my genotype? Especially considering european countries that have socialized healthcare (and thus healthcare costs cannot be affected by letting this info out). This legislation does not make sense.
What you call "nanny" others think of as civilization.
But if you want to experience complete freedom, you can always go out there in the boondocks, dance with the wolves. Let that experience teach you the value and importance of the "nanny".
Fill me in on what value you see on having taxpayer money go towards making sure novelty DNA tests are "regulated" (and what could you even meaningfully regulate?)
This is what's known as a compound question, and it's almost like a way of straw-manning someone. GP didn't say he wants to regulate "novelty" DNA tests. That's the spin you're putting on his position, and you've asked your question in a way that he can't answer it without tacitly approving of your spin. It's like the classic "When did you stop beating your wife?"
Are these tests just novelty tests? Are they intended to be? Is it clearly communicated to consumers that they are just novelty tests?
If they are just novelty tests, but they're not being upfront about that when they take consumers' money, is that acceptable public behavior? If that's not acceptable behavior, what is the right social response? That's the point in the discussion where GP appears to be.
If these are novelty tests, and are being marketed as such, would you mind linking to a page on, say, 23andMe's website that describes them as such?
If not - if, in your own words, these are "novelty" tests but the companies involved aren't being forthright about that - do you think a government response is so inappropriate as to dismiss it out of hand?
>> I really don't want to pay more taxes for the Central Bureau of Making Sure You Know You Might Be 7% Less Eastern European than Your DNA Test Indicates.
I'm not sure if you're serious, but 23andMe also offers tests for genetic risk factors of some serious diseases. I don't think people can verify the accuracy of these tests in their kitchen.
> What is up with this incessant clamoring for a nanny to supervise every little thing?
The incessant clamoring of these companies to sell themselves as a medical device platform.
Snark aside, it's fine that they have things set up the way they do for ancestry reporting. A couple of percentages here or there do not make a lot of difference when you are using it in a recreational use case. When you start getting into reporting potential medical associations with SNP variants which could affect clinical treatment decisions, it stats to matter a lot.
I wonder why they didn't mention male microchimerism, a still poorly understand phenomenon, in which male DNA is found in females. Possible sources of gene transfer are " unrecognized spontaneous abortion, vanished male twin, an older brother transferred by the maternal circulation, or sexual intercourse"
311 comments
[ 3.5 ms ] story [ 289 ms ] threadIf the post analysis shows nearly identical DNA that's great.
The whole '% from this or that country' has to be necessarily fuzzy, because that's just a probability game anyhow.
Part of the old-school American tradition many regions (mostly rural, which is where over 80% of residents lived at one time) was to have a bunch of kids and encourage them to spread out as far & wide as they wanted; this still continues in some places. The logistics of 'keeping in touch with family' have some pretty hard limits past a certain point, and recordkeeping can be haphazard/fragile.
A great many people in North America have significant trouble tracing their ancestry back past their great-grandparents.
Have you looked at things like thermometers? They don't offer more than +- 2F accuracy for those under $100. Some meters, etc only offer +- 5% accuracy or worse. Those aren't shams.
Depending on the actual goal of having people submit all their information for entertainment purposes
And new relatives
Could have been worse- you could have said "female, we know..."
Besides the needlessly dismissive tone, you are neglecting how know where the family came from is important to some people
I thought it was pretty well known that these services are estimates only to a large degree? These services only take samples, they don't sequence the entire genome. Of course there will be errors when you have to extrapolate from a sample.
Even for me personally, my results changed when my parents did it and linked up to me, because their results were able to flow into mine.
I think part of why they used twins is to say its two different people so they'd have to treat it different.
I'm actually curious how repeatable the results are. If they are using sampling of portions of the DNA (are they?) how much variability is there in a single dna sampling enun through the algorithm 10 times? Similarly for how much variability in 10 dna samples from the same person?
Not if you sign-up with someone else's identity and pay with a different card. I very much doubt they go to any effort to verify the human the sample came from is the same human who paid and who the results are set to be sent to.
And there's a difference between "not perfect" and "doesn't provide the same result twice".
"doesn't provide the same results twice" is the honest acknowledgement that the concept of perfection is itself what is flawed here because it creates false expectations of what science can give us in this arena
The accuracy of the broad conclusions drawn from population wide samples about any single individual’s ethno-historical origins is a falsehood that science cannot ethically or logically support
Not getting the same result twice is pretty much 100% failure :-)
Frankly I am skeptical that they could even ascertain any semblance of your heritage but I'm definitely a layperson.
Reading other replies there seems to be a cheaper less accurate tech used for these home testing kits.
Basically all forensic science has turned out to be crap or misapplication of good science except for DNA which a) exonerates a heck of a lot of people and b) was a concept that matured in biology before being adopted by forensic scientists.
So are other forensic methods, like fingerprint[1] and hair[2] analysis. Turns out that forensic evidence isn't very scientific nor accurate.
[1] http://theconversation.com/fingerprinting-to-solve-crimes-no...
[2] https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-9...
DNA fingerprinting involves:
* Sequence amplification of polymorphic sites by a panel of well-characterized primers, many of which cross-check each other by amplifying over the same locus.
* These amplicons are then characterized by size via restriction digest and simple electrophoresis. No sequencing is performed, only the length of the DNA fragments is characterized.
* Due to population-level variation of short-tandem repeats, analysis of ~20 loci can uniquely identify an individual with a desired level of statistical confidence. The kind of data used to confirm a match in e.g. a murder trial will be absurdly robust.
DNA fingerprinting is limited by the quality of sample collection and handling (i.e. don't mix up or contaminate samples), but not the accuracy of the method.
Even sequencing a genome is not perfect - the current most common method works by cutting the genome into pieces, sequencing those and then reassembling them. A pretty obvious limitation is that you can't detect repeated sections.
I'm not sure what method the police use but it definitely isn't the same as what 23andme do.
So were the results of FBI hair and fiber analysis, and look how that turned out.
https://www.washingtonpost.com/local/crime/fbi-overstated-fo...
So I picked up a mid-range automatic bp cuff, and started to measure it more regularly. The numbers were all over the place and completely unrepeatable unless I maintained exactly the same position/breathing/etc. I can literally swing my BP by 30+ mmHg, by re-positioning my body, drinking a cup of coffee, etc, all while officially still maintaining proper technique for measuring it. And yes, moving up a cuff size also tends to remove a ~5mmHg from my measurement.
I could write a book about how to get any BP you want out of the machine. Its the same with weight, I can shift my weight +-10% by eating a bowl of salty popcorn drinking a few 64oz cups of ice tea, then eating a large meal. Then I can go for a 3 hour bike ride, use the toilet and drop it all back off.
So, its sort of surprises me the fine grained aspect of DR's always measuring your BP/Weight, neither of which really mean anything until they get into the extreem ranges, and in the weight case you can get a better feel for it just by looking at someone...
We would schedule people for Nurse only BP checks to help establish the pattern.
This statistical anbiguity is why the medical community as a whole is often sceptical of the Direct to consumer DNA testing and SNP analysis. They report data in a convincing way which is statistically misleading and in no way plainly reveals what is within measurement error.
>An entire DNA sample is made up of about three billion parts, but companies that provide ancestry tests look at about 700,000 of those to spot genetic differences.
>According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.
>Still, none of the five companies provided the same ancestry breakdown for the twins.
So these people getting results and finding out they aren't related to their parents, are the results suspect?
Ditto the results about genes associated with dementia or heart disease?
I'm aware of the companies having disclaimers, I thought that was on the basis of them not really know what the genes did, not that the test itself was rubbish.
Is it a case of them adding too many decimal places to the percentages? Making statistics out of noise?
These SNP arrays, however, are exceedingly accurate and reproducible in their calls. A common medium-density SNP array used for human genetics is the Illumina CytoSNP-850K [1], which routinely gets >99.5% accuracy on calls. This is at least as good, or better than, whole genome sequencing.
The challenge in ancestry assignment is not determining what is in the sample, but rather in the interpretation and assignment of ancestral haplotypes.
[1] https://www.illumina.com/content/dam/illumina-marketing/docu...
>99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers.
Or potentially put people in jail based on these results...
You can get considerably better accuracy, you just have to be willing to fork over $1500 rather than $60. How much are you willing to pay to know if your 31% German vs 30%?
There was a good 5-10 years where microarrays were the go-to method for genomic investigation. Since RNA-seq has supplanted it, there has been no revolution in overturning those old results. At best, RNA-seq reveals what microarrays cannot: novel sequences, specific sequences that have difficulty hybridizing, etc.
The variation between these services is due primarily to how the data is interpreted and presented to customers, not the reads themselves.
In this case "SNP microarray" is a "digital" (or binary) readout of the genomic DNA sequence; specifically the array has attached to it sequences that match both the "reference allele" (e.g., a "G") and the "alternate allele" (e.g., "A" or some other base). Note that across the world's population, most sites in the human genome are "bi-allelic" meaning that there's really only two variants at each site, though of course it is possible to have tri-allelic or even fully polymorphic (A, T, C, or G) sites. For the purposes of ancestry, bi-allelic identification of commonly polymorphic sites is adequate, though the more sites and the rarer those sites are in world populations, the more specific you can be re: ancestry. This is why the gold standard for ancestry and haplotyping is whole genome sequencing-- you simply get far more informative sites to map to other populations.
When my doctor asks how tall I am, if I'm 6'1", then 6'1" is a good answer. 6' (or even better: about 6') can also be an acceptable answer, as it's an approximation. I wouldn't say I'm 6'8", and then try to rationalize it by saying my tape measure is cheap, and anyway it's still right to about 10%.
Reporting what you don't know can be just as important as reporting what you do know. It doesn't matter how good your laboratory method is if you can't report results in a way that doesn't mislead customers.
You're not going to get a very high degree of accuracy from a ~$100 test. They're not sequencing your entire genome for that amount of money. It is still good enough to determine the broad strokes of someone's ancestry, to within a few percentage points as you see in the linked article.
If you're willing to pay for full sequencing then you're going to get a lot more accuracy and repeatability. You're also going to pay a lot more for it than would be warranted just to satisfy your curiosity as to where your ancestors came from. These kinds of tests would really only be warranted if you have some rare, difficult-to-treat genetic disease. That might not even help you directly, but rather, be done to contribute to the body of scientific knowledge of the disease.
If you do it 10 times, you will always get the same buckets, but the percentages will differ back and forth for a bit. If you're goal it so find out your heritage, which these services are offering, then you are getting the right heritage repeatedly.
If your goal is to find out how many 0.1% of you is of an exact descendance, then you're going in with the wrong expectations.
When you sample a genome, you're checking for one specific string. Traditionally this is done by mixing a sample of the DNA with an enzyme that breaks apart a specific DNA sequence and then measuring whether the size of the DNA molecules have changed. Take the string that I mentioned above (FOO). Suppose that I mix it with an enzyme that cleaves along "GATTACA" such that "GATTACA" becomes "GAT| |TACA". If I mix a DNA sample containing the string FOO with the enzyme I just mentioned, then the molecule would be split into: "....GAAAAGATACCCACAGAGAT| |TACAAAAC...." If I mix a DNA sample that does _not_ contain FOO with the enzyme, no change occurs.
You know when you watch CSI shows* and the DNA evidence is presented as a bunch of light and dark bands? Those are produced by mixing the suspect's DNA sample with a standard cocktail of enzymes, treating them so that the DNA molecules are electrically charged (DNA might already have a charge, not sure), placing them at one end of a container of a standard gelatin with known pore size, and applying an electric field for a standard amount of time. The cocktail of enzymes cleaves the DNA at multiple different substrings and breaks down the DNA strands into multiple smaller substrings. Different size molecules will progress through the gelatin at different rates, so you can effectively compare two people's DNA to see whether the substrings match. You _don't_ know the person's genetic sequence, just whether or not his or her's DNA contains the same substrings as the DNA of the person that you're comparing it to.
* I'm assuming those TV shows present the DNA evidence this way. I don't watch them.
Disclaimer: I'm not a biochemist by any stretch. This is based on my memory from school, so don't take my explanation as absolute truth :-)
You have described sequencing and fingerprinting. These services use SNP sampling.
Sampling has been described several places in this thread so I won’t repeat that here;
Sequencing: https://en.wikipedia.org/wiki/Whole_genome_sequencing
Microarrays: https://en.wikipedia.org/wiki/DNA_microarray
* Full DNA sequencing. This is a multi-million dollar effort taken on by large collaborations, usually to generate a reference genome sequence for a species or particular line. Typically a variety of methods are used, from 'shotgun' style illumina, 454 longer reads, and now some newer even-longer-read methods. Long reads are important for generating accurate assemblies. Generally, some tricky portions will need to be meticulously amplified and sequenced with traditional Sanger sequencing to clean up. This gives you a complete sequence, including all inter-gene sequences, repetitive portions, etc.
* RNA sequencing. This method is the core of many molecular biology labs; you take expressed sequence (i.e. RNA that has been transcribed from DNA), reverse transcribe it to DNA, and then sequence the DNA. This gives you sequence information about the expressed part of genes (this is limited, though, because during the assembly process you'll reject many reads that don't map onto the reference sequence in standard analysis) but mostly gives you an idea of the relative expression of different genes. When people talk about gene expression going up or down in certain mutants, conditions, etc., they're usually talking about RNAseq (or qPCR if only a few loci are involved).
* 'Sampling' in this case refers to chips with various DNA fragments on it. You pass the sample DNA over it, and compatible sequences hybridize together, which is then detected via florescence. These chips are designed to carry sequence fragments that vary in the population. Together, these variants form a DNA profile. The quality of this profile depends on the design of the chip. The sequences included are generally loci that are known to affect particular diseases or to be especially variable in the population for assessing ancestry.
In cost magnitude, this goes from millions to thousands to tens.
The price of a whole genome sequence has fallen dizzyingly fast, faster than any technology of which I’m aware. We’re close to the point where it’s priced like any basic blood test. Exciting times.
sampling is a high resolution photo with large holes in it, a full sequencing is a full photograph with varying resolution and quality etc. There are still some areas of the genome that are inaccessible even with full sequencing. Improvements are being made with long read sequencing.
I've heard stories of affairs being discovered through DNA testing. Curious if the denials might have validity.
If two samples had enough similarity to say someone is a child of someone else, my guess is that's pretty accurate.
The whole concept of geographic ancestry may have fundamental limitations at the level where people are quoting percentages.
Do you have any personal experience with this? Or anything you can cite? I'd be interested.
The credit given to the intelligence of society at large here falls somewhere between "cute" and "astounding".
I would recommend flipping through a tabloid and then considering such content sells millions of copies each print. And then reconsidering your thought that it's "pretty well known that these services are estimates only to a large degree".
You could easily make the point that many people won't understand something without coming across as judgmental and elitist, but failed to do so.
He said "Of course there will be errors when you have to extrapolate from a sample." The common man doesn't know what extrapolate from a sample even means!
I'm sorry but suggesting society as a whole understands genetic testing methods and techniques is just so far absolutely removed from reality (you're perhaps justified if you work for Illumina or are a faculty member of a University) that the point needed to be made even if harshly. Maybe I just interact more with "non hacker news" types than the average poster here, but that post quite frankly seemed so absurdly out of touch as to remind me of the Lucille Bluth quote from Arrested Development - "I mean, it's one banana Michael. How much could it possibly cost? 10 dollars?"
Sorry I couldn't help myself. :)
For example, you're happy to fire off about how dumb everyone is for reading tabloids because the finer points of how people unlike you consume media are not particularly important. And, likewise, the typical person buying a DNA test as largely a conversation starter isn't going to invest a great deal of effort to understand how DNA tests work.
Tragically people are notoriously bad at long term predictions of what will matter to them and how that relates to a broad corpus of knowledge or habit of skepticism. The result is that people are so ignorant they’re under an illusion of being knowledgeable, and at that point they’re more or less a lost cause.
https://en.m.wikipedia.org/wiki/Dunning–Kruger_effect
It might help to know enough about DNA to wish to avoid these services, especially given how they cooperate with law enforcement. Mostly though, it’s just really hard to calibrate your corpus of useful knowledge from a position of profound ignorance. Plus a lot of people really are quite stupid, and ignoring that in favor of some superficially positive message about mutual respect is potentially dangerous.
I'm not a biologist, but apparently it's not the case that twins have exactly identical DNA:
"Identical Twins' Genes Are Not Identical" ( https://www.scientificamerican.com/article/identical-twins-g... )
My wife did the same with the same result, as did a friend who had purchased the original 23andMe kit and wanted to get the newer data from the new chip.
So while I don't doubt the accuracy of the linked article, my anecdotal experience is that the 23andMe DNA tests have repeatability at least.
(1) reasonably guarantee that my data not only won't be distributed - but will be destroyed (on their side) within a short time frame
(2) give me access to all of the raw data and methodologies used in their analysis
Does anyone know of such a service?
The fact that none of these services allow for anonymous submission should be a huge tell about their real motivations.
But ultimately any kind of DNA analysis (that exposes the raw data) is intrinsically de-anonymizable, no?
Sure, you can be anonymous, but you're the son of John and Jane Tompkins, the brother of Emily Tompkins, the cousin of Mary Smith, and the father of Dave and Buster Tompkins.... huh, I wonder who you are.
You're incorrect. At the company I work for, which is one of them mentioned in the article, a customer pays for DNA sequencing and analysis. Who pays for it is often separate from who's being analysed. We often have customers who pay for products for their family, friends, or otherwise. The name on the DNA sample is not verified in any way shape or form (and how could we verify it beyond just billing information?) You can opt-in or opt-out of public matching. You can download your raw data anyway (GDPR requires this if you're an EU citizen).
> should be a huge tell about their real motivations
I would like it if you'd elaborate on this.
I don't have an opinion about this, but they're probably implying that the business model is the same as Google, Facebook, et. al., which is selling user data to third parties.
Not surprisingly, the results are not reliable.
Feed the unreliable result into any classification algorithm, and you still get unreliably results (actually I was expecting worse)
Not only that, but you only read small parts- SNPs. So there is no bijection; just a rough correspondence. On the bright side, it's good for privacy.
I think you’re describing short-read sequencing here but this isn’t what any of these services does. They do microarray genotyping rather than sequencing. And in neither case is there “inference”. True there’s something called imputation but it works quite differently and I don’t think (though I don’t know) that 23andme uses it for ancestry. I’m also not sure what you mean by “bijection” in this context.
Carly's results have a larger 'Broadly European' percentage, but that is a catchall that includes: Italian, Eastern European, Balkan, French & German, Iberian (Others category), and Broadly Southern European (Others category). Therefore Carly's 'Broadly European' traits could be used to match Charlsie's results.
It is unfair to call the 23andMe results 'different'. They are getting the same results, however one twin is getting more specificity.
There was a controversy about Korean & Japanese ancestry results from 23andMe where ethnic Koreans are sometimes classified as >30% Japanese. These two groups shared common ancestors since ancient humans migrated via the Korean Peninsula to the Japanese archipelago centuries before the modern nations of Korea and Japan came into existence. Slapping either one of these specific labels on shared ancestry is wildly inaccurate and short sighted.
Years of highly visible advertising by these companies which implies they're accurate? All of those “discovery your ancestry” posters they put up around here had specific countries and locations.
The landing page shows me this headline "This new year, commit to a healthier you - inspired by your DNA." I'm sure they have it legally covered somewhere in their TOS but it's still not good if these are inaccurate or estimates.
This back of the envelope estimate suggests that 99.6% similarity for somatic samples of identical twins is not unexpected.
The other factors you discuss are more likely to cause issues. One major one is the error rate of the microarray because of the high number of genotypes being assayed. If the chip has an error rate of 0.1% and 500k tests are performed you would expect 500 mis-calls. I used to work for Illumina, who makes these chips, but I worked on the DNA sequencing side of things so I don't know the actual error rate off the top of my head.
Well, they all gave different readings. I repeated the running in place test. Some went up and down much more than the others; maybe 10 bpm more. I tried re-arranging them on my arm and doing the running in place test again, you know, maybe it only works on the wrist since the blood is closer to the surface there? Nope, the readings were all over the place still. It was not super scientific of me. I mean, I was some strange person running in place in a store with a bunch of iWatches strapped to my arm.
Maybe the watches have some sort of learning algo in them that was getting messed up with all the folks trying them on. Maybe I was wearing them 'wrong'. Maybe I should have waited 10 minutes between jogging sessions. I don't know.
But, to me, these things have a long way to go. A lot of the bio-tech and bioengineering out there seems to be a bit of snake-oil right now. I really do want it all to work, I think that would be a great boon to us all [0]. If something like the iWatch can't give repeatable readings in nearly any way, then it's all just a gimmick and not useful.
[0] I mean, can you imagine if Theranos' tech actually did work?! That would change medicine for the better in incalculable ways. I can see why that company's plan was so intoxicating.
Far from being just a doodle, the PPG readout ("ECG-like trace") comes from changes in skin reflectivity that indicate relative changes in blood pressure. [1]
Obviously this isn't as good as an actual ECG (and can be disrupted by movement), but it's far more information than is needed to accurately track heart rate. These devices can be as accurate as a pulse oximeter [2], though they don't account for the movement of the user.
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4822420/ [2] https://www.ncbi.nlm.nih.gov/pubmed/28464700
Unrelated: It is called the Apple Watch. :)
There are tons of people that test these devices against chest straps online, and almost all of these bands give results that are surprisingly accurate.. or at very least about what you'd expect out of a wrist based monitor.
ML only works for these types of measurements when you have some kind of reliable source of truth or fitness function to compare against.
I can't say that I know exactly how these things work, but I wouldn't be surprised if they were built to work with the blood vessels and skin thickness of a wrist, where watches are worn, rather than the skin thickness+hair+different vessel locations of a forearm.
Different parts of the body are used for different things. It's why blood oxygen monitors that work on fingers don't work on toes.
An iWatch is not the pinnacle of wrist biometrics, however. I'd be a lot more concerned if a high end Garmin got it wrong, that's their whole business.
Still not nearly as accurate as the chest-based models as mentioned elsewhere, but for 95% of people it's plenty.
Wrist devices will give inaccurate pulse counts if the band is too loose, skin is dirty/wet, or your arm is moving around quickly. If you need accurate numbers in all circumstances then a chest strap which detects electrical signals is a much better option. I usually wear a Garmin HRM-Tri strap while exercising and it transmits a wireless signal to my fitness tracker.
The only proper comparison would be with a chest strep, which are much more accurate.
Also, it really differs per person and how you wear the watch. Just wearing it a bit differently can change the results significantly.
Sweat also gets them messed up easily.
All reasons why, if you want accurate data for workouts, you should always use a chest strep. Even if they are much more awkward.
I've always used a chest-based heart rate sensor until I started trying wrist-based about 4-5 years ago. I've used countless brands and models (ranging in price from about $49.99 up to about $399) on my wrist - and literally none are as accurate as the chest-based strap. The chest strap responds immediately to increases and decreases in heart rate. Whereas the wrist sensors always seem to lag, sometimes by up to a minute or so.
The wrist-based sensors always seem to differ depending on where on the wrist they are worn - high, low, inside of the wrist, outside of the wrist, etc - by maybe 5 bpm or so.
In either event, if you really want to test the accuracy of the wrist-based sensor, you should really compare it to the output of a chest-based strap.
In fact, the results will largely be personal - because the electric signals may differ per person greatly in different areas of the arm.
Needless to say, my apple watch is worn daily, but when riding, I have gone back to a chest-based strap.
That all being said, there is a guy that reviews these gadgets and will often wear 2-4 sensors while working out, and post an overlay of the results from his workouts. The baseline should generally always assume that the chest-based straps are accurate. His site is called DCRainmaker, if anyone wanted to look up his comparisons/reviews.
A faster response could be got by measuring the time between heartbeats (probably a low but even number of beats).
I mentioned it to the employee who was manning that table, and she told me the demo watches had some pre-programmed data for demo purposes.
I was puzzled by this since heart rate monitoring is a key feature users care about. It occurred to me that heart rate data of everyone who has tried on the watch might count as the kind of medical information you shouldn’t publish on demo watches.
Imagine ridiculous stories like “CEO of X’s heart rate was 130bpm when he tried on the new Apple Watch, says journalist who tried it on right after.”
Although people might give you strange looks :)
https://www.macrumors.com/2017/05/24/apple-watch-heart-rate-...
That's an overly generous explanation. Fitbit-like devices are notoriously inaccurate[1], and the simplest explanation seems to be that their inaccuracy is unintentional.
[1] https://www.independent.co.uk/life-style/health-and-families...
https://hipaahealthlaw.foxrothschild.com/tags/class-ii-medic...
Are we claiming that every product needs 2 FDA applications: one for the end user and one for a special run of demo watches? I believe (but did not check) that demo watches would be either exempt or go unnoticed considering the amount of data it could gather...
The wearing a bunch of them at the same time is going to be wrong, yeah. Wrist mounted fitness devices are calibrated to read on the top side of the wrist. Even reading on the bottom side of the wrist would be a bit different, way up the arm? Very different.
It is possible to make accurate devices that read there, they just need to be constructed to do such!
Wearables are also meant to give an accurate rating over the long term. If you go running for 30 minutes, are the overall calorie burn, average HR, max HR, and min HR, all correct? Tracking in perfect sync with the heart is nice, and of course everyone tries to do that, but if the number is off for a second here or there, well so long as the overall result is the same the usefulness hasn't really changed.
Finally, due to the way optical HR works, random unexpected, movements are really hard to deal with. It can take some time for an HR sensor to lock on to the heart rate. This isn't too long (measured in seconds, and each generation of devices is getting better and better), but it relies on regular patterned movement, such as arms pumping while running or walking, the regular motion of a bench press, etc.
Stopping and checking the HR value every few seconds to see if it is correct is literally the worst case scenario for a lot of wrist mounted sensors!
Ground truth is strapping a heart rate strap to your chest, putting on the wearable, going for a run as you normally would, and comparing the two at the end.
Multiple sources have done this, and found the Apple Watch to be best in class when compared to directly measuring HR.
tl;dr: Stopping and checking wrist mounted devices can confuse them, Apple Watch, and everyone else in this field who is serious, are tested against HR straps and they come within spitting distance of accuracy.
[1] https://www.mdpi.com/2075-4426/7/2/3
What I found was that the heart rate correlated well with taking it manually and/or a finger heart monitor (pulse ox cheap from Amazon).
But with both devices you could actually tell when it was clearly wrong independent of comparison. The point is you more or less (at least I found) could figure out when it is right and wrong just based on past use and what you would expect (feeling your pulse). So if it was off it was grossly off and obvious. That is my experience with the watch that I have had for years. Either 'correct' or 'vastly off and easy to tell that is the case'.
> According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.
If they were siblings, ~50% of the (several hundred thousand) SNP calls would be the same, not 99.6%. (Which is about what you would expect from identical twins, since that's where either one disagrees with the other, and errors usually don't strike twice, so a 99.6% overlap roughly implies a per-SNP accuracy rate of sqrt(0.996) = 99.8%, which sounds reasonable given the difficulty of sequencing and the inherent noise & randomness.)
On a fun side note, identical twins are not exactly 100% genetically identical, which is how rapist/murderer identical twins are being convicted these days based on DNA evidence; but there are so few unique mutations to each twin, and the new mutations wouldn't be on the common-SNP tests, that you have to do the equivalent of like 4 whole-genome sequences (IIRC, you need coverage of ~120x vs the more usual WGS of 30x) to get enough evidence to override the prior of any difference being just a repeated sequencing error, and it's very expensive.
I was watching a BBC documentary about how Rome influenced the history of Scotland. The announcer, who has a very pronounced Scottish accent, took a DNA ancestry test. The results (IIRC) indicated Germany, Italy, and eastern europe. (The announcer interpreted this last one as Dacia) Come to think of it, he doesn't look so different from Metatron.
The twin results not matching is kinda disturbing, though.
This reminds me of when economics try to predict what the GDP will be next year. It's almost always wrong, so why would ever listen to anyone who is always wrong?
It's mostly a scam, in my opinion.
Also, you should not underestimate how not knowing where you ancestors came from is disturbing for many people.
EDIT: for those who disagree, check what just the 1st two PC can give for European DNA after a simple rotation of the axis to match a map : http://blogs.discovermagazine.com/gnxp/files/genmap1.jpg
I think it's fair to say there is a scientific base.
I don't think it's a good thing for people to attempt to derive any aspect of their identity from their genealogical ancestry, it has no bearing on who you are as a person, and has the potential for great harm. I've seen it used to give people a false sense of culture and belonging to a community that they previously had nothing to do with. If you felt a connection to a particular culture or identity, should we really be telling people that their genome is responsible for determining whether or not they can associate parts of those cultures with themselves?
Racism results from a failure to separate identity from genome. If you listen to the dogma of a white supremacist, it's heavily predicated on associating the accomplishments of their "European" forefathers with themselves. If you listen to the dogma of a black supremacist, it's heavily predicated on assuming the identity of the Egyptians or Israelites and associating their accomplishments with themselves.
I really think that if an individual is truly "disturbed" by not knowing the precise nature of their lineage, that is an individual in search of an identity on a plate, and therefore highly corruptible. The motivation to learn more about one's ancestry should ideally not surpass the level of an intellectual curiosity.
I still think it can have a positive role, if only to help the person emotional investment.
> Is the usual "I'm part Irish part German part Cherokee" meaningfully more satisfying than "I'm west African?"
For that, "I'm west African" is less satisfying than "I did trace my roots to Mali" which can lead the person to study the history of the country, Timbuktu, etc.
Theoretically, they are similar, but since you inherit a random 50% of DNA from both parents, on an individual scale, you might inherit a misleading percent from your ancestors.
Let's say 3 of your grandparents are all full-blooded Italian. Like, their families have been in the same village for centuries. And one of your grandparents is from England (and is similarly from an old family in a small village).
So, theoretically your DNA results should 75% from that region of Italy and 25% from English region.
However, lets say from your mom, the parent who is 50% English and 50% Italian, you only inherit a mix of 15% Italian and 35% English (15% + 35% = 50% mom's DNA). It's random, so this is possible, especially on a small sample of ONE person.
And from your Dad, who is 100% Italian, you obviously inherit 50% of your DNA as from that Italian region.
Now, you theoretically should be 75% Italian and 25% English, but since it's random, your true DNA reads as only 65% Italian and 35% English.
This is why it's mostly bullshit to me. It's impossible to know what percents your DNA randomly passed on to your ancestors.
Sure, in our example, the two places where correct, Italy and England, but the percents are way off.
And that's honestly WHY we get these tests. We generally know where our families come from, we just have no idea what the percents are.
So, the whole point of the service is stupid since they equate DNA with ancestry.
No, not everybody.
Even percentage that are a bit off are better than nothing at all.
It's hard for me to imagine why anyone would feel any significant connection to an ancestry that they needed a DNA test to know they had.
Some people have not even such luck, and do not know their ancestors at all. Records are incomplete and some people lie. So instead, they try to grasp to what they can, using anything they can. DNA is but a tool there.
For me, DNA offers answers to questions about my family history, things that otherwise I would be unable to ever know.
I find it psychologically helpful. It helps me relate to ancestors I didn't know I have: who they were, what they were doing. When did they immigrate, what were their hopes, their lives, their challenges.
For me it is part of a healing process.
I don't think it will turn me extremist. And the data is not perfect. But it just feels nice to be able to relate to others when they tell you they are from xx country, their father from yy, their great grandfather from zz, etc. It lets you joint the common story.
Some family have had complex paths, with records lost. Dead ancestors can't tell you much about where they came from. I think it's just sad. I would prefer anyone to have access to the possibility if they want
Identical twins are supposed to have identical DNA, right? Or did I really misunderstand a lot of school.
Personally, I would love to help create legislation aimed to protect consumers from perceived and actual problems related to DNA sequencing and analysis. I don't know how though and I fear it'd fall to the back burner in the current US political climate. Our customers are not only US but international though and that presents yet another dimension to the challenges therein.
This article states:
> Whatever your ancestry results, don't get too attached to them. They could change.
That is absolutely correct. I've stated why in a previous comment [1]; Some DNA analysis software employ stochastic algorithms. That means that the answer they provide can be different if run more than once.
The article also states:
> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.
That is also correct. I've heard peers state that the industry isn't regulated outside of the field of medicine during product discussions. I think that's something which needs to be addressed: leaving it unaddressed can encourage predatory business behavior.
[0] https://news.ycombinator.com/item?id=18196717 [1] https://news.ycombinator.com/item?id=18196984 [2] https://news.ycombinator.com/item?id=18564380 [3] https://news.ycombinator.com/item?id=18569659
They are not allowed to sell health-related reports anymore in all of europe and other countries. While i think that some regulation and standards would be good, banning it overall was a dumb measure.
https://int.customercare.23andme.com/hc/en-us/articles/21769...
It would be helpful if folks would state their credentials/experience with respect to the subject area in their comment.
Mine: none whatsoever, but am interested in reading informed discussions with folks with either credentials or experience in subject matter.
So essentially just like every thread in every online discussion group then?
> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.
> It's a situation Gravel finds troubling.
What is up with this incessant clamoring for a nanny to supervise every little thing?
This is published on a page where, thanks to "oversight", I have to click a little spam box warning that this page, like every other damn website on the internet, uses cookies. No thank you. I really don't want to pay more taxes for the Central Bureau of Making Sure You Know You Might Be 7% Less Eastern European than Your DNA Test Indicates.
And I guess we remove all oversight now because of a bad example? Pack it in, FDA. We're done monitoring food safety thanks to cookie banners.
Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside. People make lifestyle choices based on things they hear from palm readers - we don't use the government to regulate and test the validity of psychics. I think this falls into the same category.
That was my point.
> Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside.
I think we read different comments; all I got was a snarky condemnation of government oversight.
A 1974 study by University of Chicago economist Sam Peltzman concluded that since 1962 the new rules had reduced the rate of introduction of effective new drugs significantly—from an average of forty-three annually in the decade before the amendments to just sixteen annually in the ten years afterward.
... The severest criticism leveled at the drug lag is that without access to a drug available elsewhere, seriously ill patients will suffer or even die. Peltzman raised the subject in his 1974 study. He noted pharmacologist William Wardell's estimate that because the relatively safe hypnotic drug nitrazepam was not cleared for use in the United States until 1971, five years after it was available in Britain, more than 3,700 Americans may have died from less safe sedatives and hypnotics. After earning the Nobel Prize for chemistry in 1988, U.S. drug researcher George Hitchings of Burroughs Wellcome Company said of an antileukemia drug he helped develop before the 1962 amendments: "We went from synthesis to the commercial drug in three years. That is absolutely impossible today."
http://www.econlib.org/library/Enc1/DrugLag.html
At this rate, the benefits of FDA regulation relative to that in foreign countries could reasonably be put at some 5,000 casualties per decade, or 10,000 per decade for worst-case scenarios. In comparison, it has been argued above that the cost of FDA delay can be estimated at anywhere from 21,000 to 120,000 lives per decade. These figures would seem to support the conclusion that the costs of post—1962 regulation outweigh benefits by a wide margin, similar to Peltzman's results of a 4:1 cost-benefit ratio for the 1962 amendments.
"The Safety and Efficacy of New Drug Approval", https://pdfs.semanticscholar.org/feb9/53ab293f24a14b3b3d4b74...
If a company is making health claims there's already abundant regulation in place to prevent you from making unfounded statements. If you are making health decisions based on an Ancestry DNA test without consulting a doctor, you're clearly on your own.
https://www.fda.gov/ICECI/EnforcementActions/WarningLetters/...
https://int.customercare.23andme.com/hc/en-us/articles/21769...
But if you want to experience complete freedom, you can always go out there in the boondocks, dance with the wolves. Let that experience teach you the value and importance of the "nanny".
Are these tests just novelty tests? Are they intended to be? Is it clearly communicated to consumers that they are just novelty tests?
If they are just novelty tests, but they're not being upfront about that when they take consumers' money, is that acceptable public behavior? If that's not acceptable behavior, what is the right social response? That's the point in the discussion where GP appears to be.
If these are novelty tests, and are being marketed as such, would you mind linking to a page on, say, 23andMe's website that describes them as such?
If not - if, in your own words, these are "novelty" tests but the companies involved aren't being forthright about that - do you think a government response is so inappropriate as to dismiss it out of hand?
I'm not sure if you're serious, but 23andMe also offers tests for genetic risk factors of some serious diseases. I don't think people can verify the accuracy of these tests in their kitchen.
The incessant clamoring of these companies to sell themselves as a medical device platform.
Snark aside, it's fine that they have things set up the way they do for ancestry reporting. A couple of percentages here or there do not make a lot of difference when you are using it in a recreational use case. When you start getting into reporting potential medical associations with SNP variants which could affect clinical treatment decisions, it stats to matter a lot.
It is because corporations keep lying and behaving horribly.
https://www.ncbi.nlm.nih.gov/pubmed/16084184/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3458919/