"In creating a DNA profile for the victim, Kim had typed three separate samples, two from blood and another from saliva. The resulting DNA profiles, which should have been identical, varied substantially. This alone was cause for serious concern—if the tech couldn’t be trusted to get a consistent DNA profile from a single person, how could she be expected to make sense of a complex mixture like the one from the vaginal swab?"
It is NOT "false promise." (Or at least you don't yet KNOW if it will eventually be "false.") It may not be working well NOW, which is unfortunate.
Let's not forget how much people (like Marvin Minsky) crapped all over neural networks. And now they're beating professional Go players and doing Natural Language Parsing better than anyone else.
Let's not forget how much people (like Marvin Minsky) crapped all over neural networks
AIUI, Minsky and Papert "crapped all over" perceptrons specifically and not neural networks in general. It's often been said that the idea that "Minsky said NN's don't work" was an erroneous interpretation by others in the field.
Title should read, "The False Promise of Flawed DNA Testing". It isn't surprising to see prosecutors resort to underhanded methods in their zeal for imprisoning others.
The basic science is sound. However, there are gaps in it. For instance, most DNA testing techniques do not account for methylation patterns in DNA samples. This is important for profiling epigenetic traits.
I don't think anybody is arguing that DNA testing could never be useful. What this piece is exposing is that a system we trust to be 99% accurate (because the science, if properly executed is) becomes tainted by improper procedures.
We need to determine processes that make bias impossible. For example, samples should be sent without any markings, directions, or names. Simply to be "compared." Periodically, an oversight group should send known matching or not matching samples, and if an incorrect result is returned by the lab, the analyst should be fired. Should multiple analysis failures happen under one manager, than manager should be fired. This isn't rocket science.
I'm always shocked that there is no auditing going on at all. If a lab isn't blind audited periodically how can the 'evidence' be considered admissible?
There are real consequences to ascertaining the accuracy of such evidence. Cases get reopen and verdicts thrown out.
For years, prosecutors told jurors that DNA and hair tests were much more accurate than they truly were, some cases claimed that they were 100% accurate. Jurors then convicted possibly innocent people based on that lie.
The result was many convictions can now be overturned, prosecutors reputations sullied and millions of dollars wasted.
The people that pay to keep these labs running have no interest in digging that deep.
Honestly the technology is also somewhat flawed. There are much better methods that could be used, and would be less prone to flawed procedures. It's just that forensics so highly conservative and unlikely to move to new platforms.
The article points to cases especially where mixed or degraded samples can be interpreted incorrectly as the technician is forced to make a judgement call. This actually seems amenable to statistical analysis to me however, so I'm wondering why that doesn't happen here. You should still be able to assign probabilities based on the evidence.
Perhaps they still present things like 60% or 40% chance of match to the jury and they just hear "DNA match" and tune out the rest. I've said this before but I'm beginning to suspect that along with a public defender there should be a public forensic scientist assigned to defendants.
It is to be frank crazy that we're talking about anything less that a 99.9% match with today's technology.
We have platforms that for less that 1000USD can sequence (read) a complete human genome. Each individual has millions of mutations which make them unique. Each position is read with >99% accuracy. Just to make sure in general each position is read >20 times.
For forensics it would be like comparing two very long strings for (very nearly) exact identity.
The only reason we use techniques like DNA fingerprinting is because that's what was feasible in the 1980s and forensics is an extremely conservative discipline.
Maybe I should clarify my thinking here. I mean that if a jury are told that the DNA matches the prosecuted with 99% certainty, then most people would interpret that as the person is guilty without doubt. But if you said that theres a 99% statistical probability of two of the 57 persons in the same court room where born on the same date, then I think most people would be very doubtful, and even if you checked and it showed that indeed two people in this room where actually born on the same day, people would see that a coincidence, not statistical proof.
Assuming DNA matching has 99% certainity. Are you saying if I get DNA samples from 57 different people and match each of the 57 samples in all possible combination of pairs (so 57*56 tests), then there is 99% chance that I'll find 2 samples which will match?
Someone should do this experiment. I think it's very scary if this is true.
It's infuriating to read about DNA fingerprinting being used such critical cases when DNA sequencing could be used to far more accurately and robustly compare sample and targets identities.
Fingerprinting uses common relatively large changes in the genome as proxy for an individuals identity. It uses these locations because that's what you could do in the 1980s easily. It's possible to screw up both the experiments and interpenetration relatively easily.
Technology has moved on, you can now sequence a complete genome for around 1000USD. Comparing complete genomes would unambiguously assign identity. You'd effectively just be doing a string comparison over two very long, unique, strings.
The only reason we still use fingerprinting is that forensics is highly conservative. I'd guess it'll take at least another 10 years for them to move to sequencing (if not longer). It is however infuriating.
> Comparing complete genomes would unambiguously assign identity.
Would we ever get full genomes from any sample though? As far as I remember, most sequencing doesn't happen on full strings. Instead, many pieces are first unwound, then split into pieces, then sequenced one after another and the whole genome is reconstructed from overlaps. So that's still assuming you have enough samples that you can reconstruct the whole string. And that you can tell the samples apart if you have a mixture like in the first case from the article.
It looks like the lower bound for commercial whole genome sequencing is about 250 nanograms. To put this in perspective, you can get 4000 nanograms of DNA from a 2 microlitre blood draw.
Your mental model of sequencing needs an update. :) Prior to sequencing, DNA is amplified via PCR. This dramatically increases the amount of DNA you have (2^30, iirc.)
You're talking about a clean blood draw though. I don't know how well it compares to dried cells you left somewhere for hours which are possibly mixed with many other people's cells. Do you have information about that?
Yes, there would be rare edge cases. But they should be rare and well characterized. Unlike the current situation.
Practically how the comparison is done depends on the informatics (yes it's unlikely to be implemented as a X~million symbol string match, more likely as comparing lists of called SNPs against a reference).
One other point about twins. Twins while genetically similar, are not identical [1]. I'd guess that the differences are still significant enough to be able to assign identity through whole genome sequencing.
>"Geneticist Carl Bruder of the University of Alabama at Birmingham, and his colleagues closely compared the genomes of 19 sets of adult identical twins. In some cases, one twin's DNA differed from the other's at various points on their genomes. At these sites of genetic divergence, one bore a different number of copies of the same gene, a genetic state called copy number variants." //
So, it's "in some cases" monozygotic twins differ by having copy number variants [TIL! thanks]. So, not all cases (according to this citation) and so the point still stands that "Comparing complete genomes would unambiguously assign identity" appears to be false (albeit at a very low false match rate).
Interestingly the article says that copy number variants appear to arise as a subject ages. Doesn't this mean that different cells in the body will have different multiples of particular genes; so a mismatch wouldn't necessarily mean that the DNA wasn't from the same twin?
"A recent study in the journal Criminal Justice Ethics notes that in North Carolina, state and local law-enforcement agencies operating crime labs are compensated $600 for DNA analysis that results in a conviction"
This was by far the best and funniest part. What would happen if a suspect offered to pay the same lab $600 if there wasn't a conviction?
32 comments
[ 2.9 ms ] story [ 80.7 ms ] thread"In creating a DNA profile for the victim, Kim had typed three separate samples, two from blood and another from saliva. The resulting DNA profiles, which should have been identical, varied substantially. This alone was cause for serious concern—if the tech couldn’t be trusted to get a consistent DNA profile from a single person, how could she be expected to make sense of a complex mixture like the one from the vaginal swab?"
Let's not forget how much people (like Marvin Minsky) crapped all over neural networks. And now they're beating professional Go players and doing Natural Language Parsing better than anyone else.
AIUI, Minsky and Papert "crapped all over" perceptrons specifically and not neural networks in general. It's often been said that the idea that "Minsky said NN's don't work" was an erroneous interpretation by others in the field.
The basic science is sound. However, there are gaps in it. For instance, most DNA testing techniques do not account for methylation patterns in DNA samples. This is important for profiling epigenetic traits.
Ref.: http://www.atdbio.com/content/56/Epigenetics
When you're trying to determine identity I'm not sure that methylation has much value.
We need to determine processes that make bias impossible. For example, samples should be sent without any markings, directions, or names. Simply to be "compared." Periodically, an oversight group should send known matching or not matching samples, and if an incorrect result is returned by the lab, the analyst should be fired. Should multiple analysis failures happen under one manager, than manager should be fired. This isn't rocket science.
For years, prosecutors told jurors that DNA and hair tests were much more accurate than they truly were, some cases claimed that they were 100% accurate. Jurors then convicted possibly innocent people based on that lie.
The result was many convictions can now be overturned, prosecutors reputations sullied and millions of dollars wasted.
The people that pay to keep these labs running have no interest in digging that deep.
Perhaps they still present things like 60% or 40% chance of match to the jury and they just hear "DNA match" and tune out the rest. I've said this before but I'm beginning to suspect that along with a public defender there should be a public forensic scientist assigned to defendants.
We have platforms that for less that 1000USD can sequence (read) a complete human genome. Each individual has millions of mutations which make them unique. Each position is read with >99% accuracy. Just to make sure in general each position is read >20 times.
For forensics it would be like comparing two very long strings for (very nearly) exact identity.
The only reason we use techniques like DNA fingerprinting is because that's what was feasible in the 1980s and forensics is an extremely conservative discipline.
Fingerprinting uses common relatively large changes in the genome as proxy for an individuals identity. It uses these locations because that's what you could do in the 1980s easily. It's possible to screw up both the experiments and interpenetration relatively easily.
Technology has moved on, you can now sequence a complete genome for around 1000USD. Comparing complete genomes would unambiguously assign identity. You'd effectively just be doing a string comparison over two very long, unique, strings.
The only reason we still use fingerprinting is that forensics is highly conservative. I'd guess it'll take at least another 10 years for them to move to sequencing (if not longer). It is however infuriating.
Would we ever get full genomes from any sample though? As far as I remember, most sequencing doesn't happen on full strings. Instead, many pieces are first unwound, then split into pieces, then sequenced one after another and the whole genome is reconstructed from overlaps. So that's still assuming you have enough samples that you can reconstruct the whole string. And that you can tell the samples apart if you have a mixture like in the first case from the article.
Your mental model of sequencing needs an update. :) Prior to sequencing, DNA is amplified via PCR. This dramatically increases the amount of DNA you have (2^30, iirc.)
However you do it, as you say there's a bunch of amplification that happens prior to sequencing (currently).
Also, in practice you'd be doing partial string matches presumably.
Practically how the comparison is done depends on the informatics (yes it's unlikely to be implemented as a X~million symbol string match, more likely as comparing lists of called SNPs against a reference).
[1] http://www.scientificamerican.com/article/identical-twins-ge...
>"Geneticist Carl Bruder of the University of Alabama at Birmingham, and his colleagues closely compared the genomes of 19 sets of adult identical twins. In some cases, one twin's DNA differed from the other's at various points on their genomes. At these sites of genetic divergence, one bore a different number of copies of the same gene, a genetic state called copy number variants." //
So, it's "in some cases" monozygotic twins differ by having copy number variants [TIL! thanks]. So, not all cases (according to this citation) and so the point still stands that "Comparing complete genomes would unambiguously assign identity" appears to be false (albeit at a very low false match rate).
Interestingly the article says that copy number variants appear to arise as a subject ages. Doesn't this mean that different cells in the body will have different multiples of particular genes; so a mismatch wouldn't necessarily mean that the DNA wasn't from the same twin?
This was by far the best and funniest part. What would happen if a suspect offered to pay the same lab $600 if there wasn't a conviction?