>The medical examiner’s office says it switched from FST to a new program, STRmix, because of changing FBI standards and not because of any deficiencies with FST.
Is there any effort towards independent examination of this new tool? I would intuitively assume that any process used to exonerate or incarcerate someone based on evidence would be available for inspection by the defense, and should certainly be available for public inspection by experts.
It seems dangerously opaque to have criminal trials decided by the results of a blackbox system... or is this really so commonplace that my ideas of criminal justice are romanticized?
Edit: STRmix has a document [0] that states that defense teams in an ongoing case can request access to source code as well as intermediary analysis results. They also list some papers that purport to go over the math, biology, and performance. However, the source code access terms are pretty restrictive. I think software this important to society should be made open-source by mandate.
> They also list some papers that purport to go over the math, biology, and performance. However, the source code access terms are pretty restrictive.
Holy shit, you weren't kidding about restrictive terms. Actual terms in the agreement (Discloser = ESR who make the STRmix software, Recipient = Defense expert witness):
1. Discloser will only provide source code to an expert witness who is retained by the defense.
2. Recipient must sign confidentiality agreement.
3. Costs of the disclosure will be recoverable from the Recipient by the Discloser.
4. When source code is disclosed, a representative of the Discloser must be in the room at all times.
5. A stand-alone computer will be provided by the Discloser, and it will not accept USB drives or CD-ROMs.
6. No photographic devices are allowed in the room, including tablets and phones.
7. The only permitted notes are hand-written.
Good luck finding any bugs using a company-provided computer, under the watch of a company representative without the ability to take anything except for handwritten notes while you are being billed for the time that you spend.
Companies hide behind "code is proprietary information" ALL THE TIME, especially when it comes to government purchases. And the government agrees because they don't know any better.
Putting aside every other disturbing issue about this case, the claim about the "security of IT assets" quoted below strikes me as very odd - why should an algorithm for matching DNA have any inseparable IT security issues? The response is so self-serving that I would not be surprised if someone is being disingenuous in her explanation to the court as to why the request is being denied - perhaps through an invalid argument from the specific (some source code has security issues) to the general (source code may have security issues) to the specific (we suggest that this particular source code has security issues).
From the article:
The medical examiner’s office denied the request, citing its “sensitive nature” and writing that “source code consists of information that, ‘if disclosed, would jeopardize the capacity of [OCME[Office of the City Medical Examiner?]] to guarantee the security of its information technology assets.’”
consists of information that, ‘if disclosed, would jeopardize the capacity of [OCME[Office of the City Medical Examiner?]] to guarantee the security of its information technology assets
... Hardcoded passwords?
... The source code itself is such an asset, and is no longer "secure"-as-in-secret once disclosed?
I would be highly skeptical of any claim that any access control is so interwoven with the DNA matching algorithm that it is not a trivial problem to separate them. Furthermore, if the OCME fears that there are passwords in plaintext, and given that it presumably knows what the administrator's password is, it shouldn't be hard to find them. Furthermore, this presumably is not (or anyway should not be) a publicly-accessible program.
The second argument would be using 'security' to disguise a proprietary-software claim, and if it is what the quoted sentence actually means, the OCME should say so specifically, instead of making vague claims about security. I don't think there is any valid reason for the OCME to have proprietary rights over software of this nature: it is not, or should not be, in the business of selling software or software services, and such concerns should definitely not trump the need for such software to be thoroughly audited.
If the claim that it is proprietary software comes from a vendor, that should similarly be stated directly.
If either of these claims are the basis of the quoted sentence, and the judge accepts them, then it would provide a de facto immunization of all software against public scrutiny. Does this matter? I think so, as anyone convicted through the use of poorly-verified software has, in effect, been convicted by heresay according to rules that are not known by anyone, and with no way to challenge his accusers.
8 comments
[ 3.5 ms ] story [ 36.6 ms ] threadIs there any effort towards independent examination of this new tool? I would intuitively assume that any process used to exonerate or incarcerate someone based on evidence would be available for inspection by the defense, and should certainly be available for public inspection by experts.
It seems dangerously opaque to have criminal trials decided by the results of a blackbox system... or is this really so commonplace that my ideas of criminal justice are romanticized?
Edit: STRmix has a document [0] that states that defense teams in an ongoing case can request access to source code as well as intermediary analysis results. They also list some papers that purport to go over the math, biology, and performance. However, the source code access terms are pretty restrictive. I think software this important to society should be made open-source by mandate.
[0] https://strmix.esr.cri.nz/assets/Uploads/Defence-Access-to-S...
Holy shit, you weren't kidding about restrictive terms. Actual terms in the agreement (Discloser = ESR who make the STRmix software, Recipient = Defense expert witness):
1. Discloser will only provide source code to an expert witness who is retained by the defense.
2. Recipient must sign confidentiality agreement.
3. Costs of the disclosure will be recoverable from the Recipient by the Discloser.
4. When source code is disclosed, a representative of the Discloser must be in the room at all times.
5. A stand-alone computer will be provided by the Discloser, and it will not accept USB drives or CD-ROMs.
6. No photographic devices are allowed in the room, including tablets and phones.
7. The only permitted notes are hand-written.
Good luck finding any bugs using a company-provided computer, under the watch of a company representative without the ability to take anything except for handwritten notes while you are being billed for the time that you spend.
Companies hide behind "code is proprietary information" ALL THE TIME, especially when it comes to government purchases. And the government agrees because they don't know any better.
From the article: The medical examiner’s office denied the request, citing its “sensitive nature” and writing that “source code consists of information that, ‘if disclosed, would jeopardize the capacity of [OCME[Office of the City Medical Examiner?]] to guarantee the security of its information technology assets.’”
... Hardcoded passwords?
... The source code itself is such an asset, and is no longer "secure"-as-in-secret once disclosed?
The second argument would be using 'security' to disguise a proprietary-software claim, and if it is what the quoted sentence actually means, the OCME should say so specifically, instead of making vague claims about security. I don't think there is any valid reason for the OCME to have proprietary rights over software of this nature: it is not, or should not be, in the business of selling software or software services, and such concerns should definitely not trump the need for such software to be thoroughly audited.
If the claim that it is proprietary software comes from a vendor, that should similarly be stated directly.
If either of these claims are the basis of the quoted sentence, and the judge accepts them, then it would provide a de facto immunization of all software against public scrutiny. Does this matter? I think so, as anyone convicted through the use of poorly-verified software has, in effect, been convicted by heresay according to rules that are not known by anyone, and with no way to challenge his accusers.