Flaws uncovered in the software researchers use to analyze fM.R.I. data (nytimes.com)

The dead Salmon was taught in one of our classes in 2010, and still nobody gives a fuck, they just fMRI on. Its not a software bug, its a people bug.

We go through this with each new tech, such as with EEG and Jello.

I read a great (in the sense that it was laugh out loud preposterous) paper that used this experiment to assert that re-incarnation was a thing, as the salmon, post mortem, had been re-incarnated as a person but not yet born, their portal to the soul was now interpreting pictures as if the fish were both human and alive. It was a tour de force in circular reasoning. It may have been in the Journal of Irreproducible Results.

One way to read scientific papers is to deconstruct the apparatus in the experiment to see if it could actually produce the data in the paper, and if so, what would have to be true for it to do so. Good papers go through that in experimental design section, bad papers gloss over it or toss it out as a stipulation "We used the same instrument that everyone else uses to read brain activity the fMRI scanner." (bad) "We used a technique called fMRI scanning which measures the magnetic resonance of hemoglobin in the blood, specifically it measures the difference between oxygen rich and oxygen poor hemoglobin." (much better) Then as a reader you can see that any change in oxygen, whether it is brain activity or bacterial decomposition would show up in such a resonance scan.

This is actually a somewhat different issue. The salmon "result" is essentially a multiple comparisons problem. In a typical fMRI analysis, the researchers run a huge number of significance tests, one per voxel (i.e., 3d pixel). Even if false positives are relatively rare on a per-test basis, you're going to see a non-trivial amount of them when you run a large number of tests. This phenomenon was well-known in the statistical literature...and in the fMRI literature, but sometimes it takes a cute little parable, like the salmon, to drive something home.

In this case, however, there was one honest-to-God bug (in AFNI). The other software modeled something in a way that sounds fairly reasonable, but didn't quite line up with the way that nature (or, actually, the scanner manufacturers) behaved. Unlike the fish issue, I think you could legitimately try to do things correctly and still get bit by this sort of bug.

Link to salmon study:

http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf

Choice quotes:

"[the salmon] was not alive at the time of scanning"

"The salmon was asked to determine what emotion the individual in the photo must have been experiencing."

Please don't put leading spaces in front of quotes.

You fuck the formatting for mobile users.

Without looking into this a lot, probably not much. This parcellation is described as multi-modal, so it's not using only structural or only diffusion, but I doubt it's using fMRI much if at all. Even if so, I'm sure they validated the parcellation against pre-existing parcellations. I'll double-check this tomorrow when I can access the full paper more easily.

Yes, this article is really just a laundry list of problems. It barely even touches on the headline problem, which is the software.