imo the real value of ML in imaging is going to come from the use of variational inference in scope development. due to their raster scanned nature, off the shelf 2p scopes can deliver high resolutions or high framerates, but not both. 2p scopes using compressive sensing have been developed to get high resolution at high framerates have been developed, but there is a lot more room for optimization, and the scope i'm thinking of has convergence issues (compressive sensing generally has good problems but this one ends up being ill-posed).
variational inference is basically super-powered compressive sensing in this context. compressive sensing exploits the correlated nature of voxels in the iamge via some sparsity in some domain, it's a pretty weak prior. there's a lot more structure to be exploited. the statistics of natural tissue can be modeled with ML, as can the physics of the measurement process.
the trick is posing the problem in a way that comes with convergence guarantees. to my knowledge no one has done this yet. a lossy microscope is not very useful.
How do you avoid the problem of using data-trained algorithms to provide measurements (in that they can’t)?
The idea of classifying or localizing detections with deep learning (or any data-trained approach) seems totally reasonable, since it’s clearly making an inference, and that should be clear to the human user. Enhancing or gap-filling the measurements with data-trained approaches would turn the measurements themselves into inference, which seems in opposition of the diagnostic goals (algorithm in-painting non-sensed information learned from the training set)
carefully. that's the whole problem. the beauty of compressed sensing is that it comes with a convergence guarantee. figuring out how to provide similar (but probably not as strong guarantees) with more sophisticated models of the prior is what is going to allow this to work.
Neuroimaging has always had an issue with interpretability and susceptibility to statistical artifacts. I fear that moving to another black box is only going to complicate matters.
FWIW my wife’s a researcher at Stanford applying ML/DL to MRI to speed up acquisition times and working closely with clinicians (I think they’re actually rolling it out in the hospitals) and the results have all been positive.
I spent a year of my life working on DL for Neuro. The huge problem is data availability. It's just unfeasible to train anything "deep" with the datasets available today. Until some well funded organisation / government comes along and open sources a dataset in the correct order of magnitude of samples - it's just not going to happen.
Still - you can have great successes with classic linear models, and the underlying effects of fMRI are very real. I look forward to the day a multimodal dataset of cutting edge DTI, fMRI and MRI is released.
8 comments
[ 3.5 ms ] story [ 28.1 ms ] threadvariational inference is basically super-powered compressive sensing in this context. compressive sensing exploits the correlated nature of voxels in the iamge via some sparsity in some domain, it's a pretty weak prior. there's a lot more structure to be exploited. the statistics of natural tissue can be modeled with ML, as can the physics of the measurement process.
the trick is posing the problem in a way that comes with convergence guarantees. to my knowledge no one has done this yet. a lossy microscope is not very useful.
The idea of classifying or localizing detections with deep learning (or any data-trained approach) seems totally reasonable, since it’s clearly making an inference, and that should be clear to the human user. Enhancing or gap-filling the measurements with data-trained approaches would turn the measurements themselves into inference, which seems in opposition of the diagnostic goals (algorithm in-painting non-sensed information learned from the training set)
https://thegradient.pub/editorsnote/
...I really should one day spend those few hours to use the HN submissions to find and select sources for the Explore/Exploit exercise)
Still - you can have great successes with classic linear models, and the underlying effects of fMRI are very real. I look forward to the day a multimodal dataset of cutting edge DTI, fMRI and MRI is released.