I often wonder if research into neural networks is taking air away from other kinds of research that would yield similar results, but also yield a comprehensible takeaway in the form of principles or equations, rather than vague hints that we can (if we’re lucky) reverse engineer from the structure of an NN.
Although I completely agree with you on the general observation, here's a rare but significant exception:
In 1951, Kleene wrote Representation of Events in Nerve Nets and Finite Automata which was motivated by reverse engineering biological NNs, but turned out to have been a discovery in CS instead.
According to Agnieszka Grabska-Barwinska, a member of the team, the graph neural network learned to encode a pattern that physicists call correlation length. That is, as DeepMind’s graph neural network restructured itself to reflect the training data, it came to exhibit the following tendency: When predicting propensities at higher temperatures (where molecular movement looks more liquid-like than solid), for each node’s prediction the network depended on information from neighboring nodes two or three connections away in the graph. But at lower temperatures closer to the glass transition, that number — the correlation length — increased to five.
“We see that the network extracts, as we lower the temperature, information from larger and larger neighborhoods” of particles, said Thomas Keck, a physicist on the DeepMind team. “At these different temperatures, the glass looks, to the naked eye, just identical. But the network sees something different as we go down.”
Increased correlation length is a hallmark of phase transitions, in which particles transition from a disordered to an ordered arrangement or vice versa. It happens, for instance, when atoms in a block of iron collectively align so that the block becomes magnetized. As the block approaches this transition, each atom influences atoms farther and farther away in the block.
They used molecular dynamics (MD) simulation to train the model - couldn't the correlation length be calculated from the MD simulations directly without the graph network to gain the same insight?
The other orientation is much less stable; it’s a stationary point but if nudged will rotate away from droop-up.
Similar reason old pre-float-glass windows are mostly found thick side down; easier to fit that way because they’re more stable. The myth that they flowed over time is just that. :-)
I didn't read it that way. It sounds like it discussing the transition from liquid to glass and the way that the molecules slow down during that transition.
This could be useful if applied on vitrification of water, a critical step for cryo-electron microscopy and one that is not nearly as simple as it appears since some stable, buffered protein solutions resist proper vitrification.
We know the glass transition isn’t a true phase transition because we do not observe a discontinuity in (derivatives of) the free energy at Tg, the glass transition temperature. What results like this suggest is that in some sense the glass transition asymptotically approaches a true phase transition, which is fascinating.
>"At the molecular level, glass looks like a liquid."
[...]
>"Glass is formed by cooling certain liquids. But why the molecules in the liquid slow down so dramatically at a certain temperature, with no obvious corresponding change in their structural arrangement — a phenomenon known as the glass transition — is a major open question."
14 comments
[ 2.4 ms ] story [ 40.5 ms ] threadIn 1951, Kleene wrote Representation of Events in Nerve Nets and Finite Automata which was motivated by reverse engineering biological NNs, but turned out to have been a discovery in CS instead.
https://www.rand.org/pubs/research_memoranda/RM704.html
According to Agnieszka Grabska-Barwinska, a member of the team, the graph neural network learned to encode a pattern that physicists call correlation length. That is, as DeepMind’s graph neural network restructured itself to reflect the training data, it came to exhibit the following tendency: When predicting propensities at higher temperatures (where molecular movement looks more liquid-like than solid), for each node’s prediction the network depended on information from neighboring nodes two or three connections away in the graph. But at lower temperatures closer to the glass transition, that number — the correlation length — increased to five.
“We see that the network extracts, as we lower the temperature, information from larger and larger neighborhoods” of particles, said Thomas Keck, a physicist on the DeepMind team. “At these different temperatures, the glass looks, to the naked eye, just identical. But the network sees something different as we go down.”
Increased correlation length is a hallmark of phase transitions, in which particles transition from a disordered to an ordered arrangement or vice versa. It happens, for instance, when atoms in a block of iron collectively align so that the block becomes magnetized. As the block approaches this transition, each atom influences atoms farther and farther away in the block.
What's the 'secret structure' being revealed here?
Similar reason old pre-float-glass windows are mostly found thick side down; easier to fit that way because they’re more stable. The myth that they flowed over time is just that. :-)
We know the glass transition isn’t a true phase transition because we do not observe a discontinuity in (derivatives of) the free energy at Tg, the glass transition temperature. What results like this suggest is that in some sense the glass transition asymptotically approaches a true phase transition, which is fascinating.
[...]
>"Glass is formed by cooling certain liquids. But why the molecules in the liquid slow down so dramatically at a certain temperature, with no obvious corresponding change in their structural arrangement — a phenomenon known as the glass transition — is a major open question."