I mostly scanned the article as it's getting late but it doesn't look like they discussed the funniest part of the story. Ernst Ising himself after grad school essentially become an obscure teacher and not a researcher, and I believe only until decades later did he become aware that his model was now the quintessential simple model for classical spins. It's funny especially given his name was attached to the model.
Looked up Ising’s story on Wikipedia. It doesn’t seem funny to me that he was unaware of the progress in the literature after being forced out of his job, then his country, then finally doing forced labor for the Wehrmacht.
What's not touched on in this article and arguably explains the ubiquity of the Ising model: it's the maximum entropy model given a mean and pairwise correlations
Which entropy does “maximum entropy” refer to here? The entropies of the spins? Wouldn’t this entropy depend on temperature and coupling strength since there is a phase transition?
That doesn’t make much sense to me. The model is a model (coupled spins in a heat bath). The “solution” (the equilibrium states) can be found using maximum entropy considerations but that’s not unusual. What makes the Ising model popular is that’s simple and interesting.
That link has a couple of sentences about “maximum entropy models” and no references. From a quick search it seems that this is a terminology used in some areas thought. It’s not yet completely clear to me what does it mean. (Seems to be about the Ising model which is selected from a large set of ‘similar” Ising models; I would think of “Ising model” as the general thing.)
Statistical mechanical entropy is information entropy (and related to thermodynamical entropy). From the link: “The principle was first expounded by E. T. Jaynes in two papers in 1957[1][2] where he emphasized a natural correspondence between statistical mechanics and information theory. In particular, Jaynes offered a new and very general rationale why the Gibbsian method of statistical mechanics works. He argued that the entropy of statistical mechanics and the information entropy of information theory are basically the same thing. Consequently, statistical mechanics should be seen just as a particular application of a general tool of logical inference and information theory.”
Simplicity is a relative thing, I guess. Let’s say the interestingness/complexity ratio is high.
Statistical mechanical (Boltzmann) entropy and informational (Shannon) entropy are not the same thing, but they are closely related. Subtleties are important.
The Shannon entropy of the distribution of the positions and velocities is precisely the Boltzmann entropy of the same, but there is no statistical mechanical analog to the Shannon entropy of a (non-Boltzmann) distribution, to my knowledge.
While Boltzmann entropy was first, and an important breakthrough, it has some limitations. Statistical mechanics is actually based on Gibbs/von Neumann entropy, which is equivalent to Shannon entropy. There are many subtleties indeed but I think the relationship between statistical mechanics and information theory may be closer than you think.
“To simulate the system, waiting times w_kj are generated for all candidate states k, and the shortest waiting time i = argmin_k w_kj is chosen. We call this shortest waiting time the holding time. A transition is then performed to state i after a delay of length w_ij.”
Say there are three states A, B, C and the transition rate to C from A or B is low. Wouldn’t your “simulation” jump endlessly between A and B without ever (or never again) visiting C?
The transition rates for each individual state transition are drawn from an exponential distribution with a rate dependent on the energy difference between the states.
So, no, C will be visited with some probability.
There are many problems with that paper, but validity of the sampling algorithm is not one of them. It is an unbiased sampler and will eventually converge to the sampled distribution (as proved in the paper). How long that convergence will take is another store. It remains unclear whether this approach converges faster than your garden-variety Markov chain and if so when and why.
Thanks for your answer. Sorry, I don’t why i did interpret “waiting times” as “expected waiting times” when it was right in the preceding paragraph how they were sampled.
For others wanting a very "computer science friendly" introduction to the Ising model, Rice piles and other critical and complex systems (in the physics sense), I would recommend "Complexity and Criticality" by Christensen and Moloney.
Something I found interesting when running analysis on Ising models; if you do an FFT (2 dimensional) on it, you get what amounts to a wavey star-like blob in the middle (assuming you center the lowest frequencies)
If that blob is a small dot, or even nothing at all; you have a stable system, and if you have a sprawling kind of wavey response-pattern, then your lattice is gone critical.
(Also it looks cool, and it’s quick to diagnose by a glance.)
19 comments
[ 2.4 ms ] story [ 65.8 ms ] threadhttps://en.m.wikipedia.org/wiki/Ernst_Ising
I would not describe the Ising model as simple, not even in the 2d case and especially not the general spin glass (where W_ij can be anything).
[1] https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
Statistical mechanical entropy is information entropy (and related to thermodynamical entropy). From the link: “The principle was first expounded by E. T. Jaynes in two papers in 1957[1][2] where he emphasized a natural correspondence between statistical mechanics and information theory. In particular, Jaynes offered a new and very general rationale why the Gibbsian method of statistical mechanics works. He argued that the entropy of statistical mechanics and the information entropy of information theory are basically the same thing. Consequently, statistical mechanics should be seen just as a particular application of a general tool of logical inference and information theory.”
Simplicity is a relative thing, I guess. Let’s say the interestingness/complexity ratio is high.
The Shannon entropy of the distribution of the positions and velocities is precisely the Boltzmann entropy of the same, but there is no statistical mechanical analog to the Shannon entropy of a (non-Boltzmann) distribution, to my knowledge.
While Boltzmann entropy was first, and an important breakthrough, it has some limitations. Statistical mechanics is actually based on Gibbs/von Neumann entropy, which is equivalent to Shannon entropy. There are many subtleties indeed but I think the relationship between statistical mechanics and information theory may be closer than you think.
“To simulate the system, waiting times w_kj are generated for all candidate states k, and the shortest waiting time i = argmin_k w_kj is chosen. We call this shortest waiting time the holding time. A transition is then performed to state i after a delay of length w_ij.”
Say there are three states A, B, C and the transition rate to C from A or B is low. Wouldn’t your “simulation” jump endlessly between A and B without ever (or never again) visiting C?
I may be missing something.
The transition rates for each individual state transition are drawn from an exponential distribution with a rate dependent on the energy difference between the states.
So, no, C will be visited with some probability.
There are many problems with that paper, but validity of the sampling algorithm is not one of them. It is an unbiased sampler and will eventually converge to the sampled distribution (as proved in the paper). How long that convergence will take is another store. It remains unclear whether this approach converges faster than your garden-variety Markov chain and if so when and why.
[1] https://www.amazon.com/COMPLEXITY-CRITICALITY-Imperial-Colle...
If that blob is a small dot, or even nothing at all; you have a stable system, and if you have a sprawling kind of wavey response-pattern, then your lattice is gone critical.
(Also it looks cool, and it’s quick to diagnose by a glance.)