Skipgram data structures, which underly any common semantic characterization ML processes, include a "Boltzmann distribution regression" step. So they'll follow Fermi-Dirac distribution forms. This has been common understanding for many years: https://www.microsoft.com/en-us/research/uploads/prod/2019/0...
> OK, so how does this happen? Why were a group of physicists who are obviously skilled and mathematically competent unable to detect that they’d rediscovered known methods? How is it that none of the reviewers for their PNAS paper were able to point this out to them? I think part of the answer lies in how vast the sphere of human knowledge really is. It is beyond anyone to keep up with the details of more than small portions of it. Research communities focus on their particular sets of problems and communication between different research communities is often weak to non-existent. This is not out of laziness, incompetence, or malevolence, but because the difficulty of such communication tests our limits. Just keeping on top of ones own field and publishing consistently is challenging enough. How to properly review cross-disciplinary research is a difficult problem.
Given that the referenced article was written in good faith and published in PNAS, I thought maybe this common understanding isn't widespread enough and wrote my article in response. Edit: Also, note that the connection is older and at a more fundamental level than the Boltzmann regression step in skip-gram data structures. ET Jaynes demonstrated that Fermi-Dirac statistics can be seen as an inference problem that is equivalent to a logistic regression in the 1950s [1]. Boltzmann distribution regression is essentially another name for logistic regression as far as I'm aware [2].
2 comments
[ 4.5 ms ] story [ 15.3 ms ] thread> OK, so how does this happen? Why were a group of physicists who are obviously skilled and mathematically competent unable to detect that they’d rediscovered known methods? How is it that none of the reviewers for their PNAS paper were able to point this out to them? I think part of the answer lies in how vast the sphere of human knowledge really is. It is beyond anyone to keep up with the details of more than small portions of it. Research communities focus on their particular sets of problems and communication between different research communities is often weak to non-existent. This is not out of laziness, incompetence, or malevolence, but because the difficulty of such communication tests our limits. Just keeping on top of ones own field and publishing consistently is challenging enough. How to properly review cross-disciplinary research is a difficult problem.
Given that the referenced article was written in good faith and published in PNAS, I thought maybe this common understanding isn't widespread enough and wrote my article in response. Edit: Also, note that the connection is older and at a more fundamental level than the Boltzmann regression step in skip-gram data structures. ET Jaynes demonstrated that Fermi-Dirac statistics can be seen as an inference problem that is equivalent to a logistic regression in the 1950s [1]. Boltzmann distribution regression is essentially another name for logistic regression as far as I'm aware [2].
[1] https://bayes.wustl.edu/etj/articles/theory.1.pdf
[2] https://ieeexplore.ieee.org/document/5726803