Then why do we need customized LLM models, two of which seemed to require the resources of 2 of the wealthiest companies on earth (this and google's alphafold) to do it?
To sell computers? 20 years ago, Apple had scientific poster sessions at WWDC and worked to bring PyMol to the Mac. The pictures of proteins you see in the paper were generated with PyMol as are probably >50% of the protein images in scientific papers for the last 15 years.
I wonder why Apple can create a model to fold proteins, but still can't get Siri to control the phone competently? I'm not sure I agree with Apple's priorities. I guess these things are not synchronous and they can work on multiple things at a time.
I remember first hearing about protein folding with the Folding @Home project (https://foldingathome.org) back when I had a spare media server and energy was cheap (free) in my college dorm. I'm not knowledgable on this, but have we come a long way in terms of making protein folding simpler on today's hardware, or is this only applicable to certain types of problems?
It seems like the Folding @Home project is still around!
Apparently from a F@H blog post [1] they say it's still useful to know the dynamics of how it folded, in addition to the final folded shape. And that having ML-folded proteins is a rich target for simulation to validate and to understand how the protein works
I was curious what the protein picture was showing:
"Figure 1 Example predictions of SimpleFold on targets ... with ground truth shown in light aqua and prediction in deep teal."
and now I'm even more curious why they thought "light aqua" vs "deep teal" would be a good choice
I am curious to hear an expert weigh in on this approach's implications for protein folding research. This sounds cool but it's really unclear to me what the implications are
A paper that says: "our approach is simpler than the state of the art". But also does not loudly say "our approach is significantly behind the state of the art on all metrics". Not easy to get published, but I guess putting it as a preprint with a big company's name will help...
It's not totally novel, but it's very cool to see the continued simplification of protein folding models - AF2 -> AF3 was a reduction in model architecture complexity, and this is a another step in the direction of the bitter lesson.
One caveat that’s easy to miss: the "simple" model here didn’t just learn folding from raw experimental structures. Most of its training data comes from AlphaFold-style predictions. Millions of protein structures that were themselves generated by big MSA-based and highly engineered models.
It’s not like we can throw away all the inductive biases and MSA machinery, someone upstream still had to build and run those models to create the training corpus.
Correct. For those that might not follow, the MSA is used to generalize from known PDB structures to new sequences. If you train on AlphaFold2 results, those results include that generalization, so that your model no longer needs that capability (you can rely on rote memorization). This simple conclusion seems to have escaped the authors.
Should an entry in a field preclude other ones. I encourage you to apply reductio-ad-absurdum here. Should Pepsi exist if Coke does? Should C exist if Fortran does?
Curious since AlphaFold got released: have classical molecular dynamics sims in this area become obsolete, at least for protein folding? How does the research coming out of venues like DESRES compare? Are they working on more specific problems in the same area or are they in a different business altogether?
MD was never really a viable way to do structure prediction, so it didn't become obsolete with AlphaFold. Instead, MD is more useful for studying the physical process of protein folding (before the protein folds to its final structure, as well as once it has reached its final structure and sort of jiggles and wiggles around that).
Predicting the end-result from the sequence of protein directly is prone to miss any new phenomenon and would just regurgitate/interpolate the training datasets.
I would much prefer an approach based on first principles.
In theory folding is easy, it's just running a simulation of your protein surrounded by some water molecules for the same number of nano-seconds nature do.
The problem is that usually this take a long time because evolving a system needs to compute the energy of the system as a position of the atoms which is a complex problem involving Quantum Mechanics. It's mostly due to the behavior of the electrons, but because they are much lighter they operate on a faster timescale. You typically don't care about them, only the effect they have on your atoms.
In the past, you would use various Lennard-Jones potentials for pairs of atoms when the pair of atoms are unbounded, and other potentials when they are bonded and it would get very complex very quickly. But now there are deep-learning based approach to compute the energy of the system by using a neural network. (See (Gromacs) Neural Network Potentials https://rowansci.com/publications/introduction-to-nnps ). So you train these networks so that they learn the local interactions between atoms based on trajectories generated from ab-initio theories. This allows you to have a faster simulator which approximate the more complex physics. It's in a sort just tabulating using a neural network the effect of the electrons would have in a specific atom arrangements according to the theory you have chosen.
At any time if you have some doubt, you can always run the slower simulator in the small local neighborhood to check that the effective field neural network approximation holds.
Only then once you have your simulator which is able to fold, you can generate some dataset of pairs "sequence of protein" to "end of trajectory", to learn the shortcut like Alpha/Simple/Fold do. And when in doubt you can go back to the slower more precise method.
If you had enough data and can train perfectly a model with sufficient representation power, you could theoretically infer the correct physics just from the correspondence initial to final arrangements. But if you don't have enough data it will just learn some shortcut and accept that it will be wrong some times.
Flow-matching, the technique they describe is incredibly interesting. I studied it in the context of generative AI and found it fascinating. It’s so fitting that a technique that borrows from thermodynamics and uses Brownian motion would go full circle to solve for protein folding.
37 comments
[ 2.7 ms ] story [ 64.8 ms ] threadThen why do we need customized LLM models, two of which seemed to require the resources of 2 of the wealthiest companies on earth (this and google's alphafold) to do it?
https://arxiv.org/abs/2509.18480
It seems like the Folding @Home project is still around!
https://foldingathome.org/papers-results/?lng=en
[1] https://foldingathome.org/2024/05/02/alphafold-opens-new-opp...
and now I'm even more curious why they thought "light aqua" vs "deep teal" would be a good choice
It’s not like we can throw away all the inductive biases and MSA machinery, someone upstream still had to build and run those models to create the training corpus.
However, it seems like anyone can download the parameters for AlphaFold V2: https://github.com/google-deepmind/alphafold?tab=readme-ov-f...
Predicting the end-result from the sequence of protein directly is prone to miss any new phenomenon and would just regurgitate/interpolate the training datasets.
I would much prefer an approach based on first principles.
In theory folding is easy, it's just running a simulation of your protein surrounded by some water molecules for the same number of nano-seconds nature do.
The problem is that usually this take a long time because evolving a system needs to compute the energy of the system as a position of the atoms which is a complex problem involving Quantum Mechanics. It's mostly due to the behavior of the electrons, but because they are much lighter they operate on a faster timescale. You typically don't care about them, only the effect they have on your atoms.
In the past, you would use various Lennard-Jones potentials for pairs of atoms when the pair of atoms are unbounded, and other potentials when they are bonded and it would get very complex very quickly. But now there are deep-learning based approach to compute the energy of the system by using a neural network. (See (Gromacs) Neural Network Potentials https://rowansci.com/publications/introduction-to-nnps ). So you train these networks so that they learn the local interactions between atoms based on trajectories generated from ab-initio theories. This allows you to have a faster simulator which approximate the more complex physics. It's in a sort just tabulating using a neural network the effect of the electrons would have in a specific atom arrangements according to the theory you have chosen.
At any time if you have some doubt, you can always run the slower simulator in the small local neighborhood to check that the effective field neural network approximation holds.
Only then once you have your simulator which is able to fold, you can generate some dataset of pairs "sequence of protein" to "end of trajectory", to learn the shortcut like Alpha/Simple/Fold do. And when in doubt you can go back to the slower more precise method.
If you had enough data and can train perfectly a model with sufficient representation power, you could theoretically infer the correct physics just from the correspondence initial to final arrangements. But if you don't have enough data it will just learn some shortcut and accept that it will be wrong some times.
https://www.youtube.com/watch?v=P_fHJIYENdI