Perhaps - but is that necessarily a problem? I guess it depends on the purpose of the training. If you're training ML models to do research on ML, this bias isn't necessarily a problem.
Even if your long-term goal is to apply ML to the real problem, having intermediate research goals using a simulation might still be useful towards achieving the long-term goal.
Depends on the accuracy of the simulation similar to the error that can be made in measurement of natural phenomena. I don't have the relevant domain knowledge to assess this particular simulation however.
Reinforcement learning, for instance, is almost entirely trained on simulated environments such as the state-space of a board game or even the full set of pixels displayed by a video game.
Robotics makes use of this to e.g. teach a robotic dog how to walk in a rigid body simulation. The success of the knowledge learned being useful in the real world depends on the accuracy of the simulation and the breadth of the agent's search over the simulation's state space.
Echoing previous responses, it depends on what you want to learn. Gravity on large scales is fairly well understood (or at least many astrophysicists believe this to be the case). If there are large scale structure observables that are only dependent on cosmology (i.e. the cosmological parameters) simulations like this will be ideal. However, the major issue here is that a huge amount of the results of these simulations depend on physics that happen below the "grid scale" off the simulation. Thus I would agree that training the ML models on these simulations is the epitome of bias because it is learning a "sub grid" model rather than fundamental physics. Reading through the papers on this work, the vast majority of models that have been trained are not able to generalize between the different classes of sub grid models used which very much limits predicability. This is an inherent limitation of most cosmological simulations, not just these ones.
We'll always be biased until we directly observe Dark Matter ;)
There's another gotcha: if we are inferring the hidden from observable properties, might not the charge of "feature engineering" also creep into the model? Why human observable properties of the galaxies and not nature's implicit representations?
Pretty eye-opening to think with enough supervised training data humanity approaches a GAN for cosmology!
Discovering Symbolic Models from Deep Learning with Inductive Biases
Briefly, no. Why: they're most likely not training your average ML models on this dataset. Instead, they are likely taking a model of some physics and seeing how it performs in these simulations.
You can think of this as a form of causal inference - "if this model is true, how well does it work with our current understanding (simulations) of physics?" type of questions. There are measures of error and bias that come with evaluation of these models.
4233 low fi universe simulations. Not bad. After all, this is still within 100 years of inventing computers. So, after another 1000 years or so of r&d, just think of the fidelity of the vast number of universes that we will be able to simulate...
Of course it is reasonably likely that we are living in a similar simulation. If so, it becomes morally important to lead interesting lives (so they keep the simulation running!)
A morals and ethics officer will be waiting with questions when you awake. Please continue play until the game ends. Attempts to end the game early will be seen as intent to flee.
9 comments
[ 3.1 ms ] story [ 32.0 ms ] threadEven if your long-term goal is to apply ML to the real problem, having intermediate research goals using a simulation might still be useful towards achieving the long-term goal.
Reinforcement learning, for instance, is almost entirely trained on simulated environments such as the state-space of a board game or even the full set of pixels displayed by a video game.
Robotics makes use of this to e.g. teach a robotic dog how to walk in a rigid body simulation. The success of the knowledge learned being useful in the real world depends on the accuracy of the simulation and the breadth of the agent's search over the simulation's state space.
There's another gotcha: if we are inferring the hidden from observable properties, might not the charge of "feature engineering" also creep into the model? Why human observable properties of the galaxies and not nature's implicit representations?
Pretty eye-opening to think with enough supervised training data humanity approaches a GAN for cosmology!
Discovering Symbolic Models from Deep Learning with Inductive Biases
https://deepmind.com/research/publications/2020/Discovering-...
You can think of this as a form of causal inference - "if this model is true, how well does it work with our current understanding (simulations) of physics?" type of questions. There are measures of error and bias that come with evaluation of these models.
Of course it is reasonably likely that we are living in a similar simulation. If so, it becomes morally important to lead interesting lives (so they keep the simulation running!)