An AI model developed by Microsoft can accurately forecast weather and air pollution for the whole world in less than a minute.
predict the levels of carbon monoxide, nitrogen oxide, nitrogen dioxide, sulfur dioxide, ozone and particulate matter. Its predictions span five days...at [much less] cost than...ECMWF [model], which predicts global air-pollution levels
Aurora’s predictions were of a similar quality to those of the conventional model.
After 1st read-thru, I believed the article was touting promise as result.
As I wrote the above synopsis, I zeroed in on the part about the performance comparison to the conventional model. Not sure why I couldn't pick it out at first.
1. First, models will predict pollution. The outcomes will help shape urban policy. But these won't solve crime or stop people from driving.
2. Second, models will predict individual behavior and track person level emissions. The outcomes will force behavior changes, mostly freedom limiting.
3. Third, and finally, models will predict thoughts. The the thought of driving instead of walking might trigger a response.
It's a slippery slope and we need to draw a line between prediction and policy.
Even allowing for the ridiculously massive technical leap from 1 to 2 and then 2 to 3, it doesn't make much sense.
For one thing, if states are determined to enforce individual emissions limits, they can do it today with legislation. You don't need a predictive model. What does the model add?
Also, the only difference between 2 and 3 is whether a person acts on a thought.
So are you suggesting with #3 that predicted thoughts (e.g. not literal mind reading) which a person doesn't act upon will prompt state action?
Using the unqualified word "freedom" has an ambiguity that political actors exploit. Freedom to do something is entirely separate to "free to live in a world where ___".
To be honest, I feel the latter sense of the word is a bit of a stretch - semantically, not politically.
But you see it because "freedom" is a powerful word in politics, and rather than argue against "freedom", pundits go up the ladder of abstraction and argue the definition instead.
There's a lot of excitement in this space but a lot of weather centers are really struggling to get even a tiny fraction of the resources and talent needed to start using these results in real operational systems. Everyone is still using fortran physics code from the 90s.
One thing I found really interesting abou the Graphcast paper (I appreciate this is not graphcast, but I think it is still relevant) is that it doesn't understand climate change. The model requires the training data to be recent to get the best quality projections.
While there are some factors that influence predictability in the weather forecast, as the fortran code is based on physics (at least in a broad sense), it doesn't suffer from those issues in the same way.
This doesn't mean that the ML forecasts are wrong (obviously), just different. Given the relative computational simplicity of running them, I wonder if the issue is not just expertise, but also understanding how they can best be used to generate reliable weather forecasts?
The article is about a paper that is published in Nature. Nature will frequently do this (publish a paper and then also have their editorial staff write a more accessible piece about the paper.)
Actually, the paper isn't in press at Nature - it's only published on arXiv. It seems a little bit unusual that Nature would publish this sort of news article for a work that hasn't been peer-reviewed.
If this thing is trained upon the existing models input + output, doesn't this just mean that the AI is a duplicate/aggregate of the existing model/models? E.g. the quality should be the same if trained on a single model, might be better when trained on multiple?
> The researchers trained Aurora on more than a million hours of data from six weather and climate models. After training the model, the team tweaked it to predict pollution and weather globally. The model generates a ten-day global weather forecast alongside the air-pollution prediction.
Edit: Added the quote from the article where they confirm the model is trained on existing legacy models.
Is it possible it formed new inferences based on the analysis of the pervious models and their accuracy or errors? That would be the interesting part, but I'm not sure that was the case.
The idea generally for these models is: get an architecture that can copy outputs of a very slow process, ablate to see what you can discard from the architecture so that it will run more efficiently, train the heck out of it, then deliver something that’s 10-10000x more efficient than original process.
This is the same broad plan used for Alphafold — and for things like Stable Diffusion turbo.
It’s rare that these efforts yield interesting new output; they’re usually made specifically for the space/time speedup. In this case, weather predictions that are crazy fast/cheaper to run are seen as a public good, and so sending around a ‘compressed’ version of the big simulations is great.
This is a total layman's question, but the "legacy" models don't seem to get the weather forecast right very often. It's a running joke everywhere I've ever lived that weather forecasting is one of the few jobs where you can be wrong up to half the time and still keep your job. If you're making outdoor plans, you don't cancel your indoor backup plans until the forecast is less than 2-3 days out.
Why would training an AI on many of the models that get mediocre results somehow produce a more reliable output than the current natural intelligences are able to get from the same models?
Are experts evaluating "precision" differently from laypeople?
Like, from a layman's perspective, getting the forecasted high and low within 5 degrees and accurately predicting if it's going to rain (and if so, when) are probably the factors we're evaluating. We don't really notice or care if something like the atmospheric pressure forecast is right or wrong. We care about whether we're well-prepared for the temperature and whether we get wet, and we know from constant experience that the forecast 14 days out can change by 10+ degrees (F) and/or more than 50% chance of rain by the time it becomes tomorrow's forecast, so from our perspective, the forecast is "usually wrong" when it's more than a couple days out.
Are we really, en masse, that wrong about what seems to be constant and universal experience? Or is "precision" measured very differently by professionals than what laypeople mean when they talk about "whether the weather forecast is right or wrong"?
Given that the weather models are working on grid sizes 5km and up [1], and we often only get a report for a rough location like a whole city, meaning the prediction needs to be valid for 10s of square km, while our own personal observation range is often just a couple hundred square meters, I am not surprised that my personal sampling of a small area does not match the averaged prediction for a much larger area.
But, having said that, temperature predictions seem to be fairly ok up to a week out. Rain is something else, but since we got our dog, I don't care any longer so much. I need to go out anyway ;-) And then I just take a look at the actual rain-radar data for my location to check the actual situation and the outlook for the next hour or so.
1. any percentage prediction like "99,999999999999%" chance of rain is still 100% correct if it doesn't rain.
2. its my understanding (from a trip to the local weather bureau) that an "80% change of rain" means something like "when conditions in the past were like they are now, it rained 4 out of 5 times".
The evaluation of the air pollution / chemistry forecasts in the arxiv paper [1] is really threadbare and entirely disconnected from the actual literature of atmospheric chemical transport modeling and air quality forecasting. The highlighted success case - a sandstorm, where PM10 is the key parameter being analyzed - is probably entirely explained by adequate performance of the underlying "weather" portion of the model (e.g. there is obviously a very strong correlation between intense winds and sandstorms in a desert), but the authors don't provide a more nuanced discussion of the forecast meteorology and transport dynamics of that scenario.
It's cool that their model has learned about this relationship, but a greater editorial point is that _we already have global air pollution models_ - particularly, the one this model was trained (fine-tuned) against! That's not to discount all the really novel and interesting new developments with this paper, but the editorialization around AI applications in the weather and climate space is getting way out of hand, especially when most of these applications are incremental improvements at best over the existing technology in the field (with the key exception of cost/efficiency of generating forecasts - that's a true revolution).
The Aurora paper is really cool, but issues like these greatly temper my enthusiasm for it. I hope that the research team seeks input and collaboration with domain experts in meteorology and atmospheric chemistry in the future to find ways to leverage this new technology in impactful and useful ways that actually will benefit society.
> I hope that the research team seeks input and collaboration with domain experts
From my perspective ML in general has a weird relationship with cross-domain collaboration. The discussions around linguistics feel like they're full of similar examples and it comes off as a lack of curiosity.
40 comments
[ 3.1 ms ] story [ 114 ms ] threadpredict the levels of carbon monoxide, nitrogen oxide, nitrogen dioxide, sulfur dioxide, ozone and particulate matter. Its predictions span five days...at [much less] cost than...ECMWF [model], which predicts global air-pollution levels
Aurora’s predictions were of a similar quality to those of the conventional model.
After 1st read-thru, I believed the article was touting promise as result.
As I wrote the above synopsis, I zeroed in on the part about the performance comparison to the conventional model. Not sure why I couldn't pick it out at first.
1. First, models will predict pollution. The outcomes will help shape urban policy. But these won't solve crime or stop people from driving.
2. Second, models will predict individual behavior and track person level emissions. The outcomes will force behavior changes, mostly freedom limiting.
3. Third, and finally, models will predict thoughts. The the thought of driving instead of walking might trigger a response.
It's a slippery slope and we need to draw a line between prediction and policy.
Even allowing for the ridiculously massive technical leap from 1 to 2 and then 2 to 3, it doesn't make much sense.
For one thing, if states are determined to enforce individual emissions limits, they can do it today with legislation. You don't need a predictive model. What does the model add?
Also, the only difference between 2 and 3 is whether a person acts on a thought.
So are you suggesting with #3 that predicted thoughts (e.g. not literal mind reading) which a person doesn't act upon will prompt state action?
To be honest, I feel the latter sense of the word is a bit of a stretch - semantically, not politically.
But you see it because "freedom" is a powerful word in politics, and rather than argue against "freedom", pundits go up the ladder of abstraction and argue the definition instead.
While there are some factors that influence predictability in the weather forecast, as the fortran code is based on physics (at least in a broad sense), it doesn't suffer from those issues in the same way.
This doesn't mean that the ML forecasts are wrong (obviously), just different. Given the relative computational simplicity of running them, I wonder if the issue is not just expertise, but also understanding how they can best be used to generate reliable weather forecasts?
Oh wait, that's right. IBM owns weather.com.
but you have to admit, they are really gross to work with
In my experience stuff like this is often vastly overestimated but if it really can do what it says it's great but I would like to see some examples.
> The researchers trained Aurora on more than a million hours of data from six weather and climate models. After training the model, the team tweaked it to predict pollution and weather globally. The model generates a ten-day global weather forecast alongside the air-pollution prediction.
Edit: Added the quote from the article where they confirm the model is trained on existing legacy models.
So we'd still need to keep working on them anyway, or not?
This is the same broad plan used for Alphafold — and for things like Stable Diffusion turbo.
It’s rare that these efforts yield interesting new output; they’re usually made specifically for the space/time speedup. In this case, weather predictions that are crazy fast/cheaper to run are seen as a public good, and so sending around a ‘compressed’ version of the big simulations is great.
Why would training an AI on many of the models that get mediocre results somehow produce a more reliable output than the current natural intelligences are able to get from the same models?
Like, from a layman's perspective, getting the forecasted high and low within 5 degrees and accurately predicting if it's going to rain (and if so, when) are probably the factors we're evaluating. We don't really notice or care if something like the atmospheric pressure forecast is right or wrong. We care about whether we're well-prepared for the temperature and whether we get wet, and we know from constant experience that the forecast 14 days out can change by 10+ degrees (F) and/or more than 50% chance of rain by the time it becomes tomorrow's forecast, so from our perspective, the forecast is "usually wrong" when it's more than a couple days out.
Are we really, en masse, that wrong about what seems to be constant and universal experience? Or is "precision" measured very differently by professionals than what laypeople mean when they talk about "whether the weather forecast is right or wrong"?
But, having said that, temperature predictions seem to be fairly ok up to a week out. Rain is something else, but since we got our dog, I don't care any longer so much. I need to go out anyway ;-) And then I just take a look at the actual rain-radar data for my location to check the actual situation and the outlook for the next hour or so.
[1] "the gridboxes in weather and climate models have sides that are between 5 kilometers (3 mi) and 300 kilometers (200 mi) in length" https://en.wikipedia.org/wiki/Numerical_weather_prediction
1. any percentage prediction like "99,999999999999%" chance of rain is still 100% correct if it doesn't rain.
2. its my understanding (from a trip to the local weather bureau) that an "80% change of rain" means something like "when conditions in the past were like they are now, it rained 4 out of 5 times".
Happy to be educated otherwise.
It's cool that their model has learned about this relationship, but a greater editorial point is that _we already have global air pollution models_ - particularly, the one this model was trained (fine-tuned) against! That's not to discount all the really novel and interesting new developments with this paper, but the editorialization around AI applications in the weather and climate space is getting way out of hand, especially when most of these applications are incremental improvements at best over the existing technology in the field (with the key exception of cost/efficiency of generating forecasts - that's a true revolution).
The Aurora paper is really cool, but issues like these greatly temper my enthusiasm for it. I hope that the research team seeks input and collaboration with domain experts in meteorology and atmospheric chemistry in the future to find ways to leverage this new technology in impactful and useful ways that actually will benefit society.
[1]: https://arxiv.org/html/2405.13063v2
From my perspective ML in general has a weird relationship with cross-domain collaboration. The discussions around linguistics feel like they're full of similar examples and it comes off as a lack of curiosity.