There's been a lot talk of time series foundation models in the last year, but how well do they in fact perform on real-world operational tasks where decision makers face actual stakes?
Researchers led by Tao Han at The Hong Kong University of Science and Technology collaborated with AI/ML weather scientists in both China and Australia to answer this in a rare, one-of-its-kind study comparing the ECMWF-HRES Numerical Weather Prediction (NWP) model against a panel of nearly a dozen time series foundation models (TSFMs), and found that, while recent TSFMs perform well on benchmarks, they still lag behind operational NWP systems in capturing complex weather dynamics and extreme events.
In other words, modeling the physics of these meteorological phenomena outperforms simply generating confidence-bounded trend lines from statistical covariates and correlations.
Their work was accepted at Int'l Conference on Machine Learning (ICML) this July 2026.
Takeaway: TSFMs are still useful for quick back-of-the-envelope planning estimation at low cost, they require just a fraction of the inference compute of running NWP models; but true accuracy and skill comes from modeling the underlying causal dynamics of complex systems — especially when lives and livelihood are at risk.
1 comment
[ 0.17 ms ] story [ 10.7 ms ] threadResearchers led by Tao Han at The Hong Kong University of Science and Technology collaborated with AI/ML weather scientists in both China and Australia to answer this in a rare, one-of-its-kind study comparing the ECMWF-HRES Numerical Weather Prediction (NWP) model against a panel of nearly a dozen time series foundation models (TSFMs), and found that, while recent TSFMs perform well on benchmarks, they still lag behind operational NWP systems in capturing complex weather dynamics and extreme events.
In other words, modeling the physics of these meteorological phenomena outperforms simply generating confidence-bounded trend lines from statistical covariates and correlations.
Their work was accepted at Int'l Conference on Machine Learning (ICML) this July 2026.
Takeaway: TSFMs are still useful for quick back-of-the-envelope planning estimation at low cost, they require just a fraction of the inference compute of running NWP models; but true accuracy and skill comes from modeling the underlying causal dynamics of complex systems — especially when lives and livelihood are at risk.