Study Questions Hype Around Big AI for Climate Predictions

New research from the Massachusetts Institute of Technology suggests that bigger artificial intelligence models are not always better for predicting climate and weather changes.

The study, published today in the Journal of Advances in Modeling Earth Systems, shows that in certain scenarios, simpler physics-based models can provide more accurate forecasts than state-of-the-art deep-learning systems. The team also found that commonly used benchmarks for evaluating AI in climate science can be skewed by natural variations in the data, such as fluctuations in weather patterns, giving the impression that complex models are more accurate than they actually are.

The researchers developed a more robust evaluation method, which revealed that while deep-learning models may be better at predicting local rainfall, traditional techniques excel at estimating regional surface temperatures. They then used these insights to improve a climate emulator, a tool that simulates how human activities could influence the climate in the future.

Noelle Selin, a professor at MIT and senior author of the study, said the research offers a cautionary lesson for climate scientists. “While it might be tempting to apply the latest, large-scale AI models to climate problems, our study shows that stepping back and considering the fundamental physics is often more useful and accurate,” she said.

The study compared linear pattern scaling, a traditional approach, with a deep-learning model using a standard benchmarking dataset. Linear pattern scaling outperformed the AI model in almost all measures, including temperature and precipitation. The researchers found that high natural variability in the climate system, such as El Niño and La Niña cycles, can cause AI models to underperform and distort benchmarking results.

Using a new evaluation method that accounted for natural climate variability, the deep-learning approach slightly outperformed linear pattern scaling for local rainfall predictions, but linear pattern scaling remained more accurate for temperature forecasts.

Raffaele Ferrari, co-author and MIT professor of oceanography, said the findings highlight the importance of using the right tool for the problem. “Linear pattern scaling does have limitations, for example it cannot predict extreme weather events, but it remains highly reliable for regional temperature predictions,” he said.