Foundation models for geoscience

2024-11-14 — 2025-07-24

dynamical systems
machine learning
neural nets
sciml
SDEs
signal processing
spatial
stochastic processes
time series
Figure 1

Foundation models for the whole planet, as opposed to classical NNs, as opposed to classical geospatial techniques.

I am currently project lead for a CSIRO Geospatial Foundation Model research program, which means that I know quite a lot about this, but have little time to blog it. Yet.

2 Modelling governing equations

See foundation models for PDEs and Schmude et al. (). e.g. Prithvi WxC is a weather model surrogate, infilling “earth systems models”.

3 Incoming

4 References

Allen, Markou, Tebbutt, et al. 2025. “End-to-End Data-Driven Weather Prediction.”
Bodnar, Bruinsma, Lucic, et al. 2024. Aurora: A Foundation Model of the Atmosphere.”
Chen, Kun, Bai, Ling, et al. 2024. Towards an End-to-End Artificial Intelligence Driven Global Weather Forecasting System.”
Cheng, Min, Liu, et al. 2025. TorchDA: A Python Package for Performing Data Assimilation with Deep Learning Forward and Transformation Functions.” Computer Physics Communications.
Chen, Kang, Han, Gong, et al. 2023. FengWu: Pushing the Skillful Global Medium-Range Weather Forecast Beyond 10 Days Lead.”
Chen, Lei, Zhong, Zhang, et al. 2023. FuXi: A Cascade Machine Learning Forecasting System for 15-Day Global Weather Forecast.” Npj Climate and Atmospheric Science.
Chu, Tian, Wang, et al. 2021. Twins: Revisiting the Design of Spatial Attention in Vision Transformers.” In Advances in Neural Information Processing Systems.
Duraisamy, Iaccarino, and Xiao. 2019. Turbulence Modeling in the Age of Data.” Annual Review of Fluid Mechanics.
Grohs, and Herrmann. 2022. Deep Neural Network Approximation for High-Dimensional Elliptic PDEs with Boundary Conditions.” IMA Journal of Numerical Analysis.
Guibas, Mardani, Li, et al. 2021. Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers.”
Guo, Jia, and Bai. 2022. Transformer Based on Channel-Spatial Attention for Accurate Classification of Scenes in Remote Sensing Image.” Scientific Reports.
Hoffimann, Zortea, de Carvalho, et al. 2021. Geostatistical Learning: Challenges and Opportunities.” Frontiers in Applied Mathematics and Statistics.
Huang, Gianinazzi, Yu, et al. 2024. DiffDA: A Diffusion Model for Weather-Scale Data Assimilation.”
Jakubik, Roy, Phillips, et al. 2023. Foundation Models for Generalist Geospatial Artificial Intelligence.”
Keisler. 2022. Forecasting Global Weather with Graph Neural Networks.”
Keller, and Potthast. 2024. AI-Based Data Assimilation: Learning the Functional of Analysis Estimation.”
Lam, Sanchez-Gonzalez, Willson, et al. 2023a. GraphCast: Learning Skillful Medium-Range Global Weather Forecasting.”
———, et al. 2023b. Learning Skillful Medium-Range Global Weather Forecasting.” Science.
Mahesh, Collins, Bonev, et al. 2024. Huge Ensembles Part I: Design of Ensemble Weather Forecasts Using Spherical Fourier Neural Operators.”
Manshausen, Cohen, Pathak, et al. 2024. Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales.”
McNally, Lessig, Lean, et al. 2024. Data Driven Weather Forecasts Trained and Initialised Directly from Observations.”
Mickisch, Klemmer, Teng, et al. 2025. A Joint Space-Time Encoder for Geographic Time-Series Data.” In.
Pathak, Subramanian, Harrington, et al. 2022. Fourcastnet: A Global Data-Driven High-Resolution Weather Model Using Adaptive Fourier Neural Operators.”
Price, Sanchez-Gonzalez, Alet, et al. 2024. GenCast: Diffusion-Based Ensemble Forecasting for Medium-Range Weather.”
Rasp, Hoyer, Merose, et al. 2024. WeatherBench 2: A Benchmark for the Next Generation of Data-Driven Global Weather Models.”
Schmude, Roy, Trojak, et al. 2024. Prithvi WxC: Foundation Model for Weather and Climate.”
Szwarcman, Roy, Fraccaro, et al. 2025. Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications.”
Vatsavai. 2024. Geospatial Foundation Models: Recent Advances and Applications.” In Proceedings of the 12th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data.
Wei, Bora, Oommen, et al. 2025. XAI4Extremes: An Interpretable Machine Learning Framework for Understanding Extreme-Weather Precursors Under Climate Change.”
Willig, Zečević, Dhami, et al. 2022. Can Foundation Models Talk Causality?
Zeraatkar, Faroughi, and Tešić. 2025. ViSIR: Vision Transformer Single Image Reconstruction Method for Earth System Models.”
Zhu, Cheng, Zhang, et al. 2019. An Empirical Study of Spatial Attention Mechanisms in Deep Networks.” In.