Multi fidelity models

Data-driven multi-scale sampling, multi-resolution, super-resolution

August 24, 2020 — November 20, 2023

Bayes
machine learning
physics
sciml
statmech
surrogate
Figure 1: The pun here is too delicious for me not to use this 1885 cartoon; yes, I am aware that marital dynamics amongst adherents of the LDS are not quite those that the gentleman depicted seems to believe; I am not here to endorse the opinions of fictional 19th century drunkards.

At the collision of coarse graining, sampling theory, and variational inference, we have multi-fidelity modeling, which attempts to harness the efficiency of lower-precision and higher-precision models together. This name is a Machine Learning name; I presume that this concept has been invented many times under other names, which I will add when I learn them. Possibly one of those names is learnable coarse graining.

1 GP methods

Much to say. For now, see the fascinatingly extended version in Xing, Wang, and Xing (2023).

2 Super resolution

An interesting and charismatic special case. Resolution is a special case of multi-fidelity modeling, where the lower-fidelity model is a low-resolution version of the higher-fidelity model; typically, when we talk about resolution, we are concerned specifically with a discrete lattice approximation, which is a fancy person’s way of saying pixels. This is a one-way process, going from a coarse model to a fine model, although often learning downsampling can be instrumentally helpful.

Figure 2

3 Physics constraints

TBC

4 From data alone

TBC

5 With transformers

Just heard about this. TBD.

6 As coarse-graining

See coarse graining

7 References

Altmann, Henning, and Peterseim. 2021. “Numerical Homogenization Beyond Scale Separation.” Acta Numerica.
Bao, Chen, Johnson, et al. 2022. “Physics Guided Neural Networks for Spatio-Temporal Super-Resolution of Turbulent Flows.” In Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence.
Candès, and Fernandez-Granda. 2013. “Super-Resolution from Noisy Data.” Journal of Fourier Analysis and Applications.
Chan, Cherukara, Narayanan, et al. 2019. “Machine Learning Coarse Grained Models for Water.” Nature Communications.
Chen, Bao, Givi, et al. 2023. “Reconstructing Turbulent Flows Using Physics-Aware Spatio-Temporal Dynamics and Test-Time Refinement.”
Cranmer, Brehmer, and Louppe. 2020. “The Frontier of Simulation-Based Inference.” Proceedings of the National Academy of Sciences.
Cutajar, Pullin, Damianou, et al. 2019. “Deep Gaussian Processes for Multi-Fidelity Modeling.” arXiv:1903.07320 [Cs, Stat].
Fukami, Fukagata, and Taira. 2023. “Super-Resolution Analysis via Machine Learning: A Survey for Fluid Flows.” Theoretical and Computational Fluid Dynamics.
Fu, Xie, Rebello, et al. 2022. “Simulate Time-Integrated Coarse-Grained Molecular Dynamics with Geometric Machine Learning.” In.
Gao, Sun, and Wang. 2021. “Super-Resolution and Denoising of Fluid Flow Using Physics-Informed Convolutional Neural Networks Without High-Resolution Labels.” Physics of Fluids.
Greener, and Jones. 2021. “Differentiable Molecular Simulation Can Learn All the Parameters in a Coarse-Grained Force Field for Proteins.” PLOS ONE.
Hirche. 2023. “Application and Extension of a Super Resolution Physics-Informed Convolutional Neural Network to Groundwater Modelling.”
Joshi, and Deshmukh. 2021. “A Review of Advancements in Coarse-Grained Molecular Dynamics Simulations.” Molecular Simulation.
Kennedy, M. C., and O’Hagan. 2000. “Predicting the Output from a Complex Computer Code When Fast Approximations Are Available.” Biometrika.
Kennedy, Marc C., and O’Hagan. 2001. “Bayesian Calibration of Computer Models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology).
Kochkov, Smith, Alieva, et al. 2021. “Machine Learning–Accelerated Computational Fluid Dynamics.” Proceedings of the National Academy of Sciences.
Kontolati, Alix-Williams, Boffi, et al. 2021. “Manifold Learning for Coarse-Graining Atomistic Simulations: Application to Amorphous Solids.” Acta Materialia.
Kuleshov, Enam, and Ermon. 2017. “Audio Super-Resolution Using Neural Nets.” In Proceedings of International Conference on Learning Representations (ICLR) 2017.
Ledig, Theis, Huszár, et al. 2017. “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network.” In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Lepcha, Goyal, Dogra, et al. 2023. “Image Super-Resolution: A Comprehensive Review, Recent Trends, Challenges and Applications.” Information Fusion.
Lienen, and Günnemann. 2021. “Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks.” In International Conference on Learning Representations.
Li, Yang, Chang, et al. 2022. “SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models.” Neurocomputing.
Ma, Wang, Kim, et al. 2021. “Transfer Learning of Memory Kernels for Transferable Coarse-Graining of Polymer Dynamics.” Soft Matter.
Medan, Yair, and Chazan. 1991. “Super Resolution Pitch Determination of Speech Signals.” IEEE Transactions on Signal Processing.
Meng, Babaee, and Karniadakis. 2021. “Multi-Fidelity Bayesian Neural Networks: Algorithms and Applications.” Journal of Computational Physics.
Nguyen, Tao, and Li. 2022. “Integration of Machine Learning and Coarse-Grained Molecular Simulations for Polymer Materials: Physical Understandings and Molecular Design.” Frontiers in Chemistry.
Oladyshkin, and Nowak. 2012. “Data-Driven Uncertainty Quantification Using the Arbitrary Polynomial Chaos Expansion.” Reliability Engineering & System Safety.
Perdikaris, P., Raissi, Damianou, et al. 2017. “Nonlinear Information Fusion Algorithms for Data-Efficient Multi-Fidelity Modelling.” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.
Perdikaris, Paris, Venturi, and Karniadakis. 2016. “Multifidelity Information Fusion Algorithms for High-Dimensional Systems and Massive Data Sets.” SIAM Journal on Scientific Computing.
Perdikaris, P., Venturi, Royset, et al. 2015. “Multi-Fidelity Modelling via Recursive Co-Kriging and Gaussian–Markov Random Fields.” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.
Pestourie, Mroueh, Rackauckas, et al. 2022. “Physics-Enhanced Deep Surrogates for PDEs.”
Popov. 2022. “Combining Data-Driven and Theory-Guided Models in Ensemble Data Assimilation.” ETD.
Raissi, and Karniadakis. 2016. “Deep Multi-Fidelity Gaussian Processes.” arXiv:1604.07484 [Cs, Stat].
Razavi, Tolson, and Burn. 2012. “Review of Surrogate Modeling in Water Resources.” Water Resources Research.
Ren, Rao, Liu, et al. 2023. “PhySR: Physics-Informed Deep Super-Resolution for Spatiotemporal Data.” Journal of Computational Physics.
Saharia, Chan, Chang, et al. 2022. “Palette: Image-to-Image Diffusion Models.” In ACM SIGGRAPH 2022 Conference Proceedings. SIGGRAPH ’22.
Sarkar, and Joly. 2019. “Multi-FIdelity Learning with Heterogeneous Domains.” In NeurIPS.
Shen, and Shen. 2024. “PSRFlow: Probabilistic Super Resolution with Flow-Based Models for Scientific Data.” IEEE Transactions on Visualization and Computer Graphics.
Tu, Rowley, Luchtenburg, et al. 2014. “On Dynamic Mode Decomposition: Theory and Applications.” Journal of Computational Dynamics.
Wang, Chulin, Bentivegna, Zhou, et al. 2020. “Physics-Informed Neural Network Super Resolution for Advection-Diffusion Models.” In.
Wang, Jiang, Chmiela, Müller, et al. 2020. “Ensemble Learning of Coarse-Grained Molecular Dynamics Force Fields with a Kernel Approach.” The Journal of Chemical Physics.
Wang, Jiang, Olsson, Wehmeyer, et al. 2019. “Machine Learning of Coarse-Grained Molecular Dynamics Force Fields.” ACS Central Science.
Wang, Yuxin, Xing, and Xing. 2023. “GAR: Generalized Autoregression for Multi-Fidelity Fusion.”
White. 2021. “Deep Learning for Molecules and Materials.” Living Journal of Computational Molecular Science.
Wurster, Guo, Shen, et al. 2023. “Deep Hierarchical Super Resolution for Scientific Data.” IEEE Transactions on Visualization and Computer Graphics.
Xie, Franz, Chu, et al. 2018. “tempoGAN: A Temporally Coherent, Volumetric GAN for Super-Resolution Fluid Flow.” ACM Transactions on Graphics.
Xing, Wang, and Xing. 2023. “ContinuAR: Continuous Autoregression For Infinite-Fidelity Fusion.” In.
Ye, Xian, and Li. 2021. “Machine Learning of Coarse-Grained Models for Organic Molecules and Polymers: Progress, Opportunities, and Challenges.” ACS Omega.
Zammit-Mangion, and Rougier. 2019. “Multi-Scale Process Modelling and Distributed Computation for Spatial Data.”