X-ray crystallography is a classic artisanal inverse problem, so having a picture related to that makes me look clever.
Robert Ackroyd introduces some handy phrasing for the connections between statistical estimation theory and inverse problem solving.
Photogrammetry, MRIs, SLAM, volumetric reconstruction and X-ray crystallography are all examples of inverse problems. any of these can be constructed as classical belief propagaation, especially Gaussian BP, or in a basic case least squares.
I happen to think that this is a case where it is much easier to explain in terms of Bayesian inference, so my attempt at an actual explanation is under Bayesian inverse problems.
We can do it in terms of frequentist methods, but it does not add much in the way of explanatory value; we end up considering regularizers instead of priors, but the working in between is pretty much the same. (IMO) However, in doing so we focus on point estimates rather than entire densities, which encourages us to solve the problem by optimisation rather than integration, which is a useful insight, for example, when we consider Laplace approximations.
Domain-specific model inversion
PEST, PEST++, and pyemu are some integrated systems for uncertainty quantification that use some weird terminology, such a FOSM (First-order-second-moment) models. They use various linear-algebra tricks to find plausible subspaces and samples.
Interesting specific techniques
Leaning to reconstruct introduces partly-learned, partly designed reconstruction operator trick. 🏗️
Radiance fields
A fun way of reconstructing objects from photos; differentiable photogrammetry.
References
Adler, Jonas, and Ozan Öktem. 2018. “Learned Primal-Dual Reconstruction.” IEEE Transactions on Medical Imaging 37 (6): 1322–32.
Alberti, Giovanni S., Ernesto De Vito, Matti Lassas, Luca Ratti, and Matteo Santacesaria. 2021. “Learning the Optimal Regularizer for Inverse Problems.” arXiv:2106.06513 [Cs, Math, Stat], June.
Aster, Richard C., Brian Borchers, and Clifford H. Thurber. 2019. Parameter Estimation and Inverse Problems. Third. Elsevier.
Basir, Shamsulhaq, and Inanc Senocak. 2022. “Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-Fidelity Data Fusion.” Journal of Computational Physics 463 (August): 111301.
Bissantz, Nicolai, Thorsten Hohage, and Axel Munk. 2004. “Consistency and Rates of Convergence of Nonlinear Tikhonov Regularization with Random Noise.” Inverse Problems 20 (6): 1773–89.
Borcea, Liliana, Vladimir Druskin, and Leonid Knizhnerman. 2005. “On the Continuum Limit of a Discrete Inverse Spectral Problem on Optimal Finite Difference Grids.” Communications on Pure and Applied Mathematics 58 (9): 1231–79.
Borgerding, Mark, and Philip Schniter. 2016. “Onsager-Corrected Deep Networks for Sparse Linear Inverse Problems.” arXiv:1612.01183 [Cs, Math], December.
Brehmer, Johann, Gilles Louppe, Juan Pavez, and Kyle Cranmer. 2020. “Mining Gold from Implicit Models to Improve Likelihood-Free Inference.” Proceedings of the National Academy of Sciences 117 (10): 5242–49.
Bui-Thanh, Tan. 2012. “A Gentle Tutorial on Statistical Inversion Using the Bayesian Paradigm.”
Chen, Yan, and Dean S. Oliver. 2013. “Levenberg–Marquardt Forms of the Iterative Ensemble Smoother for Efficient History Matching and Uncertainty Quantification.” Computational Geosciences 17 (4): 689–703.
Cranmer, Kyle, Johann Brehmer, and Gilles Louppe. 2020. “The Frontier of Simulation-Based Inference.” Proceedings of the National Academy of Sciences, May.
Daubechies, I., M. Defrise, and C. De Mol. 2004. “An Iterative Thresholding Algorithm for Linear Inverse Problems with a Sparsity Constraint.” Communications on Pure and Applied Mathematics 57 (11): 1413–57.
Engl, Heinz W., Andreas Hofinger, and Stefan Kindermann. 2005. “Convergence Rates in the Prokhorov Metric for Assessing Uncertainty in Ill-Posed Problems.” Inverse Problems 21 (1): 399–412.
Engl, Heinz W., and M. Zuhair Nashed. 1981. “Generalized Inverses of Random Linear Operators in Banach Spaces.” Journal of Mathematical Analysis and Applications 83 (2): 582–610.
Fernández-Martínez, J. L., Z. Fernández-Muñiz, J. L. G. Pallero, and L. M. Pedruelo-González. 2013. “From Bayes to Tarantola: New Insights to Understand Uncertainty in Inverse Problems.” Journal of Applied Geophysics 98 (November): 62–72.
Grigorievskiy, Alexander, Neil Lawrence, and Simo Särkkä. 2017. “Parallelizable Sparse Inverse Formulation Gaussian Processes (SpInGP).” In arXiv:1610.08035 [Stat].
Holl, Philipp, Vladlen Koltun, and Nils Thuerey. 2022. “Scale-Invariant Learning by Physics Inversion.” In.
Kaipio, Jari, and E. Somersalo. 2005. Statistical and Computational Inverse Problems. Applied Mathematical Sciences. New York: Springer-Verlag.
Kaipio, Jari, and Erkki Somersalo. 2007. “Statistical Inverse Problems: Discretization, Model Reduction and Inverse Crimes.” Journal of Computational and Applied Mathematics 198 (2): 493–504.
Lehtinen, M. S., L. Paivarinta, and E. Somersalo. 1989. “Linear Inverse Problems for Generalised Random Variables.” Inverse Problems 5 (4): 599–612.
Mandelbaum, Avi. 1984. “Linear Estimators and Measurable Linear Transformations on a Hilbert Space.” Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete 65 (3): 385–97.
Mosegaard, Klaus, and Albert Tarantola. 1995. “Monte Carlo Sampling of Solutions to Inverse Problems.” Journal of Geophysical Research: Solid Earth 100 (B7): 12431–47.
———. 2002. “Probabilistic Approach to Inverse Problems.” In International Geophysics, 81:237–65. Elsevier.
Murray-Smith, Roderick, and Barak A. Pearlmutter. 2005. “Transformations of Gaussian Process Priors.” In Deterministic and Statistical Methods in Machine Learning, edited by Joab Winkler, Mahesan Niranjan, and Neil Lawrence, 110–23. Lecture Notes in Computer Science. Springer Berlin Heidelberg.
O’Callaghan, Simon Timothy, and Fabio T. Ramos. 2011. “Continuous Occupancy Mapping with Integral Kernels.” In Twenty-Fifth AAAI Conference on Artificial Intelligence.
O’Sullivan, Finbarr. 1986. “A Statistical Perspective on Ill-Posed Inverse Problems.” Statistical Science 1 (4): 502–18.
Oliver, Dean S. 2022. “Hybrid Iterative Ensemble Smoother for History Matching of Hierarchical Models.” Mathematical Geosciences 54 (8): 1289–1313.
Pikkarainen, Hanna Katriina. 2006. “State Estimation Approach to Nonstationary Inverse Problems: Discretization Error and Filtering Problem.” Inverse Problems 22 (1): 365–79.
Plumlee, Matthew. 2017. “Bayesian Calibration of Inexact Computer Models.” Journal of the American Statistical Association 112 (519): 1274–85.
Putzky, Patrick, and Max Welling. 2017. “Recurrent Inference Machines for Solving Inverse Problems.” arXiv:1706.04008 [Cs], June.
Schnell, Patrick, Philipp Holl, and Nils Thuerey. 2022. “Half-Inverse Gradients for Physical Deep Learning.” arXiv:2203.10131 [Physics], March.
Schwab, C., and A. M. Stuart. 2012. “Sparse Deterministic Approximation of Bayesian Inverse Problems.” Inverse Problems 28 (4): 045003.
Stuart, Andrew M. 2010. “Inverse Problems: A Bayesian Perspective.” Acta Numerica 19: 451–559.
Tait, Daniel J., and Theodoros Damoulas. 2020. “Variational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat], June.
Tarantola, Albert. 2005. Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM.
Tonolini, Francesco, Jack Radford, Alex Turpin, Daniele Faccio, and Roderick Murray-Smith. 2020. “Variational Inference for Computational Imaging Inverse Problems.” Journal of Machine Learning Research 21 (179): 1–46.
Tropp, J. A., and S. J. Wright. 2010. “Computational Methods for Sparse Solution of Linear Inverse Problems.” Proceedings of the IEEE 98 (6): 948–58.
Wei, Qi, Kai Fan, Lawrence Carin, and Katherine A. Heller. 2017. “An Inner-Loop Free Solution to Inverse Problems Using Deep Neural Networks.” arXiv:1709.01841 [Cs], September.
Welter, David E., John E. Doherty, Randall J. Hunt, Christopher T. Muffels, Matthew J. Tonkin, and Willem A. Schreüder. 2012. “Approaches in Highly Parameterized Inversion: PEST++, a Parameter Estimation Code Optimized for Large Environmental Models.”
Welter, David E., Jeremy T. White, Randall J. Hunt, and John E. Doherty. 2015. “Approaches in Highly Parameterized Inversion—PEST++ Version 3, a Parameter ESTimation and Uncertainty Analysis Software Suite Optimized for Large Environmental Models.” USGS Numbered Series 7-C12. Techniques and Methods. Reston, VA: U.S. Geological Survey.
White, Jeremy T. 2018. “A Model-Independent Iterative Ensemble Smoother for Efficient History-Matching and Uncertainty Quantification in Very High Dimensions.” Environmental Modelling & Software 109 (November): 191–201.
White, Jeremy T., Michael N. Fienen, Paul M. Barlow, and Dave E. Welter. 2018. “A Tool for Efficient, Model-Independent Management Optimization Under Uncertainty.” Environmental Modelling & Software 100 (February): 213–21.
White, Jeremy T., Michael N. Fienen, and John E. Doherty. 2016a. “pyEMU: A Python Framework for Environmental Model Uncertainty Analysis Version .01.” U.S. Geological Survey.
———. 2016b. “A Python Framework for Environmental Model Uncertainty Analysis.” Environmental Modelling & Software 85 (November): 217–28.
White, Jeremy T., Randall J. Hunt, Michael N. Fienen, and John E. Doherty. 2020. “Approaches to Highly Parameterized Inversion: PEST++ Version 5, a Software Suite for Parameter Estimation, Uncertainty Analysis, Management Optimization and Sensitivity Analysis.” USGS Numbered Series 7-C26. Techniques and Methods. Reston, VA: U.S. Geological Survey.
Zammit-Mangion, Andrew, Michael Bertolacci, Jenny Fisher, Ann Stavert, Matthew L. Rigby, Yi Cao, and Noel Cressie. 2021. “WOMBAT v1.0: A fully Bayesian global flux-inversion framework.” Geoscientific Model Development Discussions, July, 1–51.
Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis. 2019. “Quantifying Total Uncertainty in Physics-Informed Neural Networks for Solving Forward and Inverse Stochastic Problems.” Journal of Computational Physics 397 (November): 108850.
No comments yet. Why not leave one?