Running neural nets backwards



TODO. Various techniques in β€œrunning networks backwards”, definition later.

Prior regularisations for ill-posed inverse problems.

Deep dreams probably fit here (Mahendran and Vedaldi 2015; Simonyan, Vedaldi, and Zisserman 2014; Yosinski et al. 2015). For more on that see Inceptionism: Going Deeper into Neural Networks.

I wonder if the reversible architectures (Chang et al. 2018) method gets us anything here?

My focus is more on inverse problems.

Let us run it backwards.

References

Adler, Jonas, and Ozan Γ–ktem. 2018. β€œLearned Primal-Dual Reconstruction.” IEEE Transactions on Medical Imaging 37 (6): 1322–32.
Anantha Padmanabha, Govinda, and Nicholas Zabaras. 2021. β€œSolving Inverse Problems Using Conditional Invertible Neural Networks.” Journal of Computational Physics 433 (May): 110194.
Ardizzone, Lynton, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W. Pellegrini, Ralf S. Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich KΓΆthe. 2019. β€œAnalyzing Inverse Problems with Invertible Neural Networks.” arXiv:1808.04730 [Cs, Stat], February.
Bao, Gang, Xiaojing Ye, Yaohua Zang, and Haomin Zhou. 2020. β€œNumerical Solution of Inverse Problems by Weak Adversarial Networks.” Inverse Problems 36 (11): 115003.
Borgerding, Mark, and Philip Schniter. 2016. β€œOnsager-Corrected Deep Networks for Sparse Linear Inverse Problems.” arXiv:1612.01183 [Cs, Math], December.
Chang, Bo, Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, and Elliot Holtham. 2018. β€œReversible Architectures for Arbitrarily Deep Residual Neural Networks.” In arXiv:1709.03698 [Cs, Stat].
Depina, Ivan, Saket Jain, Sigurdur Mar Valsson, and Hrvoje Gotovac. 2021. β€œApplication of Physics-Informed Neural Networks to Inverse Problems in Unsaturated Groundwater Flow.” Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards 0 (0): 1–16.
Franklin, Joel N. 1970. β€œWell-Posed Stochastic Extensions of Ill-Posed Linear Problems.” Journal of Mathematical Analysis and Applications 31 (3): 682–716.
Genzel, Martin, Jan Macdonald, and Maximilian MΓ€rz. 2020. β€œSolving Inverse Problems With Deep Neural Networks – Robustness Included?” arXiv:2011.04268 [Cs, Math], November.
Haber, Eldad, and Lars Ruthotto. 2018. β€œStable Architectures for Deep Neural Networks.” Inverse Problems 34 (1): 014004.
Holl, Philipp, and Vladlen Koltun. 2020. β€œPhiflow: A Differentiable PDE Solving Framework for Deep Learning via Physical Simulations,” 5.
Holl, Philipp, Nils Thuerey, and Vladlen Koltun. 2020. β€œLearning to Control PDEs with Differentiable Physics.” In ICLR, 5.
Jo, Hyeontae, Hwijae Son, Hyung Ju Hwang, and Eun Heui Kim. 2020. β€œDeep Neural Network Approach to Forward-Inverse Problems.” Networks & Heterogeneous Media 15 (2): 247.
Kawar, Bahjat, Gregory Vaksman, and Michael Elad. 2021. β€œSNIPS: Solving Noisy Inverse Problems Stochastically.” In.
Khodayi-Mehr, Reza, and Michael M. Zavlanos. 2019. β€œVarNet: Variational Neural Networks for the Solution of Partial Differential Equations.” arXiv:1912.07443 [Physics, Stat], December.
Li, Housen, Johannes Schwab, Stephan Antholzer, and Markus Haltmeier. 2020. β€œNETT: Solving Inverse Problems with Deep Neural Networks.” Inverse Problems 36 (6): 065005.
Lu, Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. 2021. β€œDeepXDE: A Deep Learning Library for Solving Differential Equations.” SIAM Review 63 (1): 208–28.
Lucas, Alice, Michael Iliadis, Rafael Molina, and Aggelos K. Katsaggelos. 2018. β€œUsing Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods.” IEEE Signal Processing Magazine 35 (1): 20–36.
Lunz, Sebastian, Carola Schoenlieb, and Ozan Γ–ktem. 2018. β€œAdversarial Regularizers in Inverse Problems.” In, 10.
Mahendran, Aravindh, and Andrea Vedaldi. 2015. β€œUnderstanding Deep Image Representations by Inverting Them.” In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5188–96.
Mowlavi, Saviz, and Saleh Nabi. 2021. β€œOptimal Control of PDEs Using Physics-Informed Neural Networks.” arXiv:2111.09880 [Physics], November.
Ogawa, T., Y. Kosugi, and H. Kanada. 1998. β€œNeural Network Based Solution to Inverse Problems.” In 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227), 3:2471–2476 vol.3.
Rackauckas, Christopher, Yingbo Ma, Vaibhav Dixit, Xingjian Guo, Mike Innes, Jarrett Revels, Joakim Nyberg, and Vijay Ivaturi. 2018. β€œA Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions.” arXiv:1812.01892 [Cs], December.
Simonyan, Karen, Andrea Vedaldi, and Andrew Zisserman. 2014. β€œDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.” arXiv:1312.6034 [Cs], April.
Spantini, Alessio, Antti Solonen, Tiangang Cui, James Martin, Luis Tenorio, and Youssef Marzouk. 2015. β€œOptimal Low-Rank Approximations of Bayesian Linear Inverse Problems.” SIAM Journal on Scientific Computing 37 (6): A2451–87.
Tait, Daniel J., and Theodoros Damoulas. 2020. β€œVariational Autoencoding of PDE Inverse Problems.” arXiv:2006.15641 [Cs, Stat], June.
Wei, Qi, Kai Fan, Lawrence Carin, and Katherine A. Heller. 2017. β€œAn Inner-Loop Free Solution to Inverse Problems Using Deep Neural Networks.” arXiv:1709.01841 [Cs], September.
Williams, Christopher, Stefan Klanke, Sethu Vijayakumar, and Kian M. Chai. 2009. β€œMulti-Task Gaussian Process Learning of Robot Inverse Dynamics.” In Advances in Neural Information Processing Systems 21, edited by D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, 265–72. Curran Associates, Inc.
Yosinski, Jason, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. β€œUnderstanding Neural Networks Through Deep Visualization.” arXiv:1506.06579 [Cs], June.
Zhang, Dongkun, Ling Guo, and George Em Karniadakis. 2020. β€œLearning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks.” SIAM Journal on Scientific Computing 42 (2): A639–65.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.