Constructing a backward (P)DE which effectively gives us the gradients of the forward (P)DE. A trick in automatic differentiation which happens to be useful in differentiating likelihood (or other functions) of time-evolving systems. This is an active area of research (Kidger, Chen, and Lyons 2021; Kidger et al. 2020; Li et al. 2020; Rackauckas et al. 2018; Stapor, Fröhlich, and Hasenauer 2018; Cao et al. 2003), but also old and well-studied [@Errico (1997);
- versus autodiff: There and Back Again: A Tale of Slopes and Expectations | Mathematics for Machine Learning
References
Cao, Y., S. Li, L. Petzold, and R. Serban. 2003. “Adjoint Sensitivity Analysis for Differential-Algebraic Equations: The Adjoint DAE System and Its Numerical Solution.” SIAM Journal on Scientific Computing 24 (3): 1076–89.
Carpenter, Bob, Matthew D. Hoffman, Marcus Brubaker, Daniel Lee, Peter Li, and Michael Betancourt. 2015. “The Stan Math Library: Reverse-Mode Automatic Differentiation in C++.” arXiv Preprint arXiv:1509.07164.
Errico, Ronald M. 1997. “What Is an Adjoint Model?” Bulletin of the American Meteorological Society 78 (11): 2577–92.
Gahungu, Paterne, Christopher W. Lanyon, Mauricio A. Álvarez, Engineer Bainomugisha, Michael Thomas Smith, and Richard David Wilkinson. 2022. “Adjoint-Aided Inference of Gaussian Process Driven Differential Equations.” In.
Giles, Mike B. 2008. “Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation.” In Advances in Automatic Differentiation, edited by Christian H. Bischof, H. Martin Bücker, Paul Hovland, Uwe Naumann, and Jean Utke, 64:35–44. Berlin, Heidelberg: Springer Berlin Heidelberg.
Innes, Michael. 2018. “Don’t Unroll Adjoint: Differentiating SSA-Form Programs.” arXiv:1810.07951 [Cs], October.
Ionescu, Catalin, Orestis Vantzos, and Cristian Sminchisescu. 2016. “Training Deep Networks with Structured Layers by Matrix Backpropagation.” arXiv.
Johnson, Steven G. 2012. “Notes on Adjoint Methods for 18.335,” 6.
Kavvadias, I. S., E. M. Papoutsis-Kiachagias, and K. C. Giannakoglou. 2015. “On the Proper Treatment of Grid Sensitivities in Continuous Adjoint Methods for Shape Optimization.” Journal of Computational Physics 301 (November): 1–18.
Kidger, Patrick, Ricky T. Q. Chen, and Terry J. Lyons. 2021. “‘Hey, That’s Not an ODE’: Faster ODE Adjoints via Seminorms.” In Proceedings of the 38th International Conference on Machine Learning, 5443–52. PMLR.
Kidger, Patrick, James Morrill, James Foster, and Terry Lyons. 2020. “Neural Controlled Differential Equations for Irregular Time Series.” arXiv:2005.08926 [Cs, Stat], November.
Li, Xuechen, Ting-Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. 2020. “Scalable Gradients for Stochastic Differential Equations.” In International Conference on Artificial Intelligence and Statistics, 3870–82. PMLR.
Margossian, Charles C., Aki Vehtari, Daniel Simpson, and Raj Agrawal. 2020. “Hamiltonian Monte Carlo Using an Adjoint-Differentiated Laplace Approximation: Bayesian Inference for Latent Gaussian Models and Beyond.” arXiv:2004.12550 [Stat], October.
Mitusch, Sebastian K., Simon W. Funke, and Jørgen S. Dokken. 2019. “Dolfin-Adjoint 2018.1: Automated Adjoints for FEniCS and Firedrake.” Journal of Open Source Software 4 (38): 1292.
Papoutsis-Kiachagias, E. M., and K. C. Giannakoglou. 2016. “Continuous Adjoint Methods for Turbulent Flows, Applied to Shape and Topology Optimization: Industrial Applications.” Archives of Computational Methods in Engineering 23 (2): 255–99.
Papoutsis-Kiachagias, E. M., N. Magoulas, J. Mueller, C. Othmer, and K. C. Giannakoglou. 2015. “Noise Reduction in Car Aerodynamics Using a Surrogate Objective Function and the Continuous Adjoint Method with Wall Functions.” Computers & Fluids 122 (November): 223–32.
Papoutsis-Kiachagias, Evangelos. 2013. “Adjoint Methods for Turbulent Flows, Applied to Shape or Topology Optimization and Robust Design.”
Rackauckas, Christopher, Yingbo Ma, Vaibhav Dixit, Xingjian Guo, Mike Innes, Jarrett Revels, Joakim Nyberg, and Vijay Ivaturi. 2018. “A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions.” arXiv:1812.01892 [Cs], December.
Stapor, Paul, Fabian Fröhlich, and Jan Hasenauer. 2018. “Optimization and Uncertainty Analysis of ODE Models Using 2nd Order Adjoint Sensitivity Analysis.” bioRxiv, February, 272005.
No comments yet. Why not leave one?