Numerical PDE solvers



An numerical solver iteratively massages an initial condition into a solution with the required form

There are many methods for numerically solving partial differential equations. Finite element methods, geometric multigrid, algebraic multigrid. Stochastic methods… Building a solver from base principles is a wonderful and improving activity but something that I do not wish to do recreationally. For a list that weights them by usefulness in a machine learning context see ML PDEs.

Clawpack

Clawpack documentation — Clawpack 5.8.x documentation

Clawpack is a collection of finite volume methods for linear and nonlinear hyperbolic systems of conservation laws. Clawpack employs high-resolution Godunov-type methods with limiters in a general framework applicable to many kinds of waves. Clawpack is written in Fortran and Python.

Dedelus

Dedalus Project >Dedalus solves differential equations using spectral methods. >It’s open-source, written in Python, and MPI-parallelized. > >We develop and use Dedalus to study fluid dynamics, but it’s designed to solve initial-value, boundary-value, and eigenvalue problems involving nearly arbitrary equations sets. You build a spectrally-representable domain, symbolically specify equations and boundary conditions, select a numerical solver, and go.

If my problem happens to be neatly expressible as a product of This one looks extra nice because sometimes I just want problems solved on a raster grid without all the flexibility of the FEM solvers.

ADCME

A Julia system using Tensorflow to handle some of the autodiff mechanics.

ADCME is suitable for conducting inverse modeling in scientific computing; specifically, ADCME targets physics informed machine learning, which leverages machine learning techniques to solve challenging scientific computing problems. The purpose of the package is to:

  1. provide differentiable programming framework for scientific computing based on TensorFlow automatic differentiation (AD) backend;
  2. adapt syntax to facilitate implementing scientific computing, particularly for numerical PDE discretization schemes;
  3. supply missing functionalities in the backend (TensorFlow) that are important for engineering, such as sparse linear algebra, constrained optimization, etc.

Applications include

  • physics informed machine learning (a.k.a., scientific machine learning, physics informed learning, etc.)
  • coupled hydrological and full waveform inversion
  • constitutive modeling in solid mechanics
  • learning hidden geophysical dynamics
  • parameter estimation in stochastic processes

The package inherits the scalability and efficiency from the well-optimized backend TensorFlow. Meanwhile, it provides access to incorporate existing C/C++ codes via the custom operators. For example, some functionalities for sparse matrices are implemented in this way and serve as extendable ”plugins” for ADCME.

AFAICT this implies that GPUs are not in general supported.

JuliaFEM

Julaifem is an umbrella organisation supporting julia-backed FEM solvers. The documentation is tricksy, but check out the examples. Analyses and solvers · JuliaFEM.jl

Trixi

Trixi.jl

Trixi.jl is a numerical simulation framework for hyperbolic conservation laws written in Julia. A key objective for the framework is to be useful to both scientists and students. Therefore, next to having an extensible design with a fast implementation, Trixi is focused on being easy to use for new or inexperienced users, including the installation and postprocessing procedures.

Firedrake

Firedrake seems popular for Python projects that do not need GPU?

Firedrake is an automated system for the solution of partial differential equations using the finite element method (FEM). Firedrake uses sophisticated code generation to provide mathematicians, scientists, and engineers with a very high productivity way to create sophisticated high performance simulations.

  • Expressive specification of any PDE using the Unified Form Language from the FEniCS Project.
  • Sophisticated, programmable solvers through seamless coupling with PETSc.…
  • Sophisticated automatic optimisation, including sum factorisation for high order elements, and vectorisation.

FEniCS

Also seems to be a friendly PDE solver. Also lacking in GPU support. However, it does have an interface to pytorch, barkm/torch-fenics so is a tenable, if slow, component in ML PDE solution (Alnæs 2012; Alnæs et al. 2015, 2014, 2009; Alnæs, Logg, and Mardal 2012; Kirby 2004, 2012; Kirby and Logg 2006; Logg, Mardal, et al. 2012; Logg, Ølgaard, et al. 2012; Logg and Wells 2010; Logg, Wells, and Hake 2012).

FEniCS is a popular open-source (LGPLv3) computing platform for solving partial differential equations (PDEs). FEniCS enables users to quickly translate scientific models into efficient finite element code. With the high-level Python and C++ interfaces to FEniCS, it is easy to get started, but FEniCS offers also powerful capabilities for more experienced programmers. .FEniCS runs on a multitude of platforms ranging from laptops to high-performance clusters.

As an illustration of how to program a simple PDE model with FEniCS, consider the Stokes equations in variational form:

\[\int_{\Omega} \mathrm{grad} \, u : \mathrm{grad} \, v \,\mathrm{d}x \, – \int_{\Omega} p \, \mathrm{div} \, v \,\mathrm{d}x + \int_{\Omega} \mathrm{div} \, u \, q \,\mathrm{d}x = \int_{\Omega} f \cdot v \,\mathrm{d}x.\]

The variational problem is easily transcribed into Python using mathematical operators in FEniCS:

# Define function space
P2 = VectorElement('P', tetrahedron, 2)
P1 = FiniteElement('P', tetrahedron, 1)
TH = P2 * P1
W = FunctionSpace(mesh, TH)

# Define variational problem
(u, p) = TrialFunctions(W)
(v, q) = TestFunctions(W)
a = inner(grad(u), grad(v))*dx - p*div(v)*dx + div(u)*q*dx
L = dot(f, v)*dx

# Compute solution
w = Function(W)
solve(a == L, w, [bc1, bc0])

Ooh, pretty

Mantaflow

mantaflow - an extensible framework for fluid simulation

mantaflow is an open-source framework targeted at fluid simulation research in Computer Graphics and Machine Learning. Its parallelized C++ solver core, python scene definition interface and plugin system allow for quickly prototyping and testing new algorithms. A wide range of Navier-Stokes solver variants are included. It’s very versatile, and allows coupling and import/export with deep learning frameworks (e.g., tensorflow via numpy) or standalone compilation as matlab plugin. Mantaflow also serves as the simulation engine in Blender.

Feature list

The framework can be used with or without GUI on Linux, MacOS and Windows. Here is an incomplete list of features implemented so far:

  • Eulerian simulation using MAC Grids, PCG pressure solver and MacCormack advection
  • Flexible particle systems
  • FLIP simulations for liquids
  • Surface mesh tracking
  • Free surface simulations with levelsets, fast marching
  • Wavelet and surface turbulence
  • K-epsilon turbulence modeling and synthesis
  • Maya and Blender export for rendering

Mantaflow is associated with some projects that examine neural PDE solutions, and also with producing stunning 3d animations as an output.

scikit-fdidd

Nicolas Cellier / scikit-fdiff/Welcome to Scikit FiniteDiff’s documentation!

Scikit-fdiff is a python library that aim to solve partial derivative equations without pain. As its name says, it uses finite difference method to discretize the spatial derivative. As its name does not say, it is based on method of lines where all the dimension of the PDE but the last (the time) is discretized. That turns the PDE in a high-dimension ODE that can be solved with standard numerical integration of ODEs.

Annoyingly this does not seem to directly support adjoints, but it should not be too hard to hack.

PolyFEM

PolyFEM is a simple C++ and Python finite element library. We provide a wide set of common PDEs including:

  • Laplace
  • Helmholtz
  • Linear Elasticity
  • Saint-Venant Elasticity
  • Neo-Hookean Elasticity
  • Stokes
  • Navier Stokes [beta]

PolyFEM simplicity lies on the interface: just pick a problem, select some boundary condition, and solve. No need to construct complicated function spaces, or learn a new scripting language: everything is set-up trough a JSON interface or through the Setting class in python.

SfePy

SfePy is a (sic) software for solving systems of coupled partial differential equations (PDEs) by the finite element method in 1D, 2D and 3D. It can be viewed both as black-box PDE solver, and as a Python package which can be used for building custom applications. The word “simple” means that complex FEM problems can be coded very easily and rapidly.

See Cimrman, Lukeš, and Rohan (2019).

fipy

fipy is original school:

FiPy is an object oriented, partial differential equation (PDE) solver, written in Python, based on a standard finite volume (FV) approach. … The FiPy framework includes terms for transient diffusion, convection and standard sources, enabling the solution of arbitrary combinations of coupled elliptic, hyperbolic and parabolic PDEs. Currently implemented models include phase field treatments of polycrystalline, dendritic, and electrochemical phase transformations as well as a level set treatment of the electrodeposition process.

PyAMG

pyamg is an algebraic multigrid solver, which is … a kind of multi-resolution method?

AMG is a multilevel technique for solving large-scale linear systems with optimal or near-optimal efficiency. Unlike geometric multigrid, AMG requires little or no geometric information about the underlying problem and develops a sequence of coarser grids directly from the input matrix. This feature is especially important for problems discretised on unstructured meshes and irregular grids:

A = poisson((500,500), format='csr')      # 2D Poisson problem on 500x500 grid
ml = ruge_stuben_solver(A)                # construct the multigrid hierarchy
print ml                                  # print hierarchy information
b = rand(A.shape[0])                      # pick a random right hand side
x = ml.solve(b, tol=1e-10)                # solve Ax=b to a tolerance of 1e-8
print("residual norm is", norm(b - A*x))  # compute norm of residual vector

See Yousef Saad’s textbooks on algebraic multigrid methods (free online).

Single-purpose solvers

References

Alnæs, Martin S. 2012. “UFL: A Finite Element Form Language.” In Automated Solution of Differential Equations by the Finite Element Method, Volume 84 of Lecture Notes in Computational Science and Engineering, edited by Anders Logg, Kent-Andre Mardal, and Garth N. Wells. Springer.
Alnæs, Martin S., Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris Richardson, Johannes Ring, Marie E. Rognes, and Garth N. Wells. 2015. “The FEniCS Project Version 1.5.” Archive of Numerical Software 3 (100). https://doi.org/10.11588/ans.2015.100.20553.
Alnæs, Martin S., Anders Logg, and Kent-Andre Mardal. 2012. “UFC: A Finite Element Code Generation Interface.” In Automated Solution of Differential Equations by the Finite Element Method, Volume 84 of Lecture Notes in Computational Science and Engineering, edited by Anders Logg, Kent-Andre Mardal, and Garth N. Wells. Springer.
Alnæs, Martin S., Anders Logg, Kent-Andre Mardal, Ola Skavhaug, and Hans Petter Langtangen. 2009. “Unified Framework for Finite Element Assembly.” International Journal of Computational Science and Engineering 4 (4): 231–44. https://doi.org/10.1504/IJCSE.2009.029160.
Alnæs, Martin S., Anders Logg, Kristian B. Ølgaard, Marie E. Rognes, and Garth N. Wells. 2014. “Unified Form Language: A Domain-Specific Language for Weak Formulations of Partial Differential Equations.” ACM Transactions on Mathematical Software 40 (2). https://doi.org/10.1145/2566630.
Andersson, Joel A. E., Joris Gillis, Greg Horn, James B. Rawlings, and Moritz Diehl. 2019. “CasADi: A Software Framework for Nonlinear Optimization and Optimal Control.” Mathematical Programming Computation 11 (1): 1–36. https://doi.org/10.1007/s12532-018-0139-4.
Barba, Lorena A., and Gilbert F. Forsyth. 2018. “CFD Python: The 12 Steps to Navier-Stokes Equations.” Journal of Open Source Education 2 (16): 21. https://doi.org/10.21105/jose.00021.
Cimrman, Robert, Vladimír Lukeš, and Eduard Rohan. 2019. “Multiscale Finite Element Calculations in Python Using SfePy.” Advances in Computational Mathematics 45 (4): 1897–1921. https://doi.org/10.1007/s10444-019-09666-0.
Innes, Mike, Alan Edelman, Keno Fischer, Chris Rackauckas, Elliot Saba, Viral B. Shah, and Will Tebbutt. 2019. “A Differentiable Programming System to Bridge Machine Learning and Scientific Computing.” arXiv:1907.07587 [cs], July. http://arxiv.org/abs/1907.07587.
Kirby, Robert C. 2004. “Algorithm 839: FIAT, a New Paradigm for Computing Finite Element Basis Functions.” ACM Transactions on Mathematical Software 30 (4): 502–16. https://doi.org/10.1145/1039813.1039820.
———. 2012. “FIAT: Numerical Construction of Finite Element Basis Functions,” In Automated Solution of Differential Equations by the Finite Element Method, Volume 84 of Lecture Notes in Computational Science and Engineering, edited by Anders Logg, Kent-Andre Mardal, and Garth N. Wells. Springer.
Kirby, Robert C., and Anders Logg. 2006. “A Compiler for Variational Forms.” ACM Transactions on Mathematical Software 32 (3). https://doi.org/10.1145/1163641.1163644.
Logg, Anders, Kent-Andre Mardal, Garth N. Wells, and others. 2012. Automated Solution of Differential Equations by the Finite Element Method. Springer. https://doi.org/10.1007/978-3-642-23099-8.
Logg, Anders, and Garth N. Wells. 2010. “DOLFIN: Automated Finite Element Computing.” ACM Transactions on Mathematical Software 37 (2). https://doi.org/10.1145/1731022.1731030.
Logg, Anders, Garth N. Wells, and Johan Hake. 2012. “DOLFIN: A C++/Python Finite Element Library.” In Automated Solution of Differential Equations by the Finite Element Method, Volume 84 of Lecture Notes in Computational Science and Engineering, edited by Anders Logg, Kent-Andre Mardal, and Garth N. Wells. Springer.
Logg, Anders, Kristian B. Ølgaard, Marie E. Rognes, and Garth N. Wells. 2012. “FFC: The FEniCS Form Compiler.” In Automated Solution of Differential Equations by the Finite Element Method, Volume 84 of Lecture Notes in Computational Science and Engineering, edited by Anders Logg, Kent-Andre Mardal, and Garth N. Wells. Springer.
Mitusch, Sebastian K., Simon W. Funke, and Jørgen S. Dokken. 2019. “Dolfin-Adjoint 2018.1: Automated Adjoints for FEniCS and Firedrake.” Journal of Open Source Software 4 (38): 1292. https://doi.org/10.21105/joss.01292.
Pawar, Suraj, and Omer San. 2019. “CFD Julia: A Learning Module Structuring an Introductory Course on Computational Fluid Dynamics.” Fluids 4 (3): 159. https://doi.org/10.3390/fluids4030159.
Rackauckas, Chris, Alan Edelman, Keno Fischer, Mike Innes, Elliot Saba, Viral B Shah, and Will Tebbutt. 2020. “Generalized Physics-Informed Learning Through Language-Wide Differentiable Programming,” 6.
Rackauckas, Christopher, Yingbo Ma, Vaibhav Dixit, Xingjian Guo, Mike Innes, Jarrett Revels, Joakim Nyberg, and Vijay Ivaturi. 2018. “A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions.” arXiv:1812.01892 [cs], December. http://arxiv.org/abs/1812.01892.
Rackauckas, Christopher, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, and Alan Edelman. 2020. “Universal Differential Equations for Scientific Machine Learning.” arXiv:2001.04385 [cs, Math, q-Bio, Stat], August. http://arxiv.org/abs/2001.04385.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.