Classic robotics problem: reconstruct a scene by moving a camera about the room.
There are many cool tricks combined there — differentiable rendering etc. Hierarchical implicit representations.
Differentiable GPB solver (Jatavallabhula, Iyer, and Paull 2020).
ceres-solver, (C++), the google least squares solver, seems to solve this kind of problem. I am not sure where the covariance matrices go in. I occasionally see mention of “CUDA” in the source repo so maybe it exploits GPUs these days.