Connecting R to other things.
Rcpp seems to be how everyone invokes their favoured compiled C++ code.
There are higher level tools that do this under the hood -
rstan compiles an inner loop this for Bayesian posterior simulation and a little bit of basic variational inference.
If you want a little more freedom but still want to have automatic differentiation and linear algebra done by magic, try TMB whose name and description are both awful but which manages pretty neat reduced rank matrix and optimization tricks.
Julia is a nice language that can attain high performance, especially on numerical work, and we can use it from R.
reticulate can save and load numpy’s
Also, you can try rccpcnpy (pronounced Arrsipeekernoppy) which is a numpy-matrix-loader for R.
I have not tested the above for passing sparse matrices recently.
Last time I tried to pass sparse matrices, circa 2014, it was painful.
I developed a hdf5 hack to do it eventually.
Counter-intuitively, this FS-backed method was a couple of orders of
magnitude faster than
rpy2when the arrays totalled than a few MB of data.
Apparently you can use arrow for also these days, although I have not tried myself.