Digital scientific workbooks

Literate coding for reality

The exploratory-algorithm-person’s IDE-equivalent. Literate coding-meets-science. a.k.a. dynamic report generation, a.k.a. literate programming.

Let’s say I want to demonstrate my algorithm to my thesis advisor while he’s off at conference. I need an easily shareable demonstration. that’s why we have the internet, right? I should be able to interleave text and mathematics and also code demonstrating the thingy, maybe even some graphs of the output. It should be in a simple format that one can use/execute/edit as simply as possible. That is what a scientific workbook does; It takes a the text and renders all the graphs and tables and other experimental output as a nicely formatted document. Reproducing the document requires pressing a single button, not laboriously manually executing some inscrutable code snippets, or a bloody spreadsheet.

General

Why do this? To belatedly immanentize the prophecy that the scientific paper is dead. Yihui Xie puts it more practically: Notebook war summarises some philosophical and practical differences between the literate coding/exploratory notebook hybrid tools in use with an eye to application. Workbook systmes are also behind projects like Nextjournal, a collaborative coding machine that claims to make this easy for you and your colleagues to write in a workbook style together. Go and read Exporable explanations for some philosophy.

Jupyter

jupyter is weird enough to have its own notebook. It is somewhat python centric but pretty good with multiple languages and AFAICT secretly used inside e.g. RStudio.

Chris Sewell has produced a script called ipypublish that eases some of the pain points in producing articles starting from jupyter.

Pweave

Pweave, by Matti Pastell, is python twin to knitr, in the lineage of literate coding tools. That is to say, it does less interactive notebook stuff and more straight-up report generation stuff.

Pweave is a scientific report generator and a literate programming tool for Python. It can capture the results and plots from data analysis and works well with numpy, scipy and matplotlib.

Max Masnick gives a detailed set up example.

Weave.jl

The Julia twin to PWeave or knitr is weave.jl See also Literate.jl. It looks similar

# capturing code output

The code chunk wil be run with default options and the output captured.
<<>>==
using Gadfly
x = linspace(0, 2* pi)
println(x)
plot(x = x, y = sin(x)
@

Or you could use RMarkdown/knitr in julia mode. It’s not yet clear to me how graphing works in that case. Changcheng Li claims: “easily”.

Tangle

Tangle did well here but appears to be little-maintained. Perhaps this is just because these things are hard.

knitr/RMarkdown

See knitr/RMarkdown.

Editor/IDE support

Miscellaneous preview support scripts are given in the knitr documentation.

Atom

Atom supports a number of literate programming tools via language-weave - you might also want a full typesetting experience via the latex or atom-latex package, which can be made to support literate coding of plain LaTeX. It uses Hydrogen to provide code preview.

Setting up a latex toolchain with a literate coding tool incorporated in atom-latex is not too bad. E.g. here is one for knitr:

{
    "root": "path/to/my/file.Rnw",
    "toolchain": "Rscript -e \"library(knitr); knit('%DOC.%EXT')\" && latexmk -synctex=1 -interaction=nonstopmode -file-line-error -pdf -halt-on-error %DOC",
    "latex_ext": [".Rnw"]
}

It would probably also work for pweave or Weave.jl.

VS Code

TBD.