The exploratoryalgorithmpersonâ€™s IDEequivalent. Literate codingmeetsscience. a.k.a. dynamic report generation, a.k.a. literate programming.
Letâ€™s say I want to demonstrate my algorithm to my thesis advisor while heâ€™s off at conference. I need an easily shareable demonstration. thatâ€™s why we have the internet, right? I should be able to interleave text and mathematics and also code demonstrating the thingy, maybe even some graphs of the output. It should be in a simple format that one can use/execute/edit as simply as possible. That is what a scientific workbook does; It takes a the text and renders all the graphs and tables and other experimental output as a nicely formatted document. Reproducing the document requires pressing a single button, not laboriously manually executing some inscrutable code snippets, or a bloody spreadsheet.
Philosophy
Why do this? To belatedly immanentize the prophecy that the scientific paper is dead. Yihui Xie puts it more practically: Notebook war summarises some philosophical and practical differences between the literate coding/exploratory notebook hybrid tools in use with an eye to application. Workbook systmes are also behind projects like Nextjournal, a collaborative coding machine that claims to make this easy for you and your colleagues to write in a workbook style together. Go and read Exporable explanations for some philosophy, or explorabl.es for some hands on experiments.
Welcome to Explorable Explanations, a hub for learning through play! Weâ€™re a disorganized â€śmovementâ€ť of artists, coders & educators who want to reunite play and learning.
Glamorous Toolkit is the moldable development environment. It is a live notebook. It is a flexible search interface. It is a fancy code editor. It is a software analysis platform. It is a data visualization engine. All in one. And it is free and opensource under an MIT license.
Jeremy Kun on UI for mathematics discusses one fo the problem that scientific workbooks are implicitly attempting to solve.
Lots of people struggle with math, and a better user interface for mathematics would immediately usher in a new age of enlightenment. This isnâ€™t an idle speculation. It has happened time and time again throughout history. The Persian mathematician Muhammad ibn Musa alKhwarizmi invented algebra (though without the symbols for it) which revolutionized mathematics, elevating it above arithmetic and classical geometry, quickly scaling the globe. Make no mistake, the invention of algebra literally enabled average people to do contemporarily advanced mathematics.â€¦ Shortly after the printing press was invented French mathematicians invented modern symbolic notation for algebra, allowing mathematics to scale up in complexity. Symbolic algebra was a new user interface that birthed countless new thoughts. Without this, for example, mathematicians would never have discovered the connections between algebra and geometry that are so prevalent in modern mathematics and which lay the foundation of modern physics. Later came the invention of set theory, and shortly after category theory, which were each new and improved user interfaces that allowed mathematicians to express deeper, more unified, and more nuanced ideas than was previously possible.â€¦
In his book â€śThe Art of Doing Science and Engineering,â€ť the mathematician and computer scientist Richard Hamming put this difficulty into words quite nicely,
It has rarely proved practical to produce exactly the same product by machines as we produced by hand. Indeed, one of the major items in the conversion from hand to machine production is the imaginative redesign of an equivalent product. Thus in thinking of mechanizing a large organization, it wonâ€™t work if you try to keep things in detail exactly the same, rather there must be a larger giveandtake if there is to be a significant success. You must get the essentials of the job in mind and then design the mechanization to do that job rather than trying to mechanize the current versionâ€”if you want a significant success in the long run.
Hammingâ€™s attitude about an â€śequivalent productâ€ť summarizes the frustration of writing software. What customers want differs from what they say they want. Automating manual human processes requires arduously encoding the loose judgments made by humansâ€”often inconsistent and based on folk lore and experience. Software almost always falls short of really solving your problem. Accommodating the shortcomings requires a whole extra layer of process.
â€¦ My imagination may thus defeat itself by failing to give any ground. If a new interface is to replace pencil and paper mathematics, must I give up the ease of some routine mathematical tasks? Or remove them from my thinking style entirely? â€¦
Mathematics succeeds only insofar as it advances human understanding. Pencil and paper may be the wrong tool for the next generation of great thinkers. But if we hope to enable future insights, we must understand how and why the existing tools facilitated the great ideas of the past. We must imbue the best features of history into whatever we build. If you, dear programmer, want to build those tools, I hope you will incorporate the lessons and insights of mathematics.
Jupyter
jupyter is weird enough to have its own notebook. It is somewhat python centric but pretty good with multiple languages and AFAICT secretly used inside e.g.Â RStudio.
Chris Sewell has produced a script called ipypublish that eases some of the pain points in producing articles starting from jupyter.
Pweave
Pweave, by Matti Pastell, is python twin to knitr, in the lineage of literate coding tools. That is to say, it does less interactive notebook stuff and more straightup report generation stuff.
Pweave is a scientific report generator and a literate programming tool for Python. It can capture the results and plots from data analysis and works well with numpy, scipy and matplotlib.
Max Masnick gives a detailed set up example.
Weave.jl
The Julia twin to PWeave or knitr is weave.jl See also Literate.jl. It looks similar
# capturing code output
The code chunk wil be run with default options and the output captured.
<<>>==
using Gadfly
x = linspace(0, 2* pi)
println(x)
plot(x = x, y = sin(x)
@
Or you could use RMarkdown/knitr in julia mode. Itâ€™s not yet clear to me how graphing works in that case. Changcheng Li claims: â€śeasilyâ€ť.
Tangle
Tangle did well here but appears to be littlemaintained. Perhaps this is just because these things are hard.
knitr/RMarkdown
See knitr/RMarkdown.
MATLAB
 Neil Lawrencesâ€™s matlab sweavelike hacks
 Marking up MATLAB comments for publishing
Editor/IDE support
Miscellaneous preview support scripts are given in the knitr documentation.
Atom
Atom supports a number of literate programming tools via
languageweave

you might also want a full typesetting experience via
the latex
or atomlatex
package,
which can be made to support literate coding of plain LaTeX.
It uses Hydrogen
to provide code preview.
Setting up a latex toolchain with a literate coding tool incorporated in atomlatex
is
not too bad.
E.g. here is one for knitr
:
{
"root": "path/to/my/file.Rnw",
"toolchain": "Rscript e \"library(knitr); knit('%DOC.%EXT')\" && latexmk synctex=1 interaction=nonstopmode filelineerror pdf haltonerror %DOC",
"latex_ext": [".Rnw"]
}
It would probably also work for pweave
or Weave.jl
.
VS Code
TBD.