Academic writing workflow

Minimising the friction of advertising my thoughts in order to maximise the chance a clever thought gets advertised.

“Documenting my academic writing workflow and how to improve it”, or, “How I learned to stop worrying and love text files”.

Science blog

This blog, and virtually all my notes, are in plain text files on my computer, published online as plain html files. It’s an informal open notebook.

I had to jump through some hoops to make this work, because I need mathematical markup support and basic citation management. Vanilla, non-academic plain text blogging is simpler. But even this academic stuff is not complicated.

Why bother?

Personally speaking, it helps keeps me rigorous and coherent, knowing that the public can see what I am doing, and which half-cocked opinions I am holding. It encourages people to contact me about my ideas.

Further, having a bunch of plain text files is the most simple, convenient and reliable way of taking notes. I’d do it this way even if it weren’t going to be online.

Other examples of online notebooks

Technical details

This blog I publish online using Pelican Blogdown. I see plain text files; you see fancy online HTML. The HTML is automatically served by github pages netlify, which is fast and free. The citations are handled through Zotero. This is a work in progress.

For now I mostly edit the text using VS Code, or RStudio, both of which have integrated preview. The process looks like this:

Screencap of my text editor

Screencap of my text editor

On software choice

There are lots of tools to render academic blogs. There are hundreds of static site generators for plain text blogging. Particular ones of note for academics are…

Blogdown

Blogdown might be the most popular move now, especially if you want to access the mind-share of other academics, such as the rbind community. See also the blogdown-book. This is what I currently use for this site. It is built upon hugo which is an elegant platform. One point of friction is that hugo is well integrated with its own internal markdown processor, goldmark and much of the documentation reflects that. However, blogdown bypasses goldmark, using instead rmarkdown to render workbooks direct to HTML+images. This leads to pluses (equation and citation support) and minuses (many hugo tricks, notably shortcode) do not work. One could possibly work around this by rendering to markdown).

Pro tip for VS code. Hide extraneous crap using the following configuration:

  "files.watcherExclude": {
    "**/output/**": true
  },
  "files.exclude": {
    "content/**/*.html": true
  }

distill+radix

Another project by the crew at RStudio, distill is an academic writing system that overlaps in features and toolkit with blogdown but is oriented towards journal article-style typography. It was extracted from the journal Distill. As befits such an origin story, it has lavish article and citation metadata support. Emi Tanaka’s comparison makes the points of difference clear. (tl;dr; It makes academic output easy and other webby stuff maybe harder?)

Hugo academic

hugo-academic is oft recommended for scholarly types. Judging by the bug tracker is brittler than vanilla hugo. However it solves many niggly themeing problems at once and has a strong community. Leslie Myint’s introduction gives us the gist of it. It it not incompatible with blogdown.

pandoc-scholar

pandoc-scholar extends pandoc with useful academic features. 🏗

Pollen

Matthew Butterick’s Pollen is a dark horse.

I created Pollen so I could make my web-based books Practical Typography, Typography for Lawyers, and Beautiful Racket. Sure, go take a look. Are they better than the last digital books you encountered? Yes they are. Would you like your next digital book to work like that? If so, keep reading.

At the core of Pollen is an argument:

  1. Digital books should be the best books we’ve ever had. So far, they’re not even close.
  2. Because digital books are software, an author shouldn’t think of a book as merely data. The book is a program.
  3. The way we make digital books better than their predecessors is by exploiting this programmability.

That’s what Pollen is for.

Yes, it supports mathematical markup amongst all the other elaborate typography support.

Madoko

Daan Leijen’s Madoko is an even darker horse than Pollen. It includes some nifty publish-to-the-web features, native citation support, and fancy maths rendering.

jupyter

jupyter can render and export markdown as a flagship part of its policy of doing everything easily but nothing well.

jekyll

The classic static site generator endorsed by github can pump out academic blogs too. FWIW I found it needed too much tweaking out of the box and got bored because I am not a ruby hacker who enjoys that kind of thing. I mean, I’m even less of a go hacker but I still found hugo less trouble. 🏗

Pelican

Pelican is the python-based one that I used for this site for a ling time; works well for academia with my custom pandoc reader. Nafiul Islam gives some clever shortcuts on getting live preview using livereload. Michael Toth’s shows how to use RMarkdown+knitr to approximate blogdown.

Hakyll

Hakyll

Writing papers

Writing papers, especially collaboratively, is a whole other story; you need media management and citations etc. You might want to try a scientific notebook such as jupyter, knitr etc, which will generate the requisite diagrams etc. rticles, in particular, takes the scientific computation tools and turns them to generating photo-ready journal submissions.

Minimally painful LaTeX collaboration

And you need to work with collaborators of different technical expertise levels, across various quirky LaTeX setups, here are some options.

  • teletype enables generic live collaborative document editing (atom-only for now, no special support for latex.)

  • Overleaf (formerly ShareLaTeX) is an open source online collaborative LaTeX editor. They also have a hosted version.

    • Work together on a single version
    • View collaborator edits
    • No complicated LaTeX installation
    • Restore to any older version
    • Access your work from anywhere in the world
    • Work offline and sync your files via Dropbox and GitHub
  • papernow is more radical (and more ugly) again.

    Create, edit and (optionally) display a journal article, entirely in GitHub.

    In contrast to the more traditional process of submit > peer review > publish at PeerJ, or even the less formal preprints at PeerJ Preprints or arXiv, Paper Now is an experiment to see where the future may go with scholarly communication. Initially, it may be that co-authors collaborate either privately or publicly on GitHub and then proceed to submitting to PeerJ or other journals for formal peer-review or preprinting. Or perhaps this is where the traditional medium of publication begins to diverge. There is no end goal other than to see what the academic community wants, which is why this is completely open to fork, extend, and build upon.

Emailing word files between collaborators

You have gone beyond my ability to help you.