Zotero

The adequate citation tool


This icon is on my desktop about 90% of my working day

Zotero! My weapon of choice for citation management.

It starts with browser plugin. With Zotero, I visit an article in my browser, and a button appears in the browser to enable me to import the article into my literature database. I click it and it magically appears in my database, with all the metadata and citation information and a copy of the PDF. After a while I have a big interactive searchable database of these things with a nice user interface and browsing tools. From there I can export to various other formats, such a BibTeX. I can render bibliographies directly in my word processor. I can do lots of other stuff. But don’t listen to me blather. Read the manual. Or the other manual. Or some other person’s manual.

Zotero has an API with which you can query, read and write bibliographic entries in your database, making it easy to scrip automatic updates and such.

It has an active community around it, and I don’t feel that I am locking my data away with an untrusted party if I rely upon it. (Of course, you can always try to migrate data around from anything to anything else by exporting to BibTeX or similar, but if you have URLs in there, or consort with foreigners who dare to have diacritics in their names, this leads to trouble). I can use the API to make changes that I couldn’t make manually, without worrying about that parsing nonsense. Moreover, some other apps (Mendeley, …) already use the Zotero API so you know that there are people you can ask for help when things are broken.

Installing Zotero

Windows or macOS

Use the standard installers.

Linux

A little more tedious with Linux. retorquere’s repo of deb installers is a simple way for Debian/Ubuntu.

For non-ubuntu systems… try packaged version?

There is a snap-app package. The weird paths of snaps mean that you must do a lot of configuration to migrate to it if you were not previously using it. It began to fee like yak shaving to continue and so I did not finish said migration and thus cannot report on it. There is a cross-platform flatpak zotero, which is fine, with the usual caveats about flatpak.

flatpak install flathub org.zotero.Zotero
flatpak override --user --filesystem=/PATH/TO/ZOTEROFOLDER \
    org.zotero.Zotero

UPDATE: as of 2020-05, the flatpak seems to have been abandoned since 2019-12 and is no longer recommended. Install manually.

wget https://www.zotero.org/download/client/dl?channel=release&platform=linux-x86_64
## Untar it to, say, ~/Zotero_linux-x86_64/
./set_launcher_icon
ln -s ~/Zotero_linux-x86_64/zotero ~/bin/
ln -s ~/Zotero_linux-x86_64/zotero.desktop ~/.local/share/applications/

BetterBibTeX

Better BibTeX, a.k.a BBT, makes all the BibTeX stuff in Zotero smoother, and since BibTeX also integrates with markdown via pandoc, this is a double win.

The biggest trick is that BBT makes sensible, user-accessible citation keys for your references, and if you use these consistently to refer to your sources, life will go well for you.

The citation keys are generated magically by a format string. (Of course, there is no guarantee your colleagues will agree on a sensible standard for citation keys, but that problem is perennial.)

My citation key format strings for BibTeX is

[Auth:fold:nopunctordash][title:fold:nopunctordash:capitalize:skipwords:select,1,1][year]

which I think is approximately the same as the much shorter

[zotero:clean]

That may be the same as as this other shortish one

​[auth][veryshorttitle1_1][year]

I am not sure if the latter two options handle diacritical marks correctly; I should check. Additionally, I do not trust [zotero:clean] because occasionally its meaning changes and it ruins everything.

So, say I want to cite the classic

Ingrid Daubechies (1988) Orthonormal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics, 41(7), 909–996. DOI.

Those cite key formulae above will all ensure that I can refer to this work as DaubechiesOrthonormal1988. I could then cite it in LaTeX as \cite{DaubechiesOrthonormal1988}, in markdown as @DaubechiesOrthonormal1988 and probably in other systems using some other syntax that I have had no need to know thus far.

Zotero will also generate the necessary bibliography databases (e.g. .bib file) and optionally keep that file updated as I add more articles to Zotero. That is once again a nifty magical feature but I found it easier to write my own script to do this on my own schedule rather than relying on magic.

I am fond of the following BibLaTeX export exclusions to keep the files small.

file,abstract,note,keywords,lccn

BBT has a Cite-as-you-write feature for providing a GUI popup citation finder for generic editors. I do not use this because I export my citations to a BibLaTeX file on disk using HTTP-pull (below) and my editors already support this natively for BibLaTeX files on disk.

BBT also has HTTP-pull export support, meaning that accessing an up-to-date bib file is a matter of an HTTP request that looks like http://127.0.0.1:23119/better-bibtex/collection?/1/citation_management.biblatex to pull all the records in the citation_management collection and this works from my local copy of Zotero, so it’s fast and reliable compared to the cloud Zotero server API which depends upon my internet connection etc. I use this all the time to make this blog citation.

The pull export works better if for me if I set the advanced preference extensions.zotero.translators.better-bibtex.sorted to be true so that the reference sorting is consistent, otherwise it seems to be random each time.

citr provides an interactive citation finder for RStudio that works with BetterBibTeX.

install.packages("citr")

Pro tips

I'm not quite sure what the purpose of this Zotero presskit graphic is, unless it be to appear decoratively, and yet slightly confusingly, half way down a blog post about Zotero

Use your existing folder of PDFs

Integrate with your existing folder of PDFs? Do not wish to use Zotero’s storage system?

Richard Zach points out

… you keep your PDF directory synced across computers (e.g., if it lives in your Dropbox), linking the PDFs is just as good. If you add a PDF, Zotero will look up the metadata for you and add a reference to your database.

As a bonus, this means you don’t have to pay Zotero for storage on their cloud server. I pay them anyway because I want to support this project. Maybe I should just donate?

NB: this is likely to break native e-reader/tablet clients, if they do not support your custom sync method. As things currently stand, though, native tablet clients are not useful.

Quick copy shortcut breaks

Occasionally (for me, frequently) the Quick copy keyboard shortcut breaks. Per default this is Ctrl-Shift-C. This should be fixed by restarting Zotero. Nerds can enter the following in the developer console:

await Zotero.Translators.reinit()

This solves the problem for me.

It does not solve a related problem which arises on Ubuntu 20.04 where Ctrl-Shift-C just seems a contentious key combination to use and does weird stuff which is never what I want. It launches a terminal in VS Code, even though I disabled that shortcut, it never does anything in Zotero. Since this is only bad enough to annoy me but not so cripple me as to be worth going down a rabbit hole of GNOME keyboard shortcut debugging, I will not investigate this further. I reassigned it to a different key, Ctrl-Shift-B, and it seems to work better so far.

Hacking

I promised that Zotero’s selling point was its hackability. So I should mention how one goes about hacking it. One thing that may not be immediately clear is that there are two parts to the Zotero system.

  1. There is a client, the zotero app, that runs on your machine. Confusingly this can also run its own web server.
  2. There is the Zotero service, run on a George Mason University server somewhere.

Both these parts have their own APIs and many tasks can be accomplished through either. There are pluses and minuses to each.

Server side

The server side of Zotero has a net-facing Web API. Its virtues are that you can use your language of choice, not just javascript, and it is simple; there is no messing about with the complicated build chain of javascript apps. However, it is slow, restricted and does not have access to the UI, just the server-side data. Nonetheless, it is an easy way to get stuff done, such as mass updating of tags or spelling or whatever.

Client side

The client has a Javascript API, as it is essentially a javascript app. You can use this to develop plugins and such. More probably you don’t have time for such. You can still access it which you can access manually for one-off tricks, e.g. batch editing.

I was briefly trying to use the client API to write a custom exporter for Zotero. I did not finish because BetterBibTex, it transpires, does what I want but better. Nonetheless, if you want to reinvent a wheel, here are overview docs, detail docs, and all the code.

One problem I had was writing to output formats which need a unique citekey reference to the items in the bibliography. Here is a simple example which deals with that citekey issue (albeit with an outdated version of the citekey system) Here is a soothing walk-through of the whole process. Once again, BBT solves that problem, so I will spend no mor thought on it.

Tablet/e-reader

Not wholly satisfactory. You can use the zotero.org site, which is fine but not ideal. A native app would be nice. there have been many and they seem mostly discontinued.

Given none of the contenders seems, at a glance, amazing, I prefer no client at all. I use the Zotero plugin zotfile to synchronise a folder full of attachments to my tablet. It’s not perfect, but it’s easy and robust.

To be evaluated, the currently active clients seem to be

Blog integration

tl;dr. There are many over-engineered solutions to get zotero citations in a blog. I use pandoc markdown with Better BibTeX. Anyway, if you want a more fragile but fancier solution, there are other options below.

Via BetterBibTeX

See BetterBibTex.

Via CSL

CSL is a citation rendering mini-language used by modern journals and software to express house style. And you can use it too.

Screencap of Zotero

This is a slightly weird way to get plain-text citations out; CSL is a system for instructing your citation software how to render citations in your word processor; but it can be forced to pump out normal text. It is robust ad simple. There is a CSL editor online so this is easy-ish. One catch is that you might, like me, wish to get your BibTeX citation keys out to refer to them. But Bibtex keys are not accessible to CSL, so this doesn’t quite work.

  • Here is my docutils/ReST style file, restructuredtext.csl, which renders citations as plain text with ReST markup, including anchor links.

    It’s ugly, because it has to battle with a grumpy rich-text XML infrastructure to render plain text, but it gets the job done without any coding, and is robust against software changes.

  • Similarly, here is my Markdown style file, markdown.csl, which, likewise, renders citations as plain text with Markdown markup, including anchor links.

  • Emiliano Heyns has a BibTeX key CSL, which renders Emiliano’s preferred citekey \{LeDT06} (sadly not quite matching mine.)

CSL has a citation-label variable, but it doesn’t correspond to the bibtex keys generated by BBT, which is unsatisfactory.

Via the Atom citation picker

For Atom+Zotero+Markdown, you could try zotero citation, which adds a citation picker to the atom editor. References look like

[\(Heyns, 2014\)](#@heyns2014)
[\(Heyns, 2014\)](?@heyns2014,heyns2015)

They are rendered in the output by an in-built pandoc filter, which is installed separately. One locates the output bibliography in the document in either pandoc-YAML or plaintext format at

[#bibliography]: #

I don’t bother; BetterBibTeX works fine for my needs and across many editors.

Dynamic bibliography generation in situ

Erik Hetzner’s zotxt can avoid the need to create bibfiles, rendering bibliographies directly by querying the zotero app, rather than rendering an intermediate file. Since rendering an intermediate file is scriptable, this does not appeal to me.