Jupyter UI wrangling

1 Generative AI in

Runcell is an example of a Jupyter Notebook extension that integrates generative AI into the notebook interface. In other words, it provides functionality we get for free when we use Python in a modern IDE, but that’s hard to achieve in the hostile Jupyter notebook interface.

2 Presentations using Jupyter

$ jupyter nbconvert --to slides some_notebook.ipynb  --post serve

jupyter nbconvert Presentation.ipynb  \
    --to slides --reveal-prefix ../../reveal.js \
    --post serve --SlidesExporter.reveal_theme=league

pip install hide_code
jupyter nbextension install --py hide_code
jupyter nbextension enable --py hide_code
jupyter serverextension enable --py hide_code

conda install -c conda-forge hide_code

3 Rich display

We can show Python objects with rich display, for example with IPython.display.Image.

from IPython.display import Image
Image(filename='img/test.png')

Or you can use Markdown to display a local image.

![title](img/test.png)

If we want our objects to display richly, we can implement the appropriate magic methods:

class Shout(object):
    def __init__(self, text):
        self.text = text

    def _repr_html_(self):
        return "<h1>" + self.text + "</h1>"

I used this to build a LaTeX renderer called latex_fragment that we should totally check out for rendering inline algorithms or emitting SVG equations.

Alternatively, we can use the inbuilt Jupyter LaTeX renderer:

#%%
from IPython.display import Latex
Latex('''The mass-energy equivalence is described by the famous equation

$$E=mc^2$$

discovered in 1905 by Albert Einstein.
In natural units ($c$ = 1), the formula expresses the identity

\\begin{equation}
E=m
\\end{equation}''')

4 Graphs

Set up inline plots:

%matplotlib inline

Inline SVG:

%config InlineBackend.figure_format = 'svg'

Graph sizes are controlled by matplotlib. Here’s how we make big graphs:

import matplotlib as mpl
mpl.rcParams['figure.figsize'] = (10.0, 8.0)

Interesting-looking other graphing options:

• asymptote vector plotting plugin
• Graphviz renders in Jupyter.

Jupyter lab includes such nifty features as a diagram editor which you can install using jupyter labextension install jupyterlab-drawio

5 Citations and other academic writing in Jupyter

I did this for

1. my blog — using simple Zotero markdown citation export, which is not great for inline citations but fine for bibliographies, and easy and robust.

2. my papers — abandoning Jupyter in favour of Pweave+pandoc, which works amazingly for everything if you use pandoc tricks for your citations.

I couldn’t find a unified approach for these two different use cases which didn’t sound like more work than it was worth. At least, many academics seem to have way more tedious and error-prone workflows than this, so I’m just being a fancypants if I try to tweak it further.

More recently, there has appeared jupyterbook which enables notebook-based blog rendering, including citations. This is built using the ruby site generator jekyll-scholar so is heavy in dependencies but it seems to work. A hyped example of this in action is You Only Write Thrice.

Chris Sewell has produced a script called ipypublish that eases some pain points in producing articles. It’s an impressive piece of work.

My own latex_fragment allows you to insert one-off LaTeX fragments into Jupyter and Pweave (e.g. algorithmic environments or some weird TikZ thing.)

Jean-François Bercher’s jupyter_latex_envs reimplements various LaTeX markup as native Jupyter including \cite.

Sylvain Deville recommends treating Jupyter as a glorified markdown editor and then using pandoc, which is an OK workflow if you are producing a once-off paper, but not for a repeatedly updated blog.

nbconvert has built-in citation support but only for LaTeX output. Citations look like this in markup:

<cite data-cite="granger2013">(Granger, 2013)</cite>

or even

<strong data-cite="granger2013">(Granger, 2013)</strong>

The template defines the bibliography source and looks like this:

((*- extends 'article.tplx' -*))

((* block bibliography *))j
((( super () )))
\bibliographystyle{unsrt}
\bibliography{refs}
((* endblock bibliography *))

And the build looks like this:

jupyter nbconvert --to latex --template=print.tplx mynotebook.ipynb

As above, it’s helpful to know how the document templates work.

Note that even in the best case we don’t have access to BibTeX-style citations, so auto-year citation styles will look funky.

Speaking of custom templates, the nbconvert setup is customizable for more than just LaTeX.

{% extends 'full.tpl'%}
{% block any_cell %}
    <div style="border:thin solid red">
        {{ super() }}
    </div>
{% endblock any_cell %}

But how about online? cite2c seems to do this by inserting citations live from Zotero, including author–year details. (Requires Jupyter Notebook 4.2 or later, which might require a pip install --upgrade notebook)

Julius Schulz provides a comprehensive config for this and everything else.

This workflow is smooth for directed citing, but there’s no way to include a bibliography except via citations, so we have to name-check every article. The citation keys it uses are Zotero citation keys, which are nothing like our BibTeX keys, so they can’t really be edited manually.

If you are customizing the output of Jupyter’s nbconvert, be aware that the {% block output_prompt %} override doesn’t actually do anything in the templates I use (slides, HTML, LaTeX). Instead, we need to use a config option:

$ jupyter nbconvert --to slides some_notebook.ipynb \
   --TemplateExporter.exclude_output_prompt=True \
    --post serve

I had to use the source to discover this.

Here’s what ipyBibtex.ipynb looks like:

%%cite
Lorem ipsum dolor sit amet
__\citep{hansen1982,crealkoopmanlucas2013}__,
consectetuer adipiscing elit,
sed diam nonummy nibh euismod tincidunt
ut laoreet dolore magna aliquam erat volutpat.

So it supports author–year citations, but it’s a small, unmaintained package and therefore risky.

🏗 Work out how Mark Masden got citations working?

6 Interacting

Jupyter supports interactions! For some special cases with backing by a major firm or dev team—like dashboards—this can be an easy way to interact with Python.

In the general case, not so much. For a while I thought building widgets in Jupyter might be useful or convenient, but after many days of effort I concluded my early experiments weren’t convenient. The API changes so frequently that maintaining a UI long‑term would have to be effortless to be viable. These days I regard using Jupyter for interactions as adding extra failure points to an app that would be better off without it.

from IPython.display import clear_output
## Run code

for i in range(10):
    clear_output(True)
    plt.plot(x, y)
    plt.show()

pip install ipywidgets
jupyter nbextension enable --py widgetsnbextension

from ipywidgets import widget
widget.Widget.widget_types

from ipywidgets import interact, IntSlider

pip install pixiedust
jupyter pixiedust install

from zmq.eventloop import ioloop
ioloop.install()