Jupyter UI wrangling

What happens inside those notebook cells doesnt stay inside tgose notebook cells

February 9, 2017 — February 13, 2023

faster pussycat
premature optimization
python

Jupyter is complicated so I segregated information about UI, rendering etc from the main notebook. Here is the stuff about how to use the jupyter system to support non-standard user interactions.

Figure 1: Jupyter is dynamic thanks to the interaction between javascript (left) and python (right)

1 Presentations using Jupyter

1.1 Basic: export to reveal.js

Jupyter can export a notebook as a browser-based presentation! The easiest is Classic reveal.js mode. tl;dr:

$ jupyter nbconvert --to slides some_notebook.ipynb  --post serve

You might want to make various improvements, such as tweaking the reveal.js settings in jupyter slideshows.

If you aren’t running a coding class, you will want to hide the input cells from your IPython slides by customising the output templates, or you can suppress all code by using output format hide_code_slides.

These kind of custom tweaks are not too crazy but a copy of the reveal.js source code is needed for some of them. As an example, a more comprehensive version using a custom theme and the reveal.js source looks like

jupyter nbconvert Presentation.ipynb  \
    --to slides --reveal-prefix ../../reveal.js \
    --post serve --SlidesExporter.reveal_theme=league

1.2 Fancier: integrated slideshows using RISE

Fancier again: interactive slideshows using RISE.

To meet your house style requirements it is usually sufficient to customise some decorations and alter some css.

Major plus: you can execute code while running the slide!

Major minus: there is no facility that I can see to style your cover slides differently, which is incompatible with, e.g., my university’s style guide.

If I don’t wish to display inline input code, O can avoid it with hide_code.

Install using pip:

pip install hide_code
jupyter nbextension install --py hide_code
jupyter nbextension enable --py hide_code
jupyter serverextension enable --py hide_code

Install using conda:

conda install -c conda-forge hide_code

2 Rich display

Various objects support rich display of python objects e.g. IPython.display.Image

from IPython.display import Image
Image(filename='img/test.png')

or you can use markdown for local image display

![title](img/test.png)

If you want to make your own objects display, uh, richly, you can implement the appropriate magical methods:

class Shout(object):
    def __init__(self, text):
        self.text = text

    def _repr_html_(self):
        return "<h1>" + self.text + "</h1>"

I leveraged this to make a latex renderer called latex_fragment which you should totally check out for rendering inline algorithms, or for emitting SVG equations.

Alternatively one can use the inbuilt jupyter LaTeX renderer:

#%%
from IPython.display import Latex
Latex('''The mass-energy equivalence is described by the famous equation

$$E=mc^2$$

discovered in 1905 by Albert Einstein.
In natural units ($c$ = 1), the formula expresses the identity

\\begin{equation}
E=m
\\end{equation}''')

3 Graphs

Set up inline plots:

%matplotlib inline

inline svg:

%config InlineBackend.figure_format = 'svg'

Graph sizes are controlled by matplotlib. Here’s how to make big graphs:

import matplotlib as mpl
mpl.rcParams['figure.figsize'] = (10.0, 8.0)

Interesting-looking other graphing options:

Jupyter lab includes such nifty features as a diagram editor which you can install using jupyter labextension install jupyterlab-drawio

4 Citations and other academic writing in Jupyter

I did this for

  1. my blog — using simple Zotero markdown citation export, which is not great for inline citations but fine for bibliographies, and easy and robust.

  2. my papers — abandoning jupyter in favour of Pweave+pandoc, which works amazingly for everything if you use pandoc tricks for your citations.

I couldn’t find a unified approach for these two different use cases which didn’t sound like more work than it was worth. At least, many academics seem to have way more tedious and error-prone workflows than this, so I’m just being a fancypants if I try to tweak it further.

More recently, there has appeared jupyterbook which enables notebook-based blog rendering, including citations. This is built using the ruby site generator jekyll-scholar so is heavy in dependencies but it seems to work. A hyped example of this in action is You Only Write Thrice.

Chris Sewell has produced a script called ipypublish that eases some pain points in producing articles. It’s an impressive piece of work.

My own latex_fragment allows you to insert one-off latex fragments into jupyter and pweave (e.g. algorithmic environments or some weird tikz thing.)

Jean-François Bercher’s jupyter_latex_envs reimplements various latex markup as native jupyter including \cite. I

Sylvain Deville recommends treating jupyter as a glorified markdown editor and then using pandoc, which is an OK workflow if you are producing a once-off paper, but not for a repeatedly updated blog.

nbconvert has built-in citation support but only for LaTeX output. Citations look like this in markup:

<cite data-cite="granger2013">(Granger, 2013)</cite>

or even

<strong data-cite="granger2013">(Granger, 2013)</strong>

The template defines the bibliography source and looks like:

((*- extends 'article.tplx' -*))

((* block bibliography *))j
((( super () )))
\bibliographystyle{unsrt}
\bibliography{refs}
((* endblock bibliography *))

And building looks like:

jupyter nbconvert --to latex --template=print.tplx mynotebook.ipynb

As above, it helps to know how the document templates work.

Note that even in the best case you don’t have access to BibTeX-style citation, so auto-year citation styles will look funky.

Speaking of custom templates, the nbconvert setup is customisable for more than latex.

{% extends 'full.tpl'%}
{% block any_cell %}
    <div style="border:thin solid red">
        {{ super() }}
    </div>
{% endblock any_cell %}

But how about for online? cite2c seems to do this by live inserting citations from zotero, including author-year stuff. (Requires Jupyter notebook 4.2 or better which might require a pip install --upgrade notebook)

Julius Schulz gives a comprehensive config for this and everything else.

This workflow is smooth for directed citing, but note that there is no way to include a bibliography except by citation, so you have to namecheck every article; and the citation keys it uses are zotero citation keys which are nothing like your BibTeX keys so can’t really be manually edited.

if you are customising the output of jupyter’s nbconvert, you should be aware that the {% block output_prompt %} override doesn’t actually do anything in the templates I use. (Slides, HTML, LaTeX). Instead you need to use a config option:

$ jupyter nbconvert --to slides some_notebook.ipynb \
   --TemplateExporter.exclude_output_prompt=True \
    --post serve

I had to use the source to discover this.

ipyBibtex.ipynb? Looks like this:

%%cite
Lorem ipsum dolor sit amet
__\citep{hansen1982,crealkoopmanlucas2013}__,
consectetuer adipiscing elit,
sed diam nonummy nibh euismod tincidunt
ut laoreet dolore magna aliquam erat volutpat.

So it supports author-year citations! But it’s a small, unmaintained package so is risky.

🏗 Work out how Mark Masden got citations working?

5 Interacting

Jupyter allows interactions! For certain special cases with backing by a major firm or dev team such as dashboards, this can an easy way of interacting with python.

In the general case, not so much. For a while I thought building widgets in jupyter might be a useful or convenient thing to do, but many days of effort later I must conclude that not only were my early experiments not convenient, but also that the API changes are so frequent that it would need to be effortless to be viable to maintain a UI in an ongoing way. These days, I regard using jupyter for interactions a way of adding extra failure points to an app which would be better without jupyter.

5.1 Gradio

Specifically for python ML apps, Gradio seems to work in jupyter, or at least in colab. TBD

5.2 Dashboards

See voilà. Also there an example of basic interaction here.

5.3 Updating progress graphs

from IPython.display import clear_output
## Run code

for i in range(10):
    clear_output(True)
    plt.plot(x, y)
    plt.show()

5.4 Widgets

Official Manual: ipywidgets.

pip install ipywidgets
jupyter nbextension enable --py widgetsnbextension

See also the announcement: Jupyter supports interactive JS widgets, where they discuss the data binding module in terms of javascript UI thingies.

Pro tip: If you want a list of widgets

from ipywidgets import widget
widget.Widget.widget_types

Then you use them.

from ipywidgets import interact, IntSlider

5.5 PixieApps

An alternative (?) or perhaps complementary system, PixieApps:

PixieApps are Python classes used to write UI elements for your analytics, and they run directly in a Jupyter notebook.

PixieApps are designed to be easy to build. Mostly, you’ll only need to write HTML and CSS with some custom attributes, along with some Python for the business logic. Except in rare cases, you won’t have to write JavaScript. The PixieDust JS runtime will automatically listen to key events and manage transactions to the Jupyter kernel appropriately.

pip install pixiedust
jupyter pixiedust install

The whole kit is rather heavy, requires Java and installs Scala, which feels OTT.

5.6 External event loops

External event loops are now easy and documented. What they don’t say outright is that if you want to use the tornado event loop, relax because both the jupyter server and the ipython kernel already use the pyzmq event loop which subclasses the tornado one.

If you want you make this work smoothly without messing around with passing ioloops everywhere, you should make zmq install itself as the default loop:

from zmq.eventloop import ioloop
ioloop.install()

Now, your asynchronous python should just work using tornado coroutines.

NB with the release of latest asyncio and tornado and various major version incompatibilities, I’m curious how smoothly this all still works.

5.7 Javascript from python with jupyter

As seen in art python.

Here’s how you invoke javascript from jupyter. Here is the jupyter JS source And here is the full jupyter browser JS manual, and the Jupyter JS extension guide.