IPython

Jupyter’s forgotten uncle

2011-04-18 — 2023-05-14

computers are awful
python

Suspiciously similar content

Assumed audience:

Command-line python users

Figure 1: A Hopi man in ca 1900, making the first commit to the IPython repository.

IPython was the first mass-market interactive python upgrade. The python-specific part of jupyter, which can also run without jupyter. Long story. But think of it as a REPL, a CLI-style execution environment, that is a little friendlier than naked python and has colourisation and autocomplete and such. And also is complex in confusing ways.

Here are some notes for its care and feeding.

1 ipython config

To configure python we need a config profile

ipython profile create

IPython config is per-default located in

(ipython locate profile default)/ipython_config.py

2 Debugger

This is all built upon ipython so you invoke the debugger ipython-style, specifically:

from IPython.core.debugger import Tracer; Tracer()()      # < 5.1
from IPython.core.debugger import set_trace; set_trace()  # >= v5.1

See also generic python debugging.

3 Pretty display of objects

3.1 ipython rich display

Check out the ipython rich display protocol which allows us to render objects as arbitrary graphics. This extends the __str__() and __repr__() Methods from ordinary python.

Rich display is especially useful in the jupyter frontends, which permit graphics. Some examples:

  • nbviewer examples of how to use that are helpful.
  • lovely-numpy displays many array types (e.g. numpy, pytorch etc) gracefully.
  • The Ipython display protocol was what I used to create latex_fragment which can display arbitrary latex inline.

How to display my own things nicely? The display API docs explain that you should implement methods on my objects such as, e.g., _repr_svg_.

This is how the latex_fragment library works, for example:

def _figure_data(self, format):
    fig, ax = plt.subplots()
    ax.plot(self.data, 'o')
    ax.set_title(self._repr_latex_())
    data = print_figure(fig, format)
    # We MUST close the figure, otherwise
    # IPython’s display machinery
    # will pick it up and send it as output,
    # resulting in double display
    plt.close(fig)
    return data

# Here we define the special repr methods
# that provide the IPython display protocol
# Note that for the two figures, we cache
# the figure data once computed.

def _repr_png_(self):
    if self._png_data is None:
      self._png_data = self._figure_data('png')
    return self._png_data

4 Memory leak via output history

IPython’s history obsession points out that big memory allocations can hang around in jupyter (well, ipython) for quite a while.

So the output from line 12 can be obtained as _12, Out[12] or _oh[12]. If you accidentally overwrite the Out variable you can recover it by typing Out=_oh at the prompt.

This system obviously can potentially put heavy memory demands on your system, since it prevents Python’s garbage collector from removing any previously computed results. You can control how many results are kept in memory with the configuration option InteractiveShell.cache_size. If you set it to 0, output caching is disabled. You can also use the %reset and %xdel magics to clear large items from memory.

Workarounds.

  1. in jupyter, execute %config ZMQInteractiveShell.cache_size = 0; although this does not work in all jupyter front ends
  2. edit the config file and add ZMQInteractiveShell.cache_size=0.

5 Autocomplete breaks in ipython

The ecosystem that supports tab-completion is fragile and lackadaisical. Most recently for me, autocomplete was broken because the sensitive dependencies of jedi are managed by cowboys. The fix for that particular version was

conda install jedi==0.17.2