Python

ipython config

Ipython config is per-default located in

(ipython locate)/profile_default/startup/

Autocomplete breaks

The ecosystem that support tab-completion is fragile and lackadaisical. Usually if autocomplete is broken it is because the sensitive dependencies of jedi are managed by cowboys.

Recent example iPython 7.19.0 autocomplete breaks with jedi <=0.17.0 and parso >=0.8.0.

The fix for that particular version was

conda install jedi==0.17.2

Pretty display of objects

Check out the ipython rich display protocol which allows you to render objects as arbitrary graphics. This is especially famous in the jupyter frontends. Some examples of how to use that are helpful. How to use this? The display API docs explain that you should implement methods on your objects such as, e.g., _repr_svg_.

This was what I used to create latex_fragment which can display arbitrary latex inline. This is how you use it:

def _figure_data(self, format):
    fig, ax = plt.subplots()
    ax.plot(self.data, 'o')
    ax.set_title(self._repr_latex_())
    data = print_figure(fig, format)
    # We MUST close the figure, otherwise
    # IPython’s display machinery
    # will pick it up and send it as output,
    # resulting in double display
    plt.close(fig)
    return data

# Here we define the special repr methods
# that provide the IPython display protocol
# Note that for the two figures, we cache
# the figure data once computed.

def _repr_png_(self):
    if self._png_data is None:
      self._png_data = self._figure_data('png')
    return self._png_data

For a non-graphical non-fancy terminal, the most common useful rich display probably simply want nice formatting of big data structures.

from pprint import pprint, pformat
# display it
pprint(obj)
# get a formatted representation
pretty = pformat(obj)

If I want to write my own pretty-printer, with correct indentation, I might use tiles. This has its own extension protocol.

Non obvious hacks

What a nightmare is that manual for the string formatting. While all the information I need is in there, it is arranged in perverse inversion of a comprehensible order as measured by the frequency and the priority with which I use it. See instead Marcus Kazmierczak’s cookbook.

Highlights:

## float precision
>>> print("{:.2f}".format(3.1415926))
3.14
## left padding
>>> print("{:0>2d}".format(5))
05

## power tip which the manual does not make clear:
## variable formatting of variable formatting
>>> pi = 3.1415926
>>> precision = 4
>>> print( "{:.{}f}".format( pi, precision ) )
3.1415

f-strings

f-strings make things somewhat easier for python 3.6+ because they don’t need to mess around with naming things for the .format(foo=foo) call:

>>> name = "Fred"
>>> f"He said his name is {name!r}."
"He said his name is 'Fred'."

Timestamps

Why is a now timestamp in UTC not the first line in every academic research workbook/paper/data analysis? Because it’s tedious to look up the different bits.

Here you go:

from datetime import datetime
datetime.utcnow().isoformat(timespec='seconds')

Rendering HTML output

You have a quick and dirty chunk o’ HTML you need to output? You aren’t writing some damnable webapp with a nested hierarchy of template rendering and CSS integration into some design framework? You just want to pump out some markup?

I recommend yattag which is fast, simple, good and has a 1-page manual. It works real good.

Rendering markdown as HTML

from markdown import markdown
html_string = markdown("""
# Title

Paragraph. *emphasis*.
""")

Neat python 3 stuff

Alex Rogozhnikov, Python 3 with pleasure highlights some tricks which landed in the python 3 lineage. There are a lot of those.

Useful for me is a friendlier python struct-like thing, the data class Geir Arne Hjelle explains.

Also nice, we can override module accessors.

Asynchrony is less awful (although still kind of awful).

Python 2 and 3 compatibility

TL;DR: I am no employee of giant enterprise-type business with a gigantic legacy code base, and so I don’t use python 2. My code is not python 2 compatible. Python 3 is more productive, and no-one is paying me to be less productive right now. Python 2 code is usually easy to port to python 3. It is possible to write code which is compatible with python 2 and 3, but then I would miss out on some of the work that has gone into making python 3 easier and better, and waste time porting elegant easy python 3 things to hard boring python 2 things.

🏗 six versus future.