Generic dependency managers



Reproducible deployment error

How do you install the right versions of everything for some python code you are developing? How to deploy that sustainably? How to share it with others.

Containerization

An obvious frontrunner on popularity alone. See containerization.

Nix

A package manager which also manages dependencies.

Vagrant

Dependency import error

vagrant: Virtual machines as dependency managers.

If you are a developer, Vagrant will isolate dependencies and their configuration within a single disposable, consistent environment, without sacrificing any of the tools you are used to working with (editors, browsers, debuggers, etc.). Once you or someone else creates a single Vagrantfile, you just need to vagrant up and everything is installed and configured for you to work. Other members of your team create their development environments from the same configuration, so whether you are working on Linux, Mac OS X, or Windows, all your team members are running code in the same environment, against the same dependencies, all configured the same way

I didn’t find that clear, but this example was illustrative:

$ vagrant init hashicorp/bionic64
$ vagrant up
  Bringing machine 'default' up with 'virtualbox' provider...
  ==> default: Importing base box 'hashicorp/bionic64'...
  ==> default: Forwarding ports...
  default: 22 (guest)
  => 2222 (host) (adapter 1)
  ==> default: Waiting for machine to boot...

$ vagrant ssh
  vagrant@bionic64:~$ _

Spack

For HPC specifically, there is spack, which also lets you prototype on macOS.

Spack is a package manager for supercomputers, Linux, and macOS. It makes installing scientific software easy. With Spack, you can build a package with multiple versions, configurations, platforms, and compilers, and all of these builds can coexist on the same machine.

Spack isn’t tied to a particular language; you can build a software stack in Python or R, link to libraries written in C, C++, or Fortran, and easily swap compilers. Use Spack to install in your home directory, to manage shared installations and modules on a cluster, or to build combinatorial versions of software for testing.

I do not know what particular hpc affordances it offers. p ## asdf

Have not yet used. asdf-vm/asdf: Extendable version manager with support for Ruby, Node.js, Elixir, Erlang & more.

direnv

direnv is an extension for your shell. It augments existing shells with a new feature that can load and unload environment variables depending on the current directory.

Use cases

  • Load 12factor apps environment variables
  • Create per-project isolated development environments
  • Load secrets for deployment

This does indeed create a toolkit for developing isolated environments with specific packages

Build tools

Modern build tools will also manage dependencies. e.g. google’s Bazel spend so much energy doing this when you build tensorflow that I become suspicious that it has mirred the entire internet to mu local hard drive


No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.