ssh

a.k.a. “Secure shell”, which is not actually a shell but an encrypted network protocol that people like to shell with

December 28, 2014 — February 23, 2024

computers are awful
computers are awful together
confidentiality
encryption

Assumed audience:

People who need to run processes on remote machines without sweating the details

SSH, the secure shell, is the Swiss army knife of the internet. It provides methods that allow me to shunt data from one place to another in a huge number of different ways. There are similar tools which are even less fuss, but they are gaping security holes and should not be used.

It is weird that it is named after almost the one thing that it is not: a shell. But other pedants have complained about that, I am sure.

I am writing from the perspective of someone using ssh as a client, not maintaining a server.

Figure 1

1 Suspending ssh

Use the escape character. Per default this is Enter ~. Immediately after typing it, Ctrl-Z will suspend an SSH client.

2 Extra security

SSH has shipped with some insecure authentication and encryption options historically. One should probably secure SSH by shutting down unnecessarily weak crypto options, and disabling unneeded ports and so on. If I want to be extra sensible I could secure it to modern cryptography standards, such as elliptic ciphers, and smart defaults which are suspected to be less vulnerable to quotidian NSA attacks. (With these settings we’re still screwed when cheap quantum factorization becomes a thing, though, but let us set that aside for now.)

There are a lot of other commands needed to make stuff go. Recommended /etc/ssh/sshd_config, i.e. for the server:

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
AllowGroups ssh-user
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com

Recommended /etc/ssh/ssh_config for the clients:

# Github needs diffie-hellman-group-exchange-sha1 some of the time but not always.
#Host github.com
#    KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
Host *
    KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
    PasswordAuthentication no
    ChallengeResponseAuthentication no
    PubkeyAuthentication yes
    HostKeyAlgorithms ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,ssh-rsa
    Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
    MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com
    UseRoaming no

3 SSH identities

It is fiddly to have multiple identities for the same host - in particular, using git this needs to be managed at the configuration level by host aliases. A guide to that is here. See also the lengthy stackoverflow discussion. For bonus paranoia, mac users can store SSH identities in their secure enclave doodad.

4 Key encryption with ssh-agent

If you don’t want someone to steal your private key you can encrypt it. Doing this even if you have filesystem encryption would be wise if you were in a hostile environment such as a cloud server, or use features such as ssh agent forwarding (long story), or are just generally cautious.

Constantly decrypting the key with a password is annoying; for this we use ssh-agent, which is slightly subtle and complicated and I have a feeling I am endangering myself by not understanding how it magically decrypts stuff for me.

tl;dr On bash use

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

4.1 at startup

Want to start ssh-agent on startup, but not if it’s already running? See Joseph M. Reagle’s solution:

SSH_ENV="$HOME/.ssh/environment"

function start_agent {
    echo "Initialising new SSH agent…"
    /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
    echo succeeded
    chmod 600 "${SSH_ENV}"
    . "${SSH_ENV}" > /dev/null
    /usr/bin/ssh-add;
}

# Source SSH settings, if applicable
if [ -f "${SSH_ENV}" ]; then
    . "${SSH_ENV}" > /dev/null
    #ps ${SSH_AGENT_PID} doesn’t work under cywgin
    ps -ef | grep ${SSH_AGENT_PID} | \
        grep ssh-agent$ > /dev/null || {
            start_agent;
    }
else
    start_agent;
fi

I am unclear as to how much of this can be avoided for a modern setup.

4.2 ssh-agent with fish

Easy enough; see the fish shell ssh-agent, which installs the fish_ssh_agent command.

How to use the ssh-agent with fish explains how things are supposed to work, but I found that the fish_ssh_agent solution seems to be the one that actually worked for me.

4.3 ssh-agent on macOS

How does ssh-agent work with the macOS keychain? Should it be permitted to, or is that inviting hostile actors in?

Things are weird for macOS because you can store things in ssh-agent, or osx keychain, or some weird hybrid options that make my eyes cross. Apple’s Explanation is sorta clear but it got confused and changed over time. Github has an opinion on it. Old macOS SSH behaviour for the vexed. The best and most current summary seems to be Awesome macOS CLI:

Prior to macOS Sierra, ssh would present a dialog asking for your passphrase and would offer the option to store it into the keychain. This UI was deprecated some time ago and has been removed.

Instead, a new UseKeychain option was introduced in macOS Sierra allowing users to specify whether they would like for the passphrase to be stored in the keychain. This option was enabled by default on macOS Sierra, which caused all passphrases to be stored in the keychain.

This was not the intended default behavior, so this has been changed in macOS 10.12.2. …

ssh-add -K /path/to/private_key

Then add to ~/.ssh/config:

Host server.example.com
    IdentityFile /path/to/private_key
    UseKeychain yes

UPDATE:

ssh-add --apple-use-keychain /path/to/private_key

5 SSH as VPN

sshuttle (manual) is a VPN-workalike built on SSH. As far as I can tell it’s easy for both the client and server to set up VPN this way, so I’m not sure why it is not more common. Possibly because setting up SSH shells on various servers is in itself easy to make insecure for the server? Anyway, you have a login, you might as well use it.

Installation options:

pip install shuttle
brew install sshuttle
# etc

Run:

sshuttle --dns -r username@sshserver 0/0

6 Over https

The classic is corkscrew, which sneaks SSH over, e.g. hostile web-only firewalls at the airport.

7 Tunnels

sshtunnelmanager (macos) assembles the right commands arguments to make tunnels without having to check the manual every time.

ProxyJump is a 2 step proxy for easing double tunnelling.

ssh -J your.jump.host remote.internal.host

or

scp -o 'ProxyJump your.jump.host' \
  myfile.txt remote.internal.host:/my/dir

You can dynamically add tunnels!:

<enter>~Cto bring up a console with your local SSH client (not the server). The provided console accepts a few of the ssh commands options, including -R, -L.

So, for example, if I wanted to suddenly access some service running on port 4321 on my local machine from the server, I could type <enter>~C-R 1234:localhost:4321<enter> and I would immediately have access to that resource from the server on localhost:4321 (that’s the server’s localhost).

7.1 via Microsoft’s dev tunnels

Microsoft’s Dev tunnels does not appear to be an SSH tool, but its tunnels look similar, except with certain advantages such as * Persistent SSH sessions over unreliable connections via their own SSH variant, microsoft/dev-tunnels-ssh: SSH library for dev-tunnels.

I do not know the license of this software because the license PDF is encrypted and cannot be read by me.

8 autossh

  • autossh is a program to start a copy of ssh and monitor it, restarting it as necessary should it die or stop passing traffic. The idea is from rstunnel (Reliable SSH Tunnel), but implemented in C.
  • The author’s view is that it is not as fiddly as rstunnel to get to work.
  • Connection monitoring using a loop of port forwardings or a remote echo service.
  • Backs off on rate of connection attempts when experiencing rapid failures such as connection refused.
  • Compiled and tested on OpenBSD, Linux, Solaris, Mac OS X, Cygwin, and AIX; should work on other BSDs.
  • Freeware.

Main webpage is down? See archived version, or a worked example.

9 Terminal multiplexing

screen, tmux etc. Handy for remote admin.

See terminal multiplexers.

10 Network connection multiplexing

You can share a single connection between multiple clients. Here is a worked example:

# set up master connection
ssh -f -N -M -S ~/.ssh/S.user@host.com -D 9000 user@host.com
# recycle it for a new session
ssh -S ~/.ssh/S.user@host.com dummyhost

11 Alternatives/extensions

11.1 dropbear

dropbear ssh is a minimal ssh implementation.

11.2 EternalTerminal

Eternal Terminal integrates some kind of session management into the terminal.

11.3 mosh

mosh (“mobile shell”) is also, like ssh, not a shell. It is a terminal with ssh-style tunneling for intermittent connections, including optimistic screen updates and graceful packet loss.

How is mosh better than tmux + autossh, though? I need a Venn diagram of features here. Not terribly active development since 2017.

11.4 Eternal Terminal

An SSH elaboration that support resumeable terminal sessions via clever network footwork. It also integrates with tmux (in fact, what does it offer that tmux does not?) It needs a special server process so I will not get around to having this anywhere I need it.

11.5 teleport

“Enterprise” ssh? teleport claims to offer that.

12 Incoming