Unix commands I need often

but which are tedious to work anew out each time

2015-11-06 – 2021-06-05
Maybe related 🤷:

Invocations and commands

General, not macOS-specific or Linux-specific. I try to cover mostly standard commands here but sometimes the features of the invoking shell are also important. You could optimise that also, e.g. by switching from bash to fish or PowerShell.

Nice prompts

Try Starship: Cross-Shell Prompt.

WTF

Return produces ^M instead of a newline? Try:

stty sane^J

or possibly

stty icrn

For other miscellaneous screen garbage, maybe reset will help?

reset

It is more complicated if you made a mess in tmux.

As far as I am concerned this is all dark magic. If this nonsense wastes 2 hours of my life in total it will not justify the rather lengthier process of getting a better mental model of how terminals work.

File system watching

Watch That Filesystem, by Al Williams, discusses how to trigger things based on filesystem events via inotifywait and incron.

Thanks Yohans Bastian for pointing out this handy trick to find the PID of the process using the inotify watchers.

find /proc/*/fd \
    -lname anon_inode:inotify \
    -printf '%hinfo/%f\n' 2>/dev/null \
    | xargs grep -c '^inotify'  \
    | sort -n -t: -k2 -r

rebuild on change

entr watches for changed files and recompiles your whatsit when it gets edited. This is incredibly useful for automating life.

Which file is crashing/hanging $PID?

lsof -r -p $PID | grep /path/to/file

Vars, expansions, file names, white space hell

See bash or fish depending on your choice of shell.

Or avoid both by using some other utility. rename is a script that makes renaming work how you imagine it should, avoiding the mysterious punctuation stew at least somewhat.

rename -s html txt *.html

NB depending on your distro you may get some other systutil rename which is much less powerful than the one I linked to, albeit still often powerful enough.

Text processing

Trailing whitespace

A shell script to remove trailing whitespace from a file - put this in trimspace.sh:

#!/bin/bash

# macOS version
sed -i '' -e’s/[[:space:]]*$//' "$1"
#!/bin/bash

# GNU version
sed -i -e’s/[ \t]*$//' "$1"

Then you can trim trailing whitespace from your… whatever… by putting this line in there:

find . -type f -print0 | xargs -0 -I {} trimspace.sh \{\}

Data wrangling

See text data processing.

Misc

rsync files in random order

based of Ole Tange’s tor version

Remotely

find . -maxdepth 4 -depth -type d | shuf | rsync --delete -zHax --inplace --rsync-path='mkdir -p {} && rsync' ~/{}/ remote.host:/some/dir/{}/

Presumably this works locally:

for file in (find . -maxdepth 4 -depth -type d | shuf)
  mkdir -p "$file" && rsync --delete -Hax --inplace - ./{}/ /some/dir/{}/
end

Download from command line without getting owned

Default curl is insecure.

curl --tlsv1.2 --proto=https --location \
  --remote-name-all --remote-header-name

Find common diacritics in filenames

Caught in unicode crossfire? Try

find . -iname "*[üñàáèéöäçã]*"

Which process is bound to port $PORT?

This one!

lsof -nP -i4TCP:$PORT | grep LISTEN

Sync only if drive present

test -d  /Volumes/syncdrive/ && rsync --delete -avz \
    192.168.0.1:/path/to/stuff/ /Volumes/syncdrive/

Unzip zips into subdir using fish

for foo in *.zip
    unzip $foo -d (basename $foo .zip)
end

Set operations

comm.

Meta command lines

Command lines to command your command lines.

I suppose if I were a good person I’d check if any of the above commands are novel and submit them to tldr or version my .cheats folder in a dotfile repo, or mention them in commandlinefu but I’m now exhausted and just want to go home.

Explainshell

Explainshell dissects a shell command and shows you the documentation for each part of it. Good for instilling this knowledge in your brain.

tldr

tldr:

New to the command-line world? Or just a little rusty? Or perhaps you can’t always remember the arguments to lsof, or tar?

…And the manual for your typical command is utterly incomprehensible, so remembering is obviously preferable to trying to decipher it.

First

## install
npm install -g tldr
# or
brew install tldr
# invoke
tldr tar

cheat

cheat

…allows you to create and view interactive cheatsheets on the command-line. It was designed to help remind *nix system administrators of options for commands that they use frequently, but not frequently enough to remember.

## install
pip install cheat
# or
brew install cheat
# invoke
cheat tar

how do i

howdoi:

Are you a hack programmer? Do you find yourself constantly Googling for how to do basic programming tasks?

Suppose you want to know how to format a date in bash. Why open your browser and read through blogs (risking major distraction) when you can simply stay in the console and ask howdoi

pip install howdoi
howdoi tar

Seems to search the internet for you, and not be command-line specific, which is broader in scope than some of the other entrants here, but also noisier.

commandlinefu

commandlinefu has an awful command-line interface, but the website is so good that it makes the grade.

bash hackers wiki

bash hackers wiki: avoid its the paradigmatically awful bash manual

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.