Care and feeding of macOS filesystems

1 Trash bin from the command line

AFAICT there are two interchangeable options

• sindresorhus/macos-trash: Move files and folders to the trash
• trash

brew install macos-trash

2 How do I rsync between macOS and other filesystems?

A chaos and a mess. There are two root causes AFAICT.

• macOS presents weird Unicode normalisation — NTFS stores filenames as NFC (“composed”), APFS expects NFD (“decomposed”), so accented characters like ä get mangled in transit. Do not ask me what the NF stands for; I am trying not to find out.
• macOS desktop filesystems are case-insensitive by default, while most other modern filesystems are case-sensitive, which can cause confusing filename collisions.

For the case-sensitivity issue, rsyncing from the non-Apple machine seems to help — it avoids accidentally duplicating files where only the case changed.

Now, as to the Unicode normalisation: macOS ships with openrsync, an incomplete BSD-licensed reimplementation that Apple switched to in Sequoia due to GPL licensing issues. openrsync is 💩. It handles special characters and NTFS quirks poorly.

The test to see if I’m actually using this poor tool is — if rsync --version returns something like openrsync: protocol version 29 / rsync version 2.6.9 compatible rather than a real rsync 3.x version string.

I do not know how to fix Unicode normalisation with the native, shit openrsync.

Install the real thing:

brew install rsync

Then handle Unicode normalisation mismatch with --iconv (good worked example here). Use the full Homebrew path /opt/homebrew/bin because it does not precede /usr/bin in the PATH — at least, not in my path.

The --iconv options are ordered LOCAL,REMOTE. For two local paths that is a bit odd — LOCAL means the encoding rsync uses on this machine for source paths, REMOTE means the encoding it uses for destination paths. So UTF-8,UTF-8-MAC converts NFC→NFD (NTFS→APFS), and UTF-8-MAC,UTF-8 would do the reverse. For a remote pull (rsyncing from a Linux box), LOCAL is still the local machine and REMOTE is the SSH target.

Also: trailing slash on the source directory (source/ not source/*) is the rsync idiom for “copy contents of this directory”. The star/glob form skips hidden files, which is typically not what I want.

Pulling this all together, an NTFS→APFS copy between local drives goes like so:

/opt/homebrew/bin/rsync -auv --iconv=UTF-8,UTF-8-MAC \
    /Volumes/source/ /Volumes/dest/ \
    --progress

3 Sensible Finder

I like to know which actual folder I am viewing

defaults write com.apple.finder AppleShowAllFiles false
defaults write com.apple.finder _FXShowPosixPathInTitle -bool true

4 Arsed FileVault

The best way to make sure nobody reads your confidential data is to make sure you never store it in the first place. Fortunately, there is a solution — FileVault 2! The encrypted external drives can be delicate. Expect the entire disk full of backups to become unusable next time the cable jiggles, and all your data will be, uh… securely deleted for your convenience.

If you like your data, keep at least two copies, on two different disks, to compensate for this convenience. See backups for the broader strategy — and for why the cheap portable HDDs that keep corrupting themselves do so in the first place.

But you need to take one more step; in periodically reformatting whichever drive most recently corrupted itself and cloning it, you might find that you can’t even erase it. Instead, you will get an error unable to delete core storage logical volume.

How to delete Apple encrypted drives gone AWOL:

diskutil cs list
diskutil cs deleteVolume <UUID>
diskutil cs delete UUID-OF-COREVOLUME-GROUP

That doesn’t work? Boot into Linux and nuke it that way.

Or try this cowboy option:

cat /dev/random > /dev/diskBLAH
^C

5 Rebuild Spotlight index

Is Spotlight working?

sudo mdutil -sv /

No? Turn it off and on again

sudo mdutil -E /

6 Foreign filesystems

Who do I need to mount to get access to data around here?

7 Burning bootable ISO images to USB

Via Evan Borgström.

Convert ISO to DMG:

hdiutil convert -format UDRW -o something.img \
   something.iso

Find which disk is which:

diskutil list

Say it was /dev/disk3… Unmount and clobber the right one (careful now) with the new dmg:

diskutil unmountDisk /dev/disk3
dd if=./debian-6.0.7-amd64-netinst.img.dmg of=/dev/rdisk3 bs=1m
diskutil eject /dev/disk3

Doing this without intermediate dmg output is left as an exercise for the premature optimiser.

8 Filesystem updates never triggered

If a file watcher is never triggered :

OS X FSEvents bug may prevent monitoring of certain folders OS X has a rarely-occurring bug that causes some folders to get ‘broken’ with regards to file system change monitoring via FSEvents. […]

In case you wonder, the bug is related to case (in)sensitivity of the file system. For certain folders, either realpath or FSCopyAliasInfo APIs report their names in incorrect case. This somehow causes the FSEvents system (used by LiveReload to monitor file system changes) to never report any changes for those folders and their subfolders.

(If you are really curious, more details can be found in Radar #10207999, in rb-fsevents issue #10 and in find-fsevents-bug repository.)

Note: this is not a “rarely occurring bug” in the sense that it sometimes occurs when you capitalize a letter in a filename; AFAICT it is triggered every time. Rather, it is a rarely noticed bug, since I guess either changing capitalisation or running file watchers, or the intersection thereof, is rarely noticed. Regardless, the above diagnoses seem to hold. Delete the offending folder and copy it from elsewhere.

9 TRIM and SSDs

Modern macOS handles TRIM automatically at boot for NVMe drives connected over Thunderbolt — no manual intervention needed. The old trimforce command is largely a historical curiosity at this point; it does not reliably help SATA SSDs and NVMe drives don’t need it. USB SSDs are the awkward case — they only get TRIM if the vendor ships a driver kext (Samsung does for the T5 and T7; others may not).

To check whether TRIM is active on your drives:

system_profiler SPNVMeDataType | grep -i trim
system_profiler SPSerialATADataType | grep -i trim

trimforce is still present if you need it for an unusual older internal SATA drive:

sudo trimforce enable

10 Extended attributes for users

ls -al@
xattr -l

See What does the @ symbol mean in a file’s permission settings?