equivariant

Onsite backup script for ZFS

On my home computer, I employ a 3-2-1 backup strategy1:

In this article, I describe my onsite backups.

I have since switched to using zrepl instead of a homegrown backup script. My configuration is documented here.

The script described in this article is still useful, as it is relatively simple and does not require a specialized daemon.

How it works

The backup script sends incremental backups of a dataset to another zpool using zfs send. To facilitate incremental backups without requiring the source dataset to retain a full snapshot, it uses ZFS bookmarks.

Here is what the backups look like:

NAME                                     USED  AVAIL     REFER  MOUNTPOINT
zroot/ROOT/default                       199G   169G      199G  /
zroot/ROOT/default#2022-03-12.133001        -      -      199G  -
slow/backupsend                         1.58T   772G      218G  /slow/backupsend
slow/backupsend@2022-03-08.133001       1.02G      -      216G  -
slow/backupsend@2022-03-09.133001       2.70G      -      216G  -
slow/backupsend@2022-03-10.133001       1.32G      -      216G  -
slow/backupsend@2022-03-11.133001       1.18G      -      217G  -
slow/backupsend@2022-03-12.133001          0B      -      218G  -

Here, zroot/ROOT/default is the dataset being backed up and slow/backupsend is the dataset where the daily backups are being stored. While the script is running, there will be an additional temporary snapshot and bookmark on the source dataset.

The script

Here is the script, which I saved as /usr/local/bin/zfs-rootbackupsend:

#!/bin/bash
set -euxo pipefail

if [ "$EUID" -ne 0 ]; then
    echo 'Run as root'
    exit 1
fi

src='zroot/ROOT/default'
dst='slow/backupsend'

now="$(date -u +%Y-%m-%d.%H%M%S)"

# Determine bookmark corresponding to latest snapshot.
lastbook="$(zfs list -t bookmark -H -o name -s name "$src" | tail -n 1)"

# Snapshot the source dataset. Take a bookmark so that we can send the next
# incremental snapshot from this point even after deleting the snapshot.
zfs snapshot "$src@$now"
zfs bookmark "$src@$now" "$src#$now"

if ! zfs list "$dst" >/dev/null 2>&1; then
    # Initial full send.
    zfs send "$src@$now" | zfs receive -o readonly=on "$dst@$now"
else
    # Determine the latest snapshot of the destination dataset.
    last="$(zfs list -t snapshot -H -o name -s name "$dst" | tail -n 1 | cut -d@ -f2)"

    # Send the changes since the last snapshot on the destination dataset.
    zfs send -i "$src#$last" "$src@$now" | zfs receive "$dst@$now"
fi

# Remove the snapshot of the source dataset to free up space. The bookmark
# still remains and can be used as the incremental source for the next backup.
zfs destroy "$src@$now"

# Remove bookmark corresponding to previous snapshot. Only the latest bookmark
# is needed to make an incremental snapshot, so there's no point keeping older
# bookmarks around.
if [[ -n "$lastbook" ]]; then
    zfs destroy "$lastbook"
fi

# Take a snapshot of the data that's unique to the non-root zpool.
# NOTE: If you're using this script, you probably don't need this and can
#       comment out this line.
zfs snapshot "slow/bulk@$now"

Set it as executable:

sudo chmod 755 /usr/local/bin/zfs-rootbackupsend

Add a cron entry to /etc/cron.d/zfs-rootbackupsend:

SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
30 5 * * * root nice chronic /usr/local/bin/zfs-rootbackupsend

Pruning backups

To prune old backups down to monthly:

prune() {
    zfs list -H -o name -t snapshot "$1" > snaps
    # DATASETNAME@YYYY-MM-DD
    #            \---9---/
    prefix="$(($(echo -n "$1" | wc -c)+9))"
    uniq -w "$prefix" snaps > keep
    comm -23 snaps keep > destroy
    while read s; do echo "$s"; echo sudo zfs destroy "$s"; done < destroy
}
prune slow/backupsend
prune slow/bulk
rm snaps keep destroy

  1. 3-2-1 rule: 3 copies of the data, 2 different media types, 1 offsite copy. ↩