Practicing the ZFS filesystem stable on Ubuntu
By Patineboot

The common crane flapping its wings.
News
I have published the new ZFS filesystem Article,Abstract
- Prepare hardware and software.
- Install ZFS on Linux.
- Create a ZFS dataset.
- Mount the encrypted ZFS dataset.
- Snapshot a ZFS pool.
- Set up ZED.
- Back up a ZFS pool.
- Misc
We learned ‘Configuring Samba for macOS.’
Prepare hardware and software
Hardware List
- MiniPC Barebone: ASRock DeskMini X300
- CPU: AMD Ryzen 5 PRO 4650G
- Storage: Lexar M.2 SSD 500GB
- Memory: Crucial 16GB (8GB x 2)
- Storage(ZFS): Crucial MX500 1TB SATA
- Storage(ZFS): Samsung 860 EVO 1TB SATA
- SATA Hard Drive Adapter: StarTech.com USB312SAT3CB
Software Environment
- Operating System: Ubuntu 21.04 Server
- Filesystem: ZFS on Linux 2.02
Install ZFS on Linux
-
Install ZFS on Linux with the ‘zfsutils-linux’ package on
apt
.sudo apt install zfsutils-linux whereis zfs
We can confirm to be installed correctly with
whereis zfs
.
Create a ZFS pool
Check the disk IDs by running ls
on /dev/disk/by-id/.
$ ls -la /dev/disk/by-id/
lrwxrwxrwx 1 root root 9 Aug 30 21:19 ata-CT1000MX500SSD1_XXXXXXXXXXXX -> ../../sdb
lrwxrwxrwx 1 root root 9 Aug 30 21:19 ata-Samsung_SSD_860_EVO_1TB_XXXXXXXXXXXXXXX -> ../../sda
We got the disk IDs, ata-CT1000MX500SSD1_XXXXXXXXXXXX and ata-Samsung_SSD_860_EVO_1TB_XXXXXXXXXXXXXXX. The disk IDs are identical even if we connect the SSDs to other sockets.
Create a single or mirrored ZFS pool named ‘storage-pool’ from the disk IDs identifying the SSDs.
-
Create a ZFS pool with single
sudo zpool create storage-pool ata-Samsung_SSD_860_EVO_1TB_XXXXXXXXXXXXXXX
-
Create a ZFS pool with mirrored
sudo zpool create storage-pool mirror ata-Samsung_SSD_860_EVO_1TB_XXXXXXXXXXXXXXX ata-CT1000MX500SSD1_XXXXXXXXXXXX
Check the status of the ZFS pool with zpool status
.
Create a ZFS dataset
Create an encrypted ZFS dataset
Choose AES256 GCM on a cryptographic algorithm mode and passphrase to encrypt on the keyformat option.
sudo zfs create -o encryption=aes-256-gcm -o keyformat=passphrase storage-pool/multimedia
Note: Ubuntu does NOT mount the encrypted ZFS dataset while booting itself.
We manually mount the encrypted dataset with your passphrase after booting Ubuntu.
-
Load the encryption key and mount the encrypted ZFS dataset separately.
zfs load-key storage-pool/multimedia zfs mount -a
-
Mount the encrypted ZFS dataset and load the encryption key at the same time.
zfs mount -l storage-pool/multimedia
Set Properties of ZFS dataset
Set a value on properties of a ZFS dataset with zfs set
.
We can get the values with zfs get
.
-
Compress files stored in a ZFS dataset.
Setlz4
on thecompression
property by considering a trade-off between speed and compression ratio.sudo zfs set compression=lz4 storage-pool/multimedia
-
Remove redundant data from a ZFS dataset.
Seton
on thededup
property, we reduce the used size on the ZFS dataset with the deduplication.sudo zfs set dedup=on storage-pool/multimedia
e.g., Deduplication requires approximately 2.6 GiB memory if the size of the deduplication ZFS dataset is 1TiB. The size of the block is 128 KiB, which is the default value on ZFS on Linux.
Formula: 1 TiB / 128 KiB * 320 Byte = 2 684 354 560 Byte.
Mount the encrypted ZFS dataset
We recommend that “On the boot sequence” is for your convenience, “On the Samba connected” is for your safety.
On the boot sequence
We can access files on the encrypted ZFS dataset on the boot sequence with the systemd.
Create the new zfs-mount-encrypted@.service service file to mount the encrypted ZFS dataset with the passphrase.
[Unit]
Description=Mount encrypted ZFS datasets
DefaultDependencies=no
After=local-fs.target
RequiresMountsFor=/root/
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=%s -c 'cat %h/.dataset-passphrase | /sbin/zfs mount -l %I'
ExecStop=/sbin/zfs unmount -uf %I
[Install]
WantedBy=zfs.target
Create the .dataset-passphrase passphrase file and move the passphrase file to the /root directory.
# Create the passphrase file involving your passphrase on the superuser.
cat - | tee .dataset-passphrase
# Change the permission of the passphrase file to read only.
sudo chmod 400 .dataset-passphrase
# Change the owner of the passphrase file to the root user.
sudo chown root:root .dataset-passphrase
# Move the file to the root directory.
sudo mv .dataset-passphrase /root/
Enable the service to auto-run on boot and start it now.
# copy the service file to the user directory of the systemd.
sudo cp zfs-mount-encrypted@.service /etc/systemd/system/
# Enable and start now the service.
systemctl enable --now zfs-mount-encrypted@storage\\x2dpool-multimedia.service
We get the ‘storage\
x2dpool-multimedia’ systemd-escaped string by running the systemd-escape
command with the ‘storage-pool/multimedia’ dataset name.
A backslash character ‘\\
’ represents ‘\
’ on the Shell Script.
‘storage\
x2dpool-multimedia’ is escaped to ‘storage\\
x2dpool-multimedia.’
On the Samba connected
We mount the ZFS encrypted dataset when connected with Samba, and we unmount it when disconnected.
Create the /root/.dataset-passphrase passphrase file same as “On the boot sequence.”
We add the three parameters into the section in which we specify storage-pool/multimedia on /etc/samba/smb.conf.
root preexec = sh -c 'cat /root/.dataset-passphrase | zfs mount -l storage-pool/multimedia'
root preexec close = yes
root postexec = zfs unmount -u storage-pool/multimedia
Snapshot a ZFS pool
Set up zfs-auto-snapshot
zfs-auto-snapshot
, ZFS Automatic Snapshot Service for Linux, automatically creates, rotates, and destroys periodic ZFS snapshots.
Install the zfs-auto-snapshot
package:
sudo apt install zfs-auto-snapshot
The default settings of zfs-auto-snapshot
:
Create a snapshot of the ZFS pool every Interval time, keeping the numbers of Snapshots. e.g., Create a snapshot every hour, keeping 24 hourly snapshots.
Interval | Snapshots |
---|---|
15 minutes | 4 |
1 hour | 24 |
1 day | 31 |
1 week | 8 |
1 month | 12 |
We change the settings of zfs-auto-snapshot
:
- Create a snapshot every week, keeping 108 weekly snapshots
- Disable frequent snapshots and monthly snapshots
We explain the method to change them.
-
Change the
keep
option of thezfs-auto-snapshot
command in thecron
file at /etc/cron.weekly/zfs-auto-snapshot to –keep=108 from –keep=8.The changed /etc/cron.weekly/zfs-auto-snapshot file is:
#!/bin/sh # Only call zfs-auto-snapshot if it's available which zfs-auto-snapshot > /dev/null || exit 0 exec zfs-auto-snapshot --quiet --syslog --label=weekly --keep=108 //
-
Remove /etc/cron.d/zfs-auto-snapshot and /etc/cron.monthly/zfs-auto-snapshot.
-
Destroy some frequent snapshots if
zfs-auto-snapshot
have already created them.
Share the ZFS snapshots on Samba
We configure Samba to share the ZFS snapshots.
Add the snapshot
section to the /etc/samba/smb.conf.
[snapshot]
# Set xattr on fruit:resource on the snapshot section if we do not set xattr on global
fruit:resource = xattr
path = /storage-pool/multimedia/.zfs/snapshot
valid users = patine
access based share enum = yes
browseable = yes
writable = no
We share the ZFS snapshots and and can access them on an SMB client while mounting /storage-pool/multimedia.
Set up ZED
Install an SMTP client and Configure ZED. ZED stands for ZFS Event Daemon.
ZED sends a notification email when a scrub starts or our pool becomes unhealthy.
-
Install an SMTP client
We install the ‘msmtp’, ‘msmtp-mta’, and ‘bsd-mailx’ packages.
Install and configure those packages, after reading Building Home Server with Ubuntu.
-
Configure ZED
We add the following pairs of the key and the value to the configure file located at /etc/zfs/zed.d/zed.rc.
ZED_EMAIL_ADDR="xxxxx@patineboot.com" ZED_EMAIL_PROG="mail" ZED_EMAIL_OPTS="-s '@SUBJECT@' @ADDRESS@" ZED_NOTIFY_VERBOSE=1
Restart the ZED service.
sudo systemctl restart zed.service
We complete setting up the ZFS filesystem.
Back up a ZFS pool
We backup the pool named ‘storage-pool’ to the pool named ‘backup-pool’ that we create on Crucial MX500 SSD connected by SATA Hard Drive Adapter.
Create a single ZFS pool with the ‘backup-pool’ name from the disk ID identifying Crucial MX500 SSD. Set false on the com.sun:auto-snapshot property of ‘backup-pool’ to disable auto snapshot on it.
sudo zpool create backup-pool ata-CT1000MX500SSD1_XXXXXXXXXXXX
sudo zfs set com.sun:auto-snapshot=false backup-pool
Install the pv
and tmux
packages.
sudo apt install pv tmux
Install ZxyBackupCloser to back up some ZFS pools with another ZFS pool or dataset.
sudo pip3 install zxybackupcloser
Run ZxyBackupCloser
We back up the root-pool and storage-pool ZFS pools to the backup-pool ZFS pool for example:
sudo zxybackupcloser -b backup-pool root-pool storage-pool
Patineboot provides instructive bash (Bourne Again Shell) script, example.zxybackupcloser.sh to edit for your backup on the ZxyBackupCloser.
Misc
-
The useful
zpool
andzfs
commands.# Import/Export a ZFS pool # Import a ZFS pool after we attach an external storage device. sudo zpool import <pool name> # Export the ZFS pool before we detach the device. sudo zpool export <pool name> # Display all the status of the ZFS pools zpool status # Display all the capacities of the ZFS pools zpool list # Display all the capacities of the ZFS datasets zfs list # Display all snapshots of the ZFS datasets zfs list -t snapshot # Display all snapshots of the ZFS datasets with the date zfs list -r -t snapshot -o name,creation <pool name>/<dataset> # Display the variable size of the ZFS datasets. zfs list -o space,mountpoint # Mount/Unmount a ZFS pool # Mount a ZFS dataset with an encryption key sudo zfs mount -l <dataset name> # Unmount the ZFS dataset forcefully. sudo zfs unmount -u <dataset name> # Destroy a ZFS pool sudo zpool destroy <pool name>
-
The useful properties.
# Select lz4 compression into a ZFS dataset sudo zfs set compression=lz4 <pool name>/<dataset> # Enable deduplication of a ZFS dataset sudo zfs set dedup=on <pool name>/<dataset> # Select posixacl acl type into a ZFS dataset sudo zfs set acltype=posixacl <pool name>/<dataset> # Enable extended attribute stored on i-node of a ZFS dataset sudo zfs set xattr=sa <pool name>/<dataset> # Set mount point into a ZFS dataset zfs set mountpoint=<a mount point> <pool name>/<dataset> # Set disable auto-snapshot by zfs-auto-snapshot zfs set com.sun:auto-snapshot=false <pool name>/<dataset>
-
The bad way to mount the encrypted ZFS filesystem.
We do Not recommend that you describe your passphrase on the mounting service file itself. Your raw passphrase is exposed to every user on the Ubuntu server.
[Unit] Description=Load the encryption key for multimedia of ZFS storage. DefaultDependencies=no After=zfs-import.target Before=zfs-mount.service [Service] Type=oneshot RemainAfterExit=yes Environment="PASSPHRASE=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX" ExecStart=sh -c 'echo $PASSPHRASE | zfs load-key storage-pool/multimedia' [Install] WantedBy=zfs-mount.service
Notice: Patineboot masked the passphrase with some X characters.
References
- ZxyBackupCloser
A backup application to back up some ZFS pools to another ZFS pool or dataset.
https://github.com/patineboot/zxybackupcloser - Setup a ZFS storage pool
https://ubuntu.com/tutorials/setup-zfs-storage-pool#1-overview - zfs-auto-snapshot
ZFS Automatic Snapshot Service for Linux.
https://github.com/zfsonlinux/zfs-auto-snapshot