Fedora Linux 38 is now available! This release includes the latest GNOME 44 desktop as well as other user-experience enhancements.
I’ve already written an article on how to install Fedora 37 with snapshot and rollback support. When it comes to btrfs snapshot and rollback support, nothing has changed in Fedora 38. The DNF database is still in /var/lib/dnf and has not been moved to /usr.
In this updated guide on how to Install Fedora 38 with snapshot and rollback support, I’ve made some changes. The changes are as follows:
- Created new subvolumes on the system root to keep some directories out of root file system snapshots. Some of these directories contain temporary files and caches, others contain data that you do not want to lose if you roll back the system root, and some should be in Read-Write mode when booting into a Read-only snapshot.
- Created one subvolume in the user home for the Firefox browser. This prevents you from losing bookmarks, passwords, and other user data when you undo a change to the user’s home.
- Turned on automatic timeline snapshots for the system root in snapper. You can, however, turn off the timeline snapshots.
- Moved grub-btrfs from the GitHub archived ‘Add-systemd-volatile’ branch to the active master branch.
- Created a snapshot of the system root and set it as the default root file system. This makes managing snapshots and rollbacks a lot easier.
If you want to install Fedora 38 with LUKS Full Disk Encryption, Snapshot and Rollback Support, please see my other article, ‘How to Install Fedora 38 with Full Disk Encryption, Snapshot and Rollback Support‘.
So let’s get started.
Table of Contents
- 1. Disk and Subvolumes Layout for Fedora 38
- 2. Install Fedora 38 Workstation
- 3. Create the Additional Subvolumes
- 4. Install and Configure Snapper
- 5. Install and Configure Grub-Btrfs
- 6. Create a System Root Snapshot and Set It as the Default
- 7. Snapper Tests
- 8. Make a Snapshot the New System Root
- 9. Enable Automatic Timeline Snapshots
- Watch on YouTube
1. Disk and Subvolumes Layout for Fedora 38
I’ll be using a hard disk with a capacity of 100 GiB. I’ll create one 512 MiB EFI partition and use the rest of the disk space to create a btrfs volume. There is no need for a separate swap partition because Fedora creates a SwapOnZRAM during boot.
This is how the disk partition looks.
NAME SIZE FSTYPE LABEL MOUNTPOINT /dev/vda 100G ├─/dev/vda1 512M vfat EFI /boot/efi └─/dev/vda2 99.5G btrfs FEDORA /
The system root will be installed on the btrfs main volume (/dev/vda2). I’ll then create the following subvolumes on the system root to keep them out of root file system snapshots.
NAME MOUNTPOINT TYPE [main] / mainvolume home /home subvolume opt /opt subvolume cache /var/cache subvolume crash /var/crash subvolume log /var/log subvolume spool /var/spool subvolume tmp /var/tmp subvolume www /var/www subvolume AccountsService /var/lib/AccountsService subvolume gdm /var/lib/gdm subvolume images /var/lib/libvirt/images subvolume .mozilla /home/<USER>/.mozilla subvolume
The directories for which subvolumes are created and the reasons for doing so are listed below.
/home
Contains user data. It is created to keep user data separate and also to avoid loss of user data on system root rollbacks. To learn more about /home, check this link: /home.
/opt
Contains applications installed by a third party. It is excluded to prevent these applications from being uninstalled during rollbacks. To learn more about /opt, check this link: /opt.
/var/cache, /var/crash, /var/tmp
Directories contain temporary files and caches. So they are created to be excluded from snapshots. Fedora also mounts /tmp with tmpfs at boot, so a separate subvolume for /tmp is unnecessary. To learn more, check these links: /var/cache, /var/crash, /var/tmp, /tmp.
/var/log
Contains log files. It is created to avoid the loss of log data during rollbacks. To learn more about /var/log, check this link: /var/log.
/var/spool
Contains data that is awaiting some kind of later processing such as mail, mail queues, printing, printer spool, and so on. Created to avoid a loss of mail, printing, and spool data after a rollback. To learn more about /var/spool, check this link: /var/spool.
/var/www
Web server directory. It is created to keep hosted web server data separate and to avoid data loss on system root rollbacks.
/var/lib/AccountsService, /var/lib/gdm
Contains login user information as well as Gnome display information. These directories must always be writable. When you attempt to boot a snapshot from the GRUB menu, you are booting into a read-only snapshot, which causes the system to hang just before the Gnome login screen appears.
In my previous guides, I enabled the ‘systemd.volatile=state‘ kernel parameter, which should have solved this problem, but some of my readers reported that they couldn’t log in. As a result, I created subvolumes for these two directories so that they are always writable and thus permanently solve this problem.
If you are using a display manager other than Gnome, you must create a directory specific to that display manager. For XFCE, it might be /var/lib/lightdm. I haven’t tested.
/var/lib/libvirt/images
The default location for libvirt-managed virtual machine images. Excluded to avoid replacing virtual machine images with older versions during a rollback.
/home/<USER>/.mozilla
By default, Fedora Workstation includes the Firefox browser. So, if you take a snapshot of your home directory and later undo any changes, you don’t want to lose any new bookmarks, passwords, and other user data.
As a result, I created a separate subvolume for the ‘.mozilla‘ directory, keeping it separate from the /home subvolume and avoiding data loss during any undo changes.
Create a subvolume for the ‘/home/<USER>/.config/google-chrome‘ directory if you use the Chrome web browser.
2. Install Fedora 38 Workstation
Boot your system using the Fedora 38 Workstation installer in UEFI mode. On the welcome screen, select the Install to Hard Drive option. Next, select your Language, Keyboard, and configure Time & Date. Following that, from the INSTALLATION SUMMARY screen, select Installation Destination.
You should now be on the INSTALLATION DESTINATION screen. To proceed, pick the Advanced Custom (Blivet-GUI) radio button and then hit the Done button.
On the BLIVET GUI PARTITIONING screen, create the partitions, file systems, and btrfs subvolumes necessary to install Fedora 38 Workstation.
First, you need to create and mount the EFI partition. Select the free space and click the + sign to create a partition.
Set the partition Size to 512 MiB, the Filesystem to EFI System Partition, and the Mountpoint to /boot/efi.
Then, you need to create a btrfs volume where you can create all the subvolumes needed to install Fedora 38 Workstation.
Select the free space again and click on the + sign to create a Btrfs volume. Set the Filesystem to btrfs and the Mountpoint to /. I used the entire remaining space. However, you can specify the size you want for the btrfs volume.
Next, you must create a home subvolume. (1) Select the Btrfs Volume from the left panel, and (2) click on the + sign on the right panel.
Create a home subvolume. Enter the Name as home and Mountpoint as /home. Click OK to finish.
I will create the remaining subvolumes when the installation is finished. This is because the Anaconda installer does not allow you to create subvolumes with slashes (/) in their paths, such as var/log.
For now, click Done to finish creating subvolumes.
On the SUMMARY OF CHANGES screen, double-check that everything is properly defined. To finalize the changes, click the Accept Changes button.
You will be returned to the INSTALLATION SUMMARY screen. Press the Begin Installation button to start the installation process. The installation process will start. Just sit back and relax.
When the installation is finished, click the Finish Installation button and restart the system.
The last phase of the installation process will start. Click the Start Setup button to complete the remaining customization steps, such as setting a new login, password, and so on.
You will then be logged into the Fedora 38 Workstation with the all-new Gnome 44 desktop interface.
Now, open the Gnome terminal.
When booting your system, you may have noticed the following error message.
error: ../../grub-core/commands/loadenv.c:216:sparse file not allowed.
This is because you did not create a separate ext4 /boot partition and instead included it in the btrfs system root. GRUB preboot writes to /boot/grub2/grubenv if the boot was successful. This error occurs because of the GRUB btrfs.mod driver, unlike ext4, is read-only. To resolve this, execute the following command.
$ sudo grub2-editenv - unset menu_auto_hideSet the btrfs volume label. I named the btrfs volume FEDORA, but you may name it whatever you want.
$ sudo btrfs filesystem label / FEDORA
$ sudo btrfs filesystem show /
Label: 'FEDORA' uuid: 2990369d-fdc3-4c1a-b8a8-313e010434e1
Total devices 1 FS bytes used 5.93GiB
devid 1 size 99.50GiB used 8.02GiB path /dev/vda2Your setup should look something like this.
$ lsblk -p
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
/dev/zram0 251:0 0 3.8G 0 disk [SWAP]
/dev/vda 252:0 0 100G 0 disk
├─/dev/vda1 252:1 0 512M 0 part /boot/efi
└─/dev/vda2 252:2 0 99.5G 0 part /home
/And the subvolumes will look like this.
$ sudo btrfs subvolume list /
ID 256 gen 41 top level 5 path home
ID 257 gen 33 top level 5 path var/lib/machines
The var/lib/machines subvolume is created by systemd implicitly in Fedora 38 Workstation for systemd-nspawn containers.
Enable the fastest mirrors to speed up package downloads. Copy the command from sudo to EOF, paste it into your terminal, and hit <Enter>.
$ sudo bash -c 'cat >> /etc/dnf/dnf.conf' <<EOF
defaultyes=True
fastestmirror=True
max_parallel_downloads=10
EOF
Clear all previous caches and update the local DNF metadata cache.
$ sudo dnf clean all
$ sudo dnf makecacheInstall the packages listed below.
$ sudo dnf install vim inotify-tools makeNow, update your operating system.
$ sudo dnf updateAnd reboot your system.
$ sudo reboot3. Create the Additional Subvolumes
I’ll now proceed to create the remaining subvolumes on the system root. As previously stated, these subvolumes are created on the system root to avoid being included in the rollback regime because some of these directories contain temporary files and caches, others contain data that you do not want to lose if you rollback the system root, and some should be in Read-Write mode when booting into a Read-only snapshot.
Instead of manually creating subvolumes, I use the bash for loop. If you want to add or remove subvolumes, change the variable declaration.
#### Get the UUID of your btrfs system root.
$ ROOT_UUID="$(sudo grub2-probe --target=fs_uuid /)"
#### Get the btrfs subvolume mount options from your fstab.
$ OPTIONS="$(grep '/home' /etc/fstab | awk '{print $4}' | cut -d, -f2-)"
#### Create a directory path. Only for other DEs; not required for Fedora Workstation.
$ sudo mkdir -vp /var/lib/libvirt
#### Declare rest of the subvolumes you want to create in the array.
#### Copy from 'SUBVOLUMES' to ')', paste it in terminal, and hit <Enter>.
$ SUBVOLUMES=(
"opt"
"var/cache"
"var/crash"
"var/log"
"var/spool"
"var/tmp"
"var/www"
"var/lib/AccountsService"
"var/lib/gdm"
"var/lib/libvirt/images"
"home/$USER/.mozilla"
)
#### Run the for loop to create the subvolumes.
#### Copy from 'for' to 'done', paste it in terminal, and hit <Enter>.
$ for dir in "${SUBVOLUMES[@]}" ; do
if [[ -d "/${dir}" ]] ; then
sudo mv -v "/${dir}" "/${dir}-old"
sudo btrfs subvolume create "/${dir}"
sudo cp -ar "/${dir}-old/." "/${dir}/"
else
sudo btrfs subvolume create "/${dir}"
fi
sudo restorecon -RF "/${dir}"
printf "%-41s %-24s %-5s %-s %-s\n" \
"UUID=${ROOT_UUID}" \
"/${dir}" \
"btrfs" \
"subvol=${dir},${OPTIONS}" \
"0 0" | \
sudo tee -a /etc/fstab
doneYou must change the permissions of some of the directories.
$ sudo chmod 1777 /var/tmp
$ sudo chmod 1770 /var/lib/gdm
$ sudo chown -R $USER: /home/$USER/.mozillaExamine your /etc/fstab file. This is what it should look like. The UUID and username will be unique to your system.
$ cat /etc/fstab
...
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 / btrfs defaults 0 0
UUID=D17B-4753 /boot/efi vfat umask=0077,shortname=winnt 0 2
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /home btrfs subvol=home,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /opt btrfs subvol=opt,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/cache btrfs subvol=var/cache,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/crash btrfs subvol=var/crash,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/log btrfs subvol=var/log,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/spool btrfs subvol=var/spool,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/tmp btrfs subvol=var/tmp,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/www btrfs subvol=var/www,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/lib/AccountsService btrfs subvol=var/lib/AccountsService,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/lib/gdm btrfs subvol=var/lib/gdm,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /var/lib/libvirt/images btrfs subvol=var/lib/libvirt/images,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /home/madhu/.mozilla btrfs subvol=home/madhu/.mozilla,compress=zstd:1 0 0Reload /etc/fstab to mount all the subvolumes.
$ sudo systemctl daemon-reload
$ sudo mount -va
/ : ignored
/boot/efi : already mounted
/home : already mounted
/opt : successfully mounted
/var/cache : successfully mounted
/var/crash : successfully mounted
/var/log : successfully mounted
/var/spool : successfully mounted
/var/tmp : successfully mounted
/var/www : successfully mounted
/var/lib/AccountsService : successfully mounted
/var/lib/gdm : successfully mounted
/var/lib/libvirt/images : successfully mounted
/home/madhu/.mozilla : successfully mountedCheck your subvolumes.
$ sudo btrfs subvolume list /
ID 256 gen 123 top level 5 path home
ID 257 gen 123 top level 5 path var/lib/machines
ID 258 gen 104 top level 5 path opt
ID 259 gen 123 top level 5 path var/cache
ID 260 gen 106 top level 5 path var/crash
ID 261 gen 123 top level 5 path var/log
ID 262 gen 123 top level 5 path var/spool
ID 263 gen 123 top level 5 path var/tmp
ID 264 gen 110 top level 5 path var/www
ID 265 gen 123 top level 5 path var/lib/AccountsService
ID 266 gen 123 top level 5 path var/lib/gdm
ID 267 gen 123 top level 5 path var/lib/libvirt/images
ID 268 gen 123 top level 256 path home/madhu/.mozillaYour setup should now look somewhat like this.
$ lsblk -p
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
/dev/zram0 251:0 0 3.8G 0 disk [SWAP]
/dev/vda 252:0 0 100G 0 disk
├─/dev/vda1 252:1 0 512M 0 part /boot/efi
└─/dev/vda2 252:2 0 99.5G 0 part /home/madhu/.mozilla
/var/lib/libvirt/images
/var/lib/gdm
/var/lib/AccountsService
/var/www
/var/tmp
/var/spool
/var/log
/var/crash
/var/cache
/opt
/home
/Now that everything appears to be in order, you can delete the old directories. Copy from ‘for’ to ‘done’, paste it into the terminal, and hit <Enter>.
$ for dir in "${SUBVOLUMES[@]}" ; do
if [[ -d "/${dir}-old" ]] ; then
sudo rm -rf "/${dir}-old"
fi
doneAll subvolumes have now been created.
4. Install and Configure Snapper
Install snapper and the optional package python3-dnf-plugin-snapper. The python3-dnf-plugin-snapper package allows you to create pre and post snapshots every time you install a package on the system with the dnf package manager.
$ sudo dnf install snapper python3-dnf-plugin-snapperCreate new snapper configurations named root and home for the Btrfs volume at / and /home, respectively.
$ sudo snapper -c root create-config /
$ sudo snapper -c home create-config /homeVerify the snapper configuration files have been created.
$ sudo snapper list-configs
Config | Subvolume
-------+----------
home | /home
root | /Allow regular user to use snapper without requiring root privileges. Add your user name to the snapper’s root and home configurations to set the ACL on the /.snapshots and /home/.snapshots directories.
$ sudo snapper -c root set-config ALLOW_USERS=$USER SYNC_ACL=yes
$ sudo snapper -c home set-config ALLOW_USERS=$USER SYNC_ACL=yesAdd the newly created snapshot subvolumes to the /etc/fstab file. The updated lines are in amber.
$ sudo vim /etc/fstab
...
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /home/madhu/.mozilla btrfs subvol=home/madhu/.mozilla,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /.snapshots btrfs subvol=.snapshots,compress=zstd:1 0 0
UUID=2990369d-fdc3-4c1a-b8a8-313e010434e1 /home/.snapshots btrfs subvol=home/.snapshots,compress=zstd:1 0 0Reload the /etc/fstab file.
$ sudo systemctl daemon-reload
$ sudo mount -vaYour setup should look something like this.
$ lsblk -p /dev/vda
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
/dev/vda 252:0 0 100G 0 disk
├─/dev/vda1 252:1 0 512M 0 part /boot/efi
└─/dev/vda2 252:2 0 99.5G 0 part /home/.snapshots
/.snapshots
/home/madhu/.mozilla
/var/www
/var/spool
/var/tmp
/var/log
/var/lib/libvirt/images
/var/lib/gdm
/var/lib/AccountsService
/var/crash
/var/cache
/opt
/home
/And your subvolumes will look like this.
$ sudo btrfs subvolume list /
ID 256 gen 129 top level 5 path home
ID 257 gen 123 top level 5 path var/lib/machines
ID 258 gen 104 top level 5 path opt
ID 259 gen 127 top level 5 path var/cache
ID 260 gen 106 top level 5 path var/crash
ID 261 gen 131 top level 5 path var/log
ID 262 gen 123 top level 5 path var/spool
ID 263 gen 126 top level 5 path var/tmp
ID 264 gen 110 top level 5 path var/www
ID 265 gen 123 top level 5 path var/lib/AccountsService
ID 266 gen 123 top level 5 path var/lib/gdm
ID 267 gen 123 top level 5 path var/lib/libvirt/images
ID 268 gen 123 top level 256 path home/madhu/.mozilla
ID 269 gen 129 top level 5 path .snapshots
ID 270 gen 129 top level 256 path home/.snapshotsDisable the indexing of the .snapshots directories by updatedb. It is enabled by default, which can cause significant slowdown and excessive memory usage if you have a large number of snapshots.
$ echo 'PRUNENAMES = ".snapshots"' | sudo tee -a /etc/updatedb.confEnable snapshot booting by appending the SUSE_BTRFS_SNAPSHOT_BOOTING="true" option to the /etc/default/grub file.
$ echo 'SUSE_BTRFS_SNAPSHOT_BOOTING="true"' | sudo tee -a /etc/default/grubYou must also make changes to the /boot/efi/EFI/fedora/grub.cfg file now that snapshot booting is enabled.
$ sudo sed -i '1i set btrfs_relative_path="yes"' /boot/efi/EFI/fedora/grub.cfgFinally, update the grub.cfg file.
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfgLater, I’ll enable snapper timeline snapshots. For now, the snapper configuration is complete.
List the snapshots for / volume. For the root, you may use snapper -c root ls or simply snapper ls. Both provide the same output.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+------+------+---------+-------------+---------
0 | single | | | root | | current |List the snapshots for /home subvolume.
$ snapper -c home ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+------+------+---------+-------------+---------
0 | single | | | root | | current |At this stage, you do not have any snapshots, we will create some later.
5. Install and Configure Grub-Btrfs
The grub-btrfs package adds ‘Fedora Linux snapshots’ to the GRUB menu and allows you to test a snapshot in read-only mode before rolling back to it in read-write mode.
Clone the grub-btrfs repository.
$ git clone https://github.com/Antynea/grub-btrfsThe setup is pre-configured to work with Arch and Gentoo out of the box. You must make a few changes to the grub-btrfs config file to make it work with Fedora.
$ cd grub-btrfs
$ sed -i '/#GRUB_BTRFS_SNAPSHOT_KERNEL/a GRUB_BTRFS_SNAPSHOT_KERNEL_PARAMETERS="systemd.volatile=state"' config
$ sed -i '/#GRUB_BTRFS_GRUB_DIRNAME/a GRUB_BTRFS_GRUB_DIRNAME="/boot/grub2"' config
$ sed -i '/#GRUB_BTRFS_MKCONFIG=/a GRUB_BTRFS_MKCONFIG=/sbin/grub2-mkconfig' config
$ sed -i '/#GRUB_BTRFS_SCRIPT_CHECK=/a GRUB_BTRFS_SCRIPT_CHECK=grub2-script-check' configThen install it.
$ sudo make installUpdate grub.cfg and enable the grub-btrfsd.service.
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
$ sudo systemctl enable --now grub-btrfsd.serviceYou will receive the ‘No snapshots found’ error since you have not yet created any snapshots, but don’t worry, it will function properly after you do.
Your grub-btrfs installation is now complete. You may now delete the cloned grub-btrfs repository.
$ cd ..
$ rm -rvf grub-btrfs6. Create a System Root Snapshot and Set It as the Default
Now that everything in Fedora has been installed and configured, it is time to create a snapshot of the system root and make it the default. This makes it much easier to manage snapshots and rollbacks.
Create a directory named ‘1‘ in the /.snapshots directory.
$ sudo mkdir -v /.snapshots/1Create an XML file called info.xml in the /.snapshots/1/ directory. Copy the following command from sudo to EOF, paste it into your terminal, and hit the <Enter> button.
$ sudo bash -c "cat > /.snapshots/1/info.xml" <<EOF
<?xml version="1.0"?>
<snapshot>
<type>single</type>
<num>1</num>
<date>$(date -u +"%F %T")</date>
<description>first root subvolume</description>
</snapshot>
EOFNow, create a read-write subvolume snapshot of the system root in the /.snapshots/1/ directory.
$ sudo btrfs subvolume snapshot / /.snapshots/1/snapshotGet the subvolid of the /.snapshots/1/snapshot subvolume.
$ sudo btrfs inspect-internal rootid /.snapshots/1/snapshot
271Using subvolid 271, set the /.snapshots/1/snapshot subvolume as the default subvolume for the root (/) filesystem.
$ sudo btrfs subvolume set-default 271 /And reboot your system.
$ sudo rebootAfter rebooting, confirm that the /.snapshots/1/snapshot subvolume is indeed the default for the / filesystem.
$ sudo btrfs subvolume get-default /
ID 271 gen 170 top level 269 path .snapshots/1/snapshotNow check the snapshots in snapper.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+----------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 05:49:29 PM | root | | first root subvolume |As you can see, the /.snapshots/1/snapshot subvolume is also visible as snapshot #1 in snapper. The asterisk (*) indicates that this snapshot is the default and is currently active.
7. Snapper Tests
I’ll run some tests here to make sure everything is in working order and that there are no problems.
Test 1: Undo the Changes Made by the DNF Installer
In this test, I will use the dnf package manager to install a small package named ps_mem. I’ll check the system after it’s installed to see what changes were made. I’ll then attempt to undo the changes and see if I can get the system back to its pre-installation state.
Install the ps_mem package. It displays the core memory used per program (not per process).
$ sudo dnf install ps_memRun the ps_mem program to see if it was installed successfully.
$ sudo ps_mem
Private + Shared = RAM used Program
148.0 KiB + 13.5 KiB = 161.5 KiB fusermount3
196.0 KiB + 13.5 KiB = 209.5 KiB inotifywait
296.0 KiB + 15.5 KiB = 311.5 KiB mcelog
...
51.4 MiB + 3.5 MiB = 54.9 MiB python3.11
204.2 MiB + 23.5 MiB = 227.6 MiB gnome-shell
268.7 MiB + 2.0 MiB = 270.7 MiB packagekitd
---------------------------------
1.2 GiB
=================================Check the snapper for snapshots.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+-----------------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 05:49:29 PM | root | | first root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
3 | post | 2 | Friday 28 April 2023 06:50:23 PM | root | number | /usr/bin/dnf install ps_mem |As you can see, the ps_mem package has pre (#2) and post (#3) snapshots.
Let’s take a look at the changes it made to the system between snapshots #2 and #3.
$ snapper status 2..3
+..... /usr/bin/ps_mem
c..... /usr/lib/sysimage/rpm/rpmdb.sqlite-shm
c..... /usr/lib/sysimage/rpm/rpmdb.sqlite-wal
+..... /usr/share/doc/ps_mem
+..... /usr/share/doc/ps_mem/LICENSE
+..... /usr/share/man/man1/ps_mem.1.gz
c..... /var/lib/dnf/history.sqlite-shm
c..... /var/lib/dnf/history.sqlite-wal
-..... /var/lib/dnf/rpmdb_lock.pidNow I’ll undo the changes.
$ sudo snapper undochange 2..3
create:1 modify:4 delete:4Check again to ensure that the ps_mem package was successfully uninstalled.
$ sudo ps_mem
sudo: ps_mem: command not foundCool! The undo was successful. But, because I want to keep the package ps_mem, I’ll undo the changes again. This time between snapshots #3 and #2.
$ sudo snapper undochange 3..2
create:4 modify:4 delete:1Now run ps_mem again to see if it exists.
$ sudo ps_mem
Private + Shared = RAM used Program
148.0 KiB + 13.5 KiB = 161.5 KiB fusermount3
196.0 KiB + 13.5 KiB = 209.5 KiB inotifywait
296.0 KiB + 15.5 KiB = 311.5 KiB mcelog
...
51.4 MiB + 3.5 MiB = 54.9 MiB python3.11
204.2 MiB + 23.5 MiB = 227.6 MiB gnome-shell
268.7 MiB + 2.0 MiB = 270.7 MiB packagekitd
---------------------------------
1.2 GiB
=================================Test 1 is successful.
Test 2: Undo Any Changes Made to Individual Files
Assume you’re configuring something and editing a configuration file. Then you realize you messed up the configuration file. You want to see what changes you made to the original file and possibly wish you could undo the changes.
You can use snapper to undo not only all changes between two snapshots, but also a single file.
In this test, I will simply delete the last line of the /etc/hosts file. Then I’ll compare it to the same file in snapshot #3 to see what changed and if I can undo all of the changes and restore it to its original state.
This is how the original /etc/hosts file looks.
$ cat /etc/hosts
# Loopback entries; do not change.
# For historical reasons, localhost precedes localhost.localdomain:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
# See hosts(5) for proper format and other examples:
# 192.168.1.10 foo.mydomain.org foo
# 192.168.1.13 bar.mydomain.org barNow I’ll remove the last line.
$ sudo sed -i '$d' /etc/hosts
$ cat /etc/hosts
# Loopback entries; do not change.
# For historical reasons, localhost precedes localhost.localdomain:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
# See hosts(5) for proper format and other examples:
# 192.168.1.10 foo.mydomain.org fooUsing snapper, compare the /etc/hosts file with the one in snapshot #3.
$ snapper diff 3..0 /etc/hosts
--- /.snapshots/3/snapshot/etc/hosts 2022-11-27 20:56:24.000000000 +0530
+++ /etc/hosts 2023-04-28 21:14:25.578325998 +0530
@@ -4,4 +4,3 @@
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
# See hosts(5) for proper format and other examples:
# 192.168.1.10 foo.mydomain.org foo
-# 192.168.1.13 bar.mydomain.org barIn the output, 0 represents the current snapshot. As you can see in the last line, the /etc/hosts file in the current snapshot is missing that line compared to the one in snapshot #3. If you want to learn how to read the output, take a look at the man pages man snapper and man diff.
To replace the /etc/hosts file with the one in snapshot #3, use the following command.
$ sudo snapper undochange 3..0 /etc/hosts
create:0 modify:1 delete:0Check to see if the /etc/hosts file has indeed been rolled back.
$ cat /etc/hosts
# Loopback entries; do not change.
# For historical reasons, localhost precedes localhost.localdomain:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
# See hosts(5) for proper format and other examples:
# 192.168.1.10 foo.mydomain.org foo
# 192.168.1.13 bar.mydomain.org barTest 2 is successful.
Test 3: Create Manual Pre-Post Snapshots and Undo the Changes
In my previous article, I performed a manual pre-post snapshots test for installing a package with Gnome Software. The concept for both tests is the same. If you want, you can also run that test.
In this test, I will begin by creating manual ‘pre’ snapshots of the / and /home directories. Then I’ll install a small package called Color Picker (gcolor3) from GitHub by compiling it from the source file. When I’m finished, regardless of whether the compiling was successful or not, I’ll create ‘post’ snapshots for the previously created ‘pre’ snapshots. I’ll then look at the changes it made to the system between the pre-post snapshots and, finally, undo everything.
Create a manual ‘pre’ snapshot for both the / and /home directories.
$ snapper -c root create -t pre -c number -d 'Pre Color Picker'
$ snapper -c home create -t pre -c number -d 'Pre Color Picker'List the snapshots.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+-------+---------+-----------------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 05:49:29 PM | root | | first root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
3 | post | 2 | Friday 28 April 2023 06:50:23 PM | root | number | /usr/bin/dnf install ps_mem |
4 | pre | | Friday 28 April 2023 07:00:51 PM | madhu | number | Pre Color Picker |
$ snapper -c home ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+-------+---------+------------------+---------
0 | single | | | root | | current |
1 | pre | | Friday 28 April 2023 07:01:05 PM | madhu | number | Pre Color Picker |Install the pre-requisite packages before compiling the source file.
$ sudo dnf install meson gcc-c++ libhandy-devel libportal-devel \
libportal-gtk3-develCompile the source.
$ git clone https://github.com/Hjdskes/gcolor3.git
$ cd gcolor3
$ meson setup build
$ ninja-build -C build
$ sudo ninja-build -C build installMake sure the Color Picker package has been installed properly. Enter gcolor3 into the terminal and hit <Enter>. The Color Picker app must be open and appear as shown.
Now that you have successfully installed the Color Picker application, close the application and create ‘post’ snapshots for the / and /home directories.
$ snapper -c home create -t post --pre-number 1 -c number -d 'Post Color Picker'
$ snapper -c root create -t post --pre-number 4 -c number -d 'Post Color Picker'List the snapshots.
$ snapper -c root ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+-------+---------+----------------------------------------------------------------------------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 05:49:29 PM | root | | first root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
3 | post | 2 | Friday 28 April 2023 06:50:23 PM | root | number | /usr/bin/dnf install ps_mem |
4 | pre | | Friday 28 April 2023 07:00:51 PM | madhu | number | Pre Color Picker |
5 | pre | | Friday 28 April 2023 07:05:52 PM | root | number | /usr/bin/dnf install meson gcc-c++ libhandy-devel libportal-devel libportal-gtk3-devel |
6 | post | 5 | Friday 28 April 2023 07:06:27 PM | root | number | /usr/bin/dnf install meson gcc-c++ libhandy-devel libportal-devel libportal-gtk3-devel |
7 | post | 4 | Friday 28 April 2023 07:27:26 PM | madhu | number | Post Color Picker
$ snapper -c home ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+-------+---------+-------------------+---------
0 | single | | | root | | current |
1 | pre | | Friday 28 April 2023 07:01:05 PM | madhu | number | Pre Color Picker |
2 | post | 1 | Friday 28 April 2023 07:27:13 PM | madhu | number | Post Color Picker |Review the changes made to the system between the pre and post snapshots.
$ snapper -c home status 1..2
+..... /home/madhu/.config/gcolor3
+..... /home/madhu/.config/gcolor3/config.ini
+..... /home/madhu/gcolor3
+..... /home/madhu/gcolor3/build
+..... /home/madhu/gcolor3/build/build.ninja
...
+..... /home/madhu/.local/share/gnome-shell/application_state
c..... /home/madhu/.local/share/gvfs-metadata/root
-..... /home/madhu/.local/share/gvfs-metadata/root-3616f454.log
+..... /home/madhu/.local/share/gvfs-metadata/root-54385136.log
$ snapper -c root status 4..7
c..... /boot/grub2/grub-btrfs.cfg
c..... /boot/grub2/grub.cfg
+..... /etc/alternatives/ld
c..... /etc/ld.so.cache
+..... /usr/bin/addr2line
...
+..... /usr/share/xml/dbus-1/catalog.xml
+..... /usr/share/xml/dbus-1/introspect.dtd
+..... /usr/share/zsh/site-functions/_meson
+..... /usr/share/zsh/site-functions/_ninja
+..... /var/lib/alternatives/ld
c..... /var/lib/dnf/history.sqlite
c..... /var/lib/dnf/history.sqlite-shm
c..... /var/lib/dnf/history.sqlite-walThis is the total number of files added, removed, or modified.
$ snapper -c home status 1..2 | wc -l
273
$ snapper -c root status 4..7 | wc -l
14144I’ll now undo the changes in the / and /home directories.
$ sudo snapper -c home undochange 1..2
create:1 modify:1 delete:271
$ sudo snapper -c root undochange 4..7
create:0 modify:11 delete:14133The Color Picker package will be completely removed from the system.
Log out and then log back in.
$ gnome-session-quitCheck again to see if the Color Picker has been completely removed.
$ gcolor3
bash: gcolor3: command not found...Since you removed the Color Picker package, there is no reason to keep its pre-post snapshots if you do not intend to use it again. So you can delete those snapshots.
$ snapper -c home delete 1-2
$ snapper -c root delete 4-7Test 3 is successful.
Test 4: Rollback to a Previous Snapshot from the GRUB Menu
In this rollback test, I will delete the most critical files and directories on which the Linux operating system relies. These include Linux kernel and initramfs files in the /boot directory, the /etc directory, which contains all configuration files, and the /usr directory, which contains all drivers, kernel modules, and library files.
Once I delete these files and directories, I will be unable to issue any commands because the Linux OS is pretty much dead, and the system will undoubtedly fail the next time it boots. So I’ll hard boot the system and see if I can save the day by rolling back to a working snapshot.
So, before I delete them, here’s how they appear.
$ sudo du -sch /boot /etc /usr
249M /boot
32M /etc
5.8G /usr
6.1G totalNow delete the files and directories.
$ sudo rm -rvf /boot/{vmlinuz,initramfs}* /etc /usrBoom!!! All critical files have been removed. Your Linux system is no longer operational. Reboot the system. Your system should now fail to boot and display something like this.
Return to the GRUB menu and this time boot from snapshot #3.
Open the terminal and make sure everything is back to normal.
$ sudo ls /boot /etc /usrBecause you booted into the snapshot, the ‘/‘ file system is read-only.
$ sudo btrfs property get -ts /
ro=trueNow that everything appears to be operating properly, it is time to roll back this snapshot in read-write mode.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+-----------------------------+---------
0 | single | | | root | | current |
1+ | single | | Friday 28 April 2023 05:49:29 PM | root | | first root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
3- | post | 2 | Friday 28 April 2023 06:50:23 PM | root | number | /usr/bin/dnf install ps_mem |As you can see from the output, snapshot #1 has a ‘+‘ symbol, indicating that it is the default subvolume, whereas snapshot #3 has a ‘-‘ symbol, indicating that it is the currently booted subvolume.
$ sudo snapper rollback
Ambit is classic.
Creating read-only snapshot of default subvolume. (Snapshot 4.)
Creating read-write snapshot of current subvolume. (Snapshot 5.)
Setting default subvolume to snapshot 5.Now, reboot your system.
$ sudo rebootThe first thing you should do after a rollback is to update the grub.cfg file so that the GRUB menu snapshots are always in sync.
$ sudo grub2-mkconfig -o /boot/grub2/grub.cfgCheck the system once again.
$ sudo du -sch /boot /etc /usr
249M /boot
32M /etc
5.8G /usr
6.1G total
$ sudo btrfs property get -ts /
ro=false
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+-----------------------------+--------------
0 | single | | | root | | current |
1 | single | | Friday 28 April 2023 05:49:29 PM | root | number | first root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
3 | post | 2 | Friday 28 April 2023 06:50:23 PM | root | number | /usr/bin/dnf install ps_mem |
4 | single | | Friday 28 April 2023 08:21:39 PM | root | number | rollback backup of #1 | important=yes
5* | single | | Friday 28 April 2023 08:21:39 PM | root | | writable copy of #3 |
$ sudo btrfs subvolume list /
ID 256 gen 348 top level 5 path home
ID 257 gen 159 top level 5 path var/lib/machines
ID 258 gen 104 top level 5 path opt
ID 259 gen 340 top level 5 path var/cache
ID 260 gen 106 top level 5 path var/crash
ID 261 gen 351 top level 5 path var/log
ID 262 gen 340 top level 5 path var/spool
ID 263 gen 346 top level 5 path var/tmp
ID 264 gen 110 top level 5 path var/www
ID 265 gen 340 top level 5 path var/lib/AccountsService
ID 266 gen 340 top level 5 path var/lib/gdm
ID 267 gen 123 top level 5 path var/lib/libvirt/images
ID 268 gen 123 top level 256 path home/madhu/.mozilla
ID 269 gen 351 top level 5 path .snapshots
ID 270 gen 301 top level 256 path home/.snapshots
ID 271 gen 332 top level 269 path .snapshots/1/snapshot
ID 272 gen 181 top level 269 path .snapshots/2/snapshot
ID 273 gen 333 top level 269 path .snapshots/3/snapshot
ID 274 gen 332 top level 269 path .snapshots/4/snapshot
ID 275 gen 349 top level 269 path .snapshots/5/snapshot
$ sudo btrfs subvolume get-default /
ID 275 gen 349 top level 269 path .snapshots/5/snapshotYou successfully restored the system with the latest working snapshot in read and write mode.
Test 4 is successful.
8. Make a Snapshot the New System Root
Now that you’ve tested the snapper undo and rollback features and are hopefully satisfied with the results, you might want to set one of the snapshots as the new default system root and resume normal operating system use.
But first, let me show you how to determine how much space each snapshot takes up on your disk.
$ sudo btrfs filesystem du -s --human-readable /.snapshots/*/snapshot
Total Exclusive Set shared Filename
79.53MiB 0.00B 79.53MiB /.snapshots/1/snapshot
5.84GiB 36.00KiB 5.78GiB /.snapshots/2/snapshot
5.84GiB 52.00KiB 5.78GiB /.snapshots/3/snapshot
79.53MiB 0.00B 79.53MiB /.snapshots/4/snapshot
5.84GiB 228.00KiB 5.78GiB /.snapshots/5/snapshotAs you can see from the output above, snapshot #1 has an exclusive size of 0 bytes. This is because, as you may recall, I deleted all of the files recursively. Snapshot #4 is also 0 bytes in size because it is a read-only duplicate of snapshot #1.
To find out how the ‘Total’ and ‘Set shared’ values are calculated, see the man page man btrfs-filesystem.
Now, let’s return to making a snapshot the new system root.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+-----------------------------+--------------
0 | single | | | root | | current |
1 | single | | Friday 28 April 2023 05:49:29 PM | root | number | first root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
3 | post | 2 | Friday 28 April 2023 06:50:23 PM | root | number | /usr/bin/dnf install ps_mem |
4 | single | | Friday 28 April 2023 08:21:39 PM | root | number | rollback backup of #1 | important=yes
5* | single | | Friday 28 April 2023 08:21:39 PM | root | | writable copy of #3 |The output indicates that snapshot #5 is the default subvolume, and since it is operational, you can simply delete the remaining snapshots and move on.
However, I will use snapshot #2 as the default because it is a ‘pre’ snapshot that was created just before ps_mem was installed and is clean.
So, first, delete all of the snapshots except #2. You cannot delete snapshot #5 because it is the active one.
$ snapper delete 1
$ snapper delete 3-4
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+-----------------------------+---------
0 | single | | | root | | current |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
5* | single | | Friday 28 April 2023 08:21:39 PM | root | | writable copy of #3 |Create a directory named ‘1‘ in the /.snapshots directory.
$ sudo mkdir -v /.snapshots/1Copy the info.xml file from /.snapshots/2/ to /.snapshots/1/.
$ sudo cp -v /.snapshots/2/info.xml /.snapshots/1/Edit the /.snapshots/1/info.xml file and change the content from this…
$ sudo cat /.snapshots/1/info.xml
<?xml version="1.0"?>
<snapshot>
<type>pre</type>
<num>2</num>
<date>2023-04-28 13:20:21</date>
<description>/usr/bin/dnf install ps_mem</description>
<cleanup>number</cleanup>
</snapshot>…to this. Changes are in amber.
$ sudo cat /.snapshots/1/info.xml
<?xml version="1.0"?>
<snapshot>
<type>single</type>
<num>1</num>
<date>2023-04-28 13:20:21</date>
<description>new root subvolume</description>
</snapshot>Create a read-write subvolume snapshot of snapshot #2 in the /.snapshots/1/ directory.
$ sudo btrfs subvolume snapshot /.snapshots/2/snapshot /.snapshots/1/snapshotGet the subvolid of the /.snapshots/1/snapshot subvolume.
$ sudo btrfs inspect-internal rootid /.snapshots/1/snapshot
276Using subvolid 276, set the /.snapshots/1/snapshot subvolume as the default subvolume for the root (/) filesystem.
$ sudo btrfs subvolume set-default 276 /Then reboot.
$ sudo rebootAfter rebooting, confirm that the /.snapshots/1/snapshot subvolume is indeed the default for the / filesystem.
$ sudo btrfs subvolume get-default /
ID 276 gen 418 top level 269 path .snapshots/1/snapshotAnd is writable.
$ sudo btrfs property get -ts /
ro=falseTake a look at the snapper now.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+-----------------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 06:50:21 PM | root | | new root subvolume |
2 | pre | | Friday 28 April 2023 06:50:21 PM | root | number | /usr/bin/dnf install ps_mem |
5 | single | | Friday 28 April 2023 08:21:39 PM | root | | writable copy of #3 |As you can see, snapshot #1 is the default. You can now delete the remaining snapshots.
$ snapper delete 2-5
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+---------+--------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 06:50:21 PM | root | | new root subvolume |
$ sudo btrfs filesystem du -s --human-readable /.snapshots/*/snapshot
Total Exclusive Set shared Filename
5.84GiB 7.59MiB 5.78GiB /.snapshots/1/snapshotYou’re done.
9. Enable Automatic Timeline Snapshots
Now that you’ve finished everything, you can enable automatic timeline snapshots. When the timeline is enabled, a snapshot is created once every hour. Once per day, the timeline cleanup algorithm cleans up the snapshots.
It is enabled by default in both the root and home configurations. Enable it only for the system root and disable it for the home. See the Arch Wiki page on ‘Automatic timeline snapshots‘ for more information.
$ sudo snapper -c home set-config TIMELINE_CREATE=no
$ sudo systemctl enable --now snapper-timeline.timer
$ sudo systemctl enable --now snapper-cleanup.timerThat’s all. Every hour from now on, a ‘single’ snapshot will be created and cleaned up every other day.
$ snapper ls
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
---+--------+-------+----------------------------------+------+----------+--------------------+---------
0 | single | | | root | | current |
1* | single | | Friday 28 April 2023 06:50:21 PM | root | | new root subvolume |
2 | single | | Friday 28 April 2023 10:00:53 PM | root | timeline | timeline |To stop timeline snapshots, disable the snapper timers.
$ sudo systemctl disable --now snapper-timeline.timer
$ sudo systemctl disable --now snapper-cleanup.timerThe installation of Fedora 38 with snapshot and rollback support is now complete.
