ZFS as Root Filesystem

Introduction

This tutorial will show you how to install Funtoo on ZFS (rootfs).

Introduction to ZFS

For a quick tour of ZFS and have a big picture of its common operations you can consult the page ZFS Fun.

Prerequisites

   Important

ZFS designed for 64-bit systems. We only recommending and supporting 64-bit platforms and installations!

ZFS recommendation is to control entire disk, hence, guide reflecting only installing ZFS on whole disk and legacy boot. Installing on UEFI requires separate partition for /boot, formatted into FAT32 and not covered here, though, installation on UEFI is certainly possible. Guide also not describing anything related to encryption.

Downloading the ISO (With ZFS)

In order to install Funtoo on ZFS, you will need an environment such as live media with ZFS tools provided.

Creating a bootable USB from ISO (From a Linux Environment)

After you download the iso, you can do the following steps to create a bootable USB: TBA.

Creating ZFS partitions

We are letting ZFS automatically partition the drive. We use "partition" term, so it would be more familiar for many.

Creating of pool

root #  mkdir /mnt/funtoo
root #  zpool create -f -o ashift=12 -o cachefile=/tmp/zpool.cache -O normalization=formD -m none -R /mnt/funtoo -d -o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o feature@spacemap_histogram=enabled -o feature@enabled_txg=enabled -o feature@extensible_dataset=enabled -o feature@bookmarks=enabled rpool /dev/sda

   Note

ashift 12 stands for 4-K block size sectors. Such type of hard drives are common nowadays.

Pool name is irrelevant, rpool used in this guide.

Create the zfs datasets

Create rootfs

root #  zfs create -o mountpoint=none rpool/ROOT
root #  zfs create -o mountpoint=/ rpool/ROOT/funtoo

Create optional datasets

Described below datasets are examples and optional. It's up to users to perform creation of their own datasets.

Home

root #  zfs create -o mountpoint=/home rpool/HOME
root #  zfs create -o mountpoint=/root rpool/HOME/root

Portage directories

root #  zfs create -o mountpoint=none -o setuid=off rpool/FUNTOO
root #  zfs create -o mountpoint=/usr/portage -o atime=off rpool/FUNTOO/portage
root #  zfs create -o mountpoint=/usr/portage/distfiles rpool/FUNTOO/distfiles

Build directory

root #  zfs create -o mountpoint=/var/tmp/portage -o compression=lz4 -o sync=disabled rpool/FUNTOO/build

Setting bootfs

When booting from ZFS, you must specify a boot device and a root file system within the pool that was identified by the boot device. By default, the dataset selected for booting is the one identified by the pool's bootfs property.

root # zpool set bootfs=rpool/ROOT/funtoo rpool

Installing Funtoo

Now that initial ZFS pool created, datasets ready, we can perform stage3 unpack and basic system installation. This part does not differ much from regular Funtoo Linux installation. http://www.funtoo.org/Install#Installing_the_Stage_3_tarball. We recommend using funtoo-current stages for ZFS.

root # cd /mnt/funtoo
root # wget http://build.funtoo.org/funtoo-current/x86-64bit/generic_64/stage3-latest.tar.xz

Extract the contents with the following command, substituting in the actual name of your stage 3 tarball (in case of arch optimized stage).

root # tar xpf stage3-latest.tar.xz

Now we need to create chroot environment by following:

root # cd /mnt/funtoo
root # mount -t proc none proc
root # mount --rbind /sys sys
root # mount --rbind /dev dev

Important step is to copy ZFS cache we created at the very beginning of our rpool creation into chroot.

root # mkdir -p /mnt/funtoo/etc/zfs
root # cp /tmp/zpool.cache /mnt/funtoo/etc/zfs/zpool.cache

You'll also want to copy over resolv.conf in order to have proper resolution of Internet hostnames from inside the chroot:

root # cp /etc/resolv.conf /mnt/funtoo/etc/

Building kernel, initramfs and grub to work with zfs

Install GRUB 2

Install grub2:

root # echo "sys-boot/grub libzfs -truetype" >> /etc/portage/package.use
root # emerge grub

Now install grub to the drive itself (not a partition):

root # grub-install /dev/sda

Initial kernel build

Build now kernel and initramfs with --zfs

root # genkernel all --zfs --no-clean --no-mountboot --callback="emerge @module-rebuild"

Using the debian-sources, the following command may give better results:

root # genkernel all --zfs --no-clean --no-mountboot --callback="emerge spl zfs-kmod zfs"

Configuring the Bootloader

Using the genkernel you must add 'real_root=ZFS=<root>' and 'dozfs' to your params. Edit /etc/boot.conf:

   /etc/boot.conf

"Funtoo ZFS" {
        kernel kernel[-v]
        initrd initramfs-genkernel-x86_64[-v]
        params real_root=ZFS=tank/funtoo/root
        params += dozfs=force
}

The command boot-update should take care of grub configuration:

Install boot-update (if it is missing):
root # emerge boot-update

Run boot-update to update grub.cfg
root # boot-update

   Note

If boot-updatefails, try this:

root # grub-mkconfig -o /boot/grub/grub.cfg

Now you should have a new installation of the kernel, initramfs and grub which are zfs capable. The configuration files should be updated, and the system should come up during the next reboot.

Final configuration

Clean up and reboot

We are almost done, we are just going to clean up, set our root password, and unmount whatever we mounted and get out.

Delete the stage3 tarball that you downloaded earlier so it doesn't take up space.
root # cd /
root # rm stage3-latest.tar.xz

Set your root password
root # passwd
>> Enter your password, you won't see what you are writing (for security reasons), but it is there!

Get out of the chroot environment
root # exit

Unmount all the kernel filesystem stuff and boot (if you have a separate /boot)
root # umount -l proc dev sys boot

Turn off the swap
root # swapoff /dev/zvol/tank/swap

Export the zpool
root # cd /
root # zpool export tank

Reboot
root # reboot

   Important

Don't forget to set your root password as stated above before exiting chroot and rebooting. If you don't set the root password, you won't be able to log into your new system.

and that should be enough to get your system to boot on ZFS.

After reboot

Forgot to reset password?

System Rescue CD

If you aren't using bliss-initramfs, then you can reboot back into your sysresccd and reset through there by mounting your drive, chrooting, and then typing passwd.

Example:

root # zpool import -f -R /mnt/funtoo tank
root # chroot /mnt/funtoo bash -l
root # passwd
root # exit
root # zpool export -f tank
root # reboot

Create initial ZFS Snapshot

Continue to set up anything you need in terms of /etc configurations. Once you have everything the way you like it, take a snapshot of your system. You will be using this snapshot to revert back to this state if anything ever happens to your system down the road. The snapshots are cheap, and almost instant.

To take the snapshot of your system, type the following:

root # zfs snapshot -r tank@install

To see if your snapshot was taken, type:

root # zfs list -t snapshot

If your machine ever fails and you need to get back to this state, just type (This will only revert your / dataset while keeping the rest of your data intact):

root # zfs rollback tank/funtoo/root@install

   Important

For a detailed overview, presentation of ZFS' capabilities, as well as usage examples, please refer to the ZFS Fun page.

Troubleshooting

Starting from scratch

If your installation has gotten screwed up for whatever reason and you need a fresh restart, you can do the following from sysresccd to start fresh:

Destroy the pool and any snapshots and datasets it has
root # zpool destroy -R -f tank

This deletes the files from /dev/sda1 so that even after we zap, recreating the drive in the exact sector
position and size will not give us access to the old files in this partition.
root # mkfs.ext2 /dev/sda1
root # sgdisk -Z /dev/sda

Now start the guide again :).

Starting again reusing the same disk partitions and the same pool

If your installation has gotten screwed up for whatever reason and you want to keep your pole named tank than you should boou into the Rescue CD / USB as done before.

import the pool reusing all existing datasets:
root # zpool import -f -R /mnt/funtoo tank

Now you should wipe the previous installation off:

let's go to our base installation directory:
root # cd /mnt/funtoo

and delete the old installation: 
root # rm -rf *

Now start the guide again, at "Pre-Chroot"