ZFS as Root Filesystem

Introduction

This tutorial will show you how to install Funtoo on ZFS (rootfs). ZFS is designed to control entire disk , hence, guide reflecting only installing ZFS on whole disk and legacy boot. Installing on UEFI requires separate partition for /boot, formatted into FAT32 and not covered here, though, installation on UEFI is certainly possible. Guide also not describing anything related to encryption.

Introduction to ZFS

For a quick tour of ZFS and have a big picture of its common operations you can consult the page ZFS Fun.

Prerequisites

   Important

ZFS designed for 64-bit systems. We only recommending and supporting 64-bit platforms and installations!

Downloading the ISO (With ZFS)

In order to install Funtoo on ZFS, you will need an environment such as live media with ZFS tools provided.

Creating a bootable USB from ISO (From a Linux Environment)

After you download the iso, you can do the following steps to create a bootable USB: TBA.

Creating ZFS partitions

In this guide we are letting ZFS automatically partition the drive. We use "partition" term, so it would be more familiar for many.

Creating of pool

root #  zpool create -f -o ashift=12 -o cachefile=/tmp/zpool.cache -O normalization=formD -m none -R /mnt/funtoo -d -o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o feature@spacemap_histogram=enabled -o feature@enabled_txg=enabled -o feature@extensible_dataset=enabled -o feature@bookmarks=enabled rpool /dev/sda

   Note

ashift 12 is stands for 4-K block size hard drive used for installation. They are common nowadays.

Pool name is irrelevant, rpool used in this guide.

Create the zfs datasets

Create rootfs

root #  zfs create -o mountpoint=none rpool/ROOT
root #  zfs create -o mountpoint=/ rpool/ROOT/funtoo

Create optional datasets

Home

root #  zfs create -o mountpoint=/home rpool/HOME
root #  zfs create -o mountpoint=/root rpool/HOME/root

Portage directories

root #  zfs create -o mountpoint=none -o setuid=off rpool/FUNTOO
root #  zfs create -o mountpoint=/usr/portage -o atime=off rpool/FUNTOO/portage
root #  zfs create -o mountpoint=/usr/portage/distfiles rpool/FUNTOO/distfiles

Build directory

root #  zfs create -o mountpoint=/var/tmp/portage -o compression=lz4 -o sync=disabled rpool/FUNTOO/build

Installing Funtoo

Pre-Chroot

Go into the directory that you will chroot into
root # cd /mnt/funtoo

Make a boot folder and mount your boot drive
root # mkdir boot
root # mount /dev/sda1 boot

Now download and extract the Funtoo stage3 ...

   Note

It is trully recommended to use the current version and generic64. That reduces the risk of a broken build.

After successfull ZFS installation and successfull first boot, the kernel may be changed using the eselect profile set ... command. If you create a snapshot before, you may allways come back to your previous installation, with some simple steps ... (rollback your pool and in the worst case configure and install the bootloader again)

Once you've extracted the stage3, do a few more preparations and chroot into your new funtoo environment:

Bind the kernel related directories
root # mount -t proc none proc
root # mount --rbind /dev dev
root # mount --rbind /sys sys

Copy network settings
root # cp -f /etc/resolv.conf etc

Make the zfs folder in 'etc' and copy your zpool.cache
root # mkdir etc/zfs
root # cp /tmp/zpool.cache etc/zfs

Chroot into Funtoo
root # env -i HOME=/root TERM=$TERM chroot . bash -l

   Note

How to create zpool.cache file?

If no zpool.cache file is available, the following command will create one:

root # zpool set cachefile=/etc/zfs/zpool.cache tank

Install/PortageTree

Add filesystems to /etc/fstab

Before we continue to compile and or install our kernel in the next step, we will edit the /etc/fstab file because if we decide to install our kernel through portage, portage will need to know where our /boot is, so that it can place the files in there.

Edit /etc/fstab:

   /etc/fstab

# <fs>                  <mountpoint>    <type>          <opts>          <dump/pass>

/dev/sda1               /boot           ext2            defaults        0 2

# If you set up a swap partition, you will have to add this line as well
/dev/zvol/tank/swap     none            swap            defaults        0 0

Building kernel, initramfs and grub to work with zfs

Install genkernel and initial kernel build

We need to build a genkernel initially:

root # emerge genkernel

Build initial kernel (required for checks in sys-kernel/spl and sys-fs/zfs):
root # genkernel kernel --no-clean --no-mountboot 

Installing the ZFS userspace tools and kernel modules

Emerge No results. This package will bring in No results, and No results as its dependencies:

root # emerge zfs

Check to make sure that the zfs tools are working. The zpool.cache file that you copied before should be displayed.

root # zpool status
root # zfs list

   Note

If /etc/mtab is missing, these two commands will complaine. In that case solve that with:

root # grep -v rootfs /proc/mounts > /etc/mtab

Add the zfs tools to openrc.

root # rc-update add zfs-import boot
root # rc-update add zfs-mount boot
root # rc-update add zfs-share default
root # rc-update add zfs-zed default

If everything worked, continue.

Install GRUB 2

Install grub2:

root # echo "sys-boot/grub libzfs -truetype" >> /etc/portage/package.use
root # emerge grub

Now install grub to the drive itself (not a partition):

root # grub-install /dev/sda

Initial kernel build

Build now kernel and initramfs with --zfs

root # genkernel all --zfs --no-clean --no-mountboot --callback="emerge @module-rebuild"

Using the debian-sources, the following command may give better results:

root # genkernel all --zfs --no-clean --no-mountboot --callback="emerge spl zfs-kmod zfs"

Configuring the Bootloader

Using the genkernel you must add 'real_root=ZFS=<root>' and 'dozfs' to your params. Edit /etc/boot.conf:

   /etc/boot.conf

"Funtoo ZFS" {
        kernel kernel[-v]
        initrd initramfs-genkernel-x86_64[-v]
        params real_root=ZFS=tank/funtoo/root
        params += dozfs=force
}

The command boot-update should take care of grub configuration:

Install boot-update (if it is missing):
root # emerge boot-update

Run boot-update to update grub.cfg
root # boot-update

   Note

If boot-updatefails, try this:

root # grub-mkconfig -o /boot/grub/grub.cfg

Now you should have a new installation of the kernel, initramfs and grub which are zfs capable. The configuration files should be updated, and the system should come up during the next reboot.

   Note

If The luks integration works basically the same way.

Final configuration

Clean up and reboot

We are almost done, we are just going to clean up, set our root password, and unmount whatever we mounted and get out.

Delete the stage3 tarball that you downloaded earlier so it doesn't take up space.
root # cd /
root # rm stage3-latest.tar.xz

Set your root password
root # passwd
>> Enter your password, you won't see what you are writing (for security reasons), but it is there!

Get out of the chroot environment
root # exit

Unmount all the kernel filesystem stuff and boot (if you have a separate /boot)
root # umount -l proc dev sys boot

Turn off the swap
root # swapoff /dev/zvol/tank/swap

Export the zpool
root # cd /
root # zpool export tank

Reboot
root # reboot

   Important

Don't forget to set your root password as stated above before exiting chroot and rebooting. If you don't set the root password, you won't be able to log into your new system.

and that should be enough to get your system to boot on ZFS.

After reboot

Forgot to reset password?

System Rescue CD

If you aren't using bliss-initramfs, then you can reboot back into your sysresccd and reset through there by mounting your drive, chrooting, and then typing passwd.

Example:

root # zpool import -f -R /mnt/funtoo tank
root # chroot /mnt/funtoo bash -l
root # passwd
root # exit
root # zpool export -f tank
root # reboot

Create initial ZFS Snapshot

Continue to set up anything you need in terms of /etc configurations. Once you have everything the way you like it, take a snapshot of your system. You will be using this snapshot to revert back to this state if anything ever happens to your system down the road. The snapshots are cheap, and almost instant.

To take the snapshot of your system, type the following:

root # zfs snapshot -r tank@install

To see if your snapshot was taken, type:

root # zfs list -t snapshot

If your machine ever fails and you need to get back to this state, just type (This will only revert your / dataset while keeping the rest of your data intact):

root # zfs rollback tank/funtoo/root@install

   Important

For a detailed overview, presentation of ZFS' capabilities, as well as usage examples, please refer to the ZFS Fun page.

Troubleshooting

Starting from scratch

If your installation has gotten screwed up for whatever reason and you need a fresh restart, you can do the following from sysresccd to start fresh:

Destroy the pool and any snapshots and datasets it has
root # zpool destroy -R -f tank

This deletes the files from /dev/sda1 so that even after we zap, recreating the drive in the exact sector
position and size will not give us access to the old files in this partition.
root # mkfs.ext2 /dev/sda1
root # sgdisk -Z /dev/sda

Now start the guide again :).

Starting again reusing the same disk partitions and the same pool

If your installation has gotten screwed up for whatever reason and you want to keep your pole named tank than you should boou into the Rescue CD / USB as done before.

import the pool reusing all existing datasets:
root # zpool import -f -R /mnt/funtoo tank

Now you should wipe the previous installation off:

let's go to our base installation directory:
root # cd /mnt/funtoo

and delete the old installation: 
root # rm -rf *

Now start the guide again, at "Pre-Chroot"