The Funtoo Linux project has transitioned to "Hobby Mode" and this wiki is now read-only.
FLOP:Automatic FS Snapshots
Goal and context
Modern filesystem like BTRFS and ZFS supports a feature known as snapshot (some of them like NILFS2 uses that capability in a even more aggressive manner). Basically, a snapshot is nothing more than coherent "photograph" of a filesystem structures and data blocks pointed by them thus enabling the system administrator to see a filesystem in its *exact* state at the time the snapshot was taken even if some files have been deleted/changed/created since the snapshot creation.
Filesystems snapshotting is not a new feature as it is known for quite a long time in the Linux world especially by LVM users. However, one of the areas where filesystems like BTRFS shines is it uses a technique known as Copy-on-Write: while a LVM snapshot physically replicates the stripes (LVM is disconnected from the filesystems lying in the LVM logical volumes and can make no assumptions), CoW filesystems like BTRFS are clever enough to avoid unnecessary data blocks duplication. Thus, at the exact time a snapshot is taken there is no difference between a filesystem and its snapshot meaning the snapshot size is very small. As the time goes on and data blocks are modified, the filesystem will detect alterations attempt and will duplicate all the necessary data blocks to preserve the original ones. On the other side, old and obsolete snapshots must be deleted from time to time to free up the data blocks that have been "locked in the past".
Present proposal pretends to be the "big picture" of integrating snapshotting capabilities of modern CoW filesystems like BTRFS/ZFS in portage. Such an integration will enable sysamdins to easily recover from a disaster coming from a clunky update (or worse, a silent and catastrophic libpng-like update). For the project credibility, it mitigiates the degree of an major issue coming out of the blue. For Funtoo developers, the gain is they do not have to replicate their live filesystem in a chroot or play with virtual machines or jailed tests environments of any kind. In the case of something go wrong, it is simple to do a filesystem rollback just as if nothing has ever happened.
First idea
Quick functional description: the sysadmin want to install/update/delete/clean up dependencies and use the appropriate command (emerge/emerge -u/emerge -C/emerge --depclean...):
- The command starts by doing a snapshot of system directories and then installs or remove all of the specified packages/dependencies (transactional operation).
- If something goes wrong and the user resume the process (emerge --skip-first / emerge --resume) no snapshot is taken and the process goes on as it actually does
- If the something goes wrong and the user then chooses to manually emerge some packages by hand instead of doing, let's say, emerge -uaDN @world, another snapshot is taken.
This guarantee that an "image" of the filesystem is kept before each transaction (installation/removal of several packages in a single portage command). When the sysadmin is happy, he removes the now unneeded snapshots by its own.
Behavior control
As the Funtoo philosophy wants, the snapshotting capabilities integration in portage is always optional and the behaviour is governed via FEATURES (e.g. FEATURES=".... fs-snapshots"). As we can't predict what the future is made of, it is necessary to have a flexible control on the how the filesystem manipulation tools used to produce snapshots. This can be done with environment variables like:
BTRFS_PORTAGE_SNAPSHOT_CMD="btrfs subvolume snapshot " ZFS_PORTAGE_SNAPSHOT_CMD="zfs snapshot "
By default, portage will use a YYYYMMDDTHHMMSS for the snapshot name thus doing something like:
- btrfs subvolume snapshot /mountpoint /root/mountpoint-YYYYMMDDTHHMMSS (in the case of / only the YYYYMMDDTHHMMSS portion is kept)
- zfs snapshot pool/dataset@YYYYMMDDTHHMMSS (if the dataset part does not exists only the YYYYMMDDTHHMMSS portion is kept)
Approach #1: manual configuration
Portage scans a file (let says /etc/portage/fssnaps to be compliant with coming 1.0 profiles and unified configuration files) which describes what are the subvolumes to snapshot before doing a transaction:
/ /usr
Approach #2: automatic configuration
Portage scans what lies in /etc/fstab that has been defined as btrfs or zfs or anything of the same kind. If an entry describe something is located in a "well-known" directory (e.g. /lib, /usr ....) it is included in the subvolumes list that have been to be snapshotted. To allow a bit of flexibility, a list of exclusion can be specied via a variable like EXEMPT_FROM_SNAPHOT="/lib /var". This is dangerous but could be needed in some contexts.