The Funtoo Linux project has transitioned to "Hobby Mode" and this wiki is now read-only.
Difference between revisions of "User:Mrl5/Btrfs"
m |
m (editorial changes) |
||
| Line 103: | Line 103: | ||
Lets create children of the top level subvolume (ID 5). We will have: | Lets create children of the top level subvolume (ID 5). We will have: | ||
* {{c|@funtoo}} - | * {{c|@funtoo}} - used for our Funtoo Linux root filesystem: {{c|/}} | ||
* {{c|@home}} - | * {{c|@home}} - it will serve as {{c|/home}} that e.g. can be shared with other OS | ||
* {{c|snapshots}} - here backup snapshots will be stored | * {{c|snapshots}} - here backup snapshots will be stored | ||
| Line 120: | Line 120: | ||
}} | }} | ||
{{Note|This layout allows creation of granular snapshots, so that e.g. your {{c|/home}} data | {{Note|This layout allows creation of granular snapshots, so that e.g. your {{c|/home}} data don't get lost or overwritten during a roll back of {{c|/}}.}} | ||
Now lets reproduce steps from [[Install/Download and Extract Stage3]] and populate {{c|@funtoo}} subvolume with Funtoo Linux Stage3 | Now lets reproduce steps from [[Install/Download and Extract Stage3]] and populate {{c|@funtoo}} subvolume with Funtoo Linux Stage3 | ||
| Line 154: | Line 154: | ||
}} | }} | ||
At this point we can stop working on the top level subvolume (ID 5) and mount directly our {{c|@funtoo}} subvolume | At this point we can stop working on the top level subvolume (ID 5) and instead mount directly our {{c|@funtoo}} subvolume. | ||
{{console|body= | {{console|body= | ||
| Line 169: | Line 169: | ||
{{Note|Nested subvolumes are not going to be a part of snapshots created from their parent subvolume. So one typical reason is to exclude certain parts of the filesystem from being snapshot.}} | {{Note|Nested subvolumes are not going to be a part of snapshots created from their parent subvolume. So one typical reason is to exclude certain parts of the filesystem from being snapshot.}} | ||
Let's create a separate subvolumes for {{c|/tmp}}, {{c|/var/tmp}}, {{c|/var/cache}}, {{c|/var/log}}, {{c|/var/git}} and disable Copy-On-Write there. | Let's create a separate nested subvolumes for {{c|/tmp}}, {{c|/var/tmp}}, {{c|/var/cache}}, {{c|/var/log}}, {{c|/var/git}} and disable Copy-On-Write there. | ||
{{Warning|The {{c|/var}} directory as a whole must not be a separate subvolume. The reason is that portage uses {{c|/var/db/pkg}} to | {{Warning|The {{c|/var}} directory as a whole must not be a separate subvolume. The reason is that portage uses {{c|/var/db/pkg}} to manage currently installed packages. Otherwise the consistency wouldn't be preserved when restoring snapshot of {{c|/}}.}} | ||
{{console|body= | {{console|body= | ||
| Line 186: | Line 186: | ||
}} | }} | ||
You might also want to "split" | If you don't have {{c|/boot}} under separate block device (e.g. {{c|/dev/sda1}}) then you should also create a separate nested subvolume for it. You might also want to "split" other areas which are "complete" and/or "consistent" in themselves. Examples would be {{c|/var/www}}, or {{c|/var/lib/postgresql}}, which are usually more or less "independent" of the other other parts of system. | ||
{{Note|Nested subvolumes | {{Note|Nested subvolumes become automatically present if their parent subvolume is mounted.}} | ||
== /etc/fstab == <!--T:22--> | == /etc/fstab == <!--T:22--> | ||
| Line 207: | Line 207: | ||
== Wrap up == <!--T:29--> | == Wrap up == <!--T:29--> | ||
{{Important|It is recommended to run {{c|btrfs scrub}} once in a while.}} | {{Important|It is recommended to run {{c|btrfs scrub}} once in a while. E.g. every week}} | ||
Scrub is the online check and repair functionality that verifies the integrity of data and metadata, assuming the tree structure is fine. You can run it on a mounted file system; it runs as a background process during normal operation. | Scrub is the online check and repair functionality that verifies the integrity of data and metadata, assuming the tree structure is fine. You can run it on a mounted file system; it runs as a background process during normal operation. | ||
Revision as of 06:36, August 30, 2022
Btrfs is a file system based on the copy-on-write (COW) principle, initially designed at Oracle Corporation for use in Linux. The development of btrfs began in 2007, and since August 2014 the file system's on-disk format has been marked as stable.
The Funtoo Linux project recommends btrfs as a next-generation filesystem, particularly for use in production.
Btrfs is intended to address the lack of pooling, snapshots, checksums, and integral multi-device spanning in Linux file systems.
Installation
Enabling btrfs support is as simple as enabling the btrfs mix-in and running a world update:
root # epro mix-in +btrfs root # emerge -uDN @world
Btrfs is now ready for use.
Btrfs Concepts
Btrfs can be used to manage the physical disks that it uses, and physical disks are added to a Btrfs volume. Then, BTRFS can create subvolumes from the volume on which files can be stored.
Unlike traditional Linux filesystems, btrfs filesystems will allocate storage on-demand from the underlying volume.
In the btrfs world, the word volume corresponds to a storage pool (ZFS) or a volume group (LVM).
- devices - one or multiple underlying physical volumes.
- volume - one large storage pool comprised of all space of the devices and can support different redundancy levels
- subvolumes - these are what get mounted and you store files in.
- snapshots - a read-only copy of a subvolume at a given point in time and/or read-write copy of a subvolume in time (aka clone).
Creating a Volume
To create a basic btrfs volume, you will need an extra empty disk. Perform the following steps:
root # mkfs.btrfs /dev/sdxy btrfs-progs v4.17.1 See http://btrfs.wiki.kernel.org for more information. Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication. Performing full device TRIM /dev/sdj (223.57GiB) ... Label: (null) UUID: d6bcba6e-8fd5-41fc-9bb4-79628c5c928c Node size: 16384 Sector size: 4096 Filesystem size: 223.57GiB Block group profiles: Data: single 8.00MiB Metadata: single 8.00MiB System: single 4.00MiB SSD detected: yes Incompat features: extref, skinny-metadata Number of devices: 1 Devices: ID SIZE PATH 1 223.57GiB /dev/sdxy
/dev/sdxy should be an unused disk. You may need to use the following command if this disk contains any pre-existing data on it:
root # mkfs.btrfs -f /dev/sdxy
Now you can mount the created volume as you would mount any other linux filesystem.
root # mkdir /mnt/btrfs-top-level root # mount /dev/sdxy /mnt/btrfs-top-level root # mount ... /dev/sdxy on /mnt/btrfs-top-level type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/)
It is recommended that nothing is stored directly on this top-level volume (ID 5) root directory.
Creating Subvolumes
From now on, we will describe how to set up btrfs as Funtoo Linux root filesystem. However the concepts are generic and can be re-used for other usecases.
Changing subvolume layouts is made simpler by not using the top-level subvolume (ID 5) as /
Btrfs has a concept of subvolumes. Subvolume is an independently mountable POSIX filetree (but not a block device). There are several basic schemas to layout subvolumes (including snapshots) as well as mixtures thereof. The one described here was influenced by this SLES docs.
Lets create children of the top level subvolume (ID 5). We will have:
@funtoo- used for our Funtoo Linux root filesystem:/@home- it will serve as/homethat e.g. can be shared with other OSsnapshots- here backup snapshots will be stored
(later we will also create nested subvolumes)
root # cd /mnt/btrfs-top-level root # btrfs subvolume create @funtoo root # btrfs subvolume create @home root # btrfs subvolume create snapshots root # btrfs subvolume list /mnt/btrfs-top-level ID 256 gen 322336 top level 5 path @funtoo ID 257 gen 322338 top level 5 path @home ID 258 gen 322275 top level 5 path snapshots
This layout allows creation of granular snapshots, so that e.g. your /home data don't get lost or overwritten during a roll back of /.
Now lets reproduce steps from Install/Download and Extract Stage3 and populate @funtoo subvolume with Funtoo Linux Stage3
root # cd @funtoo root # wget https://build.funtoo.org/next/x86-64bit/generic_64/stage3-latest.tar.xz root # tar --numeric-owner --xattrs --xattrs-include='*' -xpf stage3-latest.tar.xz
The default Subvolume
Changing the default subvolume with btrfs subvolume default will make the top level of the filesystem accessible only when subvol or subvolid mount options are specified
When we mount a btrfs block device without specifying a subvolume the default one is used. In order to check which subvolume is currently the default one run
root # btrfs subvolume get-default /mnt/btrfs-top-level ID 5 (FS_TREE)
For the convenience lets make our @funtoo subvolume as the default one. It's good to double check the subvolume ID first. Let's use btrfs subvolume show command this time
root # btrfs subvolume show /mnt/btrfs-top-level/@funtoo ... Subvolume ID: 256
Now we can make this subvolume as a default one
root # btrfs subvolume set-default 256 /mnt/btrfs-top-level root # btrfs subvolume get-default /mnt/btrfs-top-level ID 256 gen 322336 top level 5 path @funtoo
At this point we can stop working on the top level subvolume (ID 5) and instead mount directly our @funtoo subvolume.
root # cd /mnt root # umount /mnt/btrfs-top-level root # mkdir /mnt/funtoo root # mount /dev/sdxy /mnt/funtoo root # ls -la /mnt/funtoo root # cd /mnt/funtoo
Nested Subvolumes
Nested subvolumes are not going to be a part of snapshots created from their parent subvolume. So one typical reason is to exclude certain parts of the filesystem from being snapshot.
Let's create a separate nested subvolumes for /tmp, /var/tmp, /var/cache, /var/log, /var/git and disable Copy-On-Write there.
The /var directory as a whole must not be a separate subvolume. The reason is that portage uses /var/db/pkg to manage currently installed packages. Otherwise the consistency wouldn't be preserved when restoring snapshot of /.
root # cd /mnt/funtoo root # dirs="var/cache var/log var/git var/tmp tmp" root # for dir in $dirs; do [ -d "$dir" ] && mv -v "$dir" "${dir}.old" btrfs subvolume create "$dir" chattr +C "$dir" [ -d "${dir}.old" ] && cp -a --reflink=never "${dir}.old/." "$dir" && rm -r "${dir}.old" done root # chown portage:portage var/git root # chmod 1777 tmp
If you don't have /boot under separate block device (e.g. /dev/sda1) then you should also create a separate nested subvolume for it. You might also want to "split" other areas which are "complete" and/or "consistent" in themselves. Examples would be /var/www, or /var/lib/postgresql, which are usually more or less "independent" of the other other parts of system.
Nested subvolumes become automatically present if their parent subvolume is mounted.
/etc/fstab
To automatically mount our new @funtoo and @home volumes after reboot we need to modify /etc/fstab
/dev/sdxy / btrfs subvolid=256,defaults 0 0 /dev/sdxy /home btrfs subvolid=257,defaults 0 0
According to btrfs docs, within a single file system, it is not possible to mount some subvolumes with nodatacow and others with datacow. The mount option of the first mounted subvolume applies to any other subvolumes.
Snapshots
todo
Wrap up
It is recommended to run btrfs scrub once in a while. E.g. every week
Scrub is the online check and repair functionality that verifies the integrity of data and metadata, assuming the tree structure is fine. You can run it on a mounted file system; it runs as a background process during normal operation.
To start a (background) scrub on the filesystem which contains /
root # btrfs scrub start / scrub started on /, fsid 40f8b94f-07ee-4f7e-beb1-8e686abc246d (pid=5525)
To check the status of a running scrub
root # btrfs scrub status / UUID: 40f8b94f-07ee-4f7e-beb1-8e686abc246d Scrub started: Tue Aug 30 00:38:54 2022 Status: running Duration: 0:00:15 Time left: 0:00:34 ETA: Tue Aug 30 00:39:44 2022 Total to scrub: 149.06GiB Bytes scrubbed: 44.79GiB (30.04%) Rate: 2.99GiB/s Error summary: no errors found
You should now be at the point where you can begin to use btrfs for a variety of tasks. While there is a lot more to btrfs than what is covered in this short introduction, you should now have a good understanding of the fundamental concepts on which btrfs is based.