Author: James Powell Date: 2014-08-14 License: MIT Synopsis: Using ZFS with Linux From Scratch =============================================================================== The MIT License Copyright (c) 2014 James Powell Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. =============================================================================== Description: This document is provided to demonstrate how to effectively use the ZFS File system using ZFS-On-Linux as the primary /(root) partition. ZFS is a highly advanced file system with many features not normally found on GNU/Linux, including the BtrFS file system which is still considered experimental and unstable, even though it's been supported in the Linux kernel since 2.6.xx. ZFS is licensed under the CDDL, which unfortunately in NOT compatible with the GNU's GPL license the Linux kernel is under. This means that you can not get a premade GNU/Linux distribution with built-in ZFS support. Linux From Scratch, however, is an odd exemption from this. Because LFS is not a prebuilt distribution, LFS falls outside the legal status of GPL and CDDL problematics into a sort of grey area. ZFS-On-Linux was developed by Lawrence Livermoore National Laboratories for their own usage under the umbrella of the OpenZFS Project sponsored by the Illumos Foundation for the Illumos Kernel and the OpenIndiana operating system. The OpenZFS project is under a full disclosure license agreement and OpenZFS is a fully open source project. The only implementation not sponsored or used by OpenZFS is the Oracle-ZFS implementation which is closed source, and has been developed separately from OpenZFS. ZFS-On-Linux currently uses zpool version 5000 and fs version 5. This software hint is considered experimental, so please use at your own risk on a rightful system you are willing to sacrifice if needed. =============================================================================== Prerequisites: You must first run and assign partitioning using fdisk, cfdisk, gdisk or any other partitioning tool. Make sure you assign the following such as this example: /dev/sda1: Primary Partition - 83 Partition type (Linux) - At least 512KB in size. This will serve at the primary /boot partition to where you will be installing the Linux kernel and boot loader. /dev/sda2: Primary Partition - BF Partition Type (Solaris) - Remaining disk space. This will serve as the ZFS partition space as well as the swap space. You must now have obtained the Solaris Porting Layer (SPL) and ZFS-On-Linux packages for your distribution, and have installed them. NOTE: At this point, it does not matter whether ZFS is built-into or installed as a module, you just need to have it installed. =============================================================================== Hint: For the Remainder of the hint any partitions used will be examples only. Please adjust partition values as needed. First you must create a zpool to setup where the zfs-root will go. To perform this action run the following command: zpool create -m none zfs-root /dev/sda2 This will create a blank zpool in the Solaris assigned disk space for ZFS to work with. At this point, the partition is very useless for LFS, so we'll need to make some adjustments to let LFS self manage things. Let's now set the mountpoint. First run this command: zfs get mountpoint zfs-root The result should be at first it shows ZFS has no mountpoint. Because we want the native mount tools to work with ZFS rather than the proprietary mount tools we'll change the mountpoint: zfs set mountpoint=legacy zfs-root Legacy allows ZFS to be mounted and unmounted using the traditional mount and umount commands. If you would like to see all the options of your zfs partition run: zfs get all zfs-root Now let's mount our brand new ZFS partition. mount -v zfs-root -t zfs $LFS Just like in Chapter 2: Mounting the New Partition, this will mount the ZFS partition normally. To unmount it, simply run: umount -v $LFS And the partition is unmounted just like a normal EXT*, JFS, or ReiserFS file system. Now as normal, create a boot partition: mkfs.ext4 /dev/sda1 ZFS isn't always boot friendly with various operating systems. Some like FreeBSD can boot to ZFS, but not all can. Because various Linux bootloaders like Grub don't always support every single file system, unless rebuilt for it, which you can do with Grub, you'll need a boot partition. ZFS doesn't allow swapfiles, but you can use a ZFS volume as swap. Let's create a swap space now using a zvol zfs create -V 8G -b $(getconf PAGESIZE) \ -o primarycache=metadata \ -o sync=always \ -o com.sun:auto-snapshot=false zfs-root/swap In truth, you should not require more than 8GB swap space on any computer, for any reason of usage. However, unlike a traditional file system, you can always use the zfs tools to add or reduce swap space as needed. Now we'll set the swap up like normal but using the zvol as the target: mkswap -f /dev/zvol/zfs-root/swap Now let's turn on the swap partition: swapon /dev/zvol/zfs-root/swap Now you should be able to build your system normally. When unmounting everything should remount as normal, if not, that means trouble happened and you should check your zpool status such as: zpool status zfs-root When you set your fstab, you should now take care about how you setup the ZFS parition and swap. ZFS uses these types of entries: zfs-root / zfs defaults,atime,dev,exec,suid,xattr,nomand,zfsutil 0 0 /dev/zvol/zfs-root/swap none swap discard 0 0 /dev/sda1 /boot ext4 defaults 0 2 This will setup the swap with discard feature. This will reduce fragmentation levels in the swap area if it's not full. ZFS supports a vast deal of options for the file system, as you can see, these are the recommended defaults. Now when you first boot a system with ZFS, you'll notice that the mount tools may complain about fsck.zfs missing. To be honest, there is no fsck.zfs utility period. ZFS has it's own utility, but it's best used offline. To make sure you don't get complaints on boot, let's make a stubfile: cat > /sbin/fsck.zfs << "EOF" #!/bin/sh exit 0 "EOF" That should keep the error read outs to a minimum when booting. Now let's talk about the kernel. Becasue I, personally, recommend you use a kernel with built-in modules, this next step will focus on just that. Let's first get the SPL (Solaris Porting Layer) sources: http://archive.zfsonlinux.org/downloads/zfsonlinux/spl/spl-0.6.3.tar.gz and the ZFS sources: http://archive.zfsonlinux.org/downloads/zfsonlinux/zfs/zfs-0.6.3.tar.gz Save these into /sources where you've stored everything else. Now unpack the SPL and ZFS sources, and we'll start with getting SPL ready: First go to the kernel sources and run the following against an existing kernel source that's been configured: make prepare scripts This should prepare the kernel for a source merge. Now change to the spl directory and get it prepped. ./configure \ --prefix=/ \ --libdir=/lib \ --includedir=/usr/include \ --datarootdir=/usr/share \ --enable-linux-builtin=yes \ --with-linux=/usr/src/linux-3.15.6 \ --with-linux-obj=/usr/src/linux-3.15.6 ./copy-builtin /usr/src/linux-3.15.6 make make install This will directly merge SPL into the kernel source, and build a few libraries. Now let's merge in ZFS: ./configure \ --prefix=/ \ --libdir=/lib64 \ --includedir=/usr/include \ --datarootdir=/usr/share \ --enable-linux-builtin=yes \ --with-linux=/usr/src/linux-3.15.6 \ --with-linux-obj=/usr/src/linux-3.15.6 \ --with-spl=/root/src/spl-0.6.3 --with-spl-obj=/root/src/spl-0.6.3 ./copy-builtin /usr/src/linux-3.15.6 make make install This will merge ZFS into the kernel and built what amounts to the zfsprogs package... if it did exist. Now run make menuconfig. Under the root directory you'll see SPL support as a new option in the menu. Now enable it as [*] for Built-in. Now open the File-Systems menu and you'll see ZFS in there too. Again, enable it [*] for built-in. Now build and install your kernel normally. Before you reboot, you'll need to create a hostid. To do this, you'll need to generate it but there is not utility to do so, so use this instead: cd /sources mkdir -v hostid cd hostid cat > writehostid.c << "EOF" #include #include #include int main() { int res; res = sethostid(gethostid()); if (res != 0) { switch (errno) { case EACCES: fprintf(stderr, "Error! No permission to write the" " file used to store the host ID.\n" "Are you root?\n"); break; case EPERM: fprintf(stderr, "Error! The calling process's effective" " user or group ID is not the same as" " its corresponding real ID.\n"); break; default: fprintf(stderr, "Unknown error.\n"); } return 1; } return 0; } "EOF" Now compile it with: gcc -v writehostid.c -o writehostid And run the utility: ./writehostid This will generate a proper hostid file. If all goes well, on the first system boot, you'll boot into ZFS just like any other file system, but to get subsequent reboots to work without issue we'll need to create a cache file for the zpool: zpool set cachefile=/etc/zfs/zpool.cache zfs-root Now everything should work as intended. Congradulations on your new file system, and enjoy. If you'd like to learn more information on ZFS and it's capabilities which are beyond the scope of this hint, visit: http://en.wikipedia.org/wiki/ZFS =============================================================================== Acknowledgements: I'd like to thank my partners in crime Keith Hedger and Arthur Radley for being awesome teammates in our work in GNU/Linux. I'd also like to thank SlackWiki and ArchWiki for their awesome coverage of ZFS on Linux. =============================================================================== Changelog: 2014-08-14 Version - 0.1 - Initial Draft and Public Release ===============================================================================