Update 2023: Thanks to bug fixes in the dependent projects, this process has become simpler over time. I have used struck-through text to denote information that is no longer true, and steps that are no longer required.

 

Gentoo user FearedBliss wrote the de-facto guide for installing Gentoo to boot up from a ZFS partition, alongside a tool to build an init ramdisk that support ZFS partitions. Unfortunately their init ramdisk does not fully support ZFS partitions on NVME SSD drives (something to do with how they are represented in /dev/disks/), so it only imports the root pool at system boot time. Happily, top dude Guy Robot wrote a pair of excellent Gentoo/ZFS/NVME SSD installation articles that incorporate the extra steps required to patch the init ramdisk to fix this and import all the ZFS pools you have on disk.

Time passes, and the time comes to upgrade my kernel, meaning an init ramdisk rebuild is also required. Blending together the Gentoo Kernel Upgrade Guide with Guy Robot’s articles I have synthesized the following set of kernel upgrade steps. Please note this is an upgrade article only and assumes you already have a working system that boots from a ZFS partition. If you’re starting from scratch please follow Guy Robot’s articles linked above.
If you’re already there, then there are two use cases for this article:

a. Full kernel upgrade – start from step 1.

b. ZFS has been updated and you only need to update the kernel module so that its version stays in step with the userland tools – start from step 11. If however GCC was upgraded since you built the kernel, you’ll need to also rebuild your current kernel, so do steps 4 then 9, then from 11 onwards.

 

  1. Obviously enough, start with updating the relevant packages (sys-kernel/linux-firmware, sys-kernel/gentoo-sources, sys-fs/zfs, sys-fs/zfs-kmod; note that since v0.8 ZFS no longer relies upon sys-kernel/spl so this can be unmerged). In my case I just went for a complete emerge update @world:

    emaint -a sync
    emerge –update –deep –changed-use –keep-going @world

  2. Ensure that the file /etc/portage/savedconfig/sys-kernel/linux-firmware points to (or is) a file containing the list of only the firmware blobs to compile into the kernel (the remainder should be commented out or deleted). If you are not doing so already, I recommend setting the savedconfig USE variable for the sys-kernel/linux-firmware package so that future emerges retain your config; your set of required firmware is not likely to change from one kernel build to the next.
  3. The /usr/src/linux symlink points to the kernel source package directory. This needs to be updated to point to the location we wish to use for this build. Emerging the gentoo-sources package with USE=symlink would do this for us automatically, but there are other packages out there that need to know where to find our current source, so it’s best that we manage this manually. The Gentoo way is to use the eselect command:

    eselect kernel list
    eselect kernel set <the index number of the new source dir>

  4. We’ll now do some work in the kernel dir. Change into it explicitly to ensure picking the new location of the symlink:

    cd /usr/src/linux

  5. Copy the currently-running kernel’s config to the new kernel’s source dir to use it as a basis:

    cp /boot/kernels/<currently-running kernel dir>/config /usr/src/linux/.config

  6. We already have the NVME support enabled from the original install (see Guy Robot’s article for details), so next we need to update the config to match the new kernel version (i.e. bring in options that are new since the last time we built it). In the past I have used make olddefconfig to do this, which supposedly keeps your existing settings and selects sane defaults for any new settings but I got burned badly by it once, so now I use:

    make oldconfig

    which instead prompts you to select values for any new settings that exist in the new kernel version and not in your old one; with a bit of searching it’s possible to work out what the right option should be for your hardware.

  7. Processor manufacturers periodically release microcode patches to fix bugs. Gentoo includes several methods to identify which ones apply to your processor and to build them into the kernel. The generic instructions are here and the Intel-specific procedure is here, the final steps of which involve using…
  8. …the graphical kernel config tool – maybe in browsing through it you’ll additionally find options you now want to enable, or perhaps disable:

    make menuconfig

    And do remember to save your changes before quitting the tool.

  9. Right, we’re ready to build the kernel. This takes around 45 minutes on my Intel i7-7600U. Set the -j parameter to the number of processor cores you have plus one:

    make -j5

  10. Rebuild the kernel modules (zfs-kmod and any others you’re using) against the new sources:

    emerge @module-rebuild

  11. Install the kernel and modules (they aren’t actually used until we reconfigure grub to know about them, so not to worry):

    make modules_install
    make install

  12. The modules get installed to /lib/modules/<kernel version>, and the kernel and it’s ancillary files will be installed to /boot. For neatness and the ability to roll back in case of problems we will keep the existing kernel alongside the new, so create subdirs in the form /boot/kernels/<kernel version dir> and move/rename vmlinuz, config and System.map from /boot to there.
  13. Now we need to build an init ramdisk that’ll import all our ZFS pools at boot time. First step here is to use Fearedbliss’ tool to create one that is Gentoo/ZFS-friendly (N.B. I’m assuming you’ve already added the necessary Portage overlay to your repos.conf, as explained in Guy Robot’s article and emerge’d bliss-initramfs):
    If you are using the systemd init then you can go right ahead, but if you’re using OpenRC then edit the settings to change udevProvider to /sbin/udevd Update 2022: It no longer matters whether you’re using OpenRC or systemd  init on Gentoo, since both now rely upon systemd’s default location for the udev daemon.

    bliss-initramfs -k <new kernel version>

    When it has finished, copy the resulting initrd-<kernel version> file from /usr/src/linux to /boot/kernels/<kernel version dir>/initrd

  14. If you have a single zpool on your drive, you can skip this step. If you need to import multiple pools at system boot time then read on. The initrd created by bliss-initramfs will only import the root pool. To import any other pools (/boot, in my case) it is necessary to edit the init script within the init ramdisk to have it mount additional zpools explicitly. Follow the instructions in Guy Robot’s article starting from “We now need to modify the initramfs.” and up until “Now you have a custom initramfs…”, however note:
    Update 2023: As a more fine-grained solution, instead of completely replacing the three commands in the init script with explicit pool imports, as suggested in GuyRobot’s article, insert the following new line between MountRoot and MountUserIfNeeded just to add in the additional pool imports you need:

    PrepForSafeZfsCommand && zpool import boot -R /mnt/root

  15. Next we need to let grub know about our new kernel so we can select it at boot. Edit /boot/grub/grub.cfg to set the timeout to 3 (temporarily until we’re happy with the new kernel) and add a menu item like the below, updating the kernel version number appropriately:

    menuentry “Gentoo – 4.19.52-gentoo-FC.01” {
    linux /@/kernels/4.19.52-gentoo-FC.01/vmlinuz root=rpool/ROOT/gentoo resume=/dev/nvme0n1p3 by=id elevator=noop quiet logo.nologo refresh
    initrd /@/kernels/4.19.52-gentoo-FC.01/initrd
    }

    A few notes about the kernel command line:

    • root= and by= are for the Bliss initramfs, your root value might differ
    • resume= is the hibernation partion
    • refresh= is a Bliss parameter to instruct it to flush the ZFS cache at the next reboot
  16. Take a deep breath and test your work:

    reboot

  17. We do not want the ZFS cache flushed at every boot, so go back in and edit grub.cfg to remove the ‘refresh’  option.
  18. Once you are happy with your new kernel, you will want to ensure it’s source code sticks around because some other packages depend on it when building. To prevent it being unmerged when new versions are released, preserve the gentoo-sources package version by adding it to @world:

    emerge –noreplace sys-kernel/gentoo-sources-<version>

  19. You can then do the opposite for the ‘outgoing’ kernel, so that at the next emerge @world it will be removed:

    emerge –deselect =sys-kernel/gentoo-sources-<version>

  20. You’ll need to edit grub.cfg one more time to remove the menu entry for the old kernel version and set timeout back to 0 so that you don’t see the menu on booting.
  21. Finally, delete the old kernel’s subfolders under /boot/kernels and /usr/src. You’re done! Enjoy new kernel goodness :)