arantius.com

Building a Gentoo Minimal LiveCD with ZFS support

2013-12-18 21:51 - Linux

Note, this is a 2019 rewrite of an article originally from 2013. See the original at archive.org or locally. A bit more historical context is only there now.

For several years now I've used LUKS to encrypt my linux root partition, and for almost as many years that root partition has been ZFS based. For confidence that I could always access this data, I wanted an independent boot media that can access such an encrypted ZFS data set. There's none that I know of which support both LUKS and ZFS, and it would also be nice for installing Gentoo as well, so I customized the Gentoo tooling for building their minimal install LiveCD, to add ZFS support. Here's how.

Start with the releng (release engineering) tools, which contain Catalyst. This is the tool that builds several low level Gentoo artifacts. I'm doing this from a Gentoo system (in a VM). What we really want from the sources are the scripts (inputs to Catalyst) used to build the minimal install LiveCD.

# emerge dev-vcs/git
# cd /root
# git clone https://github.com/gentoo/releng.git

Now we have the source checked out. (I wrote this from commit a04c9746e3dda7ecfc19b09746d7de21a7b76006. If you're trying this and it doesn't seem to work, maybe try from that point.) We need to prepare further inputs:

# mkdir -p /var/tmp/catalyst/builds/default
# wget http://.../stage3-amd64-....tar.xz -O /var/tmp/catalyst/builds/default/stage3-amd64-latest.tar.xz

Start from Gentoo's downloads page. Use the amd64 stage3 URL in the wget step. We're about to run catalyst, so we install it. Plus it will depend on pixz (parallel xz) later, so install that too. Continue by running cataylst:

# emerge-webrsync
# emerge catalyst pixz
# catalyst -s latest

This packages your freshly sync'ed local portage tree for use by the coming steps. I've added several utilities that I think are useful for both installation and recovery and testing scenarios. Proceed to stage one:

# cat > ~/stage1.sed
s#@REPO_DIR@#/root/releng#g
s#@TIMESTAMP@#latest#g
s#version_stamp: latest#version_stamp: 20200605.zfs#
/livecd.packages/ {
  a     app-editors/vim
  a     net-analyzer/netcat
  a     net-analyzer/mtr
  a     net-analyzer/nmap
  a     net-analyzer/tcpdump
  a     net-dns/bind-tools
  a     net-misc/telnet-bsd
  a     sys-apps/haveged
  a     sys-apps/hwinfo
  a     sys-apps/pv
  a     sys-apps/smartmontools
  a     sys-block/mbuffer
  a     sys-process/htop
}
^D
# catalyst -f <(sed -f ~/stage1.sed /root/releng/releases/specs/amd64/installcd-stage1.spec)

This will emerge over 200 packages inside a sandbox, it takes quite a while. (My time: 160 minutes.)

Aside: one of these packages is dev-libs/boost. I've (of course) got parallelism set as high as I can to reduce compile times, but (at least) this package will OOM and fail if you(r VM) doesn't have enough available RAM, with many compile processes running at once. I typically want to keep RAM low in my VMs, to make snapshots fast and small, so adding swap helps avoid OOM crashes.

Proceed to stage two:

# echo -n > ~/releng/releases/portage/isos/package.accept_keywords/zfs
# echo "GRUB_PLATFORMS='efi-64 efi-32 pc'" >> /etc/portage/make.conf
# emerge sys-apps/memtest86+ sys-boot/grub:2
# cat > ~/stage2.sed
s#@REPO_DIR@#/root/releng#g
s#@TIMESTAMP@#latest#g
s#version_stamp: latest#version_stamp: 20200605.zfs#
s/latest.iso/20200605.zfs.iso/
s/livecd-stage1-amd64-latest/livecd-stage1-amd64-20200605.zfs/
/livecd.volid/s/Gentoo amd64 latest/20200605zfs/
/boot.kernel.gentoo.config/ {
  a boot/kernel/gentoo/packages:
  a 	sys-fs/zfs
}
^D
# catalyst -f <(sed -f ~/stage2.sed /root/releng/releases/specs/amd64/installcd-stage2-minimal.spec)

This primarily builds the kernel and then the final ISO image (my time: around 50 minutes), which will be located at /var/tmp/catalyst/builds/default/install-amd64-minimal-20191116.zfs.iso. You're done!

Finally, I'm making the fully built ISO image I created (built with ZFS version 0.7.13 — the latest stable version in portage) available for download. See this salient note on licensing. Download gentoo-install-amd64-minimal-20191116.zfs.iso; MD5 4f578aef8d2a0a643e4334ac16b36c8f; SHA1 f86702efeca5aef58249398f28f8e0e476857999. See comments below for the most up-to-date version.