arantius.com

Building a Gentoo Minimal LiveCD with ZFS support

2013-12-18 21:51 - Linux

Note, this is a 2023 rewrite of an article originally from 2013. See earlier versions at archive.org.

For several years now I've used LUKS to encrypt my linux root partition, and for almost as many years that root partition has been ZFS based. For confidence that I could always access this data, I wanted an independent boot media that can access such an encrypted ZFS data set. There's none that I know of which support both LUKS and ZFS, and it would also be nice for installing Gentoo as well, so I customized the Gentoo tooling for building their minimal install LiveCD, to add ZFS support. Here's how. (2023 update: The Gentoo LiveGUI USB Image does support ZFS, but launches a full X GUI and is a 3GiB download. I'm used to and prefer the minimal installation CD — especially for recovery scenarios &mdahs; and I still want to add ZFS support to it.)

Start with the releng (release engineering) tools, which contain Catalyst. This is the tool that builds several low level Gentoo artifacts. I'm doing this from a Gentoo system (in a VM). What we really want from the sources are the scripts (inputs to Catalyst) used to build the minimal install LiveCD.

# emerge dev-vcs/git
# cd /root
# git clone https://github.com/gentoo/releng.git

Now we have the source checked out. (I wrote this from commit 895edc32f02978414e208343e55d50c46d71b47a. If you're trying this and it doesn't seem to work, maybe try from that point.) We need to prepare further inputs:

# mkdir -p /var/tmp/catalyst/builds/default
# wget http://.../stage3-amd64-....tar.xz -O /var/tmp/catalyst/builds/default/stage3-amd64-openrc-latest.tar.xz

Start from Gentoo's downloads page. Use the amd64 stage3 URL in the wget step. We're about to run catalyst, so we install it. Plus it will depend on pixz (parallel xz) later, so install that too. Continue by running cataylst:

# emerge-webrsync
# emerge catalyst pixz
# catalyst -s latest

This packages your freshly sync'ed local portage tree for use by the coming steps. I've added several utilities that I think are useful for both installation and recovery and testing scenarios. Proceed to stage one:

# cat > stage1.sed
s#@REPO_DIR@#/root/releng#g
s#@TIMESTAMP@#latest#g
s#version_stamp: latest#version_stamp: YYYYMMDD.zfs#
/livecd.packages/ {
  a     app-editors/vim
  a     net-analyzer/netcat
  a     net-analyzer/mtr
  a     net-analyzer/nmap
  a     net-analyzer/tcpdump
  a     net-dns/bind-tools
  a     net-misc/telnet-bsd
  a     sys-apps/haveged
  a     sys-apps/hwinfo
  a     sys-apps/pv
  a     sys-apps/smartmontools
  a     sys-block/mbuffer
  a     sys-process/htop
}
^D
# sed -i -e "s/YYYYMMDD/$(date '+%Y%m%d')/" stage1.sed
# catalyst -f <(sed -f stage1.sed /root/releng/releases/specs/amd64/installcd-stage1.spec)

This will emerge over 200 packages inside a sandbox, it takes quite a while. (My time: around 160 minutes.)

Aside: as menitioned I'm doing this in a resource-constrained VM. Some of these packages (boost, gcc) are big. I typically want to keep RAM low in my VMs, to make snapshots fast and small. Be sure to have some swapspace to avoid OOM crashes while building big packages. I found 4G ram plus 1G swap was not enough. Adding a few more gigs of swap helped avoid OOM crashes.

Proceed to stage two:

# echo "GRUB_PLATFORMS='efi-64 efi-32 pc'" >> /etc/portage/make.conf
# emerge sys-apps/memtest86+ sys-boot/grub:2
# cat > stage2.sed
s#@REPO_DIR@#/root/releng#g
s#@TIMESTAMP@#latest#g
s#version_stamp: latest#version_stamp: YYYYMMDD.zfs#
s/latest.iso/YYYYMMDD.zfs.iso/
s/livecd-stage1-amd64-latest/livecd-stage1-amd64-YYYYMMDD.zfs/
/livecd.volid/s/Gentoo amd64 latest/YYYYMMDDzfs/
/boot.kernel.gentoo.config/ {
  a boot/kernel/gentoo/packages:
  a 	sys-fs/zfs
}
^D
# sed -i -e "s/YYYYMMDD/$(date '+%Y%m%d')/" stage2.sed
# catalyst -f <(sed -f stage2.sed /root/releng/releases/specs/amd64/installcd-stage2-minimal.spec)

This primarily builds the kernel and then the final ISO image (my time: around 50 minutes), which will be located at /var/tmp/catalyst/builds/default/install-amd64-minimal-YYYYMMDD.zfs.iso. You're done!

Finally, I'm making the fully built ISO image I created available for download. See this salient note on licensing. Download gentoo-install-amd64-minimal-20191116.zfs.iso; MD5 4f578aef8d2a0a643e4334ac16b36c8f; SHA1 f86702efeca5aef58249398f28f8e0e476857999. See comments below for the most up-to-date version.