Building a Gentoo Minimal LiveCD with ZFS support

2013-12-18 21:51 - Linux

Note, this is a 2019 rewrite of an article originally from 2013. See the original at archive.org or locally. A bit more historical context is only there now.

For several years now I've used LUKS to encrypt my linux root partition, and for almost as many years that root partition has been ZFS based. For confidence that I could always access this data, I wanted an independent boot media that can access such an encrypted ZFS data set. There's none that I know of which support both LUKS and ZFS, and it would also be nice for installing Gentoo as well, so I customized the Gentoo tooling for building their minimal install LiveCD, to add ZFS support. Here's how.

Start with the releng (release engineering) tools, which contain Catalyst. This is the tool that builds several low level Gentoo artifacts. I'm doing this from a Gentoo system (in a VM). What we really want from the sources are the scripts (inputs to Catalyst) used to build the minimal install LiveCD.

# emerge dev-vcs/git
# cd /root
# git clone https://github.com/gentoo/releng.git

Now we have the source checked out. (I wrote this from commit a04c9746e3dda7ecfc19b09746d7de21a7b76006. If you're trying this and it doesn't seem to work, maybe try from that point.) We need to prepare further inputs:

# mkdir -p /var/tmp/catalyst/builds/default
# wget http://.../stage3-amd64-....tar.xz -O /var/tmp/catalyst/builds/default/stage3-amd64-latest.tar.xz

Start from Gentoo's downloads page. Use the amd64 stage3 URL in the wget step. We're about to run catalyst, so we install it. Plus it will depend on pixz (parallel xz) later, so install that too. Continue by running cataylst:

# emerge-webrsync
# emerge catalyst pixz
# catalyst -s latest

This packages your freshly sync'ed local portage tree for use by the coming steps. I've added several utilities that I think are useful for both installation and recovery and testing scenarios. Proceed to stage one:

# cat > ~/stage1.sed
s#@REPO_DIR@#/root/releng#g
s#@TIMESTAMP@#latest#g
s#version_stamp: latest#version_stamp: 20200605.zfs#
/livecd.packages/ {
  a     app-editors/vim
  a     net-analyzer/netcat
  a     net-analyzer/mtr
  a     net-analyzer/nmap
  a     net-analyzer/tcpdump
  a     net-dns/bind-tools
  a     net-misc/telnet-bsd
  a     sys-apps/haveged
  a     sys-apps/hwinfo
  a     sys-apps/pv
  a     sys-apps/smartmontools
  a     sys-block/mbuffer
  a     sys-process/htop
}
^D
# catalyst -f <(sed -f ~/stage1.sed /root/releng/releases/specs/amd64/installcd-stage1.spec)

This will emerge over 200 packages inside a sandbox, it takes quite a while. (My time: 160 minutes.)

Aside: one of these packages is dev-libs/boost. I've (of course) got parallelism set as high as I can to reduce compile times, but (at least) this package will OOM and fail if you(r VM) doesn't have enough available RAM, with many compile processes running at once. I typically want to keep RAM low in my VMs, to make snapshots fast and small, so adding swap helps avoid OOM crashes.

Proceed to stage two:

# echo -n > ~/releng/releases/portage/isos/package.accept_keywords/zfs
# echo "GRUB_PLATFORMS='efi-64 efi-32 pc'" >> /etc/portage/make.conf
# emerge sys-apps/memtest86+ sys-boot/grub:2
# cat > ~/stage2.sed
s#@REPO_DIR@#/root/releng#g
s#@TIMESTAMP@#latest#g
s#version_stamp: latest#version_stamp: 20200605.zfs#
s/latest.iso/20200605.zfs.iso/
s/livecd-stage1-amd64-latest/livecd-stage1-amd64-20200605.zfs/
/livecd.volid/s/Gentoo amd64 latest/20200605zfs/
/boot.kernel.gentoo.config/ {
  a boot/kernel/gentoo/packages:
  a 	sys-fs/zfs
}
^D
# catalyst -f <(sed -f ~/stage2.sed /root/releng/releases/specs/amd64/installcd-stage2-minimal.spec)

This primarily builds the kernel and then the final ISO image (my time: around 50 minutes), which will be located at /var/tmp/catalyst/builds/default/install-amd64-minimal-20191116.zfs.iso. You're done!

Finally, I'm making the fully built ISO image I created (built with ZFS version 0.7.13 — the latest stable version in portage) available for download. See this salient note on licensing. Download gentoo-install-amd64-minimal-20191116.zfs.iso; MD5 4f578aef8d2a0a643e4334ac16b36c8f; SHA1 f86702efeca5aef58249398f28f8e0e476857999. See comments below for the most up-to-date version.

Comments:

zfs on linux
2019-12-30 00:33 - kimux

Does this ISO support UEFI startup?

New Build
2020-06-06 20:14 - arantius

ZFS 0.8 reached stable some time ago. Here's an updated install CD image, built with ZFS 0.8.3: gentoo-install-amd64-minimal-20200605.zfs.iso. Its MD5 is 79a70df3956da193a73f220535a74157 and SHA1 is 37b960b58df5620919f3671e45d27516906ce2b3.

(And, belatedly: I don't know if it supports UEFI. I think that's a complicated topic, requiring support at many layers.)

Post a comment:

Username
Password
  If you do not have an account to log in to yet, register your own account. You will not enter any personal info and need not supply an email address.
Subject:
Comment:

You may use Markdown syntax in the comment, but no HTML. Hints:

If you are attempting to contact me, ask me a question, etc, please send me a message through the contact form rather than posting a comment here. Thank you. (If you post a comment anyway when it should be a message to me, I'll probably just delete your comment. I don't like clutter.)