Debian Chroot

From Crypto++ Wiki
Jump to navigation Jump to search

Debian provides a rich testing environment by way of Ports and QEMU/Chroot environments. The ports are lightweight VMs, and they include X32, IBM S/390, ARM Hard Floats (ARMHF) (used by BeagleBoards), ARM EABI (ARMEL) (used by Android), PowerPC, MIPS and MIPSEL (used for SGI and DEC workstations). This page will provide recipes for installing them for testing purposes.

There are two distinct components used for this testing. First is the host system, and it can run Debian Stable (like Buster or Bullseye), Testing or Unstable distributions. The second component is the guest, and it must run Debian Unstable (Sid) distribution. If you fail to use Unstable for a guest, then you will receive errors similar to Failed getting release file http://ftp.ports.debian.org/debian-ports/dists/buster/Release.

There is a downside to using ports and chroots. The guest binary images are native to the guest instruction set architecture (not the host ISA). The programs are interpreted so things run up to 10x slower than a native host program. If possible, you should consider using real hardware to make the testing process faster.

Debian Stable

The host or base system is a Debian x86_64 machine and can use Debian Stable, Testing or Unstable. The host can run on real hardware, or it can run in a VM like KVM, VirtualBox or VMware. The processor should provide SLAT or hardware assisted virtualization (AMD-V or Intel VT-X).

The QEMU and Chroot guests must use Debian Unstable, and not Stable or Testing. If you fail to using Unstable, then you will receive errors similar to Failed getting release file http://ftp.ports.debian.org/debian-ports/dists/buster/Release. If you encounter your error, then /etc/apt/sources.list needs to be modified to use Unstable.

After you install the Debian host system, be sure to run the following on the host to avoid a constant stream of warnings in guests because LC_ALL is not set:

debian-host# apt install locales
debian-host# dpkg-reconfigure locales
(complete the process)

Debootstrap

Debootstrap is installed on the host and used to ease the management of installing the QEMU/Chroot environment. There are six packages needed to provide a QEMU guest and they are shown below.

debian-host# apt install qemu qemu-user-static binfmt-support debootstrap debian-archive-keyring debian-ports-archive-keyring

After binfmt is installed, you can list the environments available with the following command. Note that only some of the environments are useful because some environments don't have a Debian port. If a port is missing, like MIPS64, then the environment is not useful for testing Crypto++.

debian-host$ su -
Enter password: ...

# update-binfmts --display | grep interpreter
 interpreter = /usr/bin/jexec
 interpreter = /usr/bin/python3.9
 interpreter = /usr/libexec/qemu-binfmt/aarch64-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/alpha-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/arm-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/armeb-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/cris-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/hexagon-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/hppa-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/m68k-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/microblaze-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/mips-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/mips64-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/mips64el-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/mipsel-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/mipsn32-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/mipsn32el-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/ppc-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/ppc64-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/ppc64le-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/riscv32-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/riscv64-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/s390x-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/sh4-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/sh4eb-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/sparc-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/sparc32plus-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/sparc64-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/xtensa-binfmt-P
 interpreter = /usr/libexec/qemu-binfmt/xtensaeb-binfmt-P

QEMU/Chroot Guest

The recipes to create a chroot are provided below. There's lots of them, so they were provided at the end of this article. Simply pick a recipe, and copy/paste it into host the command line.

The chroot's are simply filesystem branches stored in root's home directory:

debian-host# cd
debian-host# pwd
/root

# ls -l
drwxr-xr-x 20 root root      4096 Sep 10 06:43 debian-armel
drwxr-xr-x 20 root root      4096 Oct  5 14:27 debian-armhf
drwxr-xr-x 21 root root      4096 Aug 30 18:32 debian-s390x
drwxr-xr-x 21 root root      4096 Oct  9 16:12 debian-unstable
drwxr-xr-x 21 root root      4096 Aug 20 15:21 debian-x32

Entering a Guest

After the lightweight VM is created, you enter the guest with chroot debian-{arch}. Remember to elevate:

debian-host$ sudo su -
Password: ...

debian-host# chroot debian-s390
debian-guest#
(You are now in the lightweight VM)

Once in the QEMU/Chroot guest, the next step is to enable Ports keyring:

debian-guest# apt install debian-archive-keyring debian-ports-archive-keyring

All guests must use Unstable (and not Stable nor Testing). Modify /etc/apt/sources.list as required. You should replace buster or stable with sid or unstable. After the replacement, be sure to perform apt update && apt dist-upgrade in the guest:

debian-guest# cat /etc/apt/sources.list
deb http://ftp.ports.debian.org/debian-ports unstable main
deb-src http://ftp.ports.debian.org/debian-ports unstable main

If you need to modify sources.list and you don't have your editor of choice, try:

debian-guest# echo "deb http://ftp.debian.org/debian unstable main" > /etc/apt/sources.list

Once the Keyring is installed and Unstable is enabled, perform the necessary upgrades. Upgrades in the guest are similar to the host based system, but you will have to fetch them manually on occasion.

debian-guest# apt update
...
debian-guest# apt dist-upgrade
...

Filesystems

You can also mount the host /proc filesystem in the guest with the following commands. Also see mount dev, proc, sys in a chroot environment? on Super User.

mount -t proc proc proc/
mount --rbind /sys sys/ 
mount --rbind /dev dev/ 

You can also add an entry to mount.defaults (the Chroot equivalent to /etc/fstab).

Guest Software

Issue the following commands to fetch common development tools. You have to do it for each guest testing environment.

debian-guest# apt install locales build-essential gcc g++ gdb make cmake subversion git wget curl zip unzip

Once the initial software is installed you should configure locales to squash a nag from Perl.

debian-guest# dpkg-reconfigure locales

GDB may or may not work in a guest. If you find a GDB bug related to a platform, then please be sure to file a bug report with GDB so the Sourceware folks fix it. Also file a bug report at Debian, and reference the upstream bug. The Debian report is required so the Debian folks know they have a break that needs fixing.

Emacs users will have to install Emacs in each chroot. emacs-nox is a good choice because the window-ing gear is not available. At over 100 MB its still bloated, but its smaller than it could be. Alternately, you can build a small footprint Emacs by following Minimal emacs24 installation on Super User.

Recipes

The following are copy and paste recipes to create the guest test environments. It should be executed on the host after elevating with su - or su - root. After executing a recipe, chroot into the guest environment and install the necessary software as detailed above.

The recipes below use the {debian-archive-keyring.gpg,http://ftp.debian.org/debian} or {debian-ports-archive-keyring.gpg,http://ftp.ports.debian.org/debian-ports} pair. The keyring and release file location come in pairs.

All guests must use Unstable (and not Stable nor Testing). Modify /etc/apt/sources.list as required. You should replace buster or stable with sid or unstable. After the replacement, be sure to perform apt update && apt dist-upgrade in the guest.

X32 is an interesting platform that is a mix of X64 machine with X86 assembly language and X64 registers. To use X32, you must boot your kernel with the command line option syscall.x32=y. Debian 8 kernels provide support for the option out-of-the-box, and you can add it to GRUB_CMDLINE_LINUX or GRUB_CMDLINE_LINUX (see /etc/defaults/grub).

AMD64

AMD64 is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=amd64 --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-amd64 http://ftp.debian.org/debian

ARM64

ARM64 is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=arm64 --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-arm64 http://ftp.debian.org/debian

ARMHF

ARMHF is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=armhf --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-armhf http://ftp.debian.org/debian

ARMEL

ARMEL is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=armel --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-armel http://ftp.debian.org/debian

M68K

M68K is in Debian ports. Use http://ftp.ports.debian.org/debian-ports and debian-ports-archive-keyring.

qemu-debootstrap --arch=m68k --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-m68k http://ftp.ports.debian.org/debian-ports

MIPS

This recipe is broken. The Chroot cannot be setup using ports or non-ports.

qemu-debootstrap --arch=mips --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-mips http://ftp.ports.debian.org/debian-ports

MIPSEL

MIPSEL is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=mipsel --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-mipsel http://ftp.debian.org/debian

MIPS64

This recipe is broken. The Chroot cannot be setup using ports or non-ports.

qemu-debootstrap --arch=mips64 --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-mips64 http://ftp.ports.debian.org/debian-ports

MIPS64EL

MIPS64EL is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=mips64el --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-mips64el http://ftp.debian.org/debian

PowerPC

PowerPC is in Debian ports. Use http://ftp.ports.debian.org/debian-ports and debian-ports-archive-keyring.

qemu-debootstrap --arch=powerpc --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-powerpc http://ftp.ports.debian.org/debian-ports

S/390x

S/390x is not in ports though Debian docs say it is. Use http://ftp.debian.org/debian and debian-archive-keyring.

qemu-debootstrap --arch=s390x --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-s390x http://ftp.debian.org/debian

SH-4

SH-4 is in Debian ports. Use http://ftp.ports.debian.org/debian-ports and debian-ports-archive-keyring.

qemu-debootstrap --arch=sh4 --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-sh4 http://ftp.ports.debian.org/debian-ports

SPARC64

SPARC64 is in Debian ports. Use http://ftp.ports.debian.org/debian-ports and debian-ports-archive-keyring.

qemu-debootstrap --arch=sparc64 --keyring /usr/share/keyrings/debian-ports-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-sparc64 http://ftp.ports.debian.org/debian-ports

X32

Be sure to boot your kernel with the command line option syscall.x32=y. Note this recipe uses debootstrap, and not qemu-debootstrap

debootstrap --arch=x32 --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster unstable debian-x32 http://ftp.ports.debian.org/debian-ports

Unstable

New releases of Crypto++ enter Debian system via Unstable distribution. Also see Debian Unstable | Life Cycle on Debian's wiki.

qemu-debootstrap --arch=amd64 --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster sid debian-unstable http://ftp.ports.debian.org/debian-ports

Testing

Once Crypto++ satisfies the requirements of Debian's Unstable distribution, it is moved to Debian Testing. Also see Debian Testing | How Testing Works on Debian's wiki.

qemu-debootstrap --arch=amd64 --keyring /usr/share/keyrings/debian-archive-keyring.gpg \
  --variant=buildd --exclude=debfoster testing debian-testing http://ftp.ports.debian.org/debian-ports

Common Errors

Debian's package system has a chronic problem of missing signing keys, and inability to update the system once installed (if it installs in the first place). You will experience the goodness with errors like Release signed by unknown key (key id B4C86482705A2CE1), Release signed by unknown key (key id E852514F5DF312F6) and failures in Apt. If you find yourself fighting the Debian package system, you should (1) know you are not alone, and (2) visit bug reports like 826043 and 842307.

According to the 826043 bug, you are supposed to be able to fix it with one of the following. We know for certain the suggestions do not work under Debian Hurd (i386). We don't expect it to work in other places either, like Sparc64.

apt install --reinstall debian-archive-keyring

Or

apt install --reinstall debian-ports-archive-keyring

Or

dpkg --force-depends -P debian-archive-keyring
dpkg -i /var/cache/apt/archives/debian-archive-keyring*

Or

cd /
apt update

Or

cd ~
apt update

Running Apt with --no-check-gpg and --allow-unauthenticated does not work either.

Acknowledgements

Many thanks to László Böszörményi, Adam Borowski and Michael Tokarev of the Debian project. They were very helpful with installation and configuration of the QEMU/Chroot environments. http://ftp.ports.debian.org/debian-ports