Arch Linux ARMv8 VM on Gentoo AMD64

The post is about configuring Arch Linux ARMv8 guest virtual machine(VM) on Gentoo AMD64 host using QEMU.

VirtualBox

I'm used to making VMs in VirtualBox. It can handle a large variety of disk image formats. We can easily convert the RAW image to VDI:

$ VBoxManage convertfromraw disk.img arch.vdi --format VDI 
Converting from raw image file="disk.img" to file="arch.vdi"...
Creating dynamic image with size 4294967296 bytes (4096MB)...

However, at the time of writing I found no way to run the AARCH64 guest on AMD64 host. The boot process stuck on UEFI interactive shell which failed to detect the drive contents. All my attempts to boot from the shell had failed. I had also tried to fix it through VBoxInternal2/EfiBootArgs without success. Finally decided to give QEMU a try.

QEMU

QEMU proved to be more flexible. I had managed to boot the system by specifying initial ramdisk, firmware, kernel and kernel arguments. The following steps describe the whole process.

Install required packages

Prepare USE-flags:

$ sudo euse -p app-emulation/qemu -E qemu_softmmu_targets_aarch64 qemu_user_targets_aarch64 spice

Install QEMU as described in Gentoo Wiki.

Download firmware

Download QEMU_EFI.fd from snapshots.linaro.org.

Create disk image

$ qemu-img create -f raw disk.img 4G

Create partition table

$ parted disk.img mklabel msdos

Mount the disk image as a loopback device

$ sudo losetup -f disk.img
losetup -a
/dev/loop0: []: (/home/ruslan/aarch64-qemu/disk.img)

Create a partition

$ sudo fdisk /dev/loop0
fdisk> n
(choose defaults)
fdisk> a
fdisk> w

Detach the loop device

$ sudo losetup -d /dev/loop0

Create a loop device for the new partition

$ sudo losetup -f -P disk.img

$ losetup -a
/dev/loop0: []: (/home/ruslan/aarch64-qemu/disk.img)

Format the partition to ext4

$ sudo mke2fs -t ext4 /dev/loop0p1

Mount the new partition

$ mkdir -p mnt
$ sudo mount /dev/loop0p1 mnt

Extract the Arch files

Extract the Arch ARM Linux files from archive according to instructions:

$ sudo su -
# bsdtar -xpf ArchLinuxARM-aarch64-latest.tar.gz -C mnt
# exit

Boot in fallback mode

The system(d) failed to detect the hard drive. So I had to fix initial ramdisk hooks order through fallback mode.

$ filename="disk.img"

$ qemu-system-aarch64 -m 2048 -cpu cortex-a57 \
  -smp 1 -M virt -bios QEMU_EFI.fd -serial stdio \
  -drive if=none,file=$filename,id=hd0  \
  -device virtio-blk-device,drive=hd0 \
  -kernel mnt/boot/Image -initrd mnt/boot/initramfs-linux-fallback.img \
  -append "root=/dev/vda1"

Within the guest system change HOOKS in /etc/mkinitcpio.conf from

HOOKS="base udev block autodetect modconf filesystems keyboard fsck"

to:

HOOKS="base udev autodetect modconf block filesystems keyboard fsck"

Remount / for read-write:

# mount -o remount,rw /

Then regenerate initramfs and shutdown the guest:

# mkinitcpio -p /etc/mkinitcpio.d/linux-aarch64.preset
# shutdown now

Remount the loop device

$ sudo umount /dev/loop0p1
$ sudo losetup -d /dev/loop0
$ sudo losetup -f -P disk.img
$ sudo mount /dev/loop0p1 mnt

Finally

Run the machine in normal mode:

$ filename="disk.img"
$ qemu-system-aarch64 \
  -m 512 \
  -cpu cortex-a57 -M virt \
  -drive if=none,format=raw,file=$filename,id=hd0  \
  -device virtio-blk-device,drive=hd0 \
  -smp 1 \
  -bios QEMU_EFI.fd \
  -serial stdio \
  -kernel mnt/boot/Image \
  -initrd mnt/boot/initramfs-linux.img \
  -append "root=/dev/vda1 rootfstype=ext4 rw" \
  -netdev user,id=unet -device virtio-net-device,netdev=unet

The only drawback is that ICMP packets(ping) will not work in this mode. The network settings can be tweaked, though.

Helper scripts

To make further usage easier I've created scripts to mount and unmount the disk image, and to launch the VM.

util.sh

#!/bin/bash -

function usage()
{
  echo -e "Usage:\n$0 disk-image mount-point"
}

function error()
{
  [ $# -gt 0 ] && echo >&2 "!! $1"
}

function die()
{
  error $1
  exit 1
}

function notice()
{
  echo "* $1"
}

function checkimg()
{
  [ -e "$1" ] || die "$1 doesn't exist"
}

function checkmnt()
{
  [ -e "$1" ] || die "$1 doesn't exist"
}

mount.sh

#!/bin/bash -

source util.sh

if [ ! $# -gt 1 ]; then
  error 'Missing required arg(s)'
  usage
  die
fi

imgfile="$1"
mntdir="$2"

checkimg $imgfile
checkmnt $mntdir

free_loop_part=$(losetup --show --find --partscan "$imgfile")"p1"
[ $? -eq 0 ] || die 'Failed to create loop device'
notice "Created loop partition $free_loop_part"

mount -o loop,rw "$free_loop_part" "$mntdir"
[ $? -eq 0 ] || die "Failed to mount $imgfile to $mntdir"
notice "Associated $free_loop_part with $imgfile and mounted to $mntdir"

unmount.sh

#!/bin/bash -
# Unmounts and detaches loop devices associated with a disk image

source util.sh

if [ ! $# -gt 1 ]; then
  error 'Missing required arg(s)'
  usage
  die
fi

imgfile="$1"
mntdir="$2"

checkimg $imgfile
checkmnt $mntdir

notice "Unmounting $mntdir"
while true
do
  umount --detach-loop "$mntdir" >/dev/null 2>&1
  [ $? -eq 0 ] || break
done

losetup -j "$imgfile" -l --raw -O NAME -n | while read -r ld
do
  losetup -d "$ld"
  [ $? -eq 0 ] && notice "Detached loop device $ld"
done

run.sh

#!/bin/bash -

source util.sh

[ $# -gt 0 ] && filename="$1"
[ $# -gt 1 ] && mntdir="$2"

: ${filename:="disk.img"}
: ${mntdir:="mnt"}

checkimg $filename
checkmnt $mntdir

notice "Launching Arch aarch64, img:$filename mntdir $mntdir"

qemu-system-aarch64 \
  -m 512 \
  -cpu cortex-a57 -M virt \
  -drive if=none,format=raw,file=$filename,id=hd0  \
  -device virtio-blk-device,drive=hd0 \
  -smp 1 \
  -bios QEMU_EFI.fd \
  -serial stdio \
  -kernel $mntdir/boot/Image \
  -initrd $mntdir/boot/initramfs-linux.img \
  -append "root=/dev/vda1 rootfstype=ext4 rw" \
  -netdev user,id=unet -device virtio-net-device,netdev=unet

Usage

With the help of these scripts we can start and stop the VM easily:

$ sudo ./mount.sh disk.img mnt
* Created loop partition /dev/loop2p1
* Associated /dev/loop2p1 with disk.img and mounted to mnt
$ df
...
/dev/loop3       4061888   1128524   2707316  30% /home/ruslan/aarch64-qemu/mnt
$ losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
...
/dev/loop2         0      0         0  0 /home/ruslan/aarch64-qemu/disk.img
/dev/loop3         0      0         1  0 /dev/loop2p1

$ ./run.sh
* Launching Arch aarch64, img:disk.img mntdir mnt
...
Arch Linux 4.5.0-1-ARCH (ttyAMA0)

alarm login: root
Password: 
Last login: Tue Mar 22 17:23:48 on ttyAMA0
[root@alarm ~]# arch
-bash: arch: command not found
[root@alarm ~]# uname -a
Linux alarm 4.5.0-1-ARCH #1 SMP Mon Mar 14 18:47:45 MDT 2016 aarch64 GNU/Linux
[root@alarm ~]# shutdown now
...
$ sudo ./unmount.sh disk.img mnt
* Unmounting mnt
* Detached loop device /dev/loop2