cloud-hypervisor broken on aarch64-linux

[mikatammi]

I have a hydra setup which is building microvms. What I see is in log:

Copying a /nix/store
Creating a erofs
mkfs.erofs 1.6
Build completed.mygr3wkmv3d0-gpm-unstable-2020-06-17/share/man/man8/gpm.8.gz ...
Cleaning up store
Copying store erofs into disk
953712+0 records in
953712+0 records out
488300544 bytes (488 MB, 466 MiB) copied, 1.45549 s, 335 MB/s
1+0 records in
1+0 records out
512 bytes copied, 6.8916e-05 s, 7.4 MB/s
Creating partition table
Creating EFI partition

I have waited for days and it never gets past the "Creating EFI partition" step.

With ps ax on the builder machine I can see qemu-kvm that has been running for a long time.

 958966 ?        Ss     0:00 bash -e /nix/store/6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh
 959530 ?        S      0:00 /nix/store/vgzggbs5dv4ay3wv0dxcvc9k59zkbd8l-libguestfs-1.48.4/bin/guestfish --add /nix/store/94xchp0c0h8533mnqkdy2i64w9llzrjx-microvm-bootdisk.img run : mkfs fat /dev/sda1
 959553 ?        Sl   1086:46 /nix/store/5isknrpw41bpv4c6qcq230xa6k6kx6kd-qemu-8.0.0/bin/qemu-kvm -global virtio-blk-pci.scsi=off -no-user-config -nodefaults -display none -machine virt,accel=kvm:tcg -cpu cortex-a57 -m 1280 -no-reboot -rtc driftfix=slew -kernel /nix/store/wsb0ls0rr4v1kl5rylx
 959554 ?        S      0:00 /nix/store/vgzggbs5dv4ay3wv0dxcvc9k59zkbd8l-libguestfs-1.48.4/bin/guestfish --add /nix/store/94xchp0c0h8533mnqkdy2i64w9llzrjx-microvm-bootdisk.img run : mkfs fat /dev/sda1

I'd need to add some debug prints or something to see what is the actual command after "Creating EFI partition" that is taking so long.

This only seems to appear on aarch64-linux targets and not x86_64-linux

[astro]

I fear I broke your great aarch64-linux work. If you require something useable right now then use tag v0.3.3.

By now, the bootDisk is only required for the hypervisors that cannot load initrds by themselves. Namely: cloud-hypervisor. Their solution is https://github.com/cloud-hypervisor/rust-hypervisor-firmware which I packaged in time for nixos-23.05.

I used the last stable release 0.4.2 which supports only x86_64-linux. I had trouble packaging an unstable version but we shall try again because it brings support for aarch64 and riscv64.

Do the other hypervisors work for you? They shouldn't require the bootDisk to be built.

[mikatammi]

I have to double check this because looks like with latest versions the builds work again, but I haven't tested the functionality

[mikatammi]

My build is only using QEMU at the moment, didn't try other hypervisors yet

[mikatammi]

708f09d I guess this commit fixes this issue for other hypervisors than cloud-hypervisor

[astro]

Since 4f752ae we no longer require x86_64-only rust-hypervisor-firmware again.