ci: 'test-package' pytest is failing

CI is currently blocked because the test-package test fails. There is no useful output in gitlab CI or from pytest about the cause of this failure:

DEBUG    root:run_core.py:369 (buildroot_aarch64) % cd /home/pmos/build; busybox su pmos -c CARCH=aarch64 SUDO_APK='abuild-apk --no-progress' PATH=/native/usr/lib/crossdirect/aarch64:/usr/lib/ccache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin GOCACHE=/home/pmos/.cache/go-build HOME=/home/pmos abuild -D postmarketOS -d -f
DEBUG    root:run_core.py:248 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I haven't been able to reproduce this locally, it only seems to happen when run in gitlab CI. I've tried comparing alpine upgrades between failing jobs and previously passing ones (where nothing in pmb and pmaports changes), and rebuilding jobs in MRs that previously passed will now fail.

I traced this failure to this GCC error:

Makefile: OK
gcc -o hello-world.o -c main.c
gcc -o hello-world hello-world.o
gcc: fatal error: '-fuse-linker-plugin', but liblto_plugin.so not found
compilation terminated.
make: *** [Makefile:2: hello-world] Error 1
>>> ERROR: hello-world: build failed

Output in gitlab and from pytest is extremely limited (this will help tremendously: !2306 (merged) ). I was able to find the above compilation error by modifying pytest.sh to manually cross compile the failing package:

diff --git a/.ci/pytest.sh b/.ci/pytest.sh
index d7b5aa8d..2c605347 100755
--- a/.ci/pytest.sh
+++ b/.ci/pytest.sh
@@ -60,6 +60,9 @@ if [ -n "$(git -C "$pmaports" status --porcelain)" ]; then
        exit 1
 fi

+./pmbootstrap.py --details-to-stdout build --arch armhf --force hello-world
+exit
+
 echo "Running pytest..."
 echo "NOTE: use 'pmbootstrap log' to see the detailed log if running locally."
 pytest \

things to try next

• []

tried things

• reproduce the failure in CI without using abuild --> success! see this comment

• import env. from gitlab runner (dump with export in job) into a local pmbootstrap chroot -b armhf --> did not fail, hello-world compilation successful