Luca Weiss authored 6 months ago and Oliver Smith committed 5 months ago

Sometimes just resetting e.g. packages is desired, so add that functionality using arguments.

The images have been failing for quite a while and nobody seems to
bother to fix them, so disable building the images.

The tests seem to be randomly timing out in CI after the gitlab
migration, so let's increase the timeout.

Caleb Connolly authored 6 months ago and Pablo Correa Gomez committed 6 months ago

the prebuilts are only useful on 512 sector size devices like USB drives
anyways, so just build desktop gnome instead.

I just realized that since switching to pmbootstrap v3, the
"repo_missing" action will not output systemd packages anymore if
systemd is disabled. Explicitly enable it.

BPO holds off from building packages for other branches, if the x86_64
branch is not built yet. As this is our native branch and holds the
cross compilers. Add the missing trigger that ensures bpo checks after
publishing packages, if it can now build packages from other
architectures.

Fixes: issue 139

The sets these, make sure to reset them before running tests.

Detect when bpo has a bug that causes it to start the same job over and
over, and shut it down in that case.

Previous code set these shell variables for the sourcehut job service:

  BPO_WIP_REPO_URL: """ + shlex.quote(url_repo_wip) + """
  BPO_WIP_REPO_ARG: '-mp """ + shlex.quote(url_repo_wip) + """'

...and local job service:

  BPO_WIP_REPO_URL="" # empty, because we copy it instead
  BPO_WIP_REPO_ARG="" # empty, because we copy it instead

And then used them later on in the tasks describing the jobs. When they
were used, it was hard to understand what the actual value would be, and
the behavior differed greatly from local job service (testsuite) and
sourcehut, so it lead to bugs like issue 141.

Remove these variables, and call bpo.helpers.pmb.get_pmos_mirror()
instead.

Let the local job service always use the real server's WIP repo URL to
ensure these code paths get tested with the local job service too. To
make this work:
* When building packages: having the real WIP repo mirror enabled in
  addition to having the local WIP packages copied to pmbootstrap's work
  dir should not cause any problems as long as it exists (so not when
  testing staging branches).
* When signing the APKINDEX: download the unsigned APKINDEX from the
  WIP repo mirror first to test that code path. Then remove it and copy
  the local unsigned APKINDEX in its place.

Move bpo.helpers.job.get_pmos_mirror_for_pmbootstrap to
bpo.helpers.pmb.get_pmos_mirror. It makes sense there, it gets closer
to set_repos_task which uses it in the same file and the function name
is shorter.

Reduce complexity of the url-repo-wip parameter by merging the -http and
-https variants into one that always uses https.

I had split this in 2020 in 920e6ce7 to work around a bug that is no
longer relevant. Nowadays we always run apk natively, this was
introduced in pmbootstrap 2.2.0 (pmbootstrap MR 2242).

Move this block closer to where it is used (we can do this now, since
tasks is built iteratively).

Like the refactoring done for other jobs recently:
- Use f-strings.
- Fill tasks into the OrderedDict separately instead of doing it at
  once. This is needed because I'll add a step to add the mirrors for
  pmbootstrap v3.

Robert Mader authored 7 months ago and Newbyte committed 7 months ago

It is surprisingly well supported. Thus enable automatic builds in
preparation of moving it to community.

Related: https://gitlab.com/postmarketOS/pmaports/-/issues/3095

Use pmbootstrap 2.3.x again for master and master_staging_systemd for
now, until the fallout from switching to v3 is fixed.

Related: bpo issue 141

Luca Weiss authored 8 months ago and Oliver Smith committed 8 months ago

Without this patch, the -m and -mp arguments are at the wrong position.
They line must look like:
    pmbootstrap -m … -mp … install …
Instead of:
    pmbootstrap install … -m … -mp …

(Thanks Luca for the patch! Oliver wrote this commit msg, adjusted tests.)

This test is for running with pmbootstrap v3 (unlike ci/pytest-pmb-v2).
Set the *_PMB_BRANCH=master variables, so bpo uses the pmbootstrap v3
code paths for this, even if the default is 2.3.x again.

Fix for:
[15:14:28] WARNING: file not found: https://build.postmarketos.org/wip/master/master/aarch64/APKINDEX.tar.gz

pmbootstrap 3.0.0_alpha depends on losetup.

Related: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2423

pmbootstrap v3 doesn't create the "aports" symlink anymore that points
to the pmaports dir. Make it possible to set the pmaports dir via an
environment variable and set it from related scripts.

We don't build installer images anymore since v23.12. Remove related
code from bpo for now to make refactoring for pmb v3 easier. We can
bring this back later when making installer images again, and it will
probably be different since the previous code only worked with ondev1
and pmbootstrap v2, not with ondev2 and pmbootstrap v3.

Related: https://postmarketos.org/blog/2023/12/18/v23.12-release/#notable-changes

Instead of hardcoding the path to pmbootstrap, use the path from the
pmbootstrap argument / BPO_PMB_PATH env var.

With pmbootstrap v3, the mirror configuration has changed. Add code to
set the mirrors differently when using pmbootstrap v3.

- Use f-strings.
- Fill tasks into the OrderedDict separately instead of doing it at
  once. This is needed because I'll add a step to add the mirrors for
  pmbootstrap v3.
- Use more shlex.quote. (It was fine before, but for more consistency.)

Allow overriding the branches used for the staging branches and pmaports
master with environment variables. It will be "master" by default, but
in order to run the whole testsuite with pmbootstrap v2, it must be
2.3.x.

Make the pmbootstrap configurable per branch. Keep the default of 2.3.x,
but set it to master for pmaports master and pmaports staging branches.

When adding a new branch, it is easy to forget to add the ignore_errors
key. Make it optional with the intuitive default value false.

Add a helper script that can be used to run just one test with pytest,
but using pmbootstrap v2.

The last commit actually dropped it from the packages-to-build config,
not from the images. We need to drop it in both so it doesn't matter.

Build edge images for trailblazer every 3 days. This gives us a
reasonable cadence for linux-next while accounting for the fact that
there aren't many trailblazer users yet.

We may bump this up if it's justified later on.