Here's the problem: Imagine a queue with 2 packages in it, the 1st
package uses crossdirect and the second uses cross-native. When building
the 1st package, pmb will configure the native chroot for crossdirect
as expected. When it gets to the 2nd package, the chroot env/config
might not be appropriate for actually doing a native cross compile.

This re-inits the cross compiler stuff if the cross compile method
changes while processing the queue.

Another approach that might solve this problem is to not re-use chroots
when building packages... I didn't think this was a good way to go
because it would greatly increase runtime (having to recreate chroots
multiple times)

When running some commands like "pmbootstrap chroot" or even
"pmbootstrap work_migrate", pmbootstrap will attempt to read the
pmaports.cfg from origin/master *before* doing the work dir migration.

In order to do this, the "get_upstream_remote" function tries to find
the upstream remote by URL. Let it search with the outdated URLs too, so
this doesn't fail right before migrating to the new URLs.

Newbyte authored 5 months ago and Oliver Smith committed 5 months ago

This could be re-purposed in the future in case we migrate URLs again.

Co-authored-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>

Caleb Connolly authored 6 months ago and Oliver Smith committed 5 months ago

Now that we don't need weird apk-tools hacks for systemd, we can
re-implement this optimisation and always run apk static rather than
running apk through the chroot.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Newbyte authored 6 months ago and Oliver Smith committed 5 months ago

Otherwise Arch may be None here which results in an error.

We moved from gitlab.com to gitlab.postmarketos.org.

Newbyte authored 6 months ago and Newbyte committed 6 months ago

This is section is not meant to represent my opinions, rather I seek to
document the status quo. I base the support range on what's been
reasoned in prior merge requests such as [1] and [2], as well as [3]
which only fixed CI in the one active stable release and left old stable
releases unfixed.

While I don't necessary think this is a great situation given that old
releases may sometimes be useful, extending the time pmbootstrap has to
support old releases would incur significant additional maintenance
effort and consequently require more discussion than just documenting
what's already happening. As such, I think this is the most reasonable
thing to do for now.

 [1]: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2360#note_1999012908
 [2]: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2417#note_2135527528
 [3]: https://gitlab.com/postmarketOS/pmaports/-/merge_requests/5676

Newbyte authored 6 months ago and Oliver Smith committed 6 months ago

This is probably a leftover from automatically reformatting this file.

The previous version would fail if the suffix contained a dash as well,
e.g. postmarketos-base-ui-audio-some-new-thing. Fix it and add example
comments.

For packages selected by the user (see last commit), set the
provider_priority to 999999 instead of 999. The previous number seems
not high enough, we might set this in the APKBUILDs.

Clayton Craft authored 6 months ago and Oliver Smith committed 6 months ago

Fixes #2456
Depends on https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/72741

We plan to ship systemd with split /usr until the /usr merge is complete
in Alpine. Let's not drop all our code yet but just forcefully disable
it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Similar to pkgrel but for pkgver. Useful when dealing with e.g.
metapackages.

[ci:skip-build]: already built successfully in CI

We no longer care about supporting Python versions older than 3.10,
which is when match was introduced.

Also remove args from functions that don't actually need it to
faciliate this.

This does not attempt to fix any of the bugs with pkgrel_bump.

Otherwise we don't get type checking in these functions.

This function is used by "pmbootstrap repo_missing", which is only used
by bpo. In order to support building the split repository in bpo, this
patch is required.

Return "systemd" for the packages that are in extra-repos/systemd, and
None for all other packages (indicating that they don't get split into a
separate repository).

Checking the parent-parent dir of the APKBUILD is a bit fragile
(assuming we don't have subdirs in the systemd repository), but that
assumption was made with the previous implementation as well (would only
return "systemd" in that case). I don't see a better solution with
reasonable effort right now, given that we also support multiple
pmaports dirs. We can improve this in the future, meanwhile let's just
not use subdirs in the systemd repository.

The function is never called with must_exist=False, so drop it.

Don't attempt to find packages of other arches in repo_missing code.
This leads to unexpected results in general (we want to check if a
binary package exists for only one specific architecture!) - and in case
of master_staging_systemd, it current causes errors because pmbootstrap
tries to fetch armhf, riscv etc. APKINDEXes for which we don't build the
staging repositories.

Let the get() function pass the try_other_arches parameter when calling
itself, so it does not get lost.

Fall back to doas if sudo is not available.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

This reverts commit 099a7741.

As chroot.glob() now returns a generator, convert it to a list first to
check if it is empty. This may not be the most efficient method, but
since we expect it to be empty here it is fine.

Without this patch, it always shows the error even if there are no
locally built packages:

  ERROR: --no-local-pkgs specified, but locally built packages found. Consider 'pmbootstrap zap -p' to delete them.

Fix that pmbootstrap exits with 0 on linter failure. Let it raise an
error instead (which results in exit != 0).

Make sure the symlinks are correct when not inside the chroot, they
should be relative!

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Fix #2448

Logic for breaking dependency loops with existing binary packages is now
implemented, so remove the related comment.

When a dependency cannot be found in source or binary packages of the
selected architecture, then don't look for binary packages in other
architectures package indexes.

This leads to a confusing error down the line when building packages for
extra_repos_systemd (and e.g. stable branches), because we don't have
binary packages for all arches that pmbootstrap supports, and so it
tries to fetch a non-existing APKINDEX.

Instead, print a nice error about a non-existing dependency.

Don't just run one "mv" command to move everything from source to
target. This fails if the same directory already exists in target.
Instead, go into each directory in source, and move all files inside
that directory into the target directory structure.

Fix the following error:
  mv: cannot overwrite '/home/user/.local/var/pmbootstrap/chroot_native/usr/lib/sysctl.d': Directory not empty

Create a common merge() function that can be adjusted in the next patch,
because the logic for merging needs to be more complicated now.

Currently the usr-merge script fails, and this leads to a very confusing
error when running pmbootstrap the next time:

  (stack trace)
  KeyError: 'apk-tools'

Show a more helpful error instead.

Adjust the dependency resolver, so it uses the main pkgnames in
dependency resolution, instead of subpackages. This fixes for example
that it couldn't find -dev packages even though the binary packages for
those existed.

> systemd: missing dependency linux-pam-dev, trying to queue other packages first
> linux-pam: missing dependency systemd-dev, trying to queue other packages first
> dbus: missing dependency systemd-dev, trying to queue other packages first

Split has_cyclical_dependency logic into an extra function, so it is
easier to read.

Have messages like the following in "pmbootstrap log" to better
understand why resolving packages fails. The bug with -dev pkgs not
being recognized will be fixed in a follow-up patch.

> systemd: missing dependency linux-pam-dev, trying to queue other packages first
> linux-pam: missing dependency systemd-dev, trying to queue other packages first
> dbus: missing dependency systemd-dev, trying to queue other packages first