It may be desirable to test older versions of packages, allow them to be
built when --force is specified and print a warning that they won't be
installed automatically.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

This works around https://gitlab.postmarketos.org/postmarketOS/
pmaports/-/issues/3268, by always installing hexdump when building
packages. The full hexdump is small (like 80KB), and even though this
isn't required in most situations it seems like the best of a group
of bad workarounds, until upstream/busybox fix the bug in the pmaports
issue.

Add a test to parse some APKINDEX data.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

should be an int, but was always a string...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

I guess this is leftover from some experimentation.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Also adapt pmb/install/recovery.py to new API for variables.py.

[ci:skip-build]: already built successfully in CI

Clayton Craft authored 6 months ago and Newbyte committed 6 months ago

Sometimes these fields are not set, e.g.:

pdb> p ret
{'depends': ['build-base', 'bash', 'bison', 'findutils', 'flex', 'musl-dev', 'openssl-dev', 'perl', 'python3'], 'arch': 'noarch', 'version': '20241022.022230', 'pkgname': '.makedepends-linux-lenovo-21bx', 'provides': []}

I hit this crash when I installed the build deps for the
linux-lenovo-21bx package in the native chroot, and then ran
`pmbootstrap stats`
[ci:skip-build]: already built successfully in CI

Newbyte authored 6 months ago and Casey committed 6 months ago

It doesn't make sense to both request the nconfig UI and the xconfig UI
at the same time. Make them mutually exclusive.

Jane Rachinger authored 9 months ago and Casey committed 6 months ago

Co-authored-by: Stefan Hansson <newbyte@postmarketos.org>

Closes #2402

Luca Weiss authored 6 months ago and Casey committed 6 months ago

* Make sure all Python modules are documented by adding a bit of shell
  to .ci/docs.sh
* Remove non-existing module references from .rst
* Fix various warnings from sphinx by adjusting Python docstrings
* Add class member docs to ApkindexBlock

Clayton Craft authored 6 months ago and Casey committed 6 months ago

When a `noarch` APKBUILD is changed in local pmaports, `pmb install`
will build it automatically but image creation will fail with the
following error:

```
$ pmbootstrap install
...
[12:00:43] Build is necessary for package 'postmarketos-base-ui': binary package out of date (binary: 28-r6, local pmaports: 29-r0)
[12:00:48] => edge/postmarketos-base-ui: Done!
...
[12:01:02] (rootfs_lenovo-21bx) % apk --no-progress --repository /home/clayton/.local/var/pmbootstrap/packages/edge add -u --virtual .pmbootstrap /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-wifi-iwd-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-qt-wayland-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-qt-tweaks-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-audio-none-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-networkmanager-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-wifi-iwd-29-r0.apk --no-interactive
...
[12:01:04] ERROR: Invalid architecture: 'noarch', expected something like: x86_64, aarch64, riscv64, armhf, x86, armv7
[12:01:04] See also: <https://postmarketos.org/troubleshooting>
[12:01:04] Traceback (most recent call last):
  File "/home/clayton/src/pmbootstrap/pmb/core/arch.py", line 42, in from_str
    return Arch(arch)
           ^^^^^^^^^^
  File "/usr/lib/python3.12/enum.py", line 757, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/enum.py", line 1171, in __new__
    raise ve_exc
ValueError: 'noarch' is not a valid Arch

```

[caleb: fixed tests]

Luca Weiss authored 6 months ago and Casey committed 6 months ago

Make sure to annotate the parameters.

Luca Weiss authored 6 months ago and Casey committed 6 months ago

In the code where the object is made (pmb/parse/apkindex.py) depends
already will become an empty array when there's no dependencies.

And there's no point in having a differentiation between None and empty
array.

Try to finish the migration by changing various links and texts to point
to the new GitLab instance.

Clayton Craft authored 6 months ago and Newbyte committed 6 months ago

pmb.parse.apkindex.providers now returns a dict of string:ApkindexBlock,
probably from 71772b9b

This adds type hinting to package_provider to reflect this, and fixes
code that calls it to properly deal with the possible return types.
Without this, building FDE images is broken.

And make the docs job run in the chroot now and grab the 'public/'
directory so we have the built docs.

Since the return value is a file path, let's make it a Path object. Also
update the docstring to reflect reality.

This series breaks pmbootstrap, and that's not good[1]. A bug was filed
about this (#2465), and there have been multiple attempts to fix it (!
2435 and !2436). It kinda seems like we don't have time to fix/test this
for a while longer, which is fine, but given the impact this bug has I
think we should be revert this series until this issue is solved/tested.

1. pmb is broken in some specific cases, which means some workflows are
broken. One example is that this breaks the pmaports CI, so no work can
be done in pmaports.

Looks like an unintended leftover from debugging.

I tested $ pmbootstrap shutdown with this and observed no problems. All
the mounts disappear in lsblk as expected.

Closes postmarketOS/pmbootstrap#2423

And fix type errors.

I'm not sure why this was annotated as a Sequence rather than list. The
function clearly returns a list, not a Sequence, and the commit that
originally did it[1] provides no justification. This causes problems
with mypy checks in other places, so let's just annotate this as
returning a list.

 [1]: 198f302a

Otherwise mypy cannot figure out the type of these.

Newbyte authored 6 months ago and Newbyte committed 6 months ago

Works around circular import.

Newbyte authored 6 months ago and Newbyte committed 6 months ago

And adjust other code.

Closes postmarketOS/pmbootstrap#2455

Now that we have moved the kconfigcheck configuration into pmaports
branches via kconfigcheck.toml, it is time to get rid of the hardcoded
list of valid "pmb:kconfigcheck-…" options for APKBUILDs. Generate it
from the kconfigcheck.toml of the current branch, too.

These options are passed from "pmbootstrap lint" to "apkbuild-lint",
which we run in pmaports CI.

The output of apkbuild-lint is hard to spot among other log messages
from pmbootstrap, because it is not colorized.

Add "*** apkbuild-lint output ***"" before and after the output of
apkbuild-lint, so we get a standing out message in green around the
apkbuild-lint output.

It is not needed to run the linter as root, so don't do it. This is
probably a leftover from early days pmbootstrap, where pmaports.git
wasn't chowned by the user running pmbootstrap.

Here's the problem: Imagine a queue with 2 packages in it, the 1st
package uses crossdirect and the second uses cross-native. When building
the 1st package, pmb will configure the native chroot for crossdirect
as expected. When it gets to the 2nd package, the chroot env/config
might not be appropriate for actually doing a native cross compile.

This re-inits the cross compiler stuff if the cross compile method
changes while processing the queue.

Another approach that might solve this problem is to not re-use chroots
when building packages... I didn't think this was a good way to go
because it would greatly increase runtime (having to recreate chroots
multiple times)