Some packages in aports have their own subpackages in their
dependencies, for some reason. This causes our resolver to assume there
is a cyclical dependency.

Adjust the resolver to safely ignore these dependencies, since they'll
be built as a part of the package that depends on them.

For example, modemmanager depends on libmm-glib which is a subpackage of
modemmanager. Currently to build modemmanager in pmaports it would be
necessary to manually modify the APKBUILD after copying it from aports.
This change avoids that.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

args.autoinstall attribute might not exist, use getattr like for others,
and add a fixme about the default value...

Fixes: #2476


Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Newbyte authored 5 months ago and Oliver Smith committed 5 months ago

All call sites that use this have been removed. This was only introduced
as a stopgap until we could refactor more code to not use args
everywhere, and that has now been achived. As such, remove this so it
doesn't get used in more locations.

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

See
#2473



Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

Add a basic test to ensure that the cache does get hit as we expect.
This doesn't test for cache conflicts.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

Use a different way of keying the apkindex path cache.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

It may be desirable to test older versions of packages, allow them to be
built when --force is specified and print a warning that they won't be
installed automatically.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Vikram Alagh authored 5 months ago and Oliver Smith committed 5 months ago

Seems to be ChromeOS specific:

> this happens every time on my machine. I am on ChromeOS 129.0.6668.112
> using a Debian bookworm chroot through Crouton. I am not sure why
> this is happening nor which layer is responsible for this. I deleted
> the chroot and created another one but the problem persists.

Commit msg tweaked by Oliver.

Provides kpartx.

Looks like an unintended leftover from debugging.

---

This was needlessly reverted by
1ec55fc1. While revreting
0975d064 was good, this commit did not
cause any harm.

And fix type errors.

---

This was needlessly reverted by
1ec55fc1. While revreting
0975d064 was good, this commit did not
cause any harm.

Running these inside the chroot is tricky, and we ended up reverting
previous commits that attempted to make them run there (see
1ec55fc1). As such, just accept defeat
and ensure that the host has these.

Also see #2465

We fully expect this to happen if a user hasn't configured their host
system properly. As such, don't treat it like a programming error.

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

We now run the job in chroot, so remove the unnecessary check. If
sphinx-build is missing for some reason, it will fail a bit further in
the script anyways.

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

config.sudo_timer is a bool, and the returned value here is a bool. No
point in converting it to a str.

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

I don't think there's any case where the number of jobs would have to be
a string. It's also being assigned an integer elsewhere the code (in
ask_for_additional_options() inside of init.py), so an integer seems
like what we actually want. Also fix type errors resulting of this.

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

These are artefacts from when our line length was shorter. Ruff didn't
really do a great job at fixing them up when autoformatting.

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

For the gitlab.com -> gitlab.postmarketos.org migration we needed to
also migrate the workdir in pmb v2 since pmb v3 is not ready yet.

Therefore we need to handle that "7-2.x" workdir version and migrate
correctly when switching to pmbootstrap v3.

See !2445

This works around https://gitlab.postmarketos.org/postmarketOS/
pmaports/-/issues/3268, by always installing hexdump when building
packages. The full hexdump is small (like 80KB), and even though this
isn't required in most situations it seems like the best of a group
of bad workarounds, until upstream/busybox fix the bug in the pmaports
issue.

So we don't go through the pain of pushing 3 times to get 3 different
ruff failures in CI.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Add a test to parse some APKINDEX data.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

should be an int, but was always a string...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

I guess this is leftover from some experimentation.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Also adapt pmb/install/recovery.py to new API for variables.py.

[ci:skip-build]: already built successfully in CI

Clayton Craft authored 5 months ago and Newbyte committed 5 months ago

Sometimes these fields are not set, e.g.:

pdb> p ret
{'depends': ['build-base', 'bash', 'bison', 'findutils', 'flex', 'musl-dev', 'openssl-dev', 'perl', 'python3'], 'arch': 'noarch', 'version': '20241022.022230', 'pkgname': '.makedepends-linux-lenovo-21bx', 'provides': []}

I hit this crash when I installed the build deps for the
linux-lenovo-21bx package in the native chroot, and then ran
`pmbootstrap stats`
[ci:skip-build]: already built successfully in CI

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

It doesn't make sense to both request the nconfig UI and the xconfig UI
at the same time. Make them mutually exclusive.

Jane Rachinger authored 8 months ago and Caleb Connolly committed 5 months ago

Co-authored-by: Stefan Hansson <newbyte@postmarketos.org>

Closes #2402

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

* Make sure all Python modules are documented by adding a bit of shell
  to .ci/docs.sh
* Remove non-existing module references from .rst
* Fix various warnings from sphinx by adjusting Python docstrings
* Add class member docs to ApkindexBlock

Clayton Craft authored 5 months ago and Caleb Connolly committed 5 months ago

When a `noarch` APKBUILD is changed in local pmaports, `pmb install`
will build it automatically but image creation will fail with the
following error:

```
$ pmbootstrap install
...
[12:00:43] Build is necessary for package 'postmarketos-base-ui': binary package out of date (binary: 28-r6, local pmaports: 29-r0)
[12:00:48] => edge/postmarketos-base-ui: Done!
...
[12:01:02] (rootfs_lenovo-21bx) % apk --no-progress --repository /home/clayton/.local/var/pmbootstrap/packages/edge add -u --virtual .pmbootstrap /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-wifi-iwd-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-qt-wayland-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-qt-tweaks-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-audio-none-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-networkmanager-29-r0.apk /mnt/pmbootstrap/packages/edge/aarch64/postmarketos-base-ui-wifi-iwd-29-r0.apk --no-interactive
...
[12:01:04] ERROR: Invalid architecture: 'noarch', expected something like: x86_64, aarch64, riscv64, armhf, x86, armv7
[12:01:04] See also: <https://postmarketos.org/troubleshooting>
[12:01:04] Traceback (most recent call last):
  File "/home/clayton/src/pmbootstrap/pmb/core/arch.py", line 42, in from_str
    return Arch(arch)
           ^^^^^^^^^^
  File "/usr/lib/python3.12/enum.py", line 757, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/enum.py", line 1171, in __new__
    raise ve_exc
ValueError: 'noarch' is not a valid Arch

```

[caleb: fixed tests]

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

Make sure to annotate the parameters.

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

In the code where the object is made (pmb/parse/apkindex.py) depends
already will become an empty array when there's no dependencies.

And there's no point in having a differentiation between None and empty
array.

Try to finish the migration by changing various links and texts to point
to the new GitLab instance.

Clayton Craft authored 5 months ago and Newbyte committed 5 months ago

pmb.parse.apkindex.providers now returns a dict of string:ApkindexBlock,
probably from 71772b9b

This adds type hinting to package_provider to reflect this, and fixes
code that calls it to properly deal with the possible return types.
Without this, building FDE images is broken.

Use the new "Artifacts" directive for .ci/docs.sh and run the job in the
pmbootstrap chroot to avoid needing to install sphinx dependencies on
the host.

Fixes #2352

And make the docs job run in the chroot now and grab the 'public/'
directory so we have the built docs.