Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Since parse.deviceinfo() returns a custom class now, it can't be
trivially serialized as JSON.

This has been broken for months now so it's clear nobody is using this.
Let's take the chance to get rid of it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

For the JSON output to be machine-readable we can't have random other
logging messages in the mix. Print the metadata to stderr instead of
stdout.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Move the apk_static module from pmb/chroot to pmb/helpers which is a
more suitable location.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Since all apk invocations withing pmbootstrap now use apk.static by
default, apk_static.run() isn't called anymore, so let's remove it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

In 0b4fb911 (chroot: always run apk static v2 (MR 2423)) we adjusted
install_run_apk() to run apk static on the host and pass in the local
binary repo with "--repository". This function can call apk in two ways,
either with the progress bar handling or without, the second case was
never updated and still ran apk inside the chroot incorrectly and with
an incorrect --repository flag.

Let's finish the job by refactoring helpers/apk.py to support all our
usecases and pointing everything to it, removing the last few situations
where we call "pmb.chroot.root(["apk", ...]).

The apk_with_progress() function is replaced by a generic "run()"
function which takes a boolean to indicate if we should render apk
progress.

Additionally, a new cache_clean() function is added so that "pmbootstrap
zap --pkgs-online-mismatch" can FINALLY be refactored to not rely on a
chroot existing. This requires some hacks but nothing serious, see the
comments in the function for details.

The chroot.init() code is now simplified since handling the --root,
--arch, and --cache-dir flags is now all done by apk._prepare_cmd().

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

This function better belongs here, especially as it will be used outside
of the context of a chroot() soon.

Additionally, it's adjusted to take the rootfs path as its first
argument rather than a chroot, since it could operate on any rootfs.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Lazy load pmb.config.styles and move the pmb.__version__ print elsewhere
so the logging module is (closer to) a standalone entity. This is
necessary to be able to import it in pmb/helpers/apk.py otherwise we get
a cyclical dependency.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Make intellisense happy

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

We broadly support "Arch" objects in command invocations, adjust the
PathString type to properly represent this.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

Some packages in aports have their own subpackages in their
dependencies, for some reason. This causes our resolver to assume there
is a cyclical dependency.

Adjust the resolver to safely ignore these dependencies, since they'll
be built as a part of the package that depends on them.

For example, modemmanager depends on libmm-glib which is a subpackage of
modemmanager. Currently to build modemmanager in pmaports it would be
necessary to manually modify the APKBUILD after copying it from aports.
This change avoids that.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

args.autoinstall attribute might not exist, use getattr like for others,
and add a fixme about the default value...

Fixes: #2476


Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Newbyte authored 5 months ago and Oliver Smith committed 5 months ago

All call sites that use this have been removed. This was only introduced
as a stopgap until we could refactor more code to not use args
everywhere, and that has now been achived. As such, remove this so it
doesn't get used in more locations.

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

See
#2473



Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

Add a basic test to ensure that the cache does get hit as we expect.
This doesn't test for cache conflicts.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

Use a different way of keying the apkindex path cache.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 5 months ago and Oliver Smith committed 5 months ago

It may be desirable to test older versions of packages, allow them to be
built when --force is specified and print a warning that they won't be
installed automatically.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Vikram Alagh authored 5 months ago and Oliver Smith committed 5 months ago

Seems to be ChromeOS specific:

> this happens every time on my machine. I am on ChromeOS 129.0.6668.112
> using a Debian bookworm chroot through Crouton. I am not sure why
> this is happening nor which layer is responsible for this. I deleted
> the chroot and created another one but the problem persists.

Commit msg tweaked by Oliver.

Provides kpartx.

Looks like an unintended leftover from debugging.

---

This was needlessly reverted by
1ec55fc1. While revreting
0975d064 was good, this commit did not
cause any harm.

And fix type errors.

---

This was needlessly reverted by
1ec55fc1. While revreting
0975d064 was good, this commit did not
cause any harm.

Running these inside the chroot is tricky, and we ended up reverting
previous commits that attempted to make them run there (see
1ec55fc1). As such, just accept defeat
and ensure that the host has these.

Also see postmarketOS/pmbootstrap#2465

We fully expect this to happen if a user hasn't configured their host
system properly. As such, don't treat it like a programming error.

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

We now run the job in chroot, so remove the unnecessary check. If
sphinx-build is missing for some reason, it will fail a bit further in
the script anyways.

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

config.sudo_timer is a bool, and the returned value here is a bool. No
point in converting it to a str.

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

I don't think there's any case where the number of jobs would have to be
a string. It's also being assigned an integer elsewhere the code (in
ask_for_additional_options() inside of init.py), so an integer seems
like what we actually want. Also fix type errors resulting of this.

Newbyte authored 5 months ago and Caleb Connolly committed 5 months ago

These are artefacts from when our line length was shorter. Ruff didn't
really do a great job at fixing them up when autoformatting.

Luca Weiss authored 5 months ago and Caleb Connolly committed 5 months ago

For the gitlab.com -> gitlab.postmarketos.org migration we needed to
also migrate the workdir in pmb v2 since pmb v3 is not ready yet.

Therefore we need to handle that "7-2.x" workdir version and migrate
correctly when switching to pmbootstrap v3.

See !2445

This works around https://gitlab.postmarketos.org/postmarketOS/
pmaports/-/issues/3268, by always installing hexdump when building
packages. The full hexdump is small (like 80KB), and even though this
isn't required in most situations it seems like the best of a group
of bad workarounds, until upstream/busybox fix the bug in the pmaports
issue.

So we don't go through the pain of pushing 3 times to get 3 different
ruff failures in CI.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Add a test to parse some APKINDEX data.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

should be an int, but was always a string...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

I guess this is leftover from some experimentation.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Also adapt pmb/install/recovery.py to new API for variables.py.

[ci:skip-build]: already built successfully in CI

Clayton Craft authored 5 months ago and Newbyte committed 5 months ago

Sometimes these fields are not set, e.g.:

pdb> p ret
{'depends': ['build-base', 'bash', 'bison', 'findutils', 'flex', 'musl-dev', 'openssl-dev', 'perl', 'python3'], 'arch': 'noarch', 'version': '20241022.022230', 'pkgname': '.makedepends-linux-lenovo-21bx', 'provides': []}

I hit this crash when I installed the build deps for the
linux-lenovo-21bx package in the native chroot, and then ran
`pmbootstrap stats`
[ci:skip-build]: already built successfully in CI