pmb.config/install: add flexible provider selection for "pmbootstrap init"

The "provider selection" for pmbootstrap init added in this MR is a flexible way to offer UI/device-specific configuration options in pmbootstrap init, without hardcoding them in pmbootstrap. Instead, the options are defined entirely in pmaports using APK's virtual package provider mechanism. The code in pmbootstrap searches for available providers and displays them together with their pkgdesc.

There are many possible use cases for this but I have tested two so far:

1. Selecting root provider (sudo vs doas). This can be defined entirely in postmarketos-base, without having to handle this specifically in pmbootstrap.

   $ pmbootstrap init
   [...]
   Available providers for postmarketos-root (2):
    * sudo: Use sudo to run root commands (**default**)
    * doas: Use doas (minimal replacement for sudo) to run root commands
            (Note: Does not support all functionality of sudo)
   Provider [default]: ...

   pmaports MR: https://gitlab.com/postmarketOS/pmaports/-/merge_requests/2623

2. Device-specific options. My main motivation for working on this feature is a new configuration option for the MSM8916-based devices. It allows more control about which firmware to enable:

   $ pmbootstrap init
   [...]
   Available providers for soc-qcom-msm8916-rproc (3):
    * all: Enable all remote processors (audio goes through modem) (**default**)
    * no-modem: Disable only modem (audio bypasses modem, ~80 MiB more RAM)
    * none: Disable all remote processors (no WiFi/BT/modem, ~90 MiB more RAM)
   Provider [default]: ...

   pmaports MR: https://gitlab.com/postmarketOS/pmaports/-/merge_requests/2632

The configuration prompts show up dynamically by defining _pmb_select="<virtual packages>" in postmarketos-base, a UI PKGBUILD or the device APKBUILD. Selecting default (just pressing enter) means that no provider is selected. This allows APK to choose it automatically based on the provider_priority. It also provides compatibility with existing installation; APK will just choose the default provider when upgrading. The selection can still be changed after installation by installing another provider using "apk".

Note that at the end this is just a more convenient interface for the already existing "extra packages" prompt. When using pmbootstrap in automated scripts the providers (e.g. postmarketos-root-doas) can be simply selected through the existing extra_packages option.

changed the description

By Minecrell on 2021-10-22T11:49:21

added 3 commits

• 3512b31f - pmb.install: move install_recommends check into _install.py
• b097611b - pmb.parse.apkindex: parse "T" key for pkgdesc
• 0aac471d - pmb.config/install: add flexible provider selection for "pmbootstrap init"

Compare with previous version

By Minecrell on 2021-10-22T11:59:54

added 1 commit

• a44e9a86 - pmb.config/install: add flexible provider selection for "pmbootstrap init"

Compare with previous version

By Minecrell on 2021-10-22T12:23:23

added 1 commit

• a98f5e5d - pmb.config/install: add flexible provider selection for "pmbootstrap init"

Compare with previous version

By Minecrell on 2021-10-23T19:16:33

For reference, the selected providers are stored in pmbootstrap.cfg like this:

[pmbootstrap]
device = qemu-amd64
extra_packages = none
...

[providers]
postmarketos-root = postmarketos-root-doas
...

By Minecrell on 2021-10-23T19:18:57

Available providers for soc-qcom-msm8916-rproc (3):
 * all: Enable all remote processors (audio goes through modem) (**default**)
 * no-modem: Disable only modem (audio bypasses modem, ~80 MiB more RAM)
 * none: Disable all remote processors (no WiFi/BT/modem, ~90 MiB more RAM)

This looks like a nice prompt, and it makes sense IMHO to transition away from using the "multiple kernels" feature for all kinds of prompts to having a more generic prompt feature. And we can have multiple prompts where it makes sense, which is nice. So overall I like the feature, but this will need some more work. Thanks @Minecrell!

By Oliver Smith on 2021-10-24T11:52:36

changed milestone to %v21.12 Release

added 13 commits

• a98f5e5d...caf7973e - 3 commits from branch master
• 828757bf - pmb.config.init: parse deviceinfo only once
• 1b3cab34 - pmb.install: move install_recommends check into _install.py
• 74b945bd - pmb.parse.apkindex: parse "T" key for pkgdesc
• de43236a - pmb.config/install: add flexible provider selection for "pmbootstrap init"
• fe04b6f8 - Revert "pmb.parse.apkindex: parse "T" key for pkgdesc"
• f2f8378e - pmb.parse._apkbuild: parse provider_priority as int
• bd763577 - pmb.helpers.pmaports: separate finding pkg in APKBUILD to function
• 31c8daf4 - pmb.helpers.pmaports: parse guessed APKBUILD first for speed up
• d7a56552 - pmb.helpers.pmaports: search for provides in both root and subpackages
• 7d27d991 - pmb.config.init: get providers from APKBUILDs instead

Compare with previous version

By Minecrell on 2021-11-01T08:59:57

added 4 commits

• 2fc5c41f - pmb.helpers.pmaports: separate finding pkg in APKBUILD to function
• 6df64f1c - pmb.helpers.pmaports: parse guessed APKBUILD first for speed up
• f1ac5c5d - pmb.helpers.pmaports: search for provides in both root and subpackages
• f78ad93f - pmb.config.init: get providers from APKBUILDs instead

Compare with previous version

By Minecrell on 2021-11-01T09:03:19

I pushed some commits to improve the performance using the guess and changed the code to parse the providers from the APKBUILD. I haven't squashed the old APKINDEX code yet in case we still change our mind for some reason but you can look if you like this approach better. :)

        # Try to guess based on the subpackage name
        guess = guess_main(args, package)
        if guess:
            # ... but see if we were right

This is an awesome optimization - it will make finding providers faster in general!

So yes, I would suggest to move forward with this. I looked at the diff (not at individual commits yet) and only have some nitpicks, it looks pretty good!

Thanks a lot for the patches, looking forward to get this in :>

By Oliver Smith on 2021-11-01T20:48:33

added 5 commits

• e29cff9d - pmb.helpers.pmaports: separate finding pkg in APKBUILD to function
• 3142e852 - pmb.helpers.pmaports: parse guessed APKBUILD first for speed up
• 31d803b3 - pmb.helpers.pmaports: search for provides in both root and subpackages
• 9bb87c2e - pmb.parse._apkbuild: parse provider_priority as int
• 35056491 - pmb.config/install: add flexible provider selection for "pmbootstrap init"

Compare with previous version

By Minecrell on 2021-11-01T21:37:35

changed the description

By Minecrell on 2021-11-01T21:49:44

approved this merge request

By Oliver Smith on 2021-11-06T14:09:48

LGTM. This has been open for two weeks now, and all review comments have been addressed. So let's merge it.

Thanks @Minecrell!

By Oliver Smith on 2021-11-06T14:09:53

added 16 commits

• 35056491...05f25729 - 9 commits from branch master
• 9d724d5d - pmb.config.init: parse deviceinfo only once (MR 2132)
• 7246c325 - pmb.install: move install_recommends check into _install.py (MR 2132)
• eb3e38d1 - pmb.helpers.pmaports: separate finding pkg in APKBUILD to function (MR 2132)
• 5ed807c0 - pmb.helpers.pmaports: parse guessed APKBUILD first for speed up (MR 2132)
• 19b23296 - pmb.helpers.pmaports: search for provides in both root and subpackages (MR 2132)
• 47645f41 - pmb.parse._apkbuild: parse provider_priority as int (MR 2132)
• 8ace3611 - pmb.config/install: add flexible provider selection for "pmbootstrap init" (MR 2132)

Compare with previous version

By Oliver Smith on 2021-11-06T14:10:25

resolved all threads

By Oliver Smith on 2021-11-06T14:13:40

enabled an automatic merge when the pipeline for 8ace3611 succeeds

By Oliver Smith on 2021-11-06T14:14:12

merged

mentioned in issue pmaports#1379

By Oliver Smith on 2022-01-09T10:06:50

mentioned in issue pmaports#1911

By Oliver Smith on 2023-01-19T07:06:47