Make a consistent plan for postmarketOS installer(s) and recovery UI

Idea: Make a consistent plan for postmarketOS installer(s) and recovery UI

Summary

We have had various postmarketOS installer/TWRP-like recovery UI approaches and ideas over the years. We should reconsider them and come up with a consistent plan and figure out how we can implement it. A goal for most of these was to make it not tied to postmarketOS, but coming up with something that could work for other distros like Mobian too if they would like to use it.

Currently @craftyguy, @JustSoup321 and @caleb seem to be invested in this, maybe you could collaborate on coming up with a consistent PMCR for this?

Prior art

on device installers / recover UI

• postmarketos-ui-os-installer (pmaports!6320): boots a live gnome image with os-installer, which runs pmbootstrap under the hood to make a very nice looking installer for graphical desktops. Needs an internet connection and not sure how well it works on mobile. Looks like a nice, practical solution we can use for now but it would be great to figure out the larger story as well where we want this to go.

• ondev1: calamares based offline installer that allowed to e.g. set up an encrypted installation, set user passwords, choose the filesystem and to install from SD to eMMC for the PinePhone. It had various problems, such as having many installation paths that could not be automatically tested while at the same only supporting very few devices, calamares being rather heavy, the image build process via pmbootstrap was complicated, and too many choices during the installer that only make sense for very technical users (such as choosing the filesystem). ondev1 is not used in pmOS anymore since v23.06. (screenshots)

• ondev2: unl0kr based offline installer that addressed some of the ondev1 issues, having the choice between a simple and advanced install, being a lot more lightweight (initramfs only), an interface that works for text and screen at the same time with only one input shown at a time (compared to desktop installers which have many of those) which allows scripting the install for automated testing with expect and would later allow easy text-to-speech and speech-to-text integration for accessibility. This was never finished though. There were also future ideas of making initramfs stubs that people would download, which would contain ondev2 without an image to install, the user would then connect to wifi with it and download either a pre-built image as we have it now, or an immutable image, or possibly have apk running and assemble an installation. We could provide these stubs for lots/maybe almost all (?) devices, so users could just flash them or netboot them and install to persistent storage (or even to RAM for testing?) pmOS without using pmbootstrap; possibly even through a web usb installer. (fosdem talk)

• BuffyBox: the excellent FDE unlocking program unl0kr was moved into BuffyBox with the idea to extend its scope further while using the same code. Ideas @cherrypicker, @DylanVanAssche, @caleb and I had were making a recovery UI similar to TWRP, allowing the user to optionally encrypt their installs on first boot when using a pre-built image, and/or integrating some of the ondev2 features/planned features.

• "Live CD" support: @JustSoup321 had the idea to provide live environments. this could be particularly useful for devices where persistent storage is not working yet (iphones) or just to try various UIs out. See also pmbootstrap netboot.

desktop installers

• starting an install right from the homepage (postmarketos.org#156): some unfinished ideas for directly starting an install from the homepage, to make it as seamless as possible for users.

• ubports installer: from the nice people at Ubuntu Touch, supports installing postmarketOS for some devices such as some Fairphones. Uses index.json from bpo and downloads pre-built images from there.

• yotlip: desktop installer, unfinished

• gpmbootstrap: gtk frontend proof of concept for running pmbootstrap, unfinished

mentioned in merge request pmaports!6320

changed the description

changed the description

Thanks. I know we've discussed this a few times in the past, iirc for me the ideal way to have this work is:

• User account, locale, and encryption are all handled during first boot (using online encryption for FDE, and systemd-homed for immutable). We want as much stuff in this "common" path as possible that can be the same between desktop/laptop and phone. A UI based on buffybox (that runs in the initramfs) would be the easiest solution here
• For devices with EFI bootloaders that can boot from a USB drive (desktop/laptop/maybe some phones in the future with u-boot), we should leverage os-installer. The pmbootstrap backend is fine, we'll likely have an image-based backend in the future for immutable.
• For devices that require flashing, either with a mechanism like fastboot or writing an image to an sdcard, we should standardise on:
  • the web flasher - where possible it is the most seamless method
  • ubports installer / some CLI tool to download images and run fastboot (or DFU?) in an Alpine chroot - keep is simple for users, don't require them to install android-tools or whatever.
  • balena etcher / dd - for flashing sdcards

Essentially, we aim to get postmarketOS onto the internal storage of the target device as seamlessly as possible, and then offer a consistent setup experience.

Thanks, sounds good overall.

balena etcher

seems shady to me. the code is open source, but they do targeted advertising on their homepage and in general I get the vibe that they use this dd gui just to push their other paid products... so I'd rather not recommend this.

I commented on a previous issue on the same topic, but can't find it right now.

I'd rather see a substantially different approach to this:

• Installation images boot and then run entirely from tmpfs.
• The installer creates partitions (with FDE) as instructed by the user at this time. The flow here is extremely similar as installing a "typical" Linux distro from live media.
• pmOS is installed onto the newly created partitions, entirely overwriting the installation image.

You'll note that the result is quite similar, but there's a lot of additional flexibility here (without really having additional complexity):

• We can perform any sort of customisation before installing from this live environment. Like choosing a different file-system, or partition layout.
• The same "live environment" infrastructure is usable by both a GUI-based installation image and by a plain BSD-style terminal-based installation flow.
• An installation image running one UI can install a different UI (assuming a package cache or internet connectivity is available). This has a lot of potential, since I can give you a phone which boots into one live environment, but you can later choose to re-install another directly from the device.
• By booting the installation image, you get an ephemeral test environment where you can experiment before performing a definitive installation.

+1 from me regarding Buffybox as the first-run UI, some UIs do have this capability (GNOME, Phosh, etc.) but not all. Having a single UI here from the initramfs as first-run would be great for a consistent UX.

I made some designs/mockups in this issue: https://gitlab.com/postmarketOS/pmaports/-/issues/1356

Other mockups:

I'll put some thoughts here.

It has to be determined how recovery is built and installed. Given that boot partition sizes differ a lot amongst devices, I believe there should be at least two ways to install recovery - either in boot partition as part of initramfs or to a separate partition. The ability to install recovery in boot can be determined by deviceinfo (just like there is an option for splitting initramfs for storage-constrained devices), and then user can do something like pmbootstrap install --recovery-bootimg to generate separate image and export it later or choose to include recovery in boot image during init step.

The recovery can have a modular structure, so it includes some core functions, common for all devices, and additional ones, which could be a bunch of shell scripts of different purpose, shown in recovery as a separate list. Then, users may include their own scripts in recovery. Devices may define additional recovery scripts either as a file of device's subpackage (if it's very specific) or install recovery script packages via _pmb_recommends. If recovery is part of boot, update it with mkinitfs. Maybe that can be done for a separate recovery too.

mentioned in issue postmarketos#38 (closed)