Caleb Connolly authored 1 month ago and Clayton Craft committed 18 hours ago

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6182

[ci:skip-build]: already built successfully in CI

Caleb Connolly authored 1 month ago and Clayton Craft committed 18 hours ago

Introduce some initial unit tests for the initramfs.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6182

Caleb Connolly authored 1 month ago and Clayton Craft committed 18 hours ago

These two functions echo their output, requiring the use of a subshell
to capture it. However they also attempt to cache the output in global
variables.

Unfortunately, we can't do both. Variables set in the subshell aren't
reflected in the parent shell.

Rework these functions so they take a variable to set as a parameter
(like the "read" command). This avoids spawning a subshell and enables
the caching functionality that was added previously.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6182

David Furey authored 1 month ago and Pablo Correa Gomez committed 1 week ago

The USB network host address is used by some hosts to define the interface
name. Making the host address deterministic can simplify host network
configuration.

Part-of: !6232
[ci:skip-build]: already built successfully in CI

Felix Urbasik authored 7 months ago and Oliver Smith committed 1 week ago

This change allows mounting btrfs root filesystems which are spread
across multiple devices or partitions.

Part-of: !5508
[ci:skip-build]: already built successfully in CI

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

log splash messages with the new info log level. This can be useful when
working on headless devices.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

Support the rd.info cmdline argument to enable additional info logging,
and reduce the log level of some things to info so we print less
overall.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

Currently the initramfs is littered with various code snippets which
read /proc/cmdline. This is error prone and results in more complicated
code especially as we want to standardise all the pmos specific cmdline
arguments, and where some arguments have the same behaviour as
deviceinfo variables.

Let's do the parsing correctly, once, and set global variables to
enable/disable/configure parts of the initramfs. At the same time
introduce namespaced and standard versions of the remaining arguments
which haven't been standardised yet.

Now (for example) to check if the splash is disabled, rather than
grepping /proc/cmdline for PMOS_NOSPLASH AND checking if
deviceinfo_no_framebuffer is true, you can just check if "$nosplash" =
"y".

This will drastically simplify the process of adding additional
configuration options in the future.

Most of the code here was taken from the Arch Linux initramfs and
simplified/adjusted to better fix our use case.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

These functions are almost identical and follow the same logic.
Introduce a generic find_partition function which they can both call
into so we can abstract the logic away from the specific usage and make
everything much cleaner.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

Simplify things by putting all the log targets in one variable, and add
a special-case to log to consoles when pmos.nosplash is set and no
consoles are used by the kernel (e.g. because console=null is set).

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

Make it clearer what is exposed over USB and why rather than just
printing the cmdline option.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: !6172

It is used in the "2nd" part of the initramfs, not in the first one

[ci:skip-build]: already built successfully in CI

[ci:skip-build]: built successfully in CI for x86_64,x86,aarch64

riscv64,armv7,armhf failes at binfmt_misc, which isn't related to
this MR.

Caleb Connolly authored 2 months ago and Clayton Craft committed 1 month ago

Run e2fsck before resizing the root partition so that resize2fs will
actually succeed

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>
[ci:skip-build]: already built successfully in CI

Pretty sure when you already have a-z specified as range, second one
should be A-Z and not A-z.

Signed-off-by: Alexey Minnekhanov <alexeymin@postmarketos.org>

Caleb Connolly authored 2 months ago and Clayton Craft committed 2 months ago

when stdin is /dev/null (e.g. when console=null is on the kernel
cmdline) syslogd will call read() over and over and never handle the
SIGTERM we send it, pinning a CPU core on every boot.

Hooking up syslogd std I/O to /dev/zero prevents this.

Co-authored-by: Richard Acayan <mailingradian@gmail.com>
Co-authored-by: Clayton Craft <clayton@craftyguy.net>
[ci:skip-build]: already built successfully in CI

Minecrell authored 4 months ago and Clayton Craft committed 3 months ago

There is a special check in init_2nd.sh for older postmarketOS
installations that do not have a valid /etc/fstab file. When /boot does not
appear in fstab, it searches for the boot partition and mounts it manually.

However, /boot might be intentionally omitted in fstab, e.g. because we
just have a single rootfs partition that also contains /boot. In that case
it will wait forever, trying to find the missing boot partition.

The old installations do not have any entries in fstab, so fix this by
checking if the file is empty instead (with all comments and whitespace
removed).
[ci:skip-build]: already built successfully in CI

Minecrell authored 4 months ago and Clayton Craft committed 3 months ago

Revert "main/postmarketos-initramfs: mount subpartitions if root or boot is missing (MR 5625)" (MR 5920)

This reverts commit 1259d74b.

The main reason why that change was necessary is because we skipped
mounting subpartitions entirely if we found a potential root partition
(which could be simply a crypto_LUKS partition from a different distro).
Now that we look only for the actual root partition (based on the UUID),
this should not happen anymore.

Checking both conditions causes delays if there is no boot partition,
e.g. if pmOS was installed with --single-partition. Usually we don't need
to mount the boot partition anymore, because all needed files are part of
the initramfs. So let's drop the check for the boot partition again and
rely on the UUIDs to ensure we set up the correct partitions.

Minecrell authored 4 months ago and Clayton Craft committed 3 months ago

When using an encrypted installation of postmarketOS, the pmos_root_uuid=
on the cmdline only tells us the UUID of the crypto_LUKS partition. Once
the partition is unlocked, we perform the the old unreliable auto detection
again. This might mount the wrong partition if multiple installations of
pmOS are attached to the system.

After we unlock the root partition, we know exactly where the root
partition is supposed to be (= at /dev/mapper/root). Let's use that
directly instead of going through the whole detection sequence again.

Minecrell authored 4 months ago and Clayton Craft committed 3 months ago

Right now we fallback to the old behavior of searching for the "pmOS_root"
label when the root partition specified by pmos_root_uuid= on the cmdline
is not found. This works fine most of the time, but there are edge cases in
which this does not behave correctly. Problems occur especially if there
are multiple installations of pmOS (e.g. one on internal storage, and one
on USB drive/SD card). Depending on timing or enumeration order, the system
might boot into the wrong rootfs if the intended root partition shows up
too late.

For the boot partition we already enforce the UUID provided on the cmdline.
Let's apply the same for the root partition to ensure we always boot into
the correct root partition.

Minecrell authored 4 months ago and Alexey Minnekhanov committed 3 months ago

In addition to the pmos.debug-shell cmdline argument, add
pmos.usb-storage=<block-device> to set up the USB mass storage gadget by
default when going into debug shell. This allows building a special boot
image/option that directly exposes the storage for flashing, without having
to type in extra debug-shell commands.

Minecrell authored 4 months ago and Alexey Minnekhanov committed 3 months ago

Right now we have two similar implementations of setting up a USB mass
storage gadget, in init_functions.sh and as separate script for the
debug-shell. Given that init_functions.sh is available in debug-shell
anyway, unify both in a common setup_usb_storage_configfs() function.

Having this included in the initramfs debug-shell by default (without
installing postmarketos-mkinitfs-hook-debug-shell) also simplifies
recovery, since you can set up the USB storage for repairing the flashed
postmarketOS installation in case of boot failures. The size overhead is
minimal, since this has no extra dependencies.

Minecrell authored 4 months ago and Oliver Smith committed 3 months ago

BTRFS only supports online resizing. Right now we implement that by
temporarily mounting the file system, resizing it, and then unmounting it
again. But most of this is redundant since we mount the rootfs at /sysroot
immediately after.

Avoid this by splitting the file system resizing in offline resize (before
mount) and online resize (after mount) and resize BTRFS after mounting
/sysroot. For EXT4 we can either do offline resizing or online resizing.
Keep offline resizing for now for better compatibility with older kernels.
F2FS does not seem to support online resizing, so we need to keep the
functionality for offline resizing anyway.

Drop the splash for now, since it would introduce flickering on each boot.
We can restore it later if needed (ideally by adding a way to update the
splash message without restarting pbsplash).

Minecrell authored 4 months ago and Oliver Smith committed 3 months ago

Right now we use the resize2fs -f (force) option, which "Forces resize2fs
to proceed with the filesystem resize operation, overriding some safety
checks which resize2fs normally enforces." For example, this flag instructs
resize2fs to attempt resizing file systems with errors in some situations.

We primarily care about resizing the file system on initial boot, where we
shouldn't have any errors in the file system. For all other subsequent
boots, we shouldn't risk the filesystem data just to resize it again.
So let's drop the -f flag and only resize clean file systems.

Minecrell authored 4 months ago and Oliver Smith committed 3 months ago

At the moment, the root file system is only resized to span the entire
partition if we previously also resized the partition itself. However, if
a rootfs image without subpartitions is flashed to a pre-existing partition
this means that the installation only has very limited space available
without manual resizing.

Such a setup is used for devices using the "fastboot-bootpart" flasher
(e.g. asus-me176c), where boot and root images are flashed to two separate
partitions and the device boots directly from the boot partition.

Avoid this problem on these devices by always resizing the root file system
on each boot. All the resize tools exit early if the file system is already
expanded to the maximum size. It's faster to call resize2fs unconditionally
instead of manually checking if the file system needs resizing before.
Similar tooling like systemd-growfs also resizes unconditionally.

Oliver Smith authored 3 months ago and Anjan Momi committed 3 months ago

Ensure postmarketos-base-nofde is built before this package, otherwise
bootstrapping this on a new stable branch fails. This does not need to
be backported, as postmarketos-base-nofde has already been built for
v24.12.

Related: https://builds.sr.ht/~postmarketos/job/1384748

Minecrell authored 4 months ago and Oliver Smith committed 4 months ago

There might be additional kernel modules needed for the USB UDC to show up.
This used to work fine, but broke when the initramfs was unified with the
minimal one. We need to load modules/start udev before setting up USB.

Fix this by duplicating the setup calls in both init.sh and init_2nd.sh.
Both calls already have checks to allow calling them multiple times,
so we don't need to handle the initramfs-extra case in a special way.
[ci:skip-build]: already built successfully in CI

Changes the regex of the initramfs to make sure that it is searching for
filesystem type instead of any type (eg partition scheme type).

init_functions.sh makes use of the mass_storage and acm USB gadget
functions for error reporting, but the necessary kernel modules are not
included in the initramfs at the moment. Because of this, the functionality
works only on kernels where all these USB gadget functions are built-in and
not built as modules.

Add the missing modules and fix the sort order while at it.

[ci:skip-build]: bug in CI prevents build testing armv7, succeeded locally

Allows udev to use custom config when running from the initramfs

fixes #2989

Without these rules, the /dev/disk/by-* stuff isn't created on devices
that use subpartitions (and maybe FDE too?), and systemd boot fails
because it's unable to find the boot partition UUID under that path.

fixes !5636

[ci:skip-build]: already built successfully in CI

Fixes a bug where the kernel thinks the boot partition is still mounted
and it's unavailable for mounting in the rootfs

If you look at what the initramfs was doing before !5635, it was
explicitly umounting the boot partition from /boot in the initramfs and
then mounting it under /boot in the rootfs:
https://gitlab.postmarketos.org/postmarketOS/pmaports/-/blob/c45d7ec0a5f7ac814b50087402bb11825120f42b/main/postmarketos-initramfs/init.sh#L87

With this patch, we're now back to umounting /boot in the initramfs.
The second part of that original logic is now handled elsewhere now, and
that seems to be working OK.

The bug was reported here:
postmarketOS/pmaports#3279 (comment 454346)

[ci:skip-build]: already built successfully in CI

Caleb Connolly authored 5 months ago and Clayton Craft committed 5 months ago

importing all udev rules doesn't make sense here. Only add the needed
ones.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Co-authored-by: Clayton Craft <clayton@craftyguy.net>

[ci:ignore-count]
[ci:skip-build]: already built successfully in CI

Caleb Connolly authored 5 months ago and Clayton Craft committed 5 months ago

make the logging nicer and remove some irrelevant errors.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Co-authored-by: Clayton Craft <clayton@craftyguy.net>

Caleb Connolly authored 6 months ago and Clayton Craft committed 5 months ago

We previously forked the initramfs into a minimal variant for highly
space constrained devices, to allow the normal variant to gain
additional functionality like the debug-shell (With OSK). Initially the
hope was that only a handful of devices would need the minimal variant,
but in practise this has not been the case.

Trying to maintain two diverging forks of the initramfs is absoutely
unsustainable and (to be frank) a bit silly.

Based on discussion with craftyguy and newbyte, let's re-architect the
initramfs to run in two stages, with the second stage being part of
initramfs-extra. By itself this doesn't make a lot of sense, since
requiring the boot partition be mounted for the debug-shell to be useful
is a major downgrade.

However this can be used in tandem with pending support for a new
deviceinfo_create_initfs_extra variable, this is default false in
mkinitfs which finally gets rid of initramfs-extra entirely for devices
that don't need it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Co-authored-by: Clayton Craft <clayton@craftyguy.net>

Caleb Connolly authored 6 months ago and Clayton Craft committed 5 months ago

We need to use ls to check if the hooks dir contains any hooks
(otherwise the wildcard doesn't work). But ls prints an error if the dir
is missing or empty. Hide the error to not pollute the log.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>

Caleb Connolly authored 6 months ago and Clayton Craft committed 5 months ago

This was only added recently, is likely not used by anyone, and
introduces hard to resolve dependencies on /boot being mounted which we
may not want in the future.

Remove it to drop some complexity.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>