Draft: pmb.install: support creating fstab and crypttab

Draft because it is not yet tested in any real conditions and I want to make a boot-deploy patch before merging this to make sure this all works together

See pmaports#1531

pmb/install/_install.py

@@ -648,6 +648,73 @@ def get_partition_layout(reserve, kernel):
     return ret
 
 
+def get_uuid(args, partition):
+    """
+    Get UUID of a partition
+
+    :param partition: block device for getting UUID from
+    """
+    return pmb.chroot.root(
+        args,
+        [
+            "blkid",
+            "-s", "UUID",
+            "-o", "value",
+            partition,
+        ],
+        output_return=True
+    ).rstrip()
+
+
+def create_crypttab(args, layout, suffix):
+    """
+    Create /etc/crypttab config
+
+    :param layout: partition layout from get_partition_layout()
+    :param suffix: of the chroot, which crypttab will be created to
+    """
+
+    luks_uuid = get_uuid(args, f"/dev/installp{layout['root']}")
+
+    crypttab = f"root UUID={luks_uuid} none luks\n"
+
+    open(f"{args.work}/chroot_{suffix}/tmp/crypttab", "w").write(crypttab)
+    pmb.chroot.root(args, ["mv", "/tmp/crypttab", "/etc/crypttab"], suffix)
+
+
+def create_fstab(args, layout, suffix):
+    """
+    Create /etc/fstab config
+
+    :param layout: partition layout from get_partition_layout()
+    :param suffix: of the chroot, which fstab will be created to
+    """
+
+    # Do not install fstab into target rootfs when using on-device
+    # installer. Provide fstab only to installer suffix
+    if args.on_device_installer and "rootfs_" in suffix:
+        return
+
+    boot_dev = f"/dev/installp{layout['boot']}"
+    root_dev = f"/dev/installp{layout['root']}"
+
+    boot_mount_point = f"UUID={get_uuid(args, boot_dev)}"
+    root_mount_point = "/dev/mapper/root" if args.full_disk_encryption \
+        else f"UUID={get_uuid(args, root_dev)}"
+
+    boot_filesystem = args.deviceinfo["boot_filesystem"] or "ext2"
+    root_filesystem = pmb.install.get_root_filesystem(args)
+
+    fstab = f"""
+# <file system> <mount point> <type> <options> <dump> <pass>
+{root_mount_point} / {root_filesystem} defaults 0 0
+{boot_mount_point} /boot {boot_filesystem} defaults 0 0
+""".lstrip()
+
+    open(f"{args.work}/chroot_{suffix}/tmp/fstab", "w").write(fstab)
+    pmb.chroot.root(args, ["mv", "/tmp/fstab", "/etc/fstab"], suffix)
+
+
 def install_system_image(args, size_reserve, suffix, step, steps,
                          boot_label="pmOS_boot", root_label="pmOS_root",
                          split=False, sdcard=None):
@@ -681,6 +748,14 @@ def install_system_image(args, size_reserve, suffix, step, steps,
 
     pmb.install.format(args, layout, boot_label, root_label, sdcard)
 
+    # Create /etc/fstab and /etc/crypttab
+    create_fstab(args, layout, suffix)
+    if args.full_disk_encryption:
+        create_crypttab(args, layout, suffix)
+
+    # Run mkinitfs to pass UUIDs into cmdline
+    pmb.chroot.root(args, ["mkinitfs"], suffix)
+
     # Just copy all the files
     logging.info(f"*** ({step + 1}/{steps}) FILL INSTALL BLOCKDEVICE ***")
     copy_files_from_chroot(args, suffix)

[Administrator]

You lose the file descriptor that open(...) returns here (and for crypttab), but you have to close it like f.close(). The usual way to open/write files is a with statement:

with open(...) as f:
    f.write(fstab)

By Alper Nebi Yasak on 2022-10-26T15:09:26

[Administrator]

Can't you just write directly to /etc/fstab?

By Alper Nebi Yasak on 2022-10-26T15:04:59

[Administrator]

This will not work, because pmbootstrap is not run with root user, and pmb.chroot.root() uses sudo

By jenneron on 2022-10-26T16:01:24

[Administrator]

I couldn't get these working, but I think I know what goes wrong. The chroot is shut down above, these restart the chroot automatically, but the copy_files_from_chroot call depends on it being shut down (tries removing qemu binary stub, copying /proc and other chroot-only files).

Calling pmb.chroot.shutdown(...) after these doesn't work, because it removes the loop devices and filesystems that were mounted above with pmb.install.format(...) and others. In that case the files are copied to a non-mounted /mnt/install (not to the sdcard/image) and write_cgpt_kpart fails because installp# devices no longer exist. Re-doing the loop device mounting steps after this might work but would be a weird code duplication.

It would be easier if the chroot was directly created on the partitioned mounted target. That's what other distro bootstrapping tools do AFAIK. I don't know why postmarketOS creates the rootfs and then does partitioning and copies it to target, but I assume there's an interesting reason.

I guess what you should do is to generate random UUIDs in Python and use those for both steps. Write fstab/crypttab files in create_device_rootfs based on those, and pass the same UUIDs to the pmb.install.format like root_label when creating the partitions (which should end up in mkfs.ext4 -U <UUID>, cryptsetup luksUUID <UUID> and so on).

By Alper Nebi Yasak on 2022-10-26T15:39:20

[Administrator]

Yeah, I have already found it, but I do not know a good solution for this. I think it would be good enough to unmount qemu binary just after mkintifs, but I do not have time for figuring out how to do that

I guess what you should do is to generate random UUIDs in Python and use those for both steps

I have already thought about this. It cannot be done for F2FS which we support

By jenneron on 2022-10-26T15:46:31

[Administrator]

I think it would be good enough to unmount qemu binary

You also have to avoid copying /proc/*, /dev/* (and idk what else) to the target, maybe by passing flags to cp and rsync. Actually you should be able to exclude the qemu binary the same way, instead of removing it.

It cannot be done for F2FS which we support

That's sad, and so you have to create the partitions before fstab/crypttab.

Maybe you can split this install_system_image function into two exactly here, and run the partitioning parts before create_device_rootfs, create fstab/crypttab in create_device_rootfs, and run the rest after it. Either way it's a bit of work.

By Alper Nebi Yasak on 2022-10-26T16:12:21

[Administrator]

It would be easier if the chroot was directly created on the partitioned mounted target. That's what other distro bootstrapping tools do AFAIK. I don't know why postmarketOS creates the rootfs and then does partitioning and copies it to target, but I assume there's an interesting reason.

The reason is, so we know the size of the rootfs and can create an image that isn't much bigger.

Regarding this error:

(016469) [16:41:05] ERROR: Command failed (exit code 1): % sudo rm /home/user/.local/var/pmbootstrap/chroot_native/mnt/rootfs_pine64-pinephone/usr/bin/qemu-aarch64-static

This is because qemu-aarch64-static needs to be umounted before it can be removed. But don't use that path, this is the aarch64 rootfs inside the native chroot. Instead it would be this path:

# umount /home/user/.local/var/pmbootstrap/chroot_rootfs_pine64-pinephone/usr/bin/qemu-aarch64-static

After this, the rm command does not fail anymore. So I suggest to add the umount command before the rm command. It probably also makes sense to refactor this code block into a separate function in pmb.chroot.binfmt. I'd do that as separate commit before the other changes.

By Oliver Smith on 2022-11-02T15:50:36

[Administrator]

I still can't figure it out. I'm not sure it is doable without refactoring much of pmbootstrap code

This mkintifs execution also mounts a lot directories for chroot which affects code later. If we unmount specifying all needed paths it will become messy. If we unmount everything the later code won't work

UPD: I may have a few more things to try UPD2: It seems that I've already got it to work 3-4 hours ago. I was confused because FDE is completely broken in pmOS right now, it just destroyes encrypted partition instead of unlocking it

By jenneron on 2023-07-16T10:44:54