Image size estimation when building on zfs dataset with compression enabled is incorrect, may cause build failure

When the work path of pmbootstrap is a zfs dataset with compression enabled, the amount of disk space used by the system image is smaller than its logical size. pmbootstrap seems to use this on-disk size rather than the logical size for creating the image file for the loopback mount. This causes copying the system image to the file to fail with 'cp: write error: No space left on device', since the filesystem needs to have enough logical space to contain the image. (Despite the image file also being compressed on disk.)

The build failure can be prevented by setting compression to off for the zfs dataset. faillog.txt

Hi @futune,

in !1743 (merged) we dropped the --apparent-size parameter of du which results in smaller files on non-compressed filesystems.

What does the du --apparent-size results with the compressed zfs?

Reference: https://gitlab.postmarketos.org/postmarketos/pmbootstrap/-/blob/master/pmb/helpers/other.py#L22

By Daniele Debernardi on 2020-05-01T01:50:21

added type::bug label

After changing the related size calculation code several times, such as in the MR linked by @drebrez above, I think that we just can't get it right for all file systems. So I recommend that we add an option in pmbootstrap init to add additional space for the rootfs: #1904 (closed)

What do you think about that, and would you like to take a shot at it, @futune?

Thanks for creating the issue!

By Oliver Smith on 2020-05-10T12:13:24

Related to this - the use of --summarize in du requires that coreutils are installed, this option isn't present in busybox du and so, for example, pmbootstrap won't run on postmarketOS by default even if you install git and openssl as requested.

By Daniel Edgecumbe on 2020-06-28T18:35:23

Yup, and you also need procps, which is outlined in https://gitlab.postmarketos.org/postmarketos/pmbootstrap#requirements (although i wouldn't mind running pmbootstrap on system with only busybox too ^^)

By Dolphin von Chips on 2020-06-28T18:35:23

@dustfinger did hit this (log). Adding --apparent-size to pmbootstrap/pmb/helpers/other.py solved the problem. Maybe we should add the option back with scale factor to account for the extra space the metadata takes.

a quick test on my home directory (ext4) shows that what --aparent-size reports should be multiplied by 1.00524 to get real size back for me. I'd say adding a percent or two seems safe, especially as not doing it worked once (for plasma-mobile-extras), and given that we have some margin anyway.

By Antoine Fontaine on 2020-07-05T14:01:30

mentioned in issue #1957 (closed)

By Antoine Fontaine on 2020-08-11T09:28:04

Hey, shouldn't we close this issue since !1989 (merged) got merged? I added a section about this problem at the troubleshooting page since it took me a long time to find the issues on GitLab. As I'm not a native speaker, I would be glad if someone could read over it :) Thanks and greetings.

By Alexander Brzoska on 2020-12-23T12:58:39

Seems like we can now use workaround specified in https://wiki.postmarketos.org/index.php?title=Troubleshooting&type=revision&diff=16100&oldid=15653 , so we can probably close this?

By Alexey Min on 2020-12-23T18:47:53

closed