Package alternative mkbootimg/unpackbootimg (osm0sis' fork of the AOSP/CM C code)?

Created by: ollieparanoid

In #145 (closed) we were discussing using this fork as alternative for both mkbootimg and unpackbootimg. I have even mailed the author if he could add tags to the repository, so we could download properly versioned archives instead of a random git hash.

In the end, we didn't even need to change it, because the tool we use already outputs all needed information. But the author has responded and started to tag releases now: https://github.com/osm0sis/mkbootimg/releases

...so I feel like I should at least let everyone know that we have the option to do this!

Some properties of this tool:

• Based on AOSP's and CyanogenMod's C code for mkbootimg/unpackbootimg (the current code is based on Python, since mkbootimg in AOSP was rewritten in Python)
• Has proper version tags
• Has as goal to support all kinds of devices (also has the qtdb feature, we recently switched the mkbootimg source for that)
• Prints out more than our current unpackbootimg
• Seems to be well maintained

So what we have right now works for us and I am not that interested anymore in changing it. But if you want to have this version of mkbootimg or if you are against it with a good reason, please comment here.

An advantage to have this one instead of what we have now (python based) is that we will not install anymore the python package in the device chroot and reduce the system image. Is it correct @ollieparanoid ?

By Daniele Debernardi on 2017-08-31T21:18:11

Created by: ollieparanoid

We need Python for postmarketos-splash. If we replaced that with C/C++ code, then yes. But until then we need Python anyway.

By * postmarketOS Bot on 2017-08-31T21:52:45

@ollieparanoid why don't we build both (splashscreens and bootimg) in native chroot?

By Daniele Debernardi on 2017-08-31T22:02:09

In pmbootstrap yes, but if you update packages on your phone after the initial build you run mkinitfs and thus make-splash on the phone directly

By Martijn Braam on 2017-09-01T06:53:29

Created by: ollieparanoid

Pinging @osm0sis so he does not miss the discussion. We still did not decide whether we actually use it though. Thanks for making a release which we could package!

@drebrez: in other words, we want to be able to do all upgrades on the phone without pmbootstrap, that involves building the initramfs with the splash screens.

By * postmarketOS Bot on 2017-09-02T03:55:49

Hey, thanks for the ping, and for the consideration. Just for reference, my fork is definitely more updated than others, including proper unpackbootimg support for extended cmdline which I don't think any others have right now, and sha256 support.

By osm0sis on 2017-09-01T23:52:33

Created by: ollieparanoid

I've thought about how to package this, so we could have it side by side with the current mkbootimg. I propose we package it as cmkbootimg (because it is based on the C version of mkbootimg).

By * postmarketOS Bot on 2017-09-10T11:05:14

Created by: ghost

@osm0sis , exactly what is the license for your fork? I notice it still retains the NOTICE file from the original project but I'm not sure what that license is. The python version says it's Apache 2.0.

By * postmarketOS Bot on 2017-11-18T01:08:00

https://github.com/osm0sis/mkbootimg/blob/master/mkbootimg.c

Also Apache 2.0. I think pretty much all of AOSP is, outside of the kernel code. The lineage of it all is CyanogenMod (so a fork of AOSP) and AOSP itself, and then of course my and others' patches to add/backport features, so should be Apache all the way down the line.

In addition to the features I mentioned in my last comment it's also worth noting that my fork is the only C one supporting the os_version and os_patch_level header fields, and Pixel 2 now enforces these for its "rollback protection". There's nothing in the Python version that isn't in mine, and mine supports far more devices than the Python version so I'm really not sure why all the hesitation.

By osm0sis on 2017-11-18T01:30:01

Created by: ollieparanoid

Sorry, but no one got around packaging it yet, that's the only reason.

By * postmarketOS Bot on 2017-11-18T03:06:57

Created by: Matt07211

After seeing @Akuhak adding support for pxa layout bootimgs into Magisk it brought magiskboot to my attention, it's pretty much a renamed fork of mkbootimg with it functionality extended to handle a large variety of boot.img that don't conform to the standard boot.img layout, this includes (but not limited to), MTK images, ELF images, PXA images, Chrome OS etc.

Overall it's compatibility with multiple styles of boot.imgs should make it a prime candidate for this, as a replacement and will mostly make a few binarys redundant in the pmos repo.

By * postmarketOS Bot on 2018-01-30T21:26:47

Created by: ghost

@ollieparanoid, why don't we use plymouth for boot splash?

By * postmarketOS Bot on 2018-01-30T23:17:14

Created by: ollieparanoid

Please create a new issue to ask such questions if it has nothing to do with the current issue. I'll be happy to answer your question there.

By * postmarketOS Bot on 2018-01-30T23:28:37

Created by: ghost

We need Python for postmarketos-splash. If we replaced that with C/C++ code, then yes. But until >then we need Python anyway.

I was referring to your comment earlier in the thread, but very well, I can make a separate issue.

See https://github.com/postmarketOS/pmbootstrap/issues/1187

By * postmarketOS Bot on 2018-01-30T23:38:47

Created by: ghost

I added this package as mkbootimg-static, but it can be renamed later.

https://github.com/lramage94/pmbootstrap/commit/f8edd07642897935f32167e4ab2aeacde0725101

Should I submit a pull request?

By * postmarketOS Bot on 2018-01-31T00:51:36

Created by: Matt07211

@ollieparanoid Any objections or thights or problems or anything of using the mkbootimg code from magisk, it by far has the most support for different boot.imgs

By * postmarketOS Bot on 2018-01-31T01:29:42

You're right, Magisk has recently added all that support, but it's because I'm actively pushing for them to be added. That support was not based on the pull request you mentioned (which would have broken normal boot.img support), and they are instead being added based on my pxa-mkbootimg source, my dthbsign source, my mkbootimg with sha256 support source, etc. and that's awesome for Magisk, a rooting tool.

So it all depends on how you feel about using monolithic binaries vs, using the source they are built from... that all said, if you want to use the best maintained mkbootimg source, use mine.

By osm0sis on 2018-01-31T06:27:36

Created by: Matt07211

@osm0sis I was originally trying to merge the pxa-mkbootimg code into the normal mkbootimg tool you have, but I just wasn't able to get it working properly, and looking at the mkbootimg in code magisk, it looks like it's handles the part I was stuck on.(I made two different structures in my personal attempt, boot_img_hdr and boot_img_hdr_pxa but I just wasn't able to apply it properly in the main C code) either way I digress...

I was only suggesting that we use the mkbootimg code not the entire magisk suit.

Im happy that it's you that was pushing the changes, why don't you add that code to mkbootimg then? I personally was trying to get your/akuhaks pxa-mkbootimg code merged into your normal mkbootimg, and so far magisk mkbootimg is the only tools source code that I can see that has it merged together.

There's very minor differences between pxa and normal mkbootimgs that it won't add to much bloat but if you feel different about that and want to keep it separate then it's your choice, but I personally just wanted to minimise the amount of almost similar tools needed.

/rant

By * postmarketOS Bot on 2018-01-31T10:54:26

I prefer to keep different format tools separate. AOSP-PXA is a big enough departure from the AOSP standard format that I believe it deserves its own tools, rather than muddying up mainline mkbootimg.

By osm0sis on 2018-01-31T11:05:50

Created by: Matt07211

So it's pxa-mkbootimg and mkbootimg, eh? @ollieparanoid

By * postmarketOS Bot on 2018-01-31T11:15:41

Created by: ollieparanoid

My two cents on this would be, that I trust @osm0sis recommendation here, especially if you were the one who pushed various formats into Magisk. So my recommendation would be to replace mkbootimg which we have right now with the version from him, and packaging pxa-mkbootimg as well.

Nevertheless we should check if the output from the new mkbootimg is binary-identically with the output we have with the previous tool. That can be done with a shell script, that builds the boot.img file for each boot.img device with both tools, then compares the result (I could do that if necessary).

@lramage94: Thanks for packaging osm0sis' version of mkbootimg. Could you explain why you've compiled it statically?

Should I submit a pull request?

That would be great!

@Matt07211: do you want to take a shot at packaging pxa-mkbootimg? (@osm0sis: maybe you could tag a release there as well?)

@osm0sis: also thanks for your patience, I know we have this issue open for quite some time now, but we should be able to integrate this pretty soon, and it's good to have someone around who knows about all these formats and maintains these tools!

(@lramage94: I still think it was good to split the Plymouth idea into an extra issue to not clutter this one up, but also I did not realize that you were referring to what I wrote in this issue. Sorry if that sounded a bit unfriendly earlier, that was not intended!)

Thanks for sharing your thoughts everyone

By * postmarketOS Bot on 2018-01-31T19:31:31

Created by: ghost

I was trying to figure out how to make a distiction between the python version and I saw it had the option for statically compiling it.

By * postmarketOS Bot on 2018-01-31T19:36:12

Created by: ollieparanoid

Okay. I suggest we only keep one version in the repository - do you want to make a PR that replaces the version we currently have and compile it dynamically? (Linux distros like Alpine compile all programs dynamically unless they have a good reason not to, such as bootstrapping with apk-tools-static.)

By * postmarketOS Bot on 2018-01-31T20:09:29

Do you guys support any PXA devices to actually need pxa-mkbootimg? They're not super common.

Yes, I can throw a tag on whatever you need if needed.

My next tricks will be getting John to add rkcrc for RockChip KRNL and mkimage/dumpimage for U-Boot uImage support, then that's pretty much everything AnyKernel2 supports built into Magisk.

By osm0sis on 2018-01-31T23:52:42

Created by: Matt07211

@ollieparanoid Yea, I'll get the environment setup tonight and do it then.

@osm0sis Yea, me, with the Samsung xCover 3.

By * postmarketOS Bot on 2018-02-01T03:17:14

Ah fair enough! Tag pushed.

By osm0sis on 2018-02-01T10:41:32

mentioned in commit 665ce286

By Oliver Smith on 2018-08-04T11:07:33