Failure to boot after install on PinePhone Pro

Did you install Tow-Boot?

By Newbyte on 2022-04-21T12:30:09

I forgot to mention, but yes I am running Tow-Boot.

By hyper division on 2022-04-21T12:30:09

added category::device device-pine64-pinephonepro type::support labels

Yesterday I migrated my PPP on TowBoot and installed pmOS on the eMMC successfully.

What version of pmbootstrap did you use to install pmOS ?
Do you have the serial cable? the errors on your picture do not show the whole picture (no pun intended), the whole boot log would help a lot

By Bobby The Builder on 2022-04-28T23:07:47

What version of pmbootstrap did you use to install pmOS ?

1.43.0

Do you have the serial cable? the errors on your picture do not show the whole picture (no pun intended), the whole boot log would help a lot

Yes, but I got no output from it when trying to boot pmOS. I do get output when booting Manjaro however, so the serial cable seems to be working.

By hyper division on 2022-04-28T23:08:00

Well it seems the baud rate has changed with TowBoot ...

screen /dev/ttyUSB1 1500000 echoes nothing for my PPP whereas screen /dev/ttyUSB1 115200 works well ...

Could you confirm ?

By Bobby The Builder on 2022-04-28T23:16:27

Yes you're right. The wiki says 1500000, but 115200 is what worked. See the attached file for boot log. It just keeps going like that forever.

pmos_sd.log

By hyper division on 2022-04-30T18:01:01

Thanks, here's where your boot log differs from mine:

Yours:

[    0.490806] pwm-backlight backlight: supply power not found, using dummy regulator
[    0.525355] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    0.564700] rk818-battery rk818-battery: DMA mask not set
[    0.612138] rk818-charger rk818-charger: DMA mask not set
[    0.622189] dw_wdt ff848000.watchdog: No valid TOPs array specified
[    0.644445] rk3399-dmc-freq ffa80000.dmc: supply center not found, using dummy regulator
>[    0.704818] mmc1: Problem switching card into high-speed mode!
[    0.770086] mmc0: running CQE recovery

Mine:

[    0.491553] pwm-backlight backlight: supply power not found, using dummy regulator
[    0.526437] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    0.565576] rk818-battery rk818-battery: DMA mask not set
[    0.616106] rk818-charger rk818-charger: DMA mask not set
[    0.626032] dw_wdt ff848000.watchdog: No valid TOPs array specified
[    0.648412] rk3399-dmc-freq ffa80000.dmc: supply center not found, using dummy regulator
>[    1.818298] [drm] unsupported AFBC format[3231564e]
[    1.826847] cdn-dp fec00000.dp: Direct firmware load for rockchip/dptx.bin failed with error -2
### postmarketOS initramfs ###

And this Unexpected interrupt latency is worrisome:

[    0.776816] ------------[ cut here ]------------
[    0.782046] WARNING: CPU: 0 PID: 129 at drivers/mmc/host/cqhci-core.c:786 cqhci_irq+0x4c4/0x630
[    0.789676] dwmmc_rockchip fe310000.mmc: Unexpected interrupt latency
[    0.799587] ---[ end trace e07e4f8564e3ce6c ]---

Before I cc more knowledgeable people:

Can you confirm which pmaports rev you used ?
Is anything's plugged in the USB port while you boot?
Can you send the boot log while booting on a mSD card ?

By Bobby The Builder on 2022-04-28T23:53:00

Also, if you still have a mSD card inserted with an old pmOS intall while you try to boot on eMMC it'll lead to weird behavior. Perhaps related to #1485 (closed)

By Bobby The Builder on 2022-04-29T00:05:58

Side note: For the baud rate, it was changed a month ago (!3018 (merged)):

Fix the console baud rate to match tow-boot which is 115200. This also enables earlycon to debug early boot failures.

By Bobby The Builder on 2022-04-29T00:16:06

Can you confirm which pmaports rev you used ?

The pmaports git commit is a57701d1.

Is anything's plugged in the USB port while you boot?

Nothing is plugged in to USB.

Can you send the boot log while booting on a mSD card ?

Sorry, I forgot to specify: The boot log I already posted was booting from mSD. I also just now tried booting from eMMC with no mSD inserted, attached is the log for that: pmos_emmc.log

By hyper division on 2022-04-30T18:00:47

@hyperdivision thanks for all the info.

Were you able to flash pmOS on the eMMC by exposing the eMMC drive through Tow-Boot ?

I guess the answer is yes, which would lead me to believe that:

Your emMC is working fine
The image built has old or inconsistent data

Could you try to nuke zap your pmbootstrap env (yes | python pmbootstrap.py zap -hc -d -p -m -o -r) and try with a new build ?

By Bobby The Builder on 2022-05-02T19:33:38

Were you able to flash pmOS on the eMMC by exposing the eMMC drive through Tow-Boot ?

Correct, that's how I flashed pmOS to the eMMC.

Your emMC is working fine

I think so too. I've run Manjaro from the eMMC with no issues, and pmOS also fails to boot when flashed to mSD.

Could you try to nuke zap your pmbootstrap env (yes | python pmbootstrap.py zap -hc -d -p -m -o -r) and try with a new build ?

I tried that just now. Same result I'm afraid.

By hyper division on 2022-05-02T19:50:05

Out of curiosity, what's the output of this command on your env:

pmbootstrap status

If I provide you with a link to an image I built and that I know it works on my PPP, are you willing to test it ?

That way we could definitively rule out any HW issue.

By Bobby The Builder on 2022-05-02T20:43:36

pmbootstrap status

$ pmbootstrap status
[09:42:29] *** CONFIG ***
[09:42:29] Device: pine64-pinephonepro (aarch64, "PINE64 PinePhone Pro")
[09:42:29] Extra packages: vim,tmux
[09:42:29] User Interface: sxmo-de-sway
[09:42:29]
[09:42:29] *** GIT REPOS ***
[09:42:29] Path: /home/user/.local/var/pmbootstrap/cache_git
[09:42:29] - pmaports (master)
[09:42:29]
[09:42:29] *** CHECKS ***
[09:42:29] All checks passed! \o/
[09:42:29]
[09:42:29] NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
[09:42:29] DONE!

If I provide you with a link to an image I built and that I know it works on my PPP, are you willing to test it ?

Sure.

By hyper division on 2022-05-03T07:44:07

Just tested this image without an issue:

https://www.najdan.com/pine64-pinephonepro.img

SHA256: 70f4bcbf670506e2674946e28e62518cd2ca81c01f2e987b2ea673bfeff82d06

*Link will stay active for the next 7 days

Steps:

Reboot PPP in flash mode (Hold Volume UP until atfer second vibration)
Flash the image dd if=pine64-pinephonepro.img of=/dev/sdX oflag=direct status=progress
Resize the root partition (I like to use gparted)
Reboot ...

By Bobby The Builder on 2022-05-08T15:22:20

Thanks for the image. I downloaded it, confirmed the SHA256, and flashed it to eMMC as instructed. I'm afraid the result was the same as earlier. I've attached the boot log.

pmos_image.log

By hyper division on 2022-05-08T22:28:23

@hyperdivision thank you for the feedback.

So it seems the problems occurs in pine64 kernel but not with kernel shipped with Mobian.

Mobian uses Megi's kernel (5.17.5) and pmOS uses pine64 kernel (5.16.7).

Let's then check if the instability is already addressed in 5.17 kernel or by Megi in his repo.

I did upgrade pine64 kernel to 5.17.6 (https://gitlab.com/BobbyTheBuilder/linux-pine64/-/tree/pine64-kernel-ppp-5.17.y)

And upgraded linux-pine64-pinephonepro with it (https://gitlab.com/BobbyTheBuilder/pmaports/-/tree/linux-pine64-5.17.y)

I tested this new kernel on my PPP and it booted fine in phosh.

You can test this kernel by either:

backport my changes in your local pmaports repo and build a new image
download the same image I built and tested

https://www.najdan.com/pine64-pinephonepro-5.17.6.img

SHA256: cd3641fff5021d4e5164e6db1d8a443178c2d86a80e0f124daadd63080e5f37c

*Link will stay active for the next 7 days

DISCLAIMER: This comes with absolutely no warranty

By Bobby The Builder on 2022-05-09T13:30:13

Thanks for the troubleshooting help so far.

I tried now with your new image. Unfortunately it produces the same result. Attached is a boot log.

pmos_5.17.log

By hyper division on 2022-05-09T23:17:38

Thanks, I'll need a few days to review the differences between Mobian, Megi and pine64 kernel custom commits.

In the meantime, and again out of curiosity, if you have the time I'd like to see serial logs of a successful boot with Mobian on your device.

By Bobby The Builder on 2022-05-09T23:32:42

Here's the serial boot log of Mobian installed to eMMC, no modifications done except booting once and changing the baud rate to 115200.

mobian.log

By hyper division on 2022-05-10T01:10:38

changed the description

By hyper division on 2022-04-28T23:42:27

changed the description

By hyper division on 2022-04-28T23:43:06

changed the description

By hyper division on 2022-04-28T23:43:45

changed the description

By hyper division on 2022-04-30T18:00:22

Hello @hyperdivision can you give it another try since !3278 (merged) is now merged.

By Bobby The Builder on 2022-07-13T19:09:49

When trying now, the results are different:

If I install with pmbootstrap to eMMC, on boot it gets stuck on "Configuring kernel firmware image search path" for a long time (at least 10 minutes), until it fails to boot with "ERROR: boot partition not found!". Serial log: pmos_5.17.9_emmc.log
If I install with pmbootstrap to SD, on boot it gets stuck on the same message for a long time, but then finally it boots successfully.
If I put the pre-built installer image (20220713-1209-postmarketOS-edge-sxmo-de-sway-1.9.0-pine64-pinephonepro-installer.img.xz) on an SD card it takes the same long amount of time to boot. When I select to install to eMMC in the installer, it fails when trying to write a partition table to the eMMC:

Error: I/O error during read on /dev/mmcblk0
Error: I/O error during write on /dev/mmcblk0
Warning: Error fsyncing/closing /dev/mmcblk0: I/O error

Seems to me like it fails to access the eMMC in all instances, and the long boot time from SD is because it's trying to read the eMMC and eventually gives up.

By hyper division on 2022-07-13T19:09:49

@hyperdivision thanks for the feedback.

At this point I can only recommend to file an issue on pine64-org/linux and link our work to help troubleshoot the behavior.

By Bobby The Builder on 2022-07-16T01:22:41

@BobbyTheBuilder Sorry, I never actually got around to filing an issue on pine64-org/linux. However, I just tested flashing the latest edge with pmbootstrap to my PinePhone Pro, and it boots without any of the issues described here! The latest v22.06 image still has the same issues, but I assume that the next release will work since edge works.

By hyper division on 2022-11-15T18:31:31

closed

No idea if it's related, but I had similar issues booting the Pro after updating. I had left it un-updated for quite some time, so I expected some issues.

I found that the /boot/grub/grub.cfg was very different from the /boot/grub/grub.cfg.backup so I just swapped the two on a whim, and it booted just fine. Wouldn't even begin to be able to tell you why that worked, but hey, it's worth a shot.

EDIT: Oops, didn't realize how old this issue was, as it just popped up in my RSS feed. Sorry.:

By Bob Boberson on 2022-11-15T19:40:40

I kind of forgot this thread...

Even if you closed it I'll add my 2 cents.

I strongly believe that your behavior was not directly related to the Kernel but to U-Boot or Tow-Boot and how it initialized the PPP hardware.

I invite you to make sure you have the latest Tow-Boot installed in your SPI before booting any recent kernel.

By Bobby The Builder on 2022-11-22T16:35:24

I don't believe it was solely caused by Tow-Boot, as I can still reproduce the issue when running the latest Tow-Boot if I boot from pmOS v22.06, but not when I boot from Edge. Either way, the problem is solved, so I suppose no further investigation is needed.

By hyper division on 2022-11-22T16:35:24

Failure to boot after install on PinePhone Pro

Describe your issue

What's the expected behaviour?

What's the current behaviour?

How to reproduce your issue?

What device are you using?

On what postmarketOS version did you encounter the issue?

On what environment did you encounter the issue?

Phone environments

Other

How did you get postmarketOS image?

What's the build date of the image? (in yyyy-mm-dd format)

Additional information

Child items ...

Activity