Project direction (don't miss this one, it has a new mid-term vision)

Wall Of Text Incoming

CC: @postmarketos

Dear postmarketOS hackers,

I've been thinking about the direction to take with this project. I mean we have the long term goal of the so often cited "daily driver", making postmarketOS usable for daily usage. And depending on how you look at it, we're getting pretty close to that. With the "one year" blog post we showed that phone calls are around the corner, right now only on very few devices, in the terminal and without audio. But once that is solved, calls are there and that's more or less what you need for a daily driver, right?

At least that seems to be the general reaction I seem to get when talking with people about the project. But there is much more to it and I would argue we are stuck in a proof of concept stage which we won't magically get out by continuing our current development workflows. So I would like to discuss a new mid-term vision here, that should get us to the long term (daily driver) one. But first let's take a look at where we're at.

Where we are now

1. What Is Really Missing For Daily Driver

Build Infrastructure That Scales

TL;DR: I'm maintaining software and hardware for the build repository on my own, as only user, on a PC with an i3 CPU in my home. This needs to be scaled up (and the feedback already looks very promising \o/)!

See #78 (closed).

Stability

Our current approach is: everyone can contribute and work on what they like, we try to support everything and everyone. And this was a great concept for the proof of concept stage, it got us to where we are now, we had some fun stuff to present and to get new developers on board, together with serious progress that improves pmbootstrap and the OS as a whole.

But the downside is, that people only maintain software when they feel like it. Oftentimes, something gets ported and then it breaks, so we don't have ressources anymore to fix stuff. LuneOS and wireguard come to mind for example. Don't get me wrong, it's amazing that we have these ports and this is a great way to make progress and try out new stuff at all. But it's not the greatest user experience if people try these out and they are just broken.

How about we clearly split stable software from experimental software?

Stable software must have maintainers that dedicate to updating them, like @PureTryOut does it for Plasma Mobile right now. Follow upstream releases, and for each release, update the packages, test them, make a merge request. I (and hopefully others ) would prioritize supporting these stable packages, helping people who use them and have problems, as well as making sure they run as smoothly as possible. Of course I would love to support everything and everbody and will try to do that to some extend, but with limited time on my hands, I think it's in the best interest of the project to put most of my focus on what's going to bring the project to its goals.

The rest of the software is experimental, and clearly marked as such I think it makes sense to ask the users to enable that experimental software explicitly. Maybe think about it like Arch Linux' AUR.

Regarding technical implementation, we could either use different folders (testing folder) or branches.

A big stability improvement would also be basing on Alpine stable instead of Alpine edge (pmaports#5 (closed)). From where we are now, I would even prefer if we only offer builds on top of Alpine stable, never on edge. Because then we should have next to zero breakage with Alpine's packages, only when we are switching to the next stable version (and we can do that on an extra branch so users won't have the breakage when they stay on the old branch - as soon as the build infra is scaling this shouldn't be a problem). Building on top of Alpine edge seems to be additional effort with no benefit to me, so it would fall into the "unsupported" category for me if someone's really interested in doing that.

Always Updated Software Packages

We need to keep our software up to date. For security reasons and to minimize technical debt. Kernel releases on kernel.org happen every few days, and all other software we are packaging, all the UIs for example, and all the components they build upon also have frequent releases.

When I did initial research for postmarketOS before starting the project, I saw that Void Linux was storing a website url together with a regex in their package build scripts. That information is able to find out the latest upstream version (download the web page from the url, then run the regular expression on it). As soon as they had it, from what I understand, they would automatically increase the package version and trigger a build. So they instantly see if the build passes through, and they can notify the maintainers either way, who can then fix up the packages as necessary, and test them. And then push the update to the package repository. I like this workflow a lot, I think this makes updating software as efficient as it gets from a maintainers point of view, and I think we should implement something like that. Automatically checking for updates also allows us to build a page where you can see instantly what is out of that and what is not.

When maintenance becomes low effort like that, then it becomes realistic to go actually through with properly providing the updates.

From the client side, we would need a good mobile friendly app that allows making updates with apk, and searching for or installing new software. Maybe we can use KDE's "discovery" for that (correct me if that is not the name of the program), we would need to provide an apk backend for it though. Adélie's also interested in that.

Working Phone Calls

This is still important of course. But when put into perspective with the other two points, I would give this the least priority and mark it as unsupported until it is much more streamlined. At least in terms of what I should be working on myself until we have that rock solid base system.

2. What postmarketOS Is Good At

We have achieved more than I ever thought we could pull of with the proof of concept we have today. Honestly, I can't thank every single contributor enough for what they have done. And here are the points where postmarketOS shines the most in my opinion:

• We have the biggest booting devices number (~100) for any non-Android Linux project on phones
• No proprietary userspace blobs necessary
• Porting new devices is pretty much streamlined with pmbootstrap
• Packaging software is easy and works on pretty much every Linux distro (with pmbootstrap)
• Huge documentation resources in the wiki
• Many mobile UIs packaged, at least partially working
• We have a big community to reach, almost 2K Reddit subscribers to our subreddit for example. If we announced tomorrow that there's a stable version of postmarketOS to test for daily usage, I bet we would get a lot of people to test it.
• As we are a downstream distro, we can provide applications for UIs that were not initially intended for them.
• Good relations to mobile UI communities such as Plasma Mobile, ubports, Maemo Leste and to Alpine and Adélie
• Quite a few developers with different expertises (kernel hacking, packaging, coding, debugging weird problems, low level hacking, artwork, ...)

Brand New Mid-Term Vision

So let's take a step back from all that and conclude what we should put our mid-term fous on. My conclusion is:

Let's become a rock stable testbed for Plasma Mobile, ubports, Phosh on roughly 100 phones. With experimental features on top (clearly marked as such) and possibly other UIs if we find maintainers (Maemo Leste, LuneOS UI?).

If we could do that, the UIs would get bigger userbases, leading to more testing, probably more developers, more stability in general and especially on postmarketOS. More popularity to our upstream friends' projects due to people being able to easily hold it in their hands. Accellerating everything non-Android Linux related that is happening on mobile phones!

Is this doable?

The biggest problem with that idea is, that basically all mobile UIs need graphical acceleration to work properly. And we don't have this on most phones, which are using downstream kernels. I'm wondering if we can use a faster software renderer and tune down the effects to make this feasible.

So we should better make a proof of concept, before deciding that this mid-term vision is where we want to go. I would highly appreciate if someone could do that, and consider doing it not on postmarketOS if it's easier. Just disable hardware accelleration on your phone where one of the mobile interfaces runs, and try to make it run as fast as you can (maybe Debian has packaged other software renderers, or there are PPAs for ubuntu or something? please fill in the research folks).

@bshah: any pointers or patches for tuning down all effects in plamo?

How About Some Dogfooding?

I think it developers should ideally use postmarketOS on a real device on a daily basis. Personally, I'm only running postmarketOS on a phone every two weeks or so, briefly, when testing a merge request. This needs to change, and I've been thinking about which tasks I could do on postmarketOS if I can't use it to make phone calls and if I can't replace my main phone with it yet.

I came up with using it as (encrypted) music player or (even better) podcast player. So I will work towards that as soon as I can fit it in with the rest I'm doing and I'm sure there are other use cases where people could already (try to) use postmarketOS today. There's also a nice YouTube video about using postmarketOS as their portable emacs machine for quickly writing notes, this would be another good use case. It would be nice if people could joint me with that, so we have a better feeling of what needs to be improved for end users.

Developer Resources

Since last months I'm working less hours per day in my (all new) day job, and I've set three hours per workday aside for exclusively developing postmarketOS. When that is done, plus the work from my day job, then I still have free time left at the end of the day and on the weekends, so I don't burn out. So I have 15 hours per week that I can consistently work on postmarketOS (plus the extra time I put in when I feel like it, like today when I'm writing this huge post). I don't expect this from anyone else, because I know that this is a big commitment.

Besides the burn out protection, the good news is, that this makes it possible to do some more detailed planning of at least what I will be working on, and I would like to do that in the near future. Then it's clearer where the project is heading and when it's expected to be there. And with an outline like that, it should be easier for other people to help out, and if they feel like it, get integrated into that planning as well.

Finishing Up

Alright, this has been a huge post. I'm curious about what you all think about this. When we have discussed this a bit, and some results in these new directions, this would also make a good base for a blog post to let people know what we're up to.

Thanks for reading!

I completely agree, this seems like a great way to move forward. Most points I've thought about but its written down here perfectly.

I also agree that this would make an awesome blog post.

About the dogfooding, I guess a major usability thing to have would be a webbrowser, if that works and wifi is working on the device then you basically have a device that could be used as the pre-n900 N-series devices by nokia; fancy internet tablets/phones.

By Martijn Braam on 2018-10-03T20:07:56

Rock stable testbed for UI developers
Providing one/two models that are guaranteed to work (e.g. Nexus 5) with mainline, SMS etc. would provide a stable and easy to use platform to test and develop UIs for UI developers (even Plasma mobile only have QEMU as official thing for testing, no real hardware which would help with testing multi-touch and other things that require real metal)

Dogfooding
Some ideas:

• low-power servers with built-in screen (like cheap Raspberry Pi replacement without GPIO)
• dedicated games emulator (like PSP/retroPie, touch-friendly emulators would be needed)

By Piotr Halama on 2018-10-03T20:11:45

My two cents, if I may:

dogfooding, just wanted to mention that I really will use my nexus 5 as portable web consultation device, whenever I'm moving around (as I don't have an Android-based smartphone for ethical reasons), even if there is still work to get it as fancy as possible (stability!), it is already usable, it's better and better every week and for me who is used to having nothing, it's very useful.

Other idea: Ideally in the long run wouldn't it be nice to provide pre-build images for phones that work well that you may flash directly with fastboot or graphical installer like ubports-installer (instead of needing to build the image with pmbootstrap)?

By Pierre Parent on 2018-10-03T21:28:31

I totally agree, having every included device being able to use all uis would also give people a lot more belief in it being usable as a daily driver.

On a side note... Mainline calls won't be happening... For a long time, MDSP stuff isn't there. However data on mainline will happen soon.

By Craig Tatlor on 2018-10-03T21:15:23

NotKit has gotten hybris merged. I haven't seen that tried much with postmarketOS devices. I tried it on mine (samsung-p4wifi) using the android system.img and kernel straight from the Halium port that I made. No modifications were necessary.

The android LXC container worked. That's as far I could that test hybris because the HWC blobs for this device don't work at all with hybris.

Other devices have had better results with hybris graphics on Halium. I wonder how well it works on musl / Alpine Linux.

By ryang on 2018-10-03T21:25:35

Awesome post, thanks for this! I really admire you for the effort and passion you put into this project. If I had the time (and a little bit more programming knowledge lol, but that's improving while I'm doing this) I would love to do just as much as you do.

How about we clearly split stable software from experimental software?

I definitely agree with you on this one. LuneUI is nice to have, but clearly unusable and hard to maintain right now, even more so because we have no one dedicated to it. I much rather have stuff like that be behind some switch.

In my opinion we should focus mainly on 3 UI's: Plasma Mobile, Phosh, and Unity8. Those are the only mobile interface's that are really developed and work without too much patching and upstream unmaintained packages. Also, you can make basically everybody happy with either of these 3. For the Qt/KDE lovers there is Plasma Mobile, for the GTK/GNOME lovers there is Phosh, and for everyone that wants a completely new phone experience there is Unity8 (I'm probably insulting someone with this comparison, that is not my intention, sorry for that).

A dedicated maintainer for each of these like you said is probably a good thing. I've always been the main guy for Plasma Mobile and I love to continue doing that. @z3ntu has put a lot of work recently into Unity 8 so he is probably a good guy to maintain that. Then we only need someone for Phosh which up till this point, I have put most effort in. I don't mind maintaining it, but seeing I'll probably not actually use it myself once it's usable, it's probably better to get someone else to do it. I'll of course help when needed. Maybe @MartijnBraam?

How About Some Dogfooding?

I'd love to use postmarketOS when I can. I already walk around with 2 phones in my pocket, one for mobile calls and WhatsApp and stuff, and one for multimedia related things (bigger screen ), so I don't mind changing one of those to a pmOS phone. However, none of my devices currently are usable for this, not like I want it too anyway. My Amami's have no properly usable UI atm as Plasma Mobile doesn't run on them due to a lack of hardware acceleration (Alpine's switch to armv7 might resolve this), my Nexus 5X is way too slow without hardware acceleration to do anything screen related, and my Jolla Phone does nothing except booting pmOS really.

But, once any of these 3 devices gets to a point where it's usable at least for playing music, I'm all for turning it into my daily music player. It's definitely something that would help the project anyway, the more people that use it to report bugs, the better.

---

I think we should really focus on getting at least one phone, actually usable as a daily driver. Having this would give a surge of interest in the project, and we could really use that. It would also be a good test bed for most user-land stuff, like the UI's. Although doing this on mainline without Android stuff would be preferable, maybe it's a good idea to get this via libhybris as well, like @ryang2678 said. Both the Nexus 5 and the Sony Xperia Z2 (phone and tablet) are really close to being usable as a daily driver on mainline. I don't know the current status, but it'd be awesome to finish either one of those.

---

@pparent we don't need to provide pre-built images even if we're using a graphical installer. It would be hard anyway, as we don't know what kind of stuff the user wants on his phone, and I don't see an on-device installation wizard happening anytime soon. Having this graphical installer would be nice though. I've been learning myself QML and Kirigami via making a potential GUI for postmarketOS, maybe that or @MartijnBraam's GTK GUI could be used for this.

That said, I don't think we should advertise a graphical installer if we don't have that stable base yet.

By Bart Ribbers on 2018-10-03T21:44:48

@PureTryOut

we don't need to provide pre-built images even if we're using a graphical installer. It would be hard >anyway, as we don't know what kind of stuff the user wants on his phone

What I meant: a curious guy may like to do, is just easy and fast flash with a given UI, with no customization. No questions and in 5 minutes it's up and running with default parameters. Pmboostrap is very dependent on platform: you can't use it on Windows for example (not that I would like to use windows, but I'm sure potential testers would), you need to use cmdline and git, image compiling can break at any moment with latest development or if alpine binary repo goes down (or as a certificate issue), and so on. It's developer-friendly but not so user-friendly.

But I agree that it is useless before a phone works well and is stable!

By Pierre Parent on 2018-10-03T22:14:04

My goal is to have no UI (just a text console or speech) and so I can help with exercising the lower levels of sms/telephony (ofono I think is used by most?) in a way that tests them and should support upper levels in UIs.

I definitely think that focusing on a couple of popular models would help. I would be glad to grab a cracked screen Nexus 5 or Sony Xperia Z2 or something else that several people are working on and is fairly popular that more folks would be attracted to pmos.

By Craig Comstock on 2018-10-04T03:06:36

One other point is that we should probably have a wiki page on how to get hybris running, it would make it a lot more accessible.

By Craig Tatlor on 2018-10-04T06:54:18

Also: I think the front site could be improved. It has photo ratio display issues on some not so old browsers (firefox on debian oldstable that I use at my work), and shows photos that are not representative of the current state of development.

Might be a detail for you, but it is the first thing people see, and it could maybe attract way more people with a better site. The first time I saw it my self, with distorted photos, I must say that I was not so attracted in going any further. And it did not seem possible to easily install pmos on a device until I digged in the wiki, so I did not think I could even do something with it.

I'm willing to make a new design for the site if you want. Here is an example of a site I made: https://www.own-mailbox.com

By Pierre Parent on 2018-10-04T07:13:34

Regarding hybris: when only looking at the mid term vision, it does seem like a good solution to get there for quite a few devices. Going straight to mainline for every device is not realistic, as sad as that is.

Also: I think the front site could be improved. It has photo ratio display issues on some not so old browsers (firefox on debian oldstable that I use at my work), and shows photos that are not representative of the current state of development.

Might be a detail for you

Not at all, I think this is very important.

I'm willing to make a new design for the site if you want.

Sure, merge requests welcome

By Oliver Smith on 2018-10-04T07:35:25

Wizzup from Maemo Leste commented:

what I am not seeing in the plan you linked, is perhaps (if not already, sorry if I didn't see that before) labelling/focussing on some devices, and mark them as 'usable-ish', or 'mainline and usable', as opposed to android-vendor ports I think UBports does this as well, with "promoted devices" or something like this

By Oliver Smith on 2018-10-15T20:40:06

That seems like a good idea to me. From all the devices we "support", only the Nexus 5 and Sony Xperia Z2 {tablet,phone} are actually "usable" right now as far as I understand, although even they have issues. It might indeed be good to let them stand out from the rest. Maybe even some page on the website rather than just the wiki? Like show those devices as "promoted", and mentioning that if a users devices isn't there, it might be less supported with a link to the wiki.

By Bart Ribbers on 2018-10-17T11:05:49

@PureTryOut yeah, that seems like the best idea right now, I guess requirements would be gpu acceleration and touch, anything else?

By Craig Tatlor on 2018-10-18T17:18:15

GPU acceleration and touch definitely yes, they make the phone "feel" usable. I think that's good enough to be honest. We can then list the phones in order of most supported. So a phone with also working audio and modem would be on top (these are just examples). A phone with just audio or just the modem would be second, etc.

By Bart Ribbers on 2018-10-18T17:57:21

I think the requirements would be

• 3D acceleration
• touch/display
• WiFi/other relevant networking

By Martijn Braam on 2018-10-18T23:01:59

Yea, display + data + input makes total sense as a baseline requirement.

By clayton craft on 2018-10-19T20:19:16

So we're certain on the following then:

• The site should contain a list of "promoted devices", based on if they have a working display, 3D acceleration and wifi, and maybe some other factors
• Each "well supported" UI (probably Plasma Mobile, Phosh and Unity 8) should have a set maintainer, others should be clearly marked as experimental
• We should get libhybris working on non-mainline devices to make postmarketOS actually usable on a lot of devices. We can possibly use Halium for this, @bshah is willing to help us here

Right?

By Bart Ribbers on 2018-10-20T20:53:36

The site should contain a list of "promoted devices", based on if they have a working >display, 3D acceleration and wifi, and maybe some other factors

But will you promote non-mainline devices having a working display, 3D acceleration and wifi, only thank's to libhybris?

(I think it comes down to a philosophical standpoint, and the aim of the project: aiming at being a "real linux distribution on the phone" with full FLOSS stack, or aiming at building another halium based distro.)

By Pierre Parent on 2018-10-21T14:22:23

mentioned in issue #86 (closed)

By Oliver Smith on 2018-10-21T17:49:45