Software Apps (GNOME Software, Plasma Discover) issues and improvements

This is mostly a documenting and roadmap issue to try to put together what is needed for GNOME Software (GS) and Plasma Discover (PD) to correctly display all metadata for all apps. Hopefully at some point it would be possible to recommend users to install and remove programs with these apps instead of with the terminal. An overview of the components that make up the stack can be found here. If anybody is aware of any other "architectural" things which need a fix, please write a comment and I will add them here.

Most importantly, I would like to ask if anybody has ideas on how to solve Icon download and would be great if people caring about PD (@minlexx, anybody else?) could verify that the new deployment of won't break anything.

Issues and fixes

So, current problems with some discussion:

• Appstream data in alpine is more than one year old, stable releases don't have metadata and many more issues. All relevant MRs[redacted](https://gitlab.alpinelinux.org/alpine/infra/compose/appstream-generator/-/merge_requests/2)[redacted](https://gitlab.alpinelinux.org/alpine/infra/compose/appstream-generator/-/merge_requests/3)[redacted](https://gitlab.alpinelinux.org/alpine/infra/docker/appstream-generator/-/merge_requests/1) have been merged. We could already ask alpine to redeploy. However, the upgrade brings some changes that might break the applications:
  • Basename for component files changes from Components-community-$ARCH.xml.gz (and main and testing) to Components-$ARCH.xml.gz. Required changes:
    • Update apk packages with new URLs.
    • GS would overwrite files in download. Upstream MR has been accepted and needs to be backported to 41.1. In upstream discussion I promised that we would also cleanup old files. So additionally we need to add a post-upgrade hook that removes the all /var/cache/app-info/xmls/org.gnome.Software-* files.
    • I haven't checked PD, but maintainers should check whether this breaks something. I have had the generator running for a while (results in https://appstream.pcorreag.tk), so you could use that data like if it were alpine's appstream data.
  • Media files that were previously not generated are now generated, but they only point to a relative URL. The property that has the actual URL was not documented until recently.
    • GS cannot currently parse it correctly. I have send a MR upstream. There is some discussion on performance, but for me it worked, so it should be possible to directly backport it to 41 without waiting for the merge.
    • PD has to be checked to work with this. The best way to check that it works is to see if all kind of screenshots load.
• Appstream data in postmarketos is more than 6 months old and does not have data for any of the two latest stable releases. Additionally, I haven't found any public repository like alpine's infra repositories, so I am really not sure of what is needed to get it up and working again. Related #699 (closed) This probably does not make much sense at this point, where most if not all apps are in alpine.
• Multiple sets of icons are missing. This is the most tricky part to fix, as according to documentation, they are expected to be placed in the filesystem by the distribution. See further discussion below.
• It would be very cool to ship a metainfo file in pmOS (maybe also in alpine?) of the kind Operating System. Debian example

Icon and component caching

According Appstream's docs: "In order to display icons in software-centers, distributors should offer a repository of cached icons for applications defined in their AppStream XML files." and "All icons of type cached must be placed in /usr/share/app-info/icons/%{origin}/%{size}/ or /var/cache/app-info/icons/%{origin}/%{size}/". At least 64x64 icons are forced to be always cached. Therefore, it doesn't matter if appstream data in both alpine and pmOS is in perfect state, those icons will always be missing (random consequence, by improving the quality of appstream data, we will have less icons showing 😢). In consequence, we need to implement some way to download the icon tarballs (like icons-64x64.tar.gz here) and place them in the corresponding location. This seems to have been discussed #506 (comment 416514735), but I haven't found any conclusion.

How are others doing it?

I tried with Fedora and Debian, but could only figure out the way Debian takes care of it. If anybody knows the process in Fedora, I would love to learn about it 😄

In Debian, the package manager, APT, takes care of it, as it has the ability to download random files. Then, the appstream package includes the APT configuration files which tell APT which files to download and where to place them. Those files are actually included upstream in the contrib folder, so they are easily discoverable and useful for other debian-based distros.

Our options

Having, like Debian, some configs upstreamed which can be included based on a configure-time option seems to really fit pmOS philosophy. However, I am not sure we can actually make that work at this point. The options I can think of:

• Create a new project (could be a simple shell script) which reads some random configuration file and downloads and places the files in the right location. It could be a cron job, but how often should it run? How to make it work with suspend? Could it instead be a daemon that watches the relevant directories and fires a download if something changes in them?
• Is it possible to use apk in a similar way as debian uses APT to download the said files? The fact that in debian whenever the software repositories get updated, so is the metadata, is wonderful.

I am really short of ideas here. So any help/suggestion is welcomed.