Moving to gitlab.postmarketos.org

We are happy to announce that postmarketOS will be moving its source repositories to a selfhosted gitlab instance at https://gitlab.postmarketos.org!

The migration was done on Sunday, 2024-10-06, see the status report.

Background story

Why?

We are very grateful to the gitlab.com folks that they are currently hosting postmarketOS, even with the "free premium features for cool open source projects" thing. However the main reason we want to move away from gitlab.com is that the requirements for signing up and using the site have changed dramatically over the last years. While in 2018 when we migrated over to gitlab.com, it was no problem to use the website and git repositories with TOR and setting up an account without giving any meaningful personal data, nowadays users are required to give a valid phone number and credit card number (!) for setting up an account. Also the website is protected by cloudflare, blocking lots of proxies and sometimes even making the site hard to use for users with their real IP who have set up an account years ago and been very active. It is very understandable that gitlab.com needs to take measures to avoid spam attacks and misuse of their very useful resources. But there has to be another way. Over and over again we heard from people who would have loved to contribute to postmarketOS, but have no interest to do so because gitlab.com requires too much of their personal information.

We highly appreciate that an Open Source Program Manager from GitLab reached out to us and asked us for our concerns (https://gitlab.postmarketos.org/postmarketos/postmarketos/-/issues/49#note_1730848871). So far the situation has not really changed, but he confirmed that we are not the only community to face these challenges with GitLab, and he tried to make his team aware of these problems.

Who is maintaining the instance?

The new gitlab.postmarketos.org instance is maintained by OSUOSL, the Oregon State University Open Source Lab. They already provide various machines and services to free and open source projects, including Alpine Linux, Chimera Linux, Gentoo, GNOME, KDE, Kodi, OpenBSD just to pick a few from this huge list. @craftyguy talked to them at the FOSSY conference and they offered to host our gitlab instance too, and provide us gitlab runners for x86_64 and aarch64 as well. We decided that we will make a one time donation (TBD).

The benefit for us is that we have mostly the same code forge we have been using for years (we already try to avoid using features that are not available in the selfhosted version), but without the mentioned problems and without the need to maintain the selfhosted version ourselves - time we would much rather spend on moving postmarketOS forward.

Why not SourceHut/Forgejo/...?

We think both SourceHut and Forgejo are amazing projects, they just aren't the right fit for us at this moment and we want to get this done.

SourceHut: we evaluated it as code forge for some time with pmbootstrap.git, but then moved it back. See the related announcement for details. postmarketOS has been using SourceHut Builds to build its binary packages and images since end of 2019 and this works great, we don't have plans to change that.

Forgejo: This is the gitea fork that codeberg uses. It is close to our requirements, but when we checked there were some details that held us back like not being able to have open discussions block merge requests / PRs. It would probably be a good fit if we invested more time and set up our own CI runners. But at this point we have been looking for an alternative to gitlab.com for two years already. We are happy that we have a solution with an easy path forward where we don't need to spend too much energy (that again we can't spend on improving postmarketOS).

Migration plan

Preparation:

• OSUOSL sets up a selfhosted gitlab instance at https://gitlab.postmarketos.org
• OSUOSL sets up CI runners
• enable admin mode
• Open up registrations (with admin approval, and we will keep it that way to prevent spam accounts)
• Create gitlab groups: Active Contributors, Trusted Contributors, Core Contributors
  • Try to do it via opentofu/terraform (see https://gitlab.postmarketos.org/postmarketos/postmarketos/-/issues/77#note_2074605226) -> https://gitlab.postmarketos.org/postmarketOS/infra/gitlab-tf-pmos
• Test imports (export and import method):
  • pmbootstrap (source code, issues, merge requests)
  • pmaports too
  • Ensure CI scripts work as expected, runners get used etc.
  • Make sure milestones are properly imported -> looks like it, at least from per-project basis. we probably need to recreate the global milestones
• Try to import via "direct transfer" method, if possible -> still in contact with gitlab.com support about this, giving them some time to configure it; will use the "export and import" method otherwise -> "direct transfer" is enabled, test import worked
• Consider setting up a separate admin account (currently @craftyguy, @ollieparanoid, @caleb, @lucaweiss are admins) -> admin mode works well
• ensure OAUTH is set up for gitlab.com and github.com (see https://gitlab.postmarketos.org/postmarketos/postmarketos/-/issues/77#note_2074536828)
• ensure e-mail confirmations arrive for new users

Migration day (~~Sunday, 2024-10-06):

• Announce on status.postmarketos.org
• Announce on mastodon and in chat
• Remove/hide test imports

00:00 CEST:

(See https://status.postmarketos.org for current status, this issue will not be updated on gitlab.com after the repository has been set to archived!)

• Archive repositories on gitlab.com/postmarketOS
• Do the "direct transfer" -> see https://status.postmarketos.org/issues/2024-10-06-migration-to-gitlab.postmarketos.org/
• recreate epics as milestones (epics is a gitlab premium feature not available in selfhosted; coordination notice: it seems @JustSoup321 has started with this)
• Tighten permissions after import, only let core team write to pmbootstrap master and pmaports stable branches (as we have it now on gitlab.com)
• Approve accounts (in progress) (if your approval mail did not arrive yet, then it is probably because of the mail issues we are having - we are working on it.)
• Change gitlab.com, pmb and pma to be read-only mirrors (so they are still available if gitlab.postmarketos.org should be down)
• Adjust all repositories where we have gitlab.com hardcoded to use the new repository locations
  • pmbootstrap: pmbootstrap!2427 (merged)
  • pmaports: pmaports!5696 (merged)
  • bpo build.postmarketos.org!113 (merged)
  • homepage: postmarketos.org!314 (merged)
  • mrhlpr / mrtest: mrhlpr!51 (merged)
  • ci-common: ci-common!9 (merged)
  • monitoring: monitoring!13 (merged)
  • (more?)
• pmbootstrap: configure tokens for deploying docs again (if needed)
• Resolve status.postmarketos.org notice when done
• Tell gitlab.com support that they can disable the "direct transfer" permissions
• Fix all open threads below with additional issues