Better /usr merge support

The existing apk-tools patches for /usr merge have a few edgecase bugs, and overall take fully the wrong approach (rewriting paths at the point the files are installed rather than at the database level, breaking apk info and apk audit as well as leading to issues like https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2388).

Upstream would prefer to implement this /usr merge support on apk3 and backport it to apk2, we may consider simply switching to apk3.

I have rewritten the /usr merge patch on apk3: https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/155

And backported it to apk2: https://gitlab.com/postmarketOS/apk-tools/-/tree/caleb/apk2-usrmerge-db?ref_type=heads

We now need to:

• Update our fork of apk-tools with the backported new patches
• Test extensively (maybe ask people for their /etc/apk/world and check in VM?)
• it

changed milestone to %initial systemd support

added systemd label

assigned to @calebccff, @craftyguy, and @ollieparanoid

By Caleb Connolly on 2024-07-06T17:20:26

no longer need the migration script in pmb

It should always be possible to bootstrap a postmarketOS repository with plain alpine, without any pre-built pmOS packages. So for that purpose we probably need to keep this in some form until usr-merge lands in alpine. But we probably only need to call it during pmbootstrap repo_bootstrap instead of every time we initialize a chroot.

we may consider simply switching to apk3.

+1

By Oliver Smith on 2024-07-07T12:43:55

As discussed in chat, let's try to switch to apk3 for everything, including openrc installs. The bootstrap path may need some thought, but Caleb already has ideas.

I'll need to make sure that bpo can also bootstrap this (but no action needed from me until MRs to use apk3 are there as I understand, so unassigning myself.)

By Oliver Smith on 2024-07-08T23:51:21

unassigned @ollieparanoid

By Oliver Smith on 2024-07-08T23:51:24

changed title from apk3 and better /usr merge support to Better /usr merge support

By Caleb Connolly on 2024-07-11T00:22:06

changed the description

By Caleb Connolly on 2024-07-11T00:22:06

After some more thought, it turns out that backporting this was really straightforward. It probably makes the most sense to switch to this on apk2, then revisit apk3 in the future (future could be in a few weeks or so...).

By Caleb Connolly on 2024-07-11T08:44:17

Excellent! Using apk2 is fine with me, and thanks for spending time on apk3.

I think getting this sorted out/fixed soon is a high priority, since it seems like it might be a bit risky. How can I help?

By clayton craft on 2024-07-11T08:44:17

I got some feedback to the upstream MR I need to address, then we should update our apk-tools and get some testing done in the MR.

By Caleb Connolly on 2024-07-12T16:48:15

which MR exactly?

By clayton craft on 2024-07-12T16:49:47

linked the description

https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/155

By Caleb Connolly on 2024-07-12T16:59:27

Ah, I saw that but wasn't sure if we would fork apktools to test/use it sooner, or wait for this to get merged upstream

...

On July 12, 2024 9:59:29 AM PDT, "Caleb Connolly (@calebccff)" gitlab@mg.gitlab.com wrote:

Caleb Connolly commented on a discussion: #2988 (comment 1997279670)

linked the description

https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/155

-- Reply to this email directly or view it on GitLab: #2988 (comment 1997279670) You're receiving this email because of your account on gitlab.com.

By clayton craft on 2024-07-12T17:09:41

we already use an apk-tools fork??? even if this landed in upstream (which will likely take a while), getting it backported will take even longer

By Caleb Connolly on 2024-07-12T17:29:00

mentioned in issue #2938 (closed)

By clayton craft on 2024-07-11T08:46:53

mentioned in commit 1820b51e

By Caleb Connolly on 2024-07-15T07:51:06

mentioned in merge request !5353 (merged)

By Caleb Connolly on 2024-07-15T07:52:44

mentioned in commit 3c8ad7df

By Caleb Connolly on 2024-07-15T21:38:03

mentioned in commit f32d72a4

By Caleb Connolly on 2024-07-20T07:26:46

mentioned in commit 4309b124

By Caleb Connolly on 2024-07-24T16:36:48

mentioned in commit 9dbbc72d

By Caleb Connolly on 2024-07-29T22:05:27

marked the checklist item Update our fork of apk-tools with the backported new patches as completed

By clayton craft on 2024-07-29T22:45:13

marked the checklist item Test extensively (maybe ask people for their /etc/apk/world and check in VM?) as completed

By clayton craft on 2024-07-29T22:45:15

marked the checklist item it as completed

By clayton craft on 2024-07-29T22:45:16

fixed by !5353 (merged)

By clayton craft on 2024-07-29T22:45:34

closed

mentioned in commit e8ea67a7

By Caleb Connolly on 2024-08-06T21:41:06

mentioned in commit 8a1435b7

By Caleb Connolly on 2024-08-15T17:40:46

mentioned in commit 5303c4f2

By Caleb Connolly on 2024-08-15T18:13:41

mentioned in commit 0a566b5e

By Caleb Connolly on 2024-08-19T22:35:31

marked this issue as blocking #2938 (closed)

By clayton craft on 2024-08-19T23:54:18

mentioned in commit d2b91aa4

By Caleb Connolly on 2024-09-02T17:25:43

mentioned in commit 53b10de9

By Caleb Connolly on 2024-09-03T23:26:54

mentioned in commit e2ac2f1a

By Caleb Connolly on 2024-09-06T06:51:35

mentioned in commit 99133fa3

By Caleb Connolly on 2024-09-09T18:02:15

mentioned in commit d98c5341

By Caleb Connolly on 2024-09-09T19:34:17

mentioned in commit b3b5e33f

By Caleb Connolly on 2024-09-09T19:40:09

mentioned in commit 69ebe97f

By Caleb Connolly on 2024-09-09T22:06:35

mentioned in commit 530035a2

By Caleb Connolly on 2024-09-10T22:44:06

mentioned in commit 92e9e000

By Caleb Connolly on 2024-09-10T23:26:48

mentioned in commit ae3b1b5a

By Caleb Connolly on 2024-09-12T16:54:50

mentioned in commit 153d4aca

By Caleb Connolly on 2024-09-12T23:00:35

mentioned in commit aa27c182

By Caleb Connolly on 2024-09-24T08:15:35

mentioned in commit 57b6a375

By Caleb Connolly on 2024-09-28T14:42:20

mentioned in commit d217d1ae

By Caleb Connolly on 2024-09-30T13:55:39

mentioned in commit 540956bf

By Caleb Connolly on 2024-09-30T14:38:31

mentioned in commit 3f32206c

By Caleb Connolly on 2024-10-02T15:30:03

mentioned in commit 2a660683

By Caleb Connolly on 2024-10-03T18:54:16

mentioned in commit 9be003ed

By Caleb Connolly on 2024-10-04T19:32:36

mentioned in commit a3603706

By Caleb Connolly on 2024-10-04T19:46:54

mentioned in commit cc8e9224

By Caleb Connolly on 2024-10-04T21:17:15

mentioned in commit eb23ddca

mentioned in commit cc8e9224

mentioned in commit d729669e

mentioned in commit b0401d31