cross/crossdirect: improve rust handling

I'm suggesting this as a temporary solution for the current issues with cross-compiling Rust programs. Most big Rust projects use proc-macro crates such as serde_derive, which currently make the rustc wrapper fall back to compiling everything in QEMU (resulting in a slow build).

These changes remove the need for the QEMU+rustc fallback in the wrapper. This works because Cargo's cross-compiling logic can be enabled even when using the same target architecture as the host architecture. Cargo will then pass a --target argument to rustc for all crates that are not going to be used on the host. If the library paths are adjusted accordingly, the native rustc will produce correct results with the arguments provided by Cargo, i.e. working native build scripts and macro crates when called without --target.

Please note that this also causes Cargo build scripts to be built for and run on the native architecture, which means that any libraries they require must be present in the native chroot. This might require a change in pmbootstrap such as this one:

--- a/pmb/build/init.py
+++ b/pmb/build/init.py
@@ -108,6 +108,6 @@ def init_compiler(args, depends, cross, arch):
     if cross == "crossdirect":
         cross_pkgs += ["crossdirect"]
         if "rust" in depends or "cargo" in depends:
-            cross_pkgs += ["rust"]
+            cross_pkgs += depends
 
     pmb.chroot.apk.install(args, cross_pkgs)

added 1 commit

• 4e367a6a - cross/crossdirect: improve rust handling

Compare with previous version

By Affe Null on 2023-07-06T13:41:04

added 1 commit

• c1756787 - cross/crossdirect: improve rust handling

Compare with previous version

By Affe Null on 2023-07-06T13:48:34

marked this merge request as ready

By Affe Null on 2023-07-07T10:12:27

added 14 commits

• c1756787...d3fd1f5e - 13 commits from branch postmarketOS:master
• f632e2ca - cross/crossdirect: improve rust handling

Compare with previous version

By Affe Null on 2023-07-09T09:28:44

resolved all threads

By Affe Null on 2023-07-09T09:34:54

@affenull2345, This is so great to see, thanks so much for working on these patches! I'd love to just have crossdirect working properly with rust!

Please note that this also causes Cargo build scripts to be built for and run on the native architecture, which means that any libraries they require must be present in the native chroot. This might require a change in pmbootstrap such as this one:

Can you make the pmbootstrap adjustment, and also ensure that the rust related test cases in pmbootstrap still pass / get adjusted to work with this crossdirect change?

By Oliver Smith on 2023-07-10T06:13:30

if you want to: it's probably worth adding a CROSSDIRECT_DEBUG env var, and if it is set, be more verbose in crossdirect so we see which path it takes. We could then set this in the APKBUILDs for debugging.

maybe like this (untested):

if [ -n "$CROSSDIRECT_DEBUG" ]; then
	set -x
fi

But adding more debug output could also be done in a separate MR later on.

By Oliver Smith on 2023-07-10T06:16:48

added 1 commit

• 0551c6ef - cross/crossdirect: improve rust handling

Compare with previous version

By Affe Null on 2023-07-10T11:52:48

mentioned in commit pmbootstrap@122b9000

By Affe Null on 2023-07-13T08:28:21

added 2 commits

• 763f75c9 - 1 commit from branch postmarketOS:master
• abf42a8b - cross/crossdirect: improve rust handling

Compare with previous version

By Affe Null on 2023-07-16T10:08:49

added 16 commits

• abf42a8b...4e0229f7 - 15 commits from branch postmarketOS:master
• cb8ae770 - cross/crossdirect: improve rust handling

Compare with previous version

By Affe Null on 2023-07-16T10:09:52

Thanks again! All feedback has been addressed and no more comments for a week already, so let's merge it!

By Oliver Smith on 2023-07-24T06:31:11

approved this merge request

By Oliver Smith on 2023-07-24T06:31:11

added 58 commits

• cb8ae770...042d577d - 57 commits from branch postmarketOS:master
• f0286576 - cross/crossdirect: improve rust handling (MR 4234)

Compare with previous version

By Oliver Smith on 2023-07-24T06:32:28