Cross compiling to ARM

I’m working on a project that Compiles to ARM and runs this compiled binary in QEMU vm connected to a GDB session. On Ubuntu all I need to do to install those programs is sudo apt install qemu gdb-multiarch gcc-8-aarch64-linux-gnu and it all works fine, but on Fedora none of these libraries are currently supported or work. There’s a gcc compiler to arm but there’s no glibc, I couldn’t find any package (available for download) similar to gdb-multiarch, though there’s a qemu version for ARM I couldn’t test it because I can’t cross compile.

To me this seems like a big issue because many embedded engineering workflows depend on being able to cross compile and test ARM binaries. Any help is appreciated!

1 Like

I don’t know if this helps you:

Fedora GDB Maintainer Guide - Fedora Project Wiki

Not much help but it seems like the complexity of maintaining the packages is too daunting … Only building kernels is currently supported. Support for cross-building user space programs is not currently provided as that would massively multiply the number of packages.

I scp my projects to the aarch64 machine and build the RPMs there. It’s slowish but is a possible solution (although probably not what you were looking for). :scream_cat:

$ dnf info gcc-arm-linux-gnu-11.2.1-1.fc35.x86_64 
Last metadata expiration check: 0:00:16 ago on Sat 06 Nov 2021 12:18:28 PM EDT.
Available Packages
Name         : gcc-arm-linux-gnu
Version      : 11.2.1
Release      : 1.fc35
Architecture : x86_64
Size         : 26 M
Source       : cross-gcc-11.2.1-1.fc35.src.rpm
Repository   : fedora
Summary      : Cross-build binary utilities for arm-linux-gnu
URL          : http://gcc.gnu.org
License      : GPLv3+ and GPLv3+ with exceptions and GPLv2+ with exceptions and LGPLv2+ and BSD
Description  : Cross-build GNU C compiler.
             : 
             : Only building kernels is currently supported.  Support for cross-building
             : user space programs is not currently provided as that would massively multiply
             : the number of packages.

1 Like

Debian and Ubuntu has cross compiler capable of building userspace because there is a support for installing target packages (check for “Debian MultiArch” thing).

Fedora has cross compiler for bare metal compilations - bootloaders, kernels only. There are no plans to provide cross compiler capable of building user space binaries.

1 Like

The other alternative is to use mock for aarch64 but its very slow because of the arm emulation.

INFO: Unable to build arch aarch64 natively on arch x86_64. Setting forcearch to use software emulation.

Fedora’s cross-toolchain is built with “sysroot” support (see cross-gcc.spec), allowing the user to provide libraries of their choice in this location. I believe embedded development usually provides their own cross-environment like for example BuildRoot.

1 Like

I’m using the copr repository lantw44:arm-linux-gnueabihf-toolchain as a better toolchain than the one present in fedora. As the fedora toolchain can only be used to cross-compile bootloader or the kernel but no userspace application (not even helloworld.c).

Cross-compiler toolchain often are provided by newlib instead of a full glibc (there is another cross toolchain as arm-none-eabi-newlib in fedora other than the cross-gcc one)

FYI, I’m toying with kernel cross-compilation at Package kernel-longterm in kwizart/cross-armhfp, so I’m able to use cross-compilation for the kernel.

1 Like

I’ve experienced some issue by using this method as the cross-kernel-headers location breaks, so I wasn’t able to cross-compile even a helloworld.c

Ideally, it should be possible to initiate an armv7hl buildroot using qemu-emulation, then pass (bind mount or else ) the installed rootfs to the sysroot expected path by the cross-compiler…

Could you file a bug for it, please? I think the toolchain guys would be interesting in the feedback.

1 Like

Seems like glibc for arm is unmaintained,
https://koji.fedoraproject.org/koji/packageinfo?packageID=22508

Without it, I’m experiencing error when trying to build any simple program, tried with:
fedpkg clone a52dec
cd a52dec
fedpkg srpm
rpmbuild --rebuild *.src.rpm --target=armv7hl -D ‘__cc arm-linux-gnu-gcc’ -D ‘_host arm-linux-gnu’ -D ‘_build x86_64-redhat-linux-gnu’

Compilation fails and config.log shows:

configure:2106: arm-linux-gnu-gcc -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong   -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld   conftest.c  >&5
/usr/bin/arm-linux-gnu-ld: cannot find Scrt1.o: No such file or directory
/usr/bin/arm-linux-gnu-ld: cannot find crti.o: No such file or directory
/usr/bin/arm-linux-gnu-ld: cannot find -lc
/usr/bin/arm-linux-gnu-ld: cannot find crtn.o: No such file or director

I haven’t seen anything about glibc/newlib aarch64 related.

Next I can try mounting a arm/aarch64 buildroot into the cross-compiler sysroot…

The arm-none-eabi- toolchain is only a little better:

configure:2103: checking for C compiler default output
configure:2106: arm-none-eabi-gcc -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong   -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld   conftest.c  >&5
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: warning: -z relro ignored
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: warning: /tmp/ccZFytg1.ltrans0.ltrans.o uses 32-bit enums yet the output is to use variable-size enums; use of enum values across objects may fail
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/thumb/v7+fp/hard/crtbegin.o: relocation R_ARM_THM_MOVW_ABS_NC against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-exit.o): relocation R_ARM_THM_MOVW_ABS_NC against `_global_impure_ptr' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-fini.o): relocation R_ARM_THM_MOVW_ABS_NC against `__fini_array_end' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-init.o): relocation R_ARM_THM_MOVW_ABS_NC against `__preinit_array_end' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-__atexit.o): relocation R_ARM_THM_MOVW_ABS_NC against `__atexit_recursive_mutex' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-__call_atexit.o): relocation R_ARM_THM_MOVW_ABS_NC against `__libc_fini' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/bin/ld: /usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-exit.o): in function `exit':
/builddir/build/BUILD/newlib-4.1.0/build-newlib/arm-none-eabi/thumb/v7+fp/hard/newlib/libc/stdlib/../../../../../../../../newlib/libc/stdlib/exit.c:64: undefined reference to `_exit'
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-__call_atexit.o):(.init_array.00000+0x0): dangerous relocation: unsupported relocation
/usr/lib/gcc/arm-none-eabi/11.1.0/../../../../arm-none-eabi/lib/thumb/v7+fp/hard/libg.a(lib_a-__call_atexit.o): in function `.LANCHOR0':
/builddir/build/BUILD/newlib-4.1.0/build-newlib/arm-none-eabi/thumb/v7+fp/hard/newlib/libc/stdlib/../../../../../../../../newlib/libc/stdlib/__call_atexit.c:17:(.data.__atexit_recursive_mutex+0x0): dangerous relocation: unsupported relocation
/usr/lib/gcc/arm-none-eabi/11.1.0/thumb/v7+fp/hard/crtbegin.o:(.init_array+0x0): dangerous relocation: unsupported relocation
/usr/lib/gcc/arm-none-eabi/11.1.0/thumb/v7+fp/hard/crtbegin.o:(.fini_array+0x0): dangerous relocation: unsupported relocation
collect2: error: ld returned 1 exit status
configure:2109: $? = 1

Seems like this toolchain default to thumb mode over arm hardfloat…(probably configurable…)

There is the option of using compilers from the PlatformIO (https://platformio.org) project, they install locally in .platofrmio/lib and you can target most platforms including arm variants. However this by default isn’t going to provide full platform support for compiling for Apple or Raspberry PI, but could most likely be extended for those use cases. There is also a clib that supports ARM that you can use to compile static files with: https://musl.libc.org