rustc 1.59/1.60 builds musl binaries that segfault, when compiling with musl-gcc wrappers, due to static-pie default

[joshtriplett]

With the x86_64-unknown-linux-musl target installed:

$ cat hello.rs 
fn main() {
    println!("Hello, world!");
}
$ rustc hello.rs --target x86_64-unknown-linux-musl && file hello && ./hello 
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), static-pie linked, BuildID[sha1]=29ba716c85a1f3f9990246f160b61003b38b63a9, with debug_info, not stripped
Hello, world!
$ rustc hello.rs --target x86_64-unknown-linux-musl -C linker=musl-gcc && file hello && ./hello 
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
Segmentation fault

A gdb backtrace shows the segfault in _start_c () at ../src_musl/crt/../ldso/dlstart.c:141.

This segfault happens because the musl-gcc wrappers (including cross-compile wrappers like x86_64-linux-musl-gcc and aarch64-linux-musl-gcc) don't support static-pie in the way rustc is invoking them.

This is a regression in Rust 1.59 and 1.60; 1.58 and before produced working binaries. 1.59 includes #70740 , which not only marked the musl targets as supporting static-pie (which they may or may not depending on the linker), but also made x86_64-unknown-linux-musl default to static-pie.

For context, the musl-gcc wrappers are commonly used for a variety of reasons, including cross-compilation (particularly when linking native C code). Attempting to link using x86_64-linux-gnu-gcc or aarch64-linux-gnu-gcc cross-compilers (rather than the corresponding -musl-gcc cross-compilers) produces compatibility issues, especially with projects that build and link native C dependencies.

I noticed this problem when working with someone trying to cross-compile a project from aarch64 to x86-64, using the x86_64-linux-musl-gcc cross-compiler wrapper, installed via Debian. (I hadn't previously noticed the regression myself, because I hadn't arranged to use the cross-compiler wrappers on x86_64 since my native host is x86_64.)

This problem doesn't currently show up when cross-compiling from x86_64 to aarch64, since aarch64 doesn't use static-pie by default, but enabling PIE with -C relocation-model=pie triggers the same bug and makes the resulting binaries segfault.

I'm currently working around this, but I think this is a stable regression.

(For reference, two possible workarounds: use -C relocation-model=static, or use linker=x86_64-linux-gnu-gcc and hope it works for you.)

[12101111]

Can you provide the linker command?
It's fixed in 2018: https://git.musl-libc.org/cgit/musl/commit/?id=7dad9c212587267818de919dd9c5886f18f99779
(Unless you are using Ubuntu 18.04 which is unfixed)

[joshtriplett]

~$ cat /usr/bin/x86_64-linux-musl-gcc
#!/bin/sh
exec "${REALGCC:-x86_64-linux-gnu-gcc}" "$@" -specs "/usr/lib/x86_64-linux-musl/musl-gcc.specs"
~$ cat /usr/lib/x86_64-linux-musl/musl-gcc.specs
%rename cpp_options old_cpp_options

*cpp_options:
-nostdinc -isystem /usr/include/x86_64-linux-musl -isystem include%s %(old_cpp_options)

*cc1:
%(cc1_cpu) -nostdinc -isystem /usr/include/x86_64-linux-musl -isystem include%s

*link_libgcc:
-L/usr/lib/x86_64-linux-musl -L .%s

*libgcc:
libgcc.a%s %:if-exists(libgcc_eh.a%s)

*startfile:
%{!shared: /usr/lib/x86_64-linux-musl/Scrt1.o} /usr/lib/x86_64-linux-musl/crti.o crtbeginS.o%s

*endfile:
crtendS.o%s /usr/lib/x86_64-linux-musl/crtn.o

*link:
-dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib %{shared:-shared} %{static:-static} %{rdynamic:-export-dynamic}

*esp_link:


*esp_options:


*esp_cpp_options:

[12101111]

rust/compiler/rustc_codegen_ssa/src/back/linker.rs

Lines 382 to 402 in 327caac

	LinkOutputKind::StaticNoPicExe => {
	// `-static` works for both gcc wrapper and ld.
	self.cmd.arg("-static");
	if !self.is_ld && self.sess.target.linker_is_gnu {
	self.cmd.arg("-no-pie");
	}
	}
	LinkOutputKind::StaticPicExe => {
	if !self.is_ld {
	// Note that combination `-static -pie` doesn't work as expected
	// for the gcc wrapper, `-static` in that case suppresses `-pie`.
	self.cmd.arg("-static-pie");
	} else {
	// `--no-dynamic-linker` and `-z text` are not strictly necessary for producing
	// a static pie, but currently passed because gcc and clang pass them.
	// The former suppresses the `INTERP` ELF header specifying dynamic linker,
	// which is otherwise implicitly injected by ld (but not lld).
	// The latter doesn't change anything, only ensures that everything is pic.
	self.cmd.args(&["-static", "-pie", "--no-dynamic-linker", "-z", "text"]);
	}
	}

The musl-gcc wrapper don't recongnize -static-pie/-pie/-no-pie, but do recongnize -static. The combination of -dynamic-linker and -static also break lld.

> cat hello.c
#include<stdio.h>
int main() {
  printf("Hello, world!\n");
  return 0;
}
> musl-gcc hello.c -o hello -static-pie
ld -plugin /usr/lib/gcc/x86_64-linux-gnu/10/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper -plugin-opt=-fresolution=/tmp/cciwsIZK.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -pie -o hello /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginS.o -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/10/. /tmp/ccNzdDMJ.o --start-group /usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/10/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
> file hello
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
> musl-gcc hello.c -o hello -static
ld -plugin /usr/lib/gcc/x86_64-linux-gnu/10/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper -plugin-opt=-fresolution=/tmp/cciPPY3o.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -static -o hello /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginS.o -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/10/. /tmp/cckhAQSp.o --start-group /usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/10/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
> file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped
 > musl-gcc -fuse-ld=lld -static hello.c -o hello
ld.lld -plugin /usr/lib/gcc/x86_64-linux-gnu/10/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper -plugin-opt=-fresolution=/tmp/cc788cAg.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -static -o hello /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginS.o -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/10/. /tmp/ccsEJC6i.o --start-group /usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/10/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
> file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
> ./hello
zsh: segmentation fault  ./hello

Fedora fix this https://src.fedoraproject.org/rpms/musl/blob/rawhide/f/musl-1.2.0-Support-static-pie-with-musl-gcc-specs.patch and Debian don't have this patch.

[apiraino]

thanks @12101111 for the investigation. This looks like a distro specific issue that should be probably handled by them, correct? Just trying to understand if there's an actionable on our side.

[12101111]

Yes, we can't fix the broken wrapper in our side ( we can detect it but can't fix it expect creating a new wrapper).

[joshtriplett]

@apiraino The action on the Rust side would be to roll back the use of this by default until it's more widely supported. Or, alternatively, attempt to detect the use of the wrapper and roll it back in that circumstance, but that seems fraught.

[joshtriplett]

(Also, long-term, we could fix this by making the wrappers not necessary anymore for cross compiling with musl.)

[apiraino]

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-high