fix(build): the macOS ordering shim must not be able to break the link

Two Mach-O facts the first CI round found the hard way:

  * an __asm__ label is used VERBATIM — clang does not prepend Mach-O's
    global '_'. The C++ symbol _ZNSt3__18ios_base4InitC1Ev therefore has to
    be written __ZNSt3__18ios_base4InitC1Ev, and getting it wrong is not a
    silent no-op: every macOS link failed with
    'undefined symbol: ZNSt3__18ios_base4InitC1Ev'.

  * plain __attribute__((weak)) on a declaration is NOT Mach-O's
    weak-undefined form, so it did not make the bad reference optional.
    weak_import is.

Both are now correct, but neither is the safety net. The backend only
generates the shim TU when the libc++ archive actually defines the symbol —
checked by scanning the ranlib index in the archive's first member, no
subprocess — so an unexpected libc++ spelling disables the ordering aid and
says so, instead of failing the build. A check upstream of the reference
cannot break a link the way the reference itself can.

Author: Sunrisepeak

src/build/distribution.cppm

@@ -164,6 +164,11 @@ struct MechanismInput {
     std::string      libcxxArchive;
     std::string      libcxxAbiArchive;
     std::string      libunwindArchive;
+    // macOS only: the libc++ archive actually defines the ABI symbol the
+    // initializer-ordering shim binds to. Checked against the archive rather
+    // than assumed, so an unexpected spelling disables the shim instead of
+    // producing an undefined reference at link time.
+    bool             streamInitSymbolPresent = false;
     // macOS only: a deployment floor was resolved. The static-libc++
     // mechanism exists to make that floor real, so without one there is
     // nothing to make real.
@@ -259,7 +264,21 @@ Mechanism resolve(const MechanismInput& in) {
             m.unitFlags = " -nostdlib++"
                           " -Wl,-load_hidden," + in.libcxxArchive +
                           " -Wl,-load_hidden," + in.libcxxAbiArchive;
-            m.streamInitShim = true;
+            // The ordering shim binds a libc++ INTERNAL ABI symbol, so its
+            // presence is verified against the archive instead of assumed.
+            // Getting this wrong must not break the link — hence a check
+            // here rather than a weak reference in the shim: Mach-O's
+            // weak-undefined form is `weak_import` and applies to dylib
+            // symbols, so a plain weak declaration would NOT have saved a
+            // missing archive symbol (it did not: ld64.lld errored outright).
+            m.streamInitShim = in.streamInitSymbolPresent;
+            if (!m.streamInitShim) {
+                m.diagnostic =
+                    "this libc++ does not export the stream initializer mcpp "
+                    "orders first on macOS; a global object whose constructor "
+                    "uses std::cout may crash at startup (mcpp#336). Use "
+                    "cxx_runtime = \"host-coupled\" if you hit it";
+            }
             return m;
         }
         // HostCoupled
@@ -367,12 +386,22 @@ Mechanism resolve(const MechanismInput& in) {
 // WHY C: it needs no standard library, no module flags and no C++ ABI of its
 // own — it only has to run before everything else and poke one symbol.
 //
-// WHY WEAK: `_ZNSt3__18ios_base4InitC1Ev` is a libc++ ABI symbol
-// (_LIBCPP_EXPORTED_FROM_ABI), stable for many years but not ours. A weak
-// undefined reference means a toolchain that spells it differently links
-// exactly as it does today and the shim is a no-op, instead of failing the
-// link. It also means the shim does not itself drag iostream.cpp.o out of the
-// archive: if the program never touches a stream there is nothing to order.
+// WHY THE NAME IS SPELLED WITH TWO UNDERSCORES: an `__asm__` label is used
+// VERBATIM — clang does not add Mach-O's global `_` prefix to it. The C++
+// symbol `_ZNSt3__18ios_base4InitC1Ev` therefore has to be written
+// `__ZNSt3__18ios_base4InitC1Ev` here. Getting this wrong is not a silent
+// no-op: ld64.lld reports `undefined symbol: ZNSt3__18ios_base4InitC1Ev` and
+// every link fails, which is exactly what the first CI round did.
+//
+// WHY `weak_import` AND a presence check: Mach-O's weak-undefined form is
+// `weak_import` (plain `weak` on a declaration does NOT make an undefined
+// reference optional there). Even so, the real safety net is upstream — the
+// backend only generates this TU when the archive actually defines the
+// symbol, so an unexpected libc++ spelling disables the shim rather than
+// breaking the link. The attribute is the second line of defence.
+//
+// The reference does not itself drag iostream.cpp.o out of the archive: if the
+// program never touches a stream, there is nothing to order.
 //
 // WHY IT WORKS: libc++'s `ios_base::Init::Init()` is not empty — it is a
 // guarded function-local static that calls `DoIOSInit::DoIOSInit()`, and THAT
@@ -396,7 +425,8 @@ std::string_view stream_init_shim_source() {
         " * Weak: a toolchain without this exact ABI symbol links as before.\n"
         " */\n"
         "extern void mcpp_libcxx_ios_init(void *)\n"
-        "    __attribute__((weak)) __asm__(\"_ZNSt3__18ios_base4InitC1Ev\");\n"
+        "    __attribute__((weak_import))\n"
+        "    __asm__(\"__ZNSt3__18ios_base4InitC1Ev\");\n"
         "\n"
         "static char mcpp_libcxx_ios_init_storage[8];\n"
         "\n"

src/build/flags.cppm

@@ -564,12 +564,12 @@ CompileFlags compute_flags(const BuildPlan& plan) {
         // macOS packages) or under lib/<llvm-triple>/ (the Linux ones), so try
         // both rather than hard-coding one layout. Sorted so the choice cannot
         // depend on directory iteration order.
-        auto find_archive = [&](std::string_view name) -> std::string {
+        auto find_archive = [&](std::string_view name) -> std::filesystem::path {
             if (llvmRootForStdlib.empty()) return {};
             std::error_code ec;
             auto libDir = llvmRootForStdlib / "lib";
             auto direct = libDir / name;
-            if (std::filesystem::exists(direct, ec)) return escape_path(direct);
+            if (std::filesystem::exists(direct, ec)) return direct;
             std::vector<std::filesystem::path> hits;
             for (auto& e : std::filesystem::directory_iterator(libDir, ec)) {
                 std::error_code de;
@@ -578,7 +578,28 @@ CompileFlags compute_flags(const BuildPlan& plan) {
                 if (std::filesystem::exists(p, de)) hits.push_back(p);
             }
             std::ranges::sort(hits);
-            return hits.empty() ? std::string{} : escape_path(hits.front());
+            return hits.empty() ? std::filesystem::path{} : hits.front();
+        };
+
+        // Does this archive define a given symbol? Answered by scanning the
+        // ranlib symbol index, which a BSD/Mach-O archive keeps in its FIRST
+        // member — so a bounded read of the head is enough and no toolchain
+        // subprocess is involved. Used for exactly one thing: refusing to
+        // generate the macOS ordering shim against a libc++ that does not
+        // export the symbol it binds. That failure mode is not hypothetical —
+        // the first CI round of #336 turned every macOS link into `undefined
+        // symbol` — and a check here cannot fail the build the way a bad
+        // reference in the generated TU can.
+        auto archive_defines = [](const std::filesystem::path& p,
+                                  std::string_view sym) -> bool {
+            if (p.empty()) return false;
+            std::ifstream is(p, std::ios::binary);
+            if (!is) return false;
+            constexpr std::streamsize kHead = 8 << 20;
+            std::string head(static_cast<std::size_t>(kHead), '\0');
+            is.read(head.data(), kHead);
+            head.resize(static_cast<std::size_t>(is.gcount()));
+            return head.find(sym) != std::string::npos;
         };
 
         dist::MechanismInput mi;
@@ -601,12 +622,21 @@ CompileFlags compute_flags(const BuildPlan& plan) {
              || testsContract == dist::Contract::SelfContained)
             && caps.stdlib_id == "libc++";
         if (wantsArchives) {
-            mi.libcxxArchive    = find_archive("libc++.a");
-            mi.libcxxAbiArchive = find_archive("libc++abi.a");
+            auto libcxxA    = find_archive("libc++.a");
+            auto libcxxAbiA = find_archive("libc++abi.a");
+            mi.libcxxArchive    = libcxxA.empty()    ? std::string{} : escape_path(libcxxA);
+            mi.libcxxAbiArchive = libcxxAbiA.empty() ? std::string{} : escape_path(libcxxAbiA);
             // ELF only: without it the "self-contained" binary still pulls
             // libunwind.so.1. Mach-O's libc++abi.a carries its own unwinder.
-            if (mi.format == dist::Format::Elf)
-                mi.libunwindArchive = find_archive("libunwind.a");
+            if (mi.format == dist::Format::Elf) {
+                auto unwindA = find_archive("libunwind.a");
+                if (!unwindA.empty()) mi.libunwindArchive = escape_path(unwindA);
+            } else {
+                // Searched without the leading underscore so the same needle
+                // matches the ELF (`_ZN...`) and Mach-O (`__ZN...`) spellings.
+                mi.streamInitSymbolPresent =
+                    archive_defines(libcxxA, "ZNSt3__18ios_base4InitC1Ev");
+            }
         }
 
         // "Explicit" = a human wrote it down. `static_stdlib = false` counts:

tests/unit/test_distribution.cpp

@@ -22,6 +22,7 @@ dist::MechanismInput macos_input() {
     in.macosFloor       = true;
     in.libcxxArchive    = "/tc/lib/libc++.a";
     in.libcxxAbiArchive = "/tc/lib/libc++abi.a";
+    in.streamInitSymbolPresent = true;
     return in;
 }
 
@@ -271,6 +272,7 @@ TEST(Distribution, TableIsTotalAndEveryDowngradeExplainsItself) {
             in.libcxxArchive    = "/a.a";
             in.libcxxAbiArchive = "/b.a";
             in.libunwindArchive = "/c.a";
+            in.streamInitSymbolPresent = true;
         }
         auto m = dist::resolve(in);
         auto where = std::format("contract={} format={} stdlib={} role={} archives={}",
@@ -298,16 +300,35 @@ TEST(Distribution, TableIsTotalAndEveryDowngradeExplainsItself) {
 // differently). See issue #336 for the disassembly this rests on.
 TEST(Distribution, StreamInitShimKeepsItsThreeLoadBearingProperties) {
     auto src = std::string(dist::stream_init_shim_source());
-    // 1. weak — a toolchain without this ABI symbol must still link.
-    EXPECT_NE(src.find("__attribute__((weak))"), std::string::npos);
+    // 1. weak_import — Mach-O's weak-UNDEFINED form. Plain `weak` is not it;
+    //    the first CI round proved that by failing every macOS link.
+    EXPECT_NE(src.find("__attribute__((weak_import))"), std::string::npos);
     // 2. ios_base::Init::Init, NOT DoIOSInit::DoIOSInit — the former is
     //    guarded by __cxa_guard, so libc++'s own initializer later becomes a
     //    no-op instead of placement-new'ing over live streams.
-    EXPECT_NE(src.find("_ZNSt3__18ios_base4InitC1Ev"), std::string::npos);
+    //    ...spelled with TWO leading underscores: an __asm__ label is used
+    //    verbatim, so Mach-O's global `_` prefix has to be written out.
+    EXPECT_NE(src.find("\"__ZNSt3__18ios_base4InitC1Ev\""), std::string::npos);
     EXPECT_EQ(src.find("DoIOSInit"), std::string::npos);
     // 3. a constructor, and it must be guarded by the weak null check.
     EXPECT_NE(src.find("__attribute__((constructor))"), std::string::npos);
     EXPECT_NE(src.find("if (mcpp_libcxx_ios_init)"), std::string::npos);
     // C, not C++: no standard library, no module flags, no ABI of its own.
     EXPECT_EQ(src.find("#include"), std::string::npos);
 }
+
+// The shim binds a libc++ INTERNAL symbol. If that symbol is not in the
+// archive, mcpp must NOT generate the reference — an undefined symbol fails
+// the link outright (ld64.lld does not treat a weak declaration as an
+// optional undefined). The absence is reported instead, because the startup
+// hazard is still there.
+TEST(Distribution, ShimIsNotGeneratedWhenTheSymbolIsAbsent) {
+    auto in = macos_input();
+    in.streamInitSymbolPresent = false;
+    auto m = dist::resolve(in);
+    EXPECT_FALSE(m.streamInitShim);
+    EXPECT_FALSE(m.diagnostic.empty());
+    // The contract itself is still honored — only the ordering aid is gone.
+    EXPECT_EQ(m.effective, dist::Contract::SelfContained);
+    EXPECT_NE(m.unitFlags.find("-load_hidden"), std::string::npos);
+}