build: LLD based macOS toolchain #21778
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code CoverageFor detailed information about the code coverage, see the test coverage report. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
EDIT: Just a misunderstanding |
I might be misunderstanding your comment, but the opposite is happening here. We're migrating away from using Apples, irregularly open sourced tools (cctools, ld64), to just vanilla LLVM (lld), which is entirely open source. |
My bad! I thought this was switching from an unsupported apple open-source toolchain to an supported apple closed source toolchain, not to a vanilla toolchain. My blinding hate of apple has kept me from seeing reality! Thanks for the clarification. This all seems very good. |
|
Very nice! Strong concept ACK, but obviously we can't use this until all features we need are supported. But it's great to see that linking finally works! For a little extra context beyond what @fanquake said above, the macOS clang+cctools build has always been a wonky outlier because we have to build our own (very unsupported) toolchain first. LD64 is apple's libre/open-source linker, and there is a community project that maintains a build environment for cross builds. For a few years, there has been slow development on a darwin functionality for llvm's lld, but it's only really picked up speed and become usable in the last year or two. This PR would bring our Apple builds much more inline with others in that it would finally be using a supported libre/open-source compiler/linker combo. @fanquake Is cctools still needed at all? LLVM ships bitcode equivalents of most binutils tools (ar, nm, ranlib, etc), I wonder if we can use those instead. Maybe they require lto? |
@theuni no cctools is no longer required. I drop the native_cctools as part of this PR and we just use the LLVM tools. |
Apologies for reading too quickly, I see now that I only looked at the first commit. Turns out the 3rd one is the one I expected :) |
|
Rebased for #21793, and rebased on top of #19817. Made some additional changes, such as renaming the Note: I'm pretty sure the use of |
fanquake
force-pushed
the
llvm_13_ld64_lld
branch
from
b79ed8c
to
f068682
Compare
This was referenced Jul 13, 2021
fanquake
force-pushed
the
llvm_13_ld64_lld
branch
from
f068682
to
e6b20d5
Compare
fanquake
force-pushed
the
llvm_13_ld64_lld
branch
7 times, most recently
from
e1b2e6e
to
1179221
Compare
fanquake
force-pushed
the
llvm_13_ld64_lld
branch
from
b32b3a1
to
e8c25e8
Compare
|
Guix build (aarch64): Another mismatch, this time on riscv64 -_- |
I've just rebuilt on 2 different machines (one aarch64 one x86_64), and they match your output here (removing my comment above): 00becde2dd12878e3b9f50f27899a6a8b752343dade7c71781632715c3001473 guix-build-e8c25e8a35e3/output/aarch64-linux-gnu/SHA256SUMS.part
a685b9cee54014e74639be1e8db2d55b7c008fdb3b31c1c708c364a49b56759a guix-build-e8c25e8a35e3/output/aarch64-linux-gnu/bitcoin-e8c25e8a35e3-aarch64-linux-gnu-debug.tar.gz
d61228158409802e5aef11c39a0da5653a6c7e870d5f500483c32c75f319e8b6 guix-build-e8c25e8a35e3/output/aarch64-linux-gnu/bitcoin-e8c25e8a35e3-aarch64-linux-gnu.tar.gz
49447a196ead1f153cbebab2e22d229cb48f2736eec5c912c2ec5cebf8a58cad guix-build-e8c25e8a35e3/output/arm-linux-gnueabihf/SHA256SUMS.part
1e4510f6a5145810577b8ebc2eb5f38b3d2c28845e9d84ffe000d8cbb82429fa guix-build-e8c25e8a35e3/output/arm-linux-gnueabihf/bitcoin-e8c25e8a35e3-arm-linux-gnueabihf-debug.tar.gz
a3c014032ac22c2fc0034a5708b157884395f37f3d20dfa6070cfb2a6aeebdf5 guix-build-e8c25e8a35e3/output/arm-linux-gnueabihf/bitcoin-e8c25e8a35e3-arm-linux-gnueabihf.tar.gz
6377bf759602fb5dc00d7992b126ddc75cf03d71ce259c76c509299dac51be52 guix-build-e8c25e8a35e3/output/arm64-apple-darwin/SHA256SUMS.part
7191d5a512ac9b7e934e771985eb2743180a00285b3e5275de1ffdc1e504e68e guix-build-e8c25e8a35e3/output/arm64-apple-darwin/bitcoin-e8c25e8a35e3-arm64-apple-darwin-unsigned.tar.gz
73516bfd8acb17f865b53e67e421dccfd0644c6582f462d8ebe710291c2b5367 guix-build-e8c25e8a35e3/output/arm64-apple-darwin/bitcoin-e8c25e8a35e3-arm64-apple-darwin-unsigned.zip
07cc918282582193d8745cc7dcd4a4a4474e38d079b0d2831efdef59cb322449 guix-build-e8c25e8a35e3/output/arm64-apple-darwin/bitcoin-e8c25e8a35e3-arm64-apple-darwin.tar.gz
edcc1c61aa12b78c7a8e87d141832891e05f48b4a687ed7e3fae668236f339ff guix-build-e8c25e8a35e3/output/dist-archive/bitcoin-e8c25e8a35e3.tar.gz
29601464afa40104ad8c6b4b10a5fde0649aba6f4bfea1b93258b7bfcaab7a5a guix-build-e8c25e8a35e3/output/powerpc64-linux-gnu/SHA256SUMS.part
65c308cddf5cbac4132104a5b4d6cf7ec85e88b4f967c66cd942742996def56a guix-build-e8c25e8a35e3/output/powerpc64-linux-gnu/bitcoin-e8c25e8a35e3-powerpc64-linux-gnu-debug.tar.gz
513c91b558e52ac27e6e33d916bfc378a6be6b2b218fb37712f05a44746fb68d guix-build-e8c25e8a35e3/output/powerpc64-linux-gnu/bitcoin-e8c25e8a35e3-powerpc64-linux-gnu.tar.gz
72c790770e49bacd4e66eb2e93f6a3a2dc596c36254bf1ec00775bbd60121c31 guix-build-e8c25e8a35e3/output/riscv64-linux-gnu/SHA256SUMS.part
c7921f8f5bfe5c463e266b24399e600c2b67552a50dbf1414cacb5805271c561 guix-build-e8c25e8a35e3/output/riscv64-linux-gnu/bitcoin-e8c25e8a35e3-riscv64-linux-gnu-debug.tar.gz
4f106246e1184a32289d2ba5251fc00af852e38351e45b3bcd974d1b099e515b guix-build-e8c25e8a35e3/output/riscv64-linux-gnu/bitcoin-e8c25e8a35e3-riscv64-linux-gnu.tar.gz
4e26090aa5ffb709088cb4e4e6aba32c0c2017a1fa776f5ec4d7be36e3ccb8aa guix-build-e8c25e8a35e3/output/x86_64-apple-darwin/SHA256SUMS.part
87282cc09b9c57416c8f92ff5ae2614c9a9d26fdcfe3c16931df0fdedae9e1b4 guix-build-e8c25e8a35e3/output/x86_64-apple-darwin/bitcoin-e8c25e8a35e3-x86_64-apple-darwin-unsigned.tar.gz
25fd762a3a7b0d81f738c3b63272188094d42139b39c6f8dd92c30c54ccc86be guix-build-e8c25e8a35e3/output/x86_64-apple-darwin/bitcoin-e8c25e8a35e3-x86_64-apple-darwin-unsigned.zip
d8c01baf156546546620d0ab17933478772527be4701b7479881a5547c527e2a guix-build-e8c25e8a35e3/output/x86_64-apple-darwin/bitcoin-e8c25e8a35e3-x86_64-apple-darwin.tar.gz
b4eafb5e1c6730105e47cd7cd6ede361a4487250c903661c0ccebce8eeda5898 guix-build-e8c25e8a35e3/output/x86_64-linux-gnu/SHA256SUMS.part
819e54887b0eb2d659dccfbfcec9a1d49e47914ce46baae3b52b2821a844098a guix-build-e8c25e8a35e3/output/x86_64-linux-gnu/bitcoin-e8c25e8a35e3-x86_64-linux-gnu-debug.tar.gz
25e48ea3a8f2d86ebea8d8ea96a902f3494197e93a65d4ff986d37506771a43a guix-build-e8c25e8a35e3/output/x86_64-linux-gnu/bitcoin-e8c25e8a35e3-x86_64-linux-gnu.tar.gz
d73e31db2488cf422704e3abf432bb430f9155b8aafd4a9a501f826785cbe1d9 guix-build-e8c25e8a35e3/output/x86_64-w64-mingw32/SHA256SUMS.part
23818afba6bbd8de1c2c2d6d6b64c05d87400d424f4c9db516b854c75b29f50a guix-build-e8c25e8a35e3/output/x86_64-w64-mingw32/bitcoin-e8c25e8a35e3-win64-debug.zip
facdda5d82b05091bb98419c7b6913e4d0019c58f4d67f95d750fc632e32c611 guix-build-e8c25e8a35e3/output/x86_64-w64-mingw32/bitcoin-e8c25e8a35e3-win64-setup-unsigned.exe
45a1252ce93fb0d8e1ebde66c16542c1bbc8b6efd6280ec7f1553415f80461fd guix-build-e8c25e8a35e3/output/x86_64-w64-mingw32/bitcoin-e8c25e8a35e3-win64-unsigned.tar.gz
8b4bc3b6cd78713e045db74eed4b3fbd711dd7e09980da386100e72dce33097e guix-build-e8c25e8a35e3/output/x86_64-w64-mingw32/bitcoin-e8c25e8a35e3-win64.zip |
Thanks, can confirm the correct one is picked up by our configure now. |
|
Tested e8c25e8: the guix build, a local depends build and a local normal build on Intel macOS 14.5. Tested a normal build on macOS 13.6.7. Guix hashes (Ubuntu, AMD), matches what @TheCharlatan and @fanquake found above. |
|
I presume macos-cross compilation on risv64 metal still fails? If it is expected to pass now, I can re-try this. |
Quite possible it'll pass now. Given the previous failure was in compiling cctools. |
|
Did a Guix build for 6377bf759602fb5dc00d7992b126ddc75cf03d71ce259c76c509299dac51be52 guix-build-e8c25e8a35e3/output/arm64-apple-darwin/SHA256SUMS.part
7191d5a512ac9b7e934e771985eb2743180a00285b3e5275de1ffdc1e504e68e guix-build-e8c25e8a35e3/output/arm64-apple-darwin/bitcoin-e8c25e8a35e3-arm64-apple-darwin-unsigned.tar.gz
73516bfd8acb17f865b53e67e421dccfd0644c6582f462d8ebe710291c2b5367 guix-build-e8c25e8a35e3/output/arm64-apple-darwin/bitcoin-e8c25e8a35e3-arm64-apple-darwin-unsigned.zip
07cc918282582193d8745cc7dcd4a4a4474e38d079b0d2831efdef59cb322449 guix-build-e8c25e8a35e3/output/arm64-apple-darwin/bitcoin-e8c25e8a35e3-arm64-apple-darwin.tar.gz
edcc1c61aa12b78c7a8e87d141832891e05f48b4a687ed7e3fae668236f339ff guix-build-e8c25e8a35e3/output/dist-archive/bitcoin-e8c25e8a35e3.tar.gz |
|
Post-merge ACK e8c25e8 |
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jun 13, 2024
Somewhat of a followup to bitcoin#21778.
luke-jr
pushed a commit
to bitcoinknots/bitcoin
that referenced
this pull request
Jun 13, 2024
I need to try and patch around Qts buildsystem to keep bitcoin#21778 moving along (the issue being that even when you tell Qt to build using Clang on Linux, it still calls out to GCC, breaking our ability to have a macOS release build env that doesn't have a GCC toolchain installed, and thus no ld binary). Before trying to patch Qt any further, update to the latest LTS release, and update the current patch set. Github-Pull: bitcoin#29732 Rebased-From: 430f319
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jun 14, 2024
Somewhat of a followup to bitcoin#21778.
fanquake
added a commit
that referenced
this pull request
Jun 18, 2024
…or `mmacos-version-min` 7c298fe doc: rewrite some of the macdeploy docs (fanquake) d042230 depends: swap mmacosx-version-min for mmacos-version-min (fanquake) Pull request description: Whilst `-mmacosx-version-min` and `-mmacos-version-min` remain aliases for each other, the later is preferred, and I assume the former will be removed at some point in the future; see: llvm/llvm-project#95374. Somewhat of a followup to #21778. Rewrite some of the mac deploy docs. ACKs for top commit: theuni: utACK 7c298fe TheCharlatan: ACK 7c298fe hebasto: ACK 7c298fe. Tree-SHA512: 6493f087fde93e0eec319af0e105d163b3f047d8a03f7d4b0d6cd7c64b58d0a978b7d67c6b8dba5c6ccf8b10e188aab5dc98eec400b0546dc9ee801a689b4332
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jul 23, 2024
Using GCC 11 for the macOS build hasn't been required since bitcoin#21778, and at this point, given a toolchain is still needed (bitcoin#30206), it makes more sense to (re-)use 12, rather than make all builders compiler another GCC toolchain.
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jul 23, 2024
Using GCC 11 for the macOS build hasn't been required since bitcoin#21778, and at this point, given a toolchain is still needed (bitcoin#30206), it makes more sense to (re-)use 12, rather than make all builders compiler another GCC toolchain.
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jul 23, 2024
Using GCC 11 for the macOS build hasn't been required since bitcoin#21778, and at this point, given a toolchain is still needed (bitcoin#30206), it makes more sense to (re-)use 12, rather than make all builders compile another GCC toolchain.
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jul 24, 2024
Using GCC 11 for the macOS build hasn't been required since bitcoin#21778, and at this point, given a toolchain is still needed (bitcoin#30206), it makes more sense to (re-)use 12, rather than make all builders compile another GCC toolchain.
fanquake
added a commit
to fanquake/bitcoin
that referenced
this pull request
Jul 25, 2024
Using GCC 11 for the macOS build hasn't been required since bitcoin#21778, and at this point, given a toolchain is still needed (bitcoin#30206), it makes more sense to (re-)use 12, rather than make all builders compile another GCC toolchain.
fanquake
added a commit
that referenced
this pull request
Jul 25, 2024
d1592d2 guix: use gcc-12 to compile winpthreads (fanquake) b23690e guix: use GCC 12.4.0 over 12.3.0 (fanquake) 8b41ede guix: consolidate back to GCC 12 toolchain for all HOSTS (fanquake) Pull request description: This PR contains 3 changes: * Bump GCC in Guix from [12.3.0 to 12.4.0](https://gcc.gnu.org/gcc-12/). A patch was sent upstream, https://lists.gnu.org/archive/html/guix-patches/2024-06/msg01025.html, but has not landed. * Consolidate all build environments back to using a GCC 12 toolchain. After #21778, the macOS environment is no-longer pinned to 11 (12 would otherwise cause issues building cctools). So, instead of requiring all builders to compile an additional GCC toolchain, use 12. * Use GCC 12 to compile winpthreads. Currently, GCC 11 is used; which became apparent in #30452 (comment). ACKs for top commit: TheCharlatan: ACK d1592d2 hebasto: ACK d1592d2. Tree-SHA512: e3aa1fa3e69500c93180e07cb4684661247ec6bc45245f746538d81406ff1d8777131590307496dda3287a112b6633e4991168586ca4c2036fa3a57b1efa9c87
davidgumberg
pushed a commit
to davidgumberg/bitcoin
that referenced
this pull request
Jul 29, 2024
Using GCC 11 for the macOS build hasn't been required since bitcoin#21778, and at this point, given a toolchain is still needed (bitcoin#30206), it makes more sense to (re-)use 12, rather than make all builders compile another GCC toolchain.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This switches us to using a LLD based toolchain for macOS builds.
Benefits