MBGC on OS X #3
Comments
|
Hello. Yes, I have managed to run MBGC on OS X with ARM architecture. Still, I do not have a universal cmake receipt for linking openmp library. ARM version required also small modifications in the code. I can share here a working version of a mbgc project adapted to OS X, maybe it will work after small adjustments. I would appreciate help with this. |
|
Thanks for the prompt answer! Is there any chance it could be added to OS X Bioconda? This is the way how we integrate all the software into our pipelines. If there're any difficulties, in my experience the community around is always extremely helpful with guiding/helping with the creation/improvement of recipes. |
|
Adding mbgc to OS X bioconda should be doable. I am not an OS X user or developer so it poses some extra difficulty. I will surely keep this in mind, however, my capabilities in this area are unfortunately limited. |
|
@karel-brinda, could you please try running the binary: |
|
Thanks for the work @kowallus, going to try this now. Once MBGC starts working on OS X, it will be a ready-to-use software instead of just an experimental implementation. This will very strongly increase the impact of your work. |
|
Tried it, I'm getting the following error: immortalite:mbgc-2.0_x64-macos karel$ brew install libomp
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/cask-fonts, homebrew/core and homebrew/cask).
==> New Formulae
cidr ocaml@4 solo2-cli
==> New Casks
amie hapigo vimcal
brightintosh navigraph-charts wiso-steuer-2024
cardo-update navigraph-simlink
font-rubik-glitch-pop senabluetoothdevicemanager
You have 4 outdated formulae installed.
Warning: libomp 17.0.6 is already installed and up-to-date.
To reinstall 17.0.6, run:
brew reinstall libomp
immortalite:mbgc-2.0_x64-macos karel$ ./mbgc
dyld[72409]: Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
Referenced from: <81308242-71CA-37EB-B28E-42910E87869B> /private/tmp/~20231204105750/mbgc-2.0_x64-macos/mbgc
Reason: tried: '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/opt/libomp/lib/libomp.dylib' (no such file), '/usr/local/lib/libomp.dylib' (no such file), '/usr/lib/libomp.dylib' (no such file, not in dyld cache)
Abort trap: 6(including the output of Brew so that you know the version installed) |
|
Thank you. Unfortunately, it seems more difficult than it should be. |
|
What I think is needed:
|
|
See eg how it's done for COBS: https://github.com/bioconda/bioconda-recipes/blob/master/recipes/cobs/meta.yaml, which we also fixed for the use in combination with phylogenetic compression. One of the issues we were dealing with, was exactly the same one. @leoisl might be able to tell more. |
|
@karel-brinda there seems to be a temporary solution, could you check it (it works for me)? |
Chip: Apple M1 Pro |
|
@karel-brinda I hope that this build will work for you (NOTE: it should without using export command) : I have a question. You have bioconda on M1. Does Bioconda deliver binaries also for ARM? The azure pipeline (used to verify packages) seems to rely on intel architecture and I am not sure if M1 users can take advantage of Bioconda in such cases. |
|
Bioconda does not deliver binaries for Aarch64: bioconda/bioconda-recipes#23454 , bioconda/bioconda-docs#16 . It might happen soon, but not yet available |
|
Not completely straightforward (I had to manually open the two files first to confirm they can be trusted), but the binary is then working. Congrats!! I guess you might somehow needed to sign it, but I have no idea how this is usually done on OS X. Also, I've tried to compile it by myself from the current git repo, but got the following error: |
|
@karel-brinda thank you for help. It seems that I will be able to deploy macos build soon to conda. It passed the checks, but I am making some updates. I don't think that I will be able to provide 100% generic cmake (especially for macos), but I have made some corrections and would be grateful if you could try it. I will not try to "sign" my binaries. If anyone could do it easily, it wouldn't make sense ;). |
|
It's advancing: |
|
Another patch (UPDATED): I think also that you may try activating your conda environment. cmake doesn't find ENV{CONDA_PREFIX} environment variable, but it seems you have it installed at ~/miniconda. |
|
This is my output with the current version on git: |
|
Thank you sincerely @karel-brinda. The detected (default?) compiler is not clang. It might be a problem. However, it is above my pay grade :). I will focus on providing mbgc through bioconda and preparing working (yet unsigned) binaries. |
|
This is what happens when I ask for using GCC from Homebrew: |
|
One hint about conda paths – I need to specify the path manually as an environmental variable for this to work. In my case: export CONDA_PREFIX=~/miniconda |
|
I think that OPENMP 5.0 is required and this: is probably a problem with cmake. In such case, it might be better replacing: with or more generally: however, no guarantees. |
I thought so. In my case, I have this already set after installing miniconda3. Is it enough to build mbgc? |
|
Which packages are need from Conda? |
Have you used conda-forge channel? |
|
Not working completely yet, but this helped!! Indeed these packages were missing. Now I'm getting the following error: $ cmake .
-- The C compiler identification is AppleClang 15.0.0.15000040
-- The CXX compiler identification is AppleClang 15.0.0.15000040
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Processing Apple build
Found LIBOMP: /Users/karel/miniconda/lib
compilers: /Users/karel/miniconda/bin/clang;/Users/karel/miniconda/bin/clang++
/Users/karel/miniconda/bin/clang;/Users/karel/miniconda/bin;/Users/karel/miniconda
-- Found OpenMP_C: -fopenmp -Wno-unused-command-line-argument
-- Found OpenMP_CXX: -fopenmp -Wno-unused-command-line-argument
-- Found OpenMP: TRUE
-- Configuring done (17.6s)
-- Generating done (0.5s)
-- Build files have been written to: /Users/karel/github/mbgc
immortalite:mbgc karel$ make -j
[ 2%] Building CXX object CMakeFiles/mbgc.dir/main.cpp.o
[ 5%] Building CXX object CMakeFiles/mbgc.dir/utils/helper.cpp.o
[ 8%] Building C object CMakeFiles/mbgc.dir/utils/hashes/xxhash.c.o
[ 10%] Building CXX object CMakeFiles/mbgc.dir/utils/input_with_libdeflate_wrapper.cpp.o
[ 16%] Building CXX object CMakeFiles/mbgc.dir/coders/ContextAwareMismatchesCoder.cpp.o
[ 16%] Building C object CMakeFiles/mbgc.dir/coders/lzma/LzFindMt.c.o
[ 18%] Building CXX object CMakeFiles/mbgc.dir/utils/hashes/city.cpp.o
[ 21%] Building CXX object CMakeFiles/mbgc.dir/matching/copmem/CopMEMMatcher.cpp.o
[ 27%] Building C object CMakeFiles/mbgc.dir/coders/lzma/LzmaEnc.c.o
[ 27%] Building C object CMakeFiles/mbgc.dir/coders/lzma/LzFind.c.o
[ 29%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/arm/cpu_features.c.o
[ 32%] Building CXX object CMakeFiles/mbgc.dir/utils/hashes/metrohash64.cpp.o
[ 35%] Building C object CMakeFiles/mbgc.dir/coders/lzma/Threads.c.o
[ 37%] Building C object CMakeFiles/mbgc.dir/coders/lzma/LzmaDec.c.o
[ 40%] Building C object CMakeFiles/mbgc.dir/coders/lzma/Ppmd7Enc.c.o
[ 43%] Building CXX object CMakeFiles/mbgc.dir/coders/VarLenDNACoder.cpp.o
[ 45%] Building CXX object CMakeFiles/mbgc.dir/matching/SimpleSequenceMatcher.cpp.o
[ 48%] Building C object CMakeFiles/mbgc.dir/coders/lzma/Ppmd7.c.o
[ 51%] Building CXX object CMakeFiles/mbgc.dir/coders/PropsLibrary.cpp.o
[ 54%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/utils.c.o
[ 56%] Building CXX object CMakeFiles/mbgc.dir/mbgccoder/MBGC_Decoder.cpp.o
[ 59%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/gzip_compress.c.o
[ 62%] Building C object CMakeFiles/mbgc.dir/coders/lzma/Ppmd7Dec.c.o
[ 64%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/x86/cpu_features.c.o
[ 67%] Building C object CMakeFiles/mbgc.dir/coders/lzma/Alloc.c.o
[ 70%] Building CXX object CMakeFiles/mbgc.dir/coders/LzmaCoder.cpp.o
[ 72%] Building CXX object CMakeFiles/mbgc.dir/coders/CodersLib.cpp.o
[ 75%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/deflate_compress.c.o
[ 78%] Building CXX object CMakeFiles/mbgc.dir/coders/SymbolsPackingFacility.cpp.o
[ 81%] Building CXX object CMakeFiles/mbgc.dir/matching/TextMatchers.cpp.o
[ 86%] Building CXX object CMakeFiles/mbgc.dir/coders/PpmdCoder.cpp.o
[ 86%] Building CXX object CMakeFiles/mbgc.dir/matching/SlidingWindowSparseEMMatcher.cpp.o
[ 91%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/gzip_decompress.c.o
[ 91%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/deflate_decompress.c.o
[ 94%] Building CXX object CMakeFiles/mbgc.dir/mbgccoder/MBGC_Encoder.cpp.o
[ 97%] Building C object CMakeFiles/mbgc.dir/coders/libdeflate/lib/crc32.c.o
In file included from /Users/karel/github/mbgc/mbgccoder/MBGC_Decoder.cpp:1:
In file included from /Users/karel/github/mbgc/mbgccoder/MBGC_Decoder.h:7:
/Users/karel/github/mbgc/mbgccoder/MBGC_Params.h:6:10: fatal error: 'omp.h' file not found
6 | #include <omp.h>
| ^~~~~~~
In file included from /Users/karel/github/mbgc/utils/input_with_libdeflate_wrapper.cpp:2:
/Users/karel/github/mbgc/utils/../mbgccoder/MBGC_Params.h:6:10: fatal error: 'omp.h' file not found
6 | #include <omp.h>
| ^~~~~~~
In file included from /Users/karel/github/mbgc/main.cpp:4:
In file included from /Users/karel/github/mbgc/mbgccoder/MBGC_Encoder.h:8:
/Users/karel/github/mbgc/mbgccoder/MBGC_Params.h:6:10: fatal error: 'omp.h' file not found
6 | #include <omp.h>
| ^~~~~~~
In file included from /Users/karel/github/mbgc/matching/SimpleSequenceMatcher.cpp:5:
/Users/karel/github/mbgc/matching/../mbgccoder/MBGC_Params.h:6:10: fatal error: 'omp.h' file not found
6 | #include <omp.h>
| ^~~~~~~
/Users/karel/github/mbgc/matching/copmem/CopMEMMatcher.cpp:45:10: fatal error: 'omp.h' file not found
45 | #include <omp.h>
| ^~~~~~~
/Users/karel/github/mbgc/matching/SlidingWindowSparseEMMatcher.cpp:47:10: fatal error: 'omp.h' file not found
47 | #include <omp.h>
| ^~~~~~~
/Users/karel/github/mbgc/coders/CodersLib.cpp:6:10: fatal error: 'omp.h' file not found
6 | #include <omp.h>
| ^~~~~~~
In file included from /Users/karel/github/mbgc/mbgccoder/MBGC_Encoder.cpp:1:
In file included from /Users/karel/github/mbgc/mbgccoder/MBGC_Encoder.h:8:
/Users/karel/github/mbgc/mbgccoder/MBGC_Params.h:6:10: fatal error: 'omp.h' file not found
6 | #include <omp.h>
| ^~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/utils/input_with_libdeflate_wrapper.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/main.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/matching/SimpleSequenceMatcher.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/mbgccoder/MBGC_Decoder.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/matching/copmem/CopMEMMatcher.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/matching/SlidingWindowSparseEMMatcher.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/coders/CodersLib.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/mbgc.dir/mbgccoder/MBGC_Encoder.cpp.o] Error 1
make[1]: *** [CMakeFiles/mbgc.dir/all] Error 2
make: *** [all] Error 2 |
|
There is omp.h in my includes: If it is not there it might not work. |
|
In my case, it's there too: $ find ~/miniconda -name 'omp.h'
/Users/karel/miniconda/include/omp.h
/Users/karel/miniconda/pkgs/llvm-openmp-17.0.6-hb6ac08f_0/include/omp.h
/Users/karel/miniconda/pkgs/llvm-openmp-17.0.6-hb6ac08f_0/lib/clang/17.0.6/include/omp.h
/Users/karel/miniconda/pkgs/llvm-openmp-8.0.1-h770b8ee_0/include/omp.h
/Users/karel/miniconda/pkgs/llvm-openmp-8.0.1-h770b8ee_0/lib/clang/8.0.1/include/omp.h
/Users/karel/miniconda/lib/clang/17.0.6/include/omp.h |
|
I guess the include dir is just not added to the path by CMake |
Please, check if making it explicit will help by adding include_directories(${YOUR_DIRECTORY}) in cmake, e.g. after: add |
|
Now compilation finished and it only doesn't link: |
|
This seems to be an issue related to the architecture. As @leoisl wrote, Bioconda doesn't support arm yet, so everything still goes through x86... |
|
What about using clang from brew? |
But you are not using Bioconda channel. Could you verify version of your default conda compiler. |
|
Even though I'm not using the Bioconda channel here, to make my Conda installation with Bioconda, I need to fix the architecture to x86 (already at the stage of installing miniconda). In principle, it's possible within Conda to combine packages from different architectures (by switching parameters for individual environments), but it's a nightmare so prefer simplicity (i.e, x86 everywhere). On the other hand, I might be able to create an arm env with llm just for this and adjust conda pref. I'll try it. |
|
Before I forget/lose it, I'm putting the command here: And then |
|
And it works!!!!!!!!!!!!!!!! |
|
So I think if you combine everything:
it will work for:
|
|
Ok. Thanks @karel-brinda @leoisl ! |
|
I confirm that it's installable. One little issue I've noticed: it installs also I believe these two should be dependencies only for the build, but not for final bioconda package. |
|
Fixed by 46d8c6b. Moving the rest to a separate issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello, is it currently possible to run MBGC on OS X? I tested it some time ago when it clearly wasn't possible to compile the code, but now I see there's a new release.
While the method works really nicely on Linux, the lack of support for OS X has prevented us from embedding it eg directly into pipelines for phylogenetic compression, especially to the downstream of https://github.com/karel-brinda/mof-compress.
The text was updated successfully, but these errors were encountered: