MacOS Development - July 2023 #3410
Replies: 4 comments 21 replies
-
Errr, what? On a related topic, I'm very overdue to give remote access to @lucydodo to the newly set up M1 mac mini here. Um... give me a few more days on that. It shouldn't be too hard to do, I just need to focus on that for a bit. 😄 When that's done and the M1 Mac is also building macOS binaries on every new commit then the nightly builds may not be needed any more. |
Beta Was this translation helpful? Give feedback.
-
edited - (at least, that's my perception anyways) |
Beta Was this translation helpful? Give feedback.
-
I hit a road block with the build but I've documented as much as possible. We have an issue in a CMake module to clean up, I believe. There's some assumptions that tripped me up today. I can live with what is documented, so far. |
Beta Was this translation helpful? Give feedback.
-
Qt: As far as I know, official support for Qt 5 has ended, so I've been trying play around with migrating to Qt 6, Homebrew: When using Homebrew, dependency issues can be resolved by explicitly specifying version. Additionally, the SQLite package is deployed for the build using its own tab, and CMake is set up based on that package.
Anyway, my personal opinion is that the documentation is a bit lacking for the macOS build. |
Beta Was this translation helpful? Give feedback.
-
Yeah,I'm just a little pleased with myself. Now to make the lives of others easier...
Background
Reading through some of the latest bug reports, I noticed a trend in the reporting - reluctance to build the nightly in MacOS (at least, that's my perception anyways). Having gone through the process myself (and seriously, sometimes you have to stop beating your head against the wall from the blood loss), I think I understand what is happening, why it is a problem, and, most importantly, how to work around the difficulties.
I ran into all sorts of issues in trying to set up a development environment on MacOS Monterey. I have a 2014 Mac Mini I purchased recently at a vast discount(thank you, Other World Computing (MacSales) - also with some great bits to expand/upgrade Macs). The Mac Mini works well. Setting up software pieces was its own challenge. And here we are...
It didn't help matters struggling to find an IDE that was both not antagonizing, as well as, helpful. I'm disappointed with Netbeans - the move to Apache Foundation broke the C++ environment. Despite what ANY web search tells you, after almost four years and eight releases, C++ building just DOES NOT WORK. VS Code and Eclipse are both not suitable C++ environments. They each have problems for C++ development and are, I find, frustrating to use. If you're fighting with your IDE to do something simple and/or repetitive, you may have a bad IDE - or at least one that is not suitable for you.
I'm resorting to JetBrains CLion. It's functional, has a couple of workable quirks (of which I can rationalize why they exist), and is quite usable. Catch is cost - paying for software is something I haven't had to do for decades.
The major obstacle for DB4S nightly builds, I found, is the confluence of Qt and Homebrew. They have different priorities which are biting us in the back side.
Qt
Qt is blasting away pushing out Qt6. The world would be a better place, in their belief, with their latest-greatest library.
Okay... but we're not ready. We have some dependency and platform issues to work through, yet. It was starting to feel like the bad old days where Gnome shoved gtk+3 down everyone's throats, causing massive disruption. Vast swaths of open source software was abandoned because people couldn't keep up with the constant barrage of API-breaking releases. Fortunately, Qt is not breaking things but their warp-speed advancement (they're on 6.5 branch!!!) is noted and not exactly helpful.
Qt5 is currently the long term supported (or LTS) release. It is comforting that Qt is backporting some developments to the 5.15 branch. Overall, we're not being forced to change...at the moment...for now...
Homebrew
Homebrew isn't exactly helping the cause but this is due to their operating practice.
Contrasting the situation - Debian, Fedora, and the like, work on the distribution level. These organizations ensure all the software pieces work together maintaining bleeding-edge, testing (or next release), and stable distribution branches. Work on each branch involves not just integrating some software or library but also ensuring things, overall, work together.
Homebrew doesn't work like this - it is a rolling-release package distribution (think Arch Linux or similar). This leads to instances of whack-a-mole problems or dependency-hell situations. Any problems are quickly resolved but long-term maintenance of software is not exactly a priority.
Setup
Base Development Software
First, I am going on the assumption of a fresh OS install. Start from scratch. If you have Homebrew and Qt already installed - feel free to rip these pieces out in their entirety.
Uninstalling Qt is dependent on how Qt was installed. Google-Fu your situation. There are just too many options for me to cover here.
For Homebrew - see the FAQ answer here. Homebrew utilizes an uninstall script to clean things up. It is quite robust and thorough.
Ultimately, there is the Nuclear Option - re-install MacOS. The cost-benefit analysis of hunting down some make-it-work fix versus setting up the OS for a clean environment is an exercise best left to the reader. I was able to create a install USB drive in order to start with a fresh MacOS Monterey(v12.6.7) without having to go through the painful process of updating. YMMV. Caveat Emptor.
Apple
Homebrew will demand XCode be installed. Do this now. Install both
XCode
and theXCode Command Line Tools
. Homebrew install will ensure everything is correct before continuing.Homebrew
My guiding principle WRT Homebrew was to minimize ANY/ALL dependencies - period!!!!
Initially, I only installed some binary-only casks (VLC, Gimp, Inkscape or similar) that did not have dependencies.
For development software, I pulled in the following:
brew install cmake clang-format llvm meson nano ninja qt@5 shellcheck sqlite
This pulls in some important pieces (i.e.
autoconf
,glib
,python
,readline
, etc). However, the dependencies are known and manageable. No surprises or cruft. We cannot use the shortenedqt
with homebrew as that will pull-in Qt6. Again, Homebrew is latest-greatest and not what we want at the moment. Useqt@5
to explicitly install the Qt5 branch.Homebrew Qt@5
Because of Homebrew's setup, CLion was able to pick up the location of Qt5 somehow. However, if building from the command-line, you will need to execute the following ANYTIME you start to build and need to reference Homebrew's qt@5:
The instructions are derived from Homebrew's information printed to the console after qt@5 installation.
Homebrew SQLite
Unfortunately, we have to tweak the build to direct usage of Homebrew's SQLite (and NOT version installed by XCode). And this was the magic-sauce I discovered this week.
CMake can be guided to other locations, either through the command line (prepend directive with
-D
) or via the IDE, by populating CMake variables. These changes make integration with CMake more understandable.The
CMAKE_INCLUDE_PATH
andCMAKE_LIBRARY_PATH
variables are semi-colon lists of strings containing directory locations for CMake to search when generating software. To direct CMake to use the Homebrew installed SQLite we need to give the following either on the command-line or in the IDE:Again, the
-D
is prepended to variable for command-line/IDE usage. Also, for M1 Macs the path value to SQLite WILL BE different. Use$(brew --prefix sqlite)
on the command line to retrieve the exact value. Your IDE may not be flexible in allowing to script the path. Check IDE documentation.If all goes correctly, CMake should respond with something similar to...
Building should pull in the homebrew-installed SQLite library. You can confirm with the DB4S
About Dialog
:And you have a shiny, new DB4S app to work from. I celebrated with a coffee. Your celebratory beverage-of-choice may differ. Exercise caution.
Local QScintilla
This was painful. What I had to untangle was how exactly were we pulling in QScintilla in the first place. It didn't help that the directory structure of the library has changed between 2.13 and 2.14. However, I got there...
What I realized was how QScintilla was changed previously. In the project tree, under
[project_root]/libs/
were the filesDB4S_PATCH_05
andDB4S_PATCH_06
. These files documented the changes to QScintilla both to minimize installed components but to also modify the qmake build initially to generate a static library (i.e.libqscintilla2.a
). We have since generated a customCMakeList.txt
file to govern the build for QScintilla for packaging. But what if someone whats to use local...?Download the latest copy of QScintilla from upstream. For convenience of explanation, I saved it in the same directory holding the local copy of DB4S project from GitHub. Open Finder and navigate to this directory, location of the saved download.
Neat trick is to enable Finder's Path Bar (shows up at bottom of Finder window and shows the full path to what is shown/selected. Enable this feature with Finder Menu Bar ->
View
->Show Path Bar
. Now, right-click the directory where download was saved in the path bar and select "New Terminal at Folder". A terminal window will open with the current working directory already set to the download location.
Now for some command-line-fu. FIRST ensure you have setup environment to use qt@5 by executing all the export statements shown above.
Next, we need to establish install location for QScintilla. Execute:
For the
sudo
commands, enter your Mac password. I'm pulling a page from Homebrew here. First, we do not want to pollute/usr/local
. So, we'll just avoid that location and opt instead for the safer/opt/local
directory (unless you're using MacPorts, then you're on your own here). Second, we establish permissions and ownership on the directory/opt/local
similar to how Homebrew uses permissions/ownership for it's install. We're just copying the methodology for a different location.Now, let's build QScintilla. Extract the sources (either
tar xf QScintilla_src-X.Y.Z.tar.gz
orunzip QScintilla_src-X.Y.Z.zip
depending on what archive was downloaded). Change into the extracted directory(cd QScintilla_src-X.Y.Z
).We need to modify the existing sources:
As described above, we have patch files in the source tree from previous iteration. We need the diff shown above to accommodate upstream updates. Take the changes given and save this to the file
qsci.diff
in the same path as where QScintilla archive was downloaded.We can go one of two ways,...
a) directly - type into the command-line, with the terminal CWD pointing to extracted QScintilla sources,:
b) with git - again, CWD is in extracted QScintilla. Enter:
This method has the added benefit of being able to blow away any changes/problems you encounter. You can revert with the commands:
to get back to the "Baseline", or what was downloaded.
With terminal setup and sources patched, we now build:
And confirming the output with a directory listing:
Okay...move over to building DB4S. Either with the terminal or in your IDE, enter the following:
Letting CMake do its thing gets me the following:
At this point, I would be rushing to talk about how everything built JUST FINE but ran into a snag. I know exactly how it happened and why. Safe to say, we have some clean up to do in our
FindQScintilla.cmake
module.The keen observer will note something off about the
CMAKE_INCLUDE_PATH
given above. This is the full path to where theqscintilla.h
file can be found following the build instructions.If QScintilla is not installed as part of qt@5 infrastructure AND the full path to
qscintilla.h
is not explicitly set, CMake will fail. With the correct path set, CMake is obliviously happy however, the build will fail complaining about not findingQsci/qscintilla.h
.It seems our custom
FindQScintilla.cmake
module only modifies Qt5 directories with a regex command (see[project_root]/cmake/FindQScintilla.cmake
:line 50). Our cmake module is assuming QScintilla is located in the Qt5 structure. I do not think this is just a Mac or Unix problem. We've just been really lucky to date not having found this bug till now.I got a fair distance here. Just didn't hit the endzone unfortunately. I'll come back to this as time permits and/or things change.
Beta Was this translation helpful? Give feedback.
All reactions