github actions improvement

[mcuee]

Updated on 2024-Sept-01.

1. Apple Silicon macOS build -- we have to wait github infrastructure to be ready first

• Add Apple M1 silicon to Nightly-Builds #1677

2. Deploy action -- to include MSYS2 mingw64/mingw32 binaries upon formal release like 7.3.
   [Done, mingw 64bit binaries require manual upload now]

3. Deploy action for Arduino-packing github action upon formal release like 7.3.
   [Done, missing libserialport support]

4. It may be good to add github actions for BSDs.

[mcuee]

Deploy action -- to include MSYS2 mingw64/mingw32 binaries upon formal release like 7.3.

Currently the deploy github action only upload Windows MSVC binaries.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/deploy.yml
https://github.com/avrdudes/avrdude/actions/runs/7813933855

The mingw32/mingw64 github action binaries will not have the correct version number, but rather one commit after.

Edit: the above deploy github action does build MSYS2 mingw32/mingw64/clang32/clang64/ucrt64 binaries for avrdude 7.3 release. So we can decide whether to upload them or not.

[mcuee]

As for the binaries generated by arduino-packing script, as of now, the script does not deal with releases yet.So the version number will not matching -- one commit after the release tag.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml

@umbynos
The avrdude-packing script in this repo is a cut-down version of your more detailed script. One missing feature is to deal with the tagged release (say 7.3 release). Just wondering if you can help here. Thanks.

[umbynos]

Hey @mcuee what do you mean by this?

One missing feature is to deal with the tagged release (say 7.3 release).

[mcuee]

@umbynos

As of now, the action is only executed upon git push to git main, but not release tagging.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml

Your original version is okay but I do not know how to adapt it to avrdude git main.
https://github.com/arduino/avrdude-packing/blob/main/.github/workflows/release.yml

[mcuee]

Related issue:

• Avrdude 7.3 arduino/avrdude-packing#31
• Add libserialport for future avrdude release arduino/crossbuild#13

[mcuee]

As of now, the action is only executed upon git push to git main, but not release tagging.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml

@umbynos
I have created a new PR to address the above issue. But I am not so sure if this is the right solution or not. Please review. Thanks.

• Adding arduino_packing_release github action #1690

[ndim]

4. It may be good to add github actions for BSDs.

GitHub only has Ubuntu, Windows and OSX workers.

However, Cirrus CI has a free offer for Open Source Projects which includes CI builds on FreeBSD.

[mcuee]

4. It may be good to add github actions for BSDs.

GitHub only has Ubuntu, Windows and OSX workers.

However, Cirrus CI has a free offer for Open Source Projects which includes CI builds on FreeBSD.

There are some changes now. Please refer to the following discussions.

• Adding github actions for NetBSD/OpenBSD and maybe other OSes libusb/libusb#1466
• Consider 3rdparty Github Actions for *bsd builds libusb/hidapi#477

[mcuee]

I have documented steps to build libserialport for the Arduino cross-build Docker container. However, macOS support is missing (difficult to cross build libserialport under the container).

@ndim Just wondering if you can take a look here. Thanks.
arduino/avrdude-packing#31

[mcuee]

Even if we cannot get macOS libserialport cross-build to work, we may still want to improve the following two github actions to install libserialport for Linux and mingw32 cross build. But I am not so sure if we can do this before avrdude 8.0 release or not.

https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing_release.yml

Worst case, I can do manually cross build, just like avrdude 7.3 release.

More about the Arduino crossbuild container.
https://github.com/arduino/crossbuild

It is missing libserialport support.

• Add libserialport for future avrdude release arduino/crossbuild#13

[mcuee]

avrdude 8.0 release github action build:
https://github.com/avrdudes/avrdude/actions/runs/10541592535 (Deploy)
https://github.com/avrdudes/avrdude/actions/runs/10541592513 (avrdude_packing_release).

The binaries are uploaded to the release page.
https://github.com/avrdudes/avrdude/releases/tag/v8.0

[mcuee]

@ndim and @umbynos

I think we may need to improve the macOS crossbuild githun action to remove the following warnings.

Example run:
https://github.com/avrdudes/avrdude/actions/runs/10649351558/job/29519411601

[ 96%] Building C object src/CMakeFiles/avrdude.dir/whereami.c.o
cd /__w/avrdude/avrdude/avrdude/build/src && /opt/osxcross/target/bin/o64-clang -DCONFIG_DIR=\"/usr/local/etc\" -I/__w/avrdude/avrdude/avrdude/build/src -I/__w/avrdude/avrdude/avrdude -I/__w/avrdude/avrdude/avrdude/build -I/__w/avrdude/avrdude/avrdude/src  -I/opt/lib/x86_64-apple-darwin13/include -pthread -framework Foundation -framework IOKit -framework Cocoa -framework Security -DHAVE_USB_H   -Wall -Wextra -Wno-unused-parameter -Wno-format-zero-length -std=gnu11 -o CMakeFiles/avrdude.dir/whereami.c.o   -c /__w/avrdude/avrdude/avrdude/src/whereami.c
clang: warning: -framework Foundation: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -framework IOKit: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -framework Cocoa: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -framework Security: 'linker' input unused [-Wunused-command-line-argument]
[ 97%] Linking C executable avrdude
cd /__w/avrdude/avrdude/avrdude/build/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/avrdude.dir/link.txt --verbose=1
/opt/osxcross/target/bin/o64-clang -I/opt/lib/x86_64-apple-darwin13/include -pthread -framework Foundation -framework IOKit -framework Cocoa -framework Security -DHAVE_USB_H  -L/opt/lib/x86_64-apple-darwin13/lib/ CMakeFiles/avrdude.dir/main.c.o CMakeFiles/avrdude.dir/developer_opts.c.o CMakeFiles/avrdude.dir/whereami.c.o  -o avrdude  libavrdude.a -lm /opt/lib/x86_64-apple-darwin13/lib/libelf.a /opt/lib/x86_64-apple-darwin13/lib/libusb.a /opt/lib/x86_64-apple-darwin13/lib/libusb-1.0.a /opt/lib/x86_64-apple-darwin13/lib/libhidapi.a /opt/lib/x86_64-apple-darwin13/lib/libftdi1.a /opt/lib/x86_64-apple-darwin13/lib/libreadline.a /opt/lib/x86_64-apple-darwin13/lib/libncurses.a 
make[2]: Leaving directory '/__w/avrdude/avrdude/avrdude/build'

[mcuee]

https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing_release.yml

The following four seem to be for linking only.

-framework Foundation -framework IOKit -framework Cocoa -framework Security 

[ndim]

That appears to be stuff you probably need for every proper Macos program. Those are unused for building a CLI program like avrdude. Not sure what we can do there except disable the warning about unused arguments.

[mcuee]

That appears to be stuff you probably need for every proper Macos program. Those are unused for building a CLI program like avrdude. Not sure what we can do there except disable the warning about unused arguments.

I believe the following is correct.

# For darwin we disable the static flags (not supported by clang) and we make some adjustments
  
cmake -DCMAKE_C_COMPILER=${{ matrix.config.cross_compiler }} -DCMAKE_CXX_COMPILER=${{ matrix.config.cross_compiler }}++ -DCMAKE_AR=${{ matrix.config.ar }} -DCMAKE_LINKER=${{ matrix.config.ld}} -DCMAKE_C_FLAGS="-I/opt/lib/${{ matrix.config.cross_compile }}/include -pthread -DHAVE_USB_H" -DCMAKE_EXE_LINKER_FLAGS="-L/opt/lib/${{ matrix.config.cross_compile }}/lib/ -framework Foundation -framework IOKit -framework Cocoa -framework Security" -DCMAKE_PREFIX_PATH=/opt/lib/${{ matrix.config.cross_compile }}/ -DHAVE_LIBFTDI="NO" -DUSE_STATIC_LIBS="ON" -B build/

However, I have not figured out how to cross build for macOS yet to verify the correctness of the above patch. For Linux/Windows I am good.

[mcuee]

1. Apple Silicon macOS build -- we have to wait github infrastructure to be ready first

Now the infrastructure is ready. We should be able to add github action for Apple Silicon build.
https://github.com/actions/runner-images?tab=readme-ov-file#available-images

Edit to add:
Hmm, we may already build using ARM64 even though we say it is macos-x86_64:
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/build.yml

  macos-x86_64:
    runs-on: macos-latest

Image	YAML Label	Included Software	Rollout Status of Latest Image Release
macOS 14	macos-latest-large or macos-14-large	macOS-14
macOS 14 Arm64	macos-latest, macos-14, macos-latest-xlarge or macos-14-xlarge	macOS-14-arm64

[ndim]

Come to think about it... windows-latest, macos-latest, ubuntu-latest: None of those say anything about the processor architecture.

Processor architecture is always implied, and may therefore change without us noticing, unless we can either specify the arch or at least test the arch to verify our assumptions still hold before we label artifacts as x86_64.