Support arm64 #5320

harryzcy · 2024-02-24T23:40:16Z

Proposed changes

Updated workflows to also build arm64 image
Conditionally ignores the tools that do not support arm64

Fix #5070
Fix #5339

Readiness checklist

In order to have this pull request merged, complete the following tasks.

Pull request author tasks

I included all the needed documentation for this change.
I provided the necessary tests.
I squashed all the commits into a single commit.
I followed the Conventional Commit v1.0.0 spec.
I wrote the necessary upgrade instructions in the upgrade guide.
If this pull request is about and existing issue,
I added the Fix #ISSUE_NUMBER label to the description of the pull request.

Super-linter maintainer tasks

Label as breaking if this change breaks compatibility with the previous released version.
Label as either: automation, bug, documentation, enhancement, infrastructure.

ferrarimarco · 2024-02-27T10:19:55Z

Thanks for this PR!

@harryzcy instead of commenting things as a first step, I would start from making the ci and cd GitHub Actions workflow build the container image for the arm64 platform as well, so you can get continuous feedback.

ferrarimarco · 2024-02-29T17:19:39Z

scripts/linterVersions.sh

+ if [[ "${TARGETARCH}" == "amd64" ]]; then
+ LINTER_NAMES_ARRAY['ENV']="dotenv-linter"
+ LINTER_NAMES_ARRAY['POWERSHELL']="pwsh"
+ fi


These two linters are only available in the standard image. I'm wondering if we can start with the slim image only, instead of having to account for these cases here and in the Dockerfile.

Dockerfile

ferrarimarco · 2024-02-29T17:28:21Z

dependencies/python/build-venvs.sh

+if [[ "${TARGETARCH}" != "amd64" ]]; then
+ # Install Rust compiler (required by checkov on arm64) #
+ # remove this once https://github.com/bridgecrewio/checkov/pull/6045 is merged
+ apk add --no-cache curl
+ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+ export PATH=$PATH:$HOME/.cargo/bin
+fi


Let's see if that Checkov PR lands in, so we can skip this part.

ferrarimarco · 2024-02-29T17:28:51Z

dependencies/python/build-venvs.sh

-apk add --no-cache --virtual .python-build-deps \
- gcc \
- linux-headers \
- musl-dev \
+packages=(
+ gcc
+ linux-headers
+ musl-dev
 python3-dev
+)
+
+if [[ "${TARGETARCH}" != "amd64" ]]; then
+ # libffi-dev is required for building wheel for cffi,
+ # until https://github.com/python-cffi/cffi/issues/69 is merged
+ packages+=(libffi-dev)
+fi
+
+apk add --no-cache --virtual .python-build-deps \
+ "${packages[@]}"


Let's see if that cffi PR lands, so we can skip this change.

That PR landed, shall we remove this if?

Looking at the logs, cffi still need to built from source until a release is landed

#71 659.6 Building wheel for cffi (pyproject.toml): started

ferrarimarco · 2024-02-29T17:30:34Z

scripts/linterVersions.sh

@@ -58,7 +58,6 @@ LINTER_NAMES_ARRAY['R']="R"
 LINTER_NAMES_ARRAY['RAKU']="raku"
 LINTER_NAMES_ARRAY['RENOVATE']="renovate-config-validator"
 LINTER_NAMES_ARRAY['RUBY']="rubocop"
-LINTER_NAMES_ARRAY['SCALAFMT']="scalafmt"


This one is going to be a bit more challenging. Is scalafmt not available on arm64?

No, not available yet. i don't know anything about scala and is having a hard time contributing a multi-arch image for it

It seems that there's a scalafmt-macos executable available since a few releases? Maybe that's useful in this case?

scripts/install-glibc.sh

github-actions · 2024-04-05T00:10:55Z

This pull request has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this pull request should stay open, please remove the O: stale 🤖 label or comment on the pull request.

If you're a maintainer, you can stop the bot to mark this issue as stale in the future by adding the O: backlog 🤖 label`.

ferrarimarco · 2024-05-03T11:01:04Z

scripts/linterVersions.sh

 if [[ "${IMAGE}" == "standard" ]]; then
 LINTER_NAMES_ARRAY['ARM']="arm-ttk"
 LINTER_NAMES_ARRAY['CSHARP']="dotnet"
- LINTER_NAMES_ARRAY['ENV']="dotenv-linter"


It seems that dotenv-linter has an arm64 image? dotenv-linter/dotenv-linter#533

ferrarimarco · 2024-05-03T11:03:40Z

scripts/linterVersions.sh

 if [[ "${IMAGE}" == "standard" ]]; then
 LINTER_NAMES_ARRAY['ARM']="arm-ttk"
 LINTER_NAMES_ARRAY['CSHARP']="dotnet"
- LINTER_NAMES_ARRAY['ENV']="dotenv-linter"
- LINTER_NAMES_ARRAY['POWERSHELL']="pwsh"


arm64 request tracked in PowerShell/PowerShell-Docker#795

ferrarimarco · 2024-05-03T11:07:01Z

lib/functions/validation.sh

@@ -25,6 +25,24 @@ function ValidateBooleanConfigurationVariables() {
 ValidateBooleanVariable "YAML_ERROR_ON_WARNING" "${YAML_ERROR_ON_WARNING}"
 }

+function ValidatePlatformArchitecture() {


I think we shouldn't validate this at runtime, but rather when we build the image. Also, we wouldn't want to silently disable linters in the case that they aren't available on a given platform.

I would revert this change and the related function call in linter.sh. We likely need to think a bit more about how to handle this.

Thanks!

ferrarimarco · 2024-05-03T11:07:29Z

lib/linter.sh

@@ -108,6 +108,7 @@ declare -l YAML_ERROR_ON_WARNING
 YAML_ERROR_ON_WARNING="${YAML_ERROR_ON_WARNING:-false}"

 ValidateBooleanConfigurationVariables
+ValidatePlatformArchitecture


Can you revert this change as well? See my previous comment.

harryzcy force-pushed the support-arm64 branch 3 times, most recently from b287234 to 13d4e30 Compare February 27, 2024 03:50

harryzcy force-pushed the support-arm64 branch 18 times, most recently from 3a5edad to 2b614d7 Compare February 29, 2024 15:19

harryzcy marked this pull request as ready for review February 29, 2024 15:23

harryzcy requested review from zkoppert, Hanse00 and ferrarimarco as code owners February 29, 2024 15:23

harryzcy force-pushed the support-arm64 branch from 2b614d7 to ca63be2 Compare February 29, 2024 15:35

ferrarimarco requested changes Feb 29, 2024

View reviewed changes

harryzcy force-pushed the support-arm64 branch 2 times, most recently from 9e1c809 to afb0efc Compare March 5, 2024 19:17

jmeridth mentioned this pull request Mar 15, 2024

chore: add black linter to make lint github/stale-repos#102

Merged

4 tasks

github-actions bot added the O: stale 🤖 Stale issue/pr label Apr 5, 2024

This comment was marked as off-topic.

Sign in to view

github-actions bot removed the O: stale 🤖 Stale issue/pr label Apr 5, 2024

harryzcy force-pushed the support-arm64 branch from abeb9dc to 7501c72 Compare April 5, 2024 00:22

ferrarimarco added enhancement New feature or request O: backlog 🤖 Backlog, stale ignores this label labels Apr 5, 2024

harryzcy force-pushed the support-arm64 branch 8 times, most recently from eae232f to 07347cd Compare April 16, 2024 04:29

jmeridth mentioned this pull request Apr 16, 2024

Utilize github.com/super-linter/super-linter for local linting github/github-ospo#97

Open

harryzcy force-pushed the support-arm64 branch 2 times, most recently from 252ac4c to 860b230 Compare April 19, 2024 17:28

ferrarimarco force-pushed the support-arm64 branch from 860b230 to 1b33f1b Compare April 22, 2024 09:42

harryzcy force-pushed the support-arm64 branch 4 times, most recently from 9df4f73 to e35a567 Compare April 25, 2024 22:48

feat: support arm64

5a94700

harryzcy force-pushed the support-arm64 branch from e35a567 to 5a94700 Compare April 27, 2024 23:57

ferrarimarco reviewed May 3, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support arm64 #5320

Support arm64 #5320

harryzcy commented Feb 24, 2024 •

edited by ferrarimarco

ferrarimarco commented Feb 27, 2024

ferrarimarco Feb 29, 2024

ferrarimarco Feb 29, 2024

ferrarimarco Feb 29, 2024

ferrarimarco Apr 16, 2024

harryzcy Apr 16, 2024

ferrarimarco Feb 29, 2024

harryzcy Mar 1, 2024 •

edited

ferrarimarco May 3, 2024

github-actions bot commented Apr 5, 2024

This comment was marked as off-topic.

ferrarimarco May 3, 2024

ferrarimarco May 3, 2024

ferrarimarco May 3, 2024

ferrarimarco May 3, 2024

Support arm64 #5320

Are you sure you want to change the base?

Support arm64 #5320

Conversation

harryzcy commented Feb 24, 2024 • edited by ferrarimarco

Proposed changes

Readiness checklist

Pull request author tasks

Super-linter maintainer tasks

ferrarimarco commented Feb 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harryzcy Mar 1, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 5, 2024

This comment was marked as off-topic.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harryzcy commented Feb 24, 2024 •

edited by ferrarimarco

harryzcy Mar 1, 2024 •

edited