Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No ARM images available for development on Macbook Pro M1 CPU #326

Open
asim-blueprintprep opened this issue Jun 10, 2021 · 35 comments
Open

Comments

@asim-blueprintprep
Copy link

Preconditions

  1. Macbook Pro M1 cpu
  2. php 7.3
  3. redis 5
  4. elasticsearch 7.5 (attempted)
  5. varnish (attempted)

Steps to reproduce

  1. ./vendor/bin/ece-docker build:compose
  2. docker-compose up -d
  3. multiple issues arise

Expected result

  1. docker containers should be created, and all start with no errors

Actual result

  1. elasticsearch issue
    1. Error: could not find libjava.so Error: Could not find Java SE Runtime Environment
    2. No ARM elasticsearch image provided by magento here: https://registry.hub.docker.com/r/magento/magento-cloud-docker-elasticsearch
    3. workaround, to use elasticsearch own here, https://registry.hub.docker.com/_/elasticsearch?tab=tags&page=1&ordering=last_updated
      1. e.g image: 'elasticsearch:7.8.1'
  2. varnish issue
    1. varnish_1 | Error: varnish_1 | Running VCC-compiler failed, signal 5 varnish_1 | VCL compilation failed
    2. There are no ARM varnish images available in magento or varnishes own docker repos

Is there a known way of getting cloud docker to boot on a new Macbook Pro?

Or is there a way to disable varnish, and have the docker just go straight to the frontend node to serve the site without a FPC?

@shiftedreality
Copy link
Member

@asim-blueprintprep thank you for this request.

We've created an internal issue https://jira.corp.magento.com/browse/MCLOUD-7989 to get it fixed.
However, any help is appreciated!

@asim-blueprintprep
Copy link
Author

@shiftedreality thanks for repsonding so swiftly, do you know of a way of disabling varnish through docker compose, and configuring it so the site loads without varnish? As this is currently blocking development work

@drpayyne
Copy link
Contributor

Hi @shiftedreality, is there any ticket to track the dependencies of cloud-docker and their compatibility with the Apple M1 chip? Just so that it's visible on what's pending for us Magento developers to start using cloud-docker on our Apple M1 computers.

@shiftedreality
Copy link
Member

@asim-blueprintprep please use --no-varnish option

@shiftedreality
Copy link
Member

@drpayyne I'm aware only about this one. Is there anything else we missing?

@drpayyne
Copy link
Contributor

Not that I know of, @shiftedreality. Thanks for the update. Let me ask the community to see if anyone caught anything else.

@asim-blueprintprep
Copy link
Author

Hi @shiftedreality
That created great, and allowed me to get docker-compose up -d booting all containers, I then run bash ./mutagen.sh
And then try to run docker-compose run --rm deploy cloud-deploy

However, it errors: [2021-06-16 14:48:55] ERROR: [132] Can't get version of elasticsearch: cURL error 6: Could not resolve host: elasticsearch (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

I can't help but wonder, if this is because I've changed the elasticsearch image to not be a magento-elasticsearch image?

@asim-blueprintprep
Copy link
Author

asim-blueprintprep commented Jun 17, 2021

I believe the issue above is resolved by having elasticsearch in your hosts file, after adding that and rebuilding that works. However, because docker desktop is having to emulate so many amd64 packages, it runs out of memory

docker/for-mac#5204

When running docker-compose run --rm deploy cloud-deploy

It reaches:
[2021-06-17 08:49:41] INFO: Running setup upgrade.
[2021-06-17 08:49:58] ERROR: [126] The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade --keep-generated --ansi --no-interaction | tee -a /app/var/log/install_upgrade.log"" failed. mmap() failed: [12] Cannot allocate memory

This is with 6GB RAM set available for docker desktop, I tried increasing it to 12GB, and the same error happened (with qemu actually using 16GB RAM - taken from top command)

So far:

  1. varnish & elastic are the 2 containers that don't compile properly, even under emulation of amd64
  2. docker runs out of memory trying to emulate all of the containers as ARM instead of amd64

I am out of ideas on how to progress this any further, without there being any ARM builds available

Thanks,

@drpayyne
Copy link
Contributor

Hi @shiftedreality! Is there any update available for public view on this? :)

@WingmanImd
Copy link

@drpayyne no, I had to resort to manually editing the docker-compose.yml file after it's created then use modified versions of Dockerfiles for all images. Gonna fork it again and edit the Readme with the instructions later today

@antoinette-mkwapatira-at-hanes

@WingmanImd Can you share a little more info about your approach to getting set up with the M1 chip?

@AlexanderKopyl
Copy link

Hello guys any one resolved this issue?

@BaDos
Copy link
Contributor

BaDos commented Jan 21, 2022

Hi!
We're going to continue work on this issue soon.

@self-diagnostics
Copy link

Is there any news on this topic?

@MaximGns
Copy link

@shiftedreality any progress here?

@mustdobetter
Copy link

@BaDos I head up C3, an Adobe Commerce partner agency in the UK, and this is something I and my team need. I'm keen to know if I can be of help to move this forward - we have lots of Docker experience.

@mustdobetter
Copy link

mustdobetter commented Jul 5, 2022

I've just found a commit adding support: 50c77b0. Thank you @BaDos! 🙌

I've successfully built and run the PHP 7.4 FPM image on ARM (MacBook Pro Max), and all seems to be working well. I'll be building and testing some of the others shortly and will feed back on any issues etc.

@BaDos BaDos self-assigned this Jul 20, 2022
@ferroapp
Copy link

Hi, I wonder if there is any progress on this, we are stuck in the same situation, version 2.4..4, any help will be much appreciated

@BaDos
Copy link
Contributor

BaDos commented Sep 27, 2022

Hi @ferroapp
Now we have prepared Dockerfiles to build images for arm64/M1, but we do not have images on docker hub.

@ferroapp
Copy link

@BaDos that's nice, what version should we try?? 1.3.3?? thank you

@BaDos
Copy link
Contributor

BaDos commented Sep 28, 2022

@ferroapp yes. you can use 1.3.3

@mamarx
Copy link

mamarx commented Sep 29, 2022

really happy about the ARM64 support added in 1.3.3. thank you guys!

we updated to 1.3.3 and built a new docker setup but still AMD64 images. what do we have to update to use the ARM64? thanks in advance!

@BaDos
Copy link
Contributor

BaDos commented Sep 29, 2022

@mamarx docker hub contains only amd64 images, you need to build you own arm64 images using M1 machine or any arm64

@mamarx
Copy link

mamarx commented Sep 30, 2022

@BaDos thank you!

@ga3003
Copy link

ga3003 commented Nov 4, 2022

Is there any update on this issue? I am using the Mac Pro M2 and I am facing the same issue where I got a warning for ARM images and when the setup is complete, the URL is showing an error message of not reachable.

Please help, if someone has any hack to solve this.

@BaDos
Copy link
Contributor

BaDos commented Nov 4, 2022

Hi @ga3003
Please, see comments above.
Now you need to build arm64 images yourself.

@ga3003
Copy link

ga3003 commented Nov 16, 2022

Thanks, @BaDos.

@pawelskowronek
Copy link

Here is an example of how you can build arm64 images for Magento Cloud Docker 1.3.4 and Magento 2.4.5-p1:

  1. Clone magento cloud docker repository.
    git clone https://github.com/magento/magento-cloud-docker.git
  2. Change current working directory to magento cloud docker.
    cd magento-cloud-docker
  3. Build docker images.
    docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-php:8.1-cli-1.3.4 ./images/php/8.1-cli
    
    docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-opensearch:1.2-1.3.4 ./images/opensearch/1.2
    
    docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-nginx:1.19-1.3.4 ./images/nginx/1.19
    
    docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-varnish:6.6-1.3.4 ./images/varnish/6.6
    

@IgorVitol
Copy link

@mamarx docker hub contains only amd64 images, you need to build you own arm64 images using M1 machine or any arm64

@BaDos Hi! Is there any plans to build the cross-platform images and push those directly to Docker Hub to prevent any manual "build" actions by every dev? Same name/tag would be used on all platforms letting docker to download the proper platform image automatically.

This should be as simple as configuring your build pipeline to use proper BuildKit in dockerx, eg
docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t <username>/<image>:latest --push

Note: You can see example in varnish/redis official images, which has inbuilt cross-platform support:
image

@joeshelton-wagento
Copy link
Contributor

Any updates here? I'm able to locally build arm64 images. But, I can't utilize them without a manual modification of the docker-compose.yml file. (And changing permissions on the executable files copied to the image.) These manual modifications are not easy to figure out, and don't persist when updating the tool. So, not ideal for new Mac users.

@JonasWijne
Copy link

writing on jan 22 2024, is there any update?

@drpayyne
Copy link
Contributor

I wonder what's stopping the Adobe team from developing and publishing this? How are the 100s of internal Adobe developers working on Magento on their MacBook? Either everyone has the old chips, or they're all running these containers on emulation, or their multi-platform images are proprietary. :)

@Morgy93
Copy link
Member

Morgy93 commented Feb 28, 2024

What a great developer experience in 2024 with latest technology. Makes me want to jump on Magento Cloud right away!

Or that's what I'd like to say if this issue would not exist. Any news?

@BaDos BaDos removed their assignment Mar 2, 2024
@github-project-automation github-project-automation bot moved this to Ready for Grooming in Backlog Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Ready for Grooming
Development

No branches or pull requests