Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataflow fails launching on M1: FAILED_TO_LOAD_NATIVE_LIBRARY #3640

Open
briandealwis opened this issue Apr 9, 2021 · 4 comments
Open

Dataflow fails launching on M1: FAILED_TO_LOAD_NATIVE_LIBRARY #3640

briandealwis opened this issue Apr 9, 2021 · 4 comments

Comments

@briandealwis
Copy link
Member

  • Cloud Tools for Eclipse version: 1.8.4
  • Google Cloud SDK version: 334.0.0
  • Eclipse version: 2021‑03
  • OS: macOS 11.2.3 for Apple M1
  • Java version: 15.0.2

What did you do?

  1. Created new Dataflow app
  2. Tried to launch locally.

Execution failed:

Exception in thread "main" org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
	at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:342)
	at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:176)
	at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:154)
	at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
	at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:97)
	at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:89)
	at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
	at org.apache.beam.sdk.util.SerializableUtils.serializeToByteArray(SerializableUtils.java:52)
	at org.apache.beam.sdk.util.SerializableUtils.clone(SerializableUtils.java:107)
	at org.apache.beam.sdk.util.SerializableUtils.ensureSerializable(SerializableUtils.java:86)
	at org.apache.beam.sdk.io.Read$Bounded.<init>(Read.java:93)
	at org.apache.beam.sdk.io.Read$Bounded.<init>(Read.java:88)
	at org.apache.beam.sdk.io.Read.from(Read.java:47)
	at org.apache.beam.sdk.transforms.Create$Values.expand(Create.java:328)
	at org.apache.beam.sdk.transforms.Create$Values.expand(Create.java:273)
	at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
	at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:472)
	at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
	at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:173)
	at com.example.dflow.StarterPipeline.main(StarterPipeline.java:53)
@briandealwis
Copy link
Member Author

briandealwis commented Apr 9, 2021

@suztomo notes that Beam 2.29 should have an updated Snappy version with an mac/aarch64 support.

@meltsufin
Copy link
Member

@briandealwis is this still an issue?

@meltsufin meltsufin added the p3 label Aug 12, 2022
@briandealwis
Copy link
Member Author

It looks like Apache Beam 2.36.0 included support for Mac M1 (https://issues.apache.org/jira/browse/BEAM-11703). I have not tested.

@elefeint
Copy link
Contributor

We'll have to retest this scenario.

@lqiu96 lqiu96 added p4 and removed p3 labels Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants