cryptography>=35.0, plus pip>=20.3 - downloads wrong cryptography anywheel package (GLIBC_2.18 error)

[mbafford]

Looks related to this issue in the cryptography project, suggesting the whole platform wheels are being downloaded.

Downgrading to cryptography===3.4.8 resolves the issue for now.

In my particular case, the code line that triggers the error is:

from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey

Stack trace from Lambda CloudWatch logs:

[ERROR] ImportError: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /var/task/cryptography/hazmat/bindings/_rust.abi3.so)
Traceback (most recent call last):
  File "/var/task/handler.py", line 657, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 251, in lambda_handler
    handler = cls()
  File "/var/task/handler.py", line 148, in __init__
    self.app_module = importlib.import_module(self.settings.APP_MODULE)
  File "/var/lang/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/blueprints/auth0.py", line 9, in <module>
    from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey
  File "/var/task/cryptography/hazmat/primitives/asymmetric/dsa.py", line 12, in <module>
    from cryptography.hazmat.primitives.asymmetric import (
  File "/var/task/cryptography/hazmat/primitives/asymmetric/utils.py", line 6, in <module>
    from cryptography.hazmat.bindings._rust import asn1

please make sure that you are running Zappa from a virtual environment and are using Python 3.6/3.7/3.8

Confirmed both. Running in a docker build step from the official python:3.7.4 dockerhub image.

Your Environment

• Zappa version used: 0.54.0
• Operating System and Python version: dockerhub image python:3.7.4

Reproducing

Use this dockerfile:

FROM python:3.7.4

WORKDIR /opt/build

RUN python -mvenv .env/
RUN .env/bin/pip install 'pip===20.3' --upgrade
RUN .env/bin/pip install zappa==0.54.0
RUN .env/bin/pip install cryptography==35.0.0
RUN .env/bin/pip install Flask==2.0.2

RUN .env/bin/pip freeze

RUN echo '{"test": {\
    "app_function": "dashboard.app", \
    "aws_region": "us-east-1", \
    "project_name": "test-cryptography-bug", \
    "runtime": "python3.7" \
}}' > zappa_settings.json

RUN echo 'from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey\n\
from cryptography import x509\n\
from cryptography.hazmat.backends import default_backend\n\
from flask import Flask\n\

app = Flask(__name__)\n\
@app.route("/", methods=["GET"])\n\
def lambda_handler(event=None, context=None):\n\
    return "test"\n\
' > dashboard.py

ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
 
RUN bash -c ". .env/bin/activate; zappa deploy --disable_progress test"
# RUN bash -c ". .env/bin/activate; zappa update --disable_progress test"
# RUN bash -c ". .env/bin/activate; zappa undeploy --yes --remove-logs --disable_progress test"

(uncomment the applicable deploy/update line)

Build using this docker command:

docker build --progress plain --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY

The deploy/update step will have this line:

#16 15.14 Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 502 response code.

and the logs cloudwatch logs will have:

[1635434131729] [ERROR] ImportError: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /var/task/cryptography/hazmat/bindings/_    from cryptography.hazmat.bindings._rust import asn1tric/utils.py", line 6, in <module>

However, if you modify the dockerfile and change pip version to 20.2:

RUN .env/bin/pip install 'pip===20.2 --upgrade

the build and deploy runs fine, and the webapp works fine.

---

Test matrix:

Python	pip	cryptography	status
python:3.7.4	pip 20.2	cryptography 35.0.0	OK
python:3.7.4	pip 20.2	cryptography 3.4.8	OK
python:3.7.4	pip 20.3	cryptography 35.0.0	GLIBC erorr
python:3.7.4	pip 20.3	cryptography 3.4.8	OK

[phil-airship]

I hit my own issue around the cryptography pacakge, and found that starting in 3.4.8, something happened to the dependency structure that keeps Zappa from properly discovering it when iterating over native dependencies. 3.4.7 gets discovered without issue

[phil-airship]

Put in a more proper fix - #1083 . Cryptography moved to a new manylinux format that old Zappa publishing was not compatible with.

[allanlewis]

Can this now be closed?

[bas-kirill]

Hi! Also faced with this problem. It helped me to fix the version for cryptography: cryptography==3.4.8

[jithintomdeloitte]

I faced this issue today. Using cryptography==3.4.8 fixed the issue. Is there a better solution as this might lead to dependency conflicts eventually

[codeAshu]

This is happening with me also. With pip 22.2.2 and cryptography==38.0.1

[github-actions]

Hi there! Unfortunately, this Issue has not seen any activity for at least 90 days. If the Issue is still relevant to the latest version of Zappa, please comment within the next 10 days if you wish to keep it open. Otherwise, it will be automatically closed.

[wickeat]

Still encountering this issue in version 42.0.5.

[ERROR] Runtime.ImportModuleError: Unable to import module 'source.lambda_function': /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /opt/python/cryptography/hazmat/bindings/_rust.abi3.so

[wickeat]

For anyone still encountering the same issue with AWS Lambda, I worked around it by upgrading the AWS Lambda Runtime to Python 3.12. This works because the Python 3.12 runtime uses the Amazon Linux 2023 OS (vs Amazon Linux 2 in Python 3.11 and below), and the Amazon Linux 2023 contains newer version of the glibc wheel.

If not possible to upgrade to Python 3.12, another solution is also detailed in this issue.