Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG: mistral-common==1.4.1 crashes vllm] #52

Open
ggbetz opened this issue Sep 18, 2024 · 1 comment
Open

[BUG: mistral-common==1.4.1 crashes vllm] #52

ggbetz opened this issue Sep 18, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@ggbetz
Copy link

ggbetz commented Sep 18, 2024

Python -VV

File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/tokenizer.py", line 13, in <module>
    from vllm.transformers_utils.tokenizers import (BaichuanTokenizer,
  File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/tokenizers/__init__.py", line 2, in <module>
    from vllm.transformers_utils.tokenizers.mistral import MistralTokenizer
  File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/tokenizers/mistral.py", line 9, in <module>
    from mistral_common.tokens.tokenizers.mistral import ChatCompletionRequest
  File "/usr/local/lib/python3.10/dist-packages/mistral_common/tokens/tokenizers/mistral.py", line 32, in <module>
    from mistral_common.tokens.tokenizers.multimodal import (
  File "/usr/local/lib/python3.10/dist-packages/mistral_common/tokens/tokenizers/multimodal.py", line 6, in <module>
    import cv2
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 175, in bootstrap
    if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
    py_module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 171, in <module>
    LayerId = cv2.dnn.DictValue
AttributeError: module 'cv2.dnn' has no attribute 'DictValue'


### Pip Freeze

```shell
N/A

Reproduction Steps

Installing the latest version of VLLM will install mistral-common==1.4.1; which leads to crash (see above).

Expected Behavior

VLLM should work with latest mistral-common :-)

Additional Context

No response

Suggested Solutions

As a workaround, I'm installing mistral-common==1.4.0, which works fine.

Probably the latest introduction of cv2 has caused this error. Maybe use just open-cv and not the headless package version?

@ggbetz ggbetz added the bug Something isn't working label Sep 18, 2024
@copperwiring
Copy link

copperwiring commented Sep 19, 2024

For me installing mistral-common==1.4.0 also doesnt work. It also does crash vLLM. I have python 3.8.8

pip install mistral-common==1.4.0
ERROR: Could not find a version that satisfies the requirement mistral-common==1.4.0 (from versions: none)
ERROR: No matching distribution found for mistral-common==1.4.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants