Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import Error when using on Google Colab #392

Open
regantan opened this issue Mar 20, 2024 · 2 comments
Open

Import Error when using on Google Colab #392

regantan opened this issue Mar 20, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@regantan
Copy link

Version
See the console output for PyABSA, Torch, Transformers Version
The transformers version is 4.29.0, the torch version is 2.1.0+cu121 and the PyABSA version is 2.4.0.

Describe the bug
A clear and concise description of what the bug is.

Import warnings of libraries causing google colab to crash

Code To Reproduce
Paste buggy code here

!pip install pyabsa[dev] -U

importing the library for any usage will cause the error

Expected behavior
A clear and concise description of what you expected to happen.

Expected to do simple inference on sentence

Screenshots
If applicable, add screenshots to help explain your problem.

:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDrive2ImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _PyDriveImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _GenerativeAIImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _OpenCVImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: APICoreClientInfoImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _BokehImportHook.find_spec() not found; falling back to find_module()
:914: ImportWarning: _AltairImportHook.find_spec() not found; falling back to find_module()
[2024-01-11 13:01:34] (2.3.4) PyABSA(2.3.4): If your code crashes on Colab, please use the GPU runtime. Then run "pip install pyabsa[dev] -U" and restart the kernel.
Or if it does not work, you can use v1.x versions, e.g., pip install pyabsa<2.0 -U

WARNING: When you fails to load a checkpoint, e.g., Unexpected key(s),
Try to downgrade transformers<=4.29.0.
@regantan regantan added the bug Something isn't working label Mar 20, 2024
@yangheng95
Copy link
Owner

This is a known issue about colab, however, the reason for why it happens is still unknown. Have you try the instruction:

[2024-01-11 13:01:34] (2.3.4) PyABSA(2.3.4): If your code crashes on Colab, please use the GPU runtime. Then run "pip install pyabsa[dev] -U" and restart the kernel.
Or if it does not work, you can use v1.x versions, e.g., pip install pyabsa<2.0 -U

@jslegers
Copy link

jslegers commented Aug 4, 2024

I got the same error when trying to run the following code from https://docs.python.org/3/library/importlib.html#importlib.util.spec_from_loader on Google Colab :

def import_module(name, package=None):
    """An approximate implementation of import."""
    absolute_name = importlib.util.resolve_name(name, package)
    try:
        return sys.modules[absolute_name]
    except KeyError:
        pass

    path = None
    if '.' in absolute_name:
        parent_name, _, child_name = absolute_name.rpartition('.')
        parent_module = import_module(parent_name)
        path = parent_module.__spec__.submodule_search_locations
    for finder in sys.meta_path:
        spec = finder.find_spec(absolute_name, path)
        if spec is not None:
            break
    else:
        msg = f'No module named {absolute_name!r}'
        raise ModuleNotFoundError(msg, name=absolute_name)
    module = importlib.util.module_from_spec(spec)
    sys.modules[absolute_name] = module
    spec.loader.exec_module(module)
    if path is not None:
        setattr(parent_module, child_name, module)
    return module

The following line triggered the issue :

spec = finder.find_spec(absolute_name, path)

I have a similar issue when I run the following code :

def load_module(module_name : str) -> ModuleType :
  try :
    if module_name in sys.modules:
      print(f"{module_name} already in sys.modules")
      return sys.modules[module_name]
    if (spec := importlib.util.find_spec(module_name)) is not None :
      module = importlib.util.module_from_spec(spec)
      spec.loader.exec_module(module)
      return module
  except Exception as e :
    print(e)
  print("Can't load module")

This works fine for pretty much ever module I've tested :

module = importlib.import_module(name)

To support certain edge cases, you could do this :

module = importlib.import_module(name)
spec = module.__spec__
spec.loader.exec_module(module)

This also works fine for me :

source_code = f"from {module_name} import {name}"
module = import_from_string(module_name+'.'+name, source_code)
sys.modules[self.__name__ + '.' + name] = module

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jslegers @yangheng95 @regantan