ZMQ ioloop.install crash Windows 10 Python 3.8 #77

danieljfarrell · 2020-09-03T15:51:48Z

This is specific to (in the testing I have done so far):

Windows 10
Python 3.8

a workaround it to create a Python 3.7 environment

I have a clean python 3.8 environment created using conda,

conda create --name pvtrace-test-2.1.4 python=3.8
conda activate pvtrace-test-2.1.4
conda install Rtree
pip install pvtrace

I downloaded the box.py example and ran it

python box.py

and I get the following traceback,

C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py:28: DeprecationWarning: zmq.eventloop.ioloop is deprecated in pyzmq 17. pyzmq now works with default tornado and asyncio eventloops.
  ioloop.install()
Traceback (most recent call last):
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\runpy.py", line 194, in _run_module_as_main
Traceback (most recent call last):
  File "box.py", line 8, in <module>
    return _run_code(code, main_globals, None,    vis = meshcat.Visualizer().open()

  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 108, in __init__
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\runpy.py", line 87, in _run_code
    self.window = ViewerWindow(zmq_url=zmq_url, start_server=(zmq_url is None))
      File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 43, in __init__
exec(code, run_globals)
    self.zmq_url = match_zmq_url(self.server_proc.stdout.readline().strip().decode("utf-8"))
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 235, in <module>
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 26, in match_zmq_url
    return capture(r"^zmq_url=(.*)$", line)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 21, in capture
        raise ValueError("Could not match {:s} with pattern {:s}".format(s, pattern))
main()ValueError: Could not match <class 'NotImplementedError'> with pattern ^zmq_url=(.*)$

  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 226, in main
    bridge = ZMQWebSocketBridge(zmq_url=results.zmq_url)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 95, in __init__
    (self.zmq_socket, self.zmq_stream, self.zmq_url), _ = find_available_port(f, DEFAULT_ZMQ_PORT)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 46, in find_available_port
    return func(port), port
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 94, in f
    return self.setup_zmq("{:s}://{:s}:{:d}".format(DEFAULT_ZMQ_METHOD, self.host, port))
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 202, in setup_zmq
    zmq_stream = ZMQStream(zmq_socket)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\zmq\eventloop\zmqstream.py", line 127, in __init__
    self._init_io_state()
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\zmq\eventloop\zmqstream.py", line 552, in _init_io_state
    self.io_loop.add_handler(self.socket, self._handle_events, self.io_loop.READ)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\tornado\platform\asyncio.py", line 100, in add_handler
    self.asyncio_loop.add_reader(fd, self._handle_events, fd, IOLoop.READ)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError
NotImplementedError

Using Python 3.8.5

These are the installed packages

anytree          2.8.0
attrs            20.1.0
certifi          2020.6.20
chardet          3.0.4
colorama         0.4.3
colorlog         4.2.1
decorator        4.4.2
idna             2.10
jsonschema       3.2.0
lxml             4.5.2
meshcat          0.0.18
mkl-fft          1.1.0
mkl-random       1.1.1
mkl-service      2.3.0
mpmath           1.1.0
msgpack          1.0.0
networkx         2.5
numpy            1.19.1
pandas           1.1.1
Pillow           7.2.0
pip              20.2.2
pvtrace          2.1.4
pycollada        0.7.1
pyglet           1.5.7
pyrsistent       0.16.0
python-dateutil  2.8.1
pytz             2020.1
pyzmq            19.0.2
requests         2.24.0
Rtree            0.9.4
scipy            1.5.2
setuptools       49.6.0.post20200814
Shapely          1.7.1
six              1.15.0
svg.path         4.0.2
sympy            1.6.2
tornado          6.0.4
trimesh          3.8.5
u-msgpack-python 2.7.0
urllib3          1.25.10
wheel            0.35.1
wincertstore     0.2
xxhash           2.0.0

~~Any ideas on how I can work around this problem? Maybe I need to use an earlier version of meshcat?~~

This seems to be a problem with Python 3.8 on Window 10.

The text was updated successfully, but these errors were encountered:

rdeits · 2020-09-04T02:07:21Z

I think this might be fixed in 4829916#diff-47959962e9ca72578ddb3c2f786731e8 which is in master but not in the most recent released version of meshcat-python. Let me tag a new version and see if it fixes your issue...

rdeits · 2020-09-04T02:11:03Z

Ok, can you see if meshcat 0.0.19 fixes your issue?

danieljfarrell · 2020-09-04T07:50:09Z

Thanks!

I did

pip install --upgrade meshcat

in my python 3.8 environments.

Re-running box.py gets rid of the error, but I'm not sure it's still working. I was expecting a browser window to open and show the scene?

The prompt prints this and nothing else happens,

(pvtrace-test-2.1.4) C:\Users\daniel.farrell\Documents\pvtrace>python box.py
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python
Python runtime state: preinitialized

If I Ctrl-C I get the traceback

Traceback (most recent call last):
  File "box.py", line 8, in <module>
    vis = meshcat.Visualizer().open()
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 82, in __init__
    self.window = ViewerWindow(zmq_url=zmq_url, start_server=(zmq_url is None), server_args=server_args)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 21, in __init__
    self.server_proc, self.zmq_url, self.web_url = start_zmq_server_as_subprocess(
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 69, in start_zmq_server_as_subprocess
    line = server_proc.stdout.readline().strip().decode("utf-8")
KeyboardInterrupt
^C

I'm unsure about the internals of meshcat but does this indicate that it is handing while attempting to start the ZMQ subprocess? Or is this what you would expect and it's just that the browser window did not open automatically?

rdeits · 2020-09-04T13:25:41Z

Ah, that's interesting. Can you try just running `meshcat-server --open` and see what that does? In theory this should all work on Windows, but I haven't used it on Windows myself in a long time.

…

On Fri, Sep 4, 2020 at 3:50 AM Daniel ***@***.***> wrote: Thanks! I did pip install --upgrade meshcat in my python 3.8 environments. Re-running box.py gets rid of the error, but I'm not it's still working. I was expecting a browser window to open and show the scene? The prompt prints this and nothing else happens (pvtrace-test-2.1.4) C:\Users\daniel.farrell\Documents\pvtrace>python box.py Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python Python runtime state: preinitialized If I Ctrl-C I get the traceback Traceback (most recent call last): File "box.py", line 8, in <module> vis = meshcat.Visualizer().open() File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 82, in __init__ self.window = ViewerWindow(zmq_url=zmq_url, start_server=(zmq_url is None), server_args=server_args) File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\visualizer.py", line 21, in __init__ self.server_proc, self.zmq_url, self.web_url = start_zmq_server_as_subprocess( File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 69, in start_zmq_server_as_subprocess line = server_proc.stdout.readline().strip().decode("utf-8") KeyboardInterrupt ^C I'm unsure about the internals of meshcat but does this indicate that it is handing while attempting to start the ZMQ subprocess? Or is this that you would expect and it's just that the browser windows did not open automatically? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEQQDSHQAJLL7OI4WUWMITSECL4BANCNFSM4QU7D6MA> .

danieljfarrell · 2020-09-04T15:55:39Z

Ah! Our old friend has returned

(pvtrace-test-2.1.4) C:\Users\daniel.farrell\Documents\pvtrace>meshcat-server --open
<class 'NotImplementedError'>
Traceback (most recent call last):
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\daniel.farrell\AppData\Local\Continuum\anaconda3\envs\pvtrace-test-2.1.4\Scripts\meshcat-server.exe\__main__.py", line 7, in <module>
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 370, in main
    bridge = ZMQWebSocketBridge(zmq_url=results.zmq_url,
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 170, in __init__
    (self.zmq_socket, self.zmq_stream, self.zmq_url), _ = find_available_port(f, DEFAULT_ZMQ_PORT)
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 110, in find_available_port
    return func(port, **kwargs), port
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 169, in f
    return self.setup_zmq("{:s}://{:s}:{:d}".format(DEFAULT_ZMQ_METHOD, self.host, port))
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\meshcat\servers\zmqserver.py", line 337, in setup_zmq
    zmq_stream = ZMQStream(zmq_socket)
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\zmq\eventloop\zmqstream.py", line 127, in __init__
    self._init_io_state()
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\zmq\eventloop\zmqstream.py", line 552, in _init_io_state
    self.io_loop.add_handler(self.socket, self._handle_events, self.io_loop.READ)
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\site-packages\tornado\platform\asyncio.py", line 100, in add_handler
    self.asyncio_loop.add_reader(fd, self._handle_events, fd, IOLoop.READ)
  File "c:\users\daniel.farrell\appdata\local\continuum\anaconda3\envs\pvtrace-test-2.1.4\lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError
NotImplementedError

I can confirm this is using meshcat-0.0.19 on Python 3.8 on Windows 10.

danieljfarrell · 2020-09-07T16:13:15Z

This seems to be this bug in tornado tornadoweb/tornado#2608 the workaround is in the release notes https://www.tornadoweb.org/en/stable/releases/v6.0.4.html#general-changes

Binary wheels are now available for Python 3.8 on Windows. Note that it is still necessary to use asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) for this platform/version.

So the fix seems to be detecting Windows, Python version, and manually setting the event loop policy. I can prepare a pull request at some point, probably the weekend.

dcambie · 2020-11-18T18:56:07Z

@danieljfarrell The issue is fixed by tornado 6.1.0 for me, so this workaround is not needed anymore.

FYI With current meshcat-python master (i.e. after #81 was merged) I can run pvtrace (and box.py fwiw) in a fresh Python3.8 conda environment both on work and personal PC.

danieljfarrell · 2020-11-18T21:27:09Z

So if meshcat-python sets tornado>=6.1.0 in requirements.txt we should be good!

danieljfarrell changed the title ~~ZMQ ioloop.install crash~~ ZMQ ioloop.install crash Windows 10 Python 3.8 Sep 3, 2020

danieljfarrell mentioned this issue Sep 8, 2020

Fix for ZMQ crash using Python 3.8 on Windows #78

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ZMQ ioloop.install crash Windows 10 Python 3.8 #77

ZMQ ioloop.install crash Windows 10 Python 3.8 #77

danieljfarrell commented Sep 3, 2020 •

edited

Loading

rdeits commented Sep 4, 2020

rdeits commented Sep 4, 2020

danieljfarrell commented Sep 4, 2020 •

edited

Loading

rdeits commented Sep 4, 2020 via email

danieljfarrell commented Sep 4, 2020

danieljfarrell commented Sep 7, 2020

dcambie commented Nov 18, 2020

danieljfarrell commented Nov 18, 2020

ZMQ ioloop.install crash Windows 10 Python 3.8 #77

ZMQ ioloop.install crash Windows 10 Python 3.8 #77

Comments

danieljfarrell commented Sep 3, 2020 • edited Loading

rdeits commented Sep 4, 2020

rdeits commented Sep 4, 2020

danieljfarrell commented Sep 4, 2020 • edited Loading

rdeits commented Sep 4, 2020 via email

danieljfarrell commented Sep 4, 2020

danieljfarrell commented Sep 7, 2020

dcambie commented Nov 18, 2020

danieljfarrell commented Nov 18, 2020

danieljfarrell commented Sep 3, 2020 •

edited

Loading

danieljfarrell commented Sep 4, 2020 •

edited

Loading