Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test failure: Duplicate cells due to lingering node processes #168

Open
bnavigator opened this issue Jun 13, 2023 · 2 comments
Open

Flaky test failure: Duplicate cells due to lingering node processes #168

bnavigator opened this issue Jun 13, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@bnavigator
Copy link

Description

test_ypy_yjs_0[0] randomly fails in comparing the generated code cells to the reference. This behavior is flaky, though. Sometimes there is only one cell, sometimes there are multiple duplicates:

Reproduce

wget https://github.com/jupyter-server/jupyter_ydoc/archive/refs/tags/v1.0.2.tar.gz
tar xf v1.0.2.tar.gz
pushd jupyter_ydoc-1.0.2
pushd javascript
yarn
yarn build
popd
python3 -m venv testenv
testenv/bin/pip install '.[test]'
for i in 1 2 3 4 5 6; do
  testenv/bin/python -m pytest -vv
done

Result:


=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                          

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] FAILED                                                                                                      [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                                   [100%]

========================================================================= FAILURES =========================================================================
____________________________________________________________________ test_ypy_yjs_0[0] _____________________________________________________________________

yws_server = <ypy_websocket.websocket_server.WebsocketServer object at 0x7fc166656800>
yjs_client = <Popen: returncode: None args: 'yarn node /tmp/jupyter_ydoc-1.0.2/tests/yjs_...>

    @pytest.mark.asyncio
    @pytest.mark.parametrize("yjs_client", "0", indirect=True)
    async def test_ypy_yjs_0(yws_server, yjs_client):
        ydoc = Y.YDoc()
        ynotebook = YNotebook(ydoc)
        websocket = await connect("ws://localhost:1234/my-roomname")
        WebsocketProvider(ydoc, websocket)
        nb = stringify_source(json.loads((files_dir / "nb0.ipynb").read_text()))
        ynotebook.source = nb
        ytest = YTest(ydoc, 3.0)
        await ytest.change()
>       assert ytest.source == nb
E       assert {'nbformat_minor': 5, 'metadata': {'language_info': {'codemirror_mode': {'version': 3, 'name': 'ipython'}, 'file_extension': '.py', 'mimetype': 'text/x-python', 'version': '3.10.2', 'nbconvert_exporter': 'python', 'name': 'python', 'pygments_lexer': 'ipython3'}, 'kernelspec': {'language': 'python', 'display_name': 'Python 3 (ipykernel)', 'name': 'python3'}}, 'cells': [{'metadata': {}, 'source': "print('Hello, World!')", 'cell_type': 'code', 'execution_count': None, 'outputs': [], 'id': None}, {'id': None, 'metadata': {}, 'cell_type': 'code', 'execution_count': None, 'source': "print('Hello, World!')", 'outputs': []}, {'metadata': {}, 'cell_type': 'code', 'source': "print('Hello, World!')", 'outputs': [], 'id': None, 'execution_count': None}, {'source': "print('Hello, World!')", 'metadata': {}, 'outputs': [], 'id': None, 'execution_count': None, 'cell_type': 'code'}, {'metadata': {}, 'source': "print('Hello, World!')", 'outputs': [], 'execution_count': None, 'id': None, 'cell_type': 'code'}, {'execution_count': None, 'cell_type': 'code', 'source': "print('Hello, World!')", 'outputs': [], 'id': None, 'metadata': {}}], 'nbformat': 4} == {'cells': [{'cell_type': 'code', 'source': "print('Hello, World!')", 'metadata': {}, 'outputs': [], 'execution_count': None, 'id': None}, {'cell_type': 'code', 'source': "print('Hello, World!')", 'metadata': {}, 'outputs': [], 'execution_count': None, 'id': None}], 'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)', 'language': 'python', 'name': 'python3'}, 'language_info': {'codemirror_mode': {'name': 'ipython', 'version': 3}, 'file_extension': '.py', 'mimetype': 'text/x-python', 'name': 'python', 'nbconvert_exporter': 'python', 'pygments_lexer': 'ipython3', 'version': '3.10.2'}}, 'nbformat': 4, 'nbformat_minor': 5}
E         Common items:
E         {'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)',
E                                      'language': 'python',
E                                      'name': 'python3'},
E                       'language_info': {'codemirror_mode': {'name': 'ipython',
E                                                             'version': 3},
E                                         'file_extension': '.py',
E                                         'mimetype': 'text/x-python',
E                                         'name': 'python',
E                                         'nbconvert_exporter': 'python',
E                                         'pygments_lexer': 'ipython3',
E                                         'version': '3.10.2'}},
E          'nbformat': 4,
E          'nbformat_minor': 5}
E         Differing items:
E         {'cells': [{'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}, {'cell_type': 'code', 'exe...ne, 'id': None, 'metadata': {}, ...}, {'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}]} != {'cells': [{'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}, {'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}]}
E         Full diff:
E           {
E            'cells': [{'cell_type': 'code',
E         +             'execution_count': None,
E         +             'id': None,
E         +             'metadata': {},
E         +             'outputs': [],
E         +             'source': "print('Hello, World!')"},
E         +            {'cell_type': 'code',
E         +             'execution_count': None,
E         +             'id': None,
E         +             'metadata': {},
E         +             'outputs': [],
E         +             'source': "print('Hello, World!')"},
E         +            {'cell_type': 'code',
E         +             'execution_count': None,
E         +             'id': None,
E         +             'metadata': {},
E         +             'outputs': [],
E         +             'source': "print('Hello, World!')"},
E         +            {'cell_type': 'code',
E         +             'execution_count': None,
E         +             'id': None,
E         +             'metadata': {},
E         +             'outputs': [],
E         +             'source': "print('Hello, World!')"},
E         +            {'cell_type': 'code',
E                       'execution_count': None,
E                       'id': None,
E                       'metadata': {},
E                       'outputs': [],
E                       'source': "print('Hello, World!')"},
E                      {'cell_type': 'code',
E                       'execution_count': None,
E                       'id': None,
E                       'metadata': {},
E                       'outputs': [],
E                       'source': "print('Hello, World!')"}],
E            'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)',
E                                        'language': 'python',
E                                        'name': 'python3'},
E                         'language_info': {'codemirror_mode': {'name': 'ipython',
E                                                               'version': 3},
E                                           'file_extension': '.py',
E                                           'mimetype': 'text/x-python',
E                                           'name': 'python',
E                                           'nbconvert_exporter': 'python',
E                                           'pygments_lexer': 'ipython3',
E                                           'version': '3.10.2'}},
E            'nbformat': 4,
E            'nbformat_minor': 5,
E           }

tests/test_ypy_yjs.py:62: AssertionError
================================================================= short test summary info ==================================================================
FAILED tests/test_ypy_yjs.py::test_ypy_yjs_0[0] - assert {'nbformat_minor': 5, 'metadata': {'language_info': {'codemirror_mode': {'version': 3, 'name': 'ipython'}, 'file_extension': '.py', 'mimetype': ...
=============================================================== 1 failed, 1 passed in 0.36s ================================================================
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                          

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                      [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                                   [100%]

==================================================================== 2 passed in 0.12s =====================================================================
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                          

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                      [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                                   [100%]

==================================================================== 2 passed in 0.15s =====================================================================
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                          

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                      [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                                   [100%]

==================================================================== 2 passed in 0.14s =====================================================================
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                          

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                      [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                                   [100%]

==================================================================== 2 passed in 0.14s =====================================================================
Task exception was never retrieved
future: <Task finished name='Task-70' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                          

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] FAILED                                                                                                      [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                                   [100%]

========================================================================= FAILURES =========================================================================
____________________________________________________________________ test_ypy_yjs_0[0] _____________________________________________________________________

yws_server = <ypy_websocket.websocket_server.WebsocketServer object at 0x7f60ceffebf0>
yjs_client = <Popen: returncode: None args: 'yarn node /tmp/jupyter_ydoc-1.0.2/tests/yjs_...>

    @pytest.mark.asyncio
    @pytest.mark.parametrize("yjs_client", "0", indirect=True)
    async def test_ypy_yjs_0(yws_server, yjs_client):
        ydoc = Y.YDoc()
        ynotebook = YNotebook(ydoc)
        websocket = await connect("ws://localhost:1234/my-roomname")
        WebsocketProvider(ydoc, websocket)
        nb = stringify_source(json.loads((files_dir / "nb0.ipynb").read_text()))
        ynotebook.source = nb
        ytest = YTest(ydoc, 3.0)
        await ytest.change()
>       assert ytest.source == nb
E       assert {'nbformat_minor': 5, 'cells': [{'id': None, 'execution_count': None, 'source': "print('Hello, World!')", 'metadata': {}, 'cell_type': 'code', 'outputs': []}, {'outputs': [], 'id': None, 'execution_count': None, 'cell_type': 'code', 'source': "print('Hello, World!')", 'metadata': {}}, {'source': "print('Hello, World!')", 'cell_type': 'code', 'id': None, 'outputs': [], 'metadata': {}, 'execution_count': None}, {'source': "print('Hello, World!')", 'outputs': [], 'metadata': {}, 'id': None, 'cell_type': 'code', 'execution_count': None}], 'metadata': {'language_info': {'name': 'python', 'file_extension': '.py', 'version': '3.10.2', 'nbconvert_exporter': 'python', 'pygments_lexer': 'ipython3', 'mimetype': 'text/x-python', 'codemirror_mode': {'version': 3, 'name': 'ipython'}}, 'kernelspec': {'language': 'python', 'name': 'python3', 'display_name': 'Python 3 (ipykernel)'}}, 'nbformat': 4} == {'cells': [{'cell_type': 'code', 'source': "print('Hello, World!')", 'metadata': {}, 'outputs': [], 'execution_count': None, 'id': None}, {'cell_type': 'code', 'source': "print('Hello, World!')", 'metadata': {}, 'outputs': [], 'execution_count': None, 'id': None}], 'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)', 'language': 'python', 'name': 'python3'}, 'language_info': {'codemirror_mode': {'name': 'ipython', 'version': 3}, 'file_extension': '.py', 'mimetype': 'text/x-python', 'name': 'python', 'nbconvert_exporter': 'python', 'pygments_lexer': 'ipython3', 'version': '3.10.2'}}, 'nbformat': 4, 'nbformat_minor': 5}
E         Common items:
E         {'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)',
E                                      'language': 'python',
E                                      'name': 'python3'},
E                       'language_info': {'codemirror_mode': {'name': 'ipython',
E                                                             'version': 3},
E                                         'file_extension': '.py',
E                                         'mimetype': 'text/x-python',
E                                         'name': 'python',
E                                         'nbconvert_exporter': 'python',
E                                         'pygments_lexer': 'ipython3',
E                                         'version': '3.10.2'}},
E          'nbformat': 4,
E          'nbformat_minor': 5}
E         Differing items:
E         {'cells': [{'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}, {'cell_type': 'code', 'exe...ne, 'id': None, 'metadata': {}, ...}, {'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}]} != {'cells': [{'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}, {'cell_type': 'code', 'execution_count': None, 'id': None, 'metadata': {}, ...}]}
E         Full diff:
E           {
E            'cells': [{'cell_type': 'code',
E         +             'execution_count': None,
E         +             'id': None,
E         +             'metadata': {},
E         +             'outputs': [],
E         +             'source': "print('Hello, World!')"},
E         +            {'cell_type': 'code',
E         +             'execution_count': None,
E         +             'id': None,
E         +             'metadata': {},
E         +             'outputs': [],
E         +             'source': "print('Hello, World!')"},
E         +            {'cell_type': 'code',
E                       'execution_count': None,
E                       'id': None,
E                       'metadata': {},
E                       'outputs': [],
E                       'source': "print('Hello, World!')"},
E                      {'cell_type': 'code',
E                       'execution_count': None,
E                       'id': None,
E                       'metadata': {},
E                       'outputs': [],
E                       'source': "print('Hello, World!')"}],
E            'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)',
E                                        'language': 'python',
E                                        'name': 'python3'},
E                         'language_info': {'codemirror_mode': {'name': 'ipython',
E                                                               'version': 3},
E                                           'file_extension': '.py',
E                                           'mimetype': 'text/x-python',
E                                           'name': 'python',
E                                           'nbconvert_exporter': 'python',
E                                           'pygments_lexer': 'ipython3',
E                                           'version': '3.10.2'}},
E            'nbformat': 4,
E            'nbformat_minor': 5,
E           }

tests/test_ypy_yjs.py:62: AssertionError
------------------------------------------------------------------ Captured log teardown -------------------------------------------------------------------
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-128' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-116' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-120' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-124' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-115' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-119' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-123' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-127' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-114' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-118' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-122' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-126' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-117' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-121' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
ERROR    asyncio:base_events.py:1758 Task exception was never retrieved
future: <Task finished name='Task-129' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
================================================================= short test summary info ==================================================================
FAILED tests/test_ypy_yjs.py::test_ypy_yjs_0[0] - assert {'nbformat_minor': 5, 'cells': [{'id': None, 'execution_count': None, 'source': "print('Hello, World!')", 'metadata': {}, 'cell_type': 'code', '...
=============================================================== 1 failed, 1 passed in 0.16s ================================================================
Task exception was never retrieved
future: <Task finished name='Task-125' coro=<WebSocketCommonProtocol.send() done, defined at /tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py:580> exception=ConnectionClosedOK(Close(code=1001, reason=''), Close(code=1001, reason=''), False)>
Traceback (most recent call last):
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages/websockets/legacy/protocol.py", line 944, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: sent 1001 (going away); then received 1001 (going away)
[ben@skylab:/tmp/jupyter_ydoc-1.0.2]%

Full OpenSUSE Build Service log: jupyter-ydoc-obs-build.txt

Expected behavior

Consistent test suite sucess

Context

  • Operating System and version: openSUSE Tumbleweed
  • Browser and version: N/A
  • Jupyter Server version: N/A
Troubleshoot Output 
[ben@skylab:/tmp/jupyter_ydoc-1.0.2]% testenv/bin/jupyter-troubleshoot

[notice] A new release of pip is available: 23.0.1 -> 23.1.2

[notice] To update, run: python3 -m pip install --upgrade pip

$PATH:

/home/ben/bin

/home/ben/go/bin

/home/ben/.local/bin

/usr/local/bin

/usr/bin

/bin


sys.path:

/tmp/jupyter_ydoc-1.0.2/testenv/bin

/home/ben/.python-bens-sitelib

/usr/lib64/python310.zip

/usr/lib64/python3.10

/usr/lib64/python3.10/lib-dynload

/tmp/jupyter_ydoc-1.0.2/testenv/lib64/python3.10/site-packages

/tmp/jupyter_ydoc-1.0.2/testenv/lib/python3.10/site-packages


sys.executable:

/tmp/jupyter_ydoc-1.0.2/testenv/bin/python3


sys.version:

3.10.11 (main, Apr 27 2023, 21:52:36) [GCC]


platform.platform():

Linux-6.3.4-1-default-x86_64-with-glibc2.37


which -a jupyter:

/usr/bin/jupyter

/bin/jupyter


pip list:

Package        Version

-------------- -------

aiofiles       22.1.0

aiosqlite      0.19.0

cfgv           3.3.1

distlib        0.3.6

exceptiongroup 1.1.1

filelock       3.12.2

identify       2.5.24

iniconfig      2.0.0

jupyter_core   5.3.0

jupyter-ydoc   1.0.2

nodeenv        1.8.0

packaging      23.1

pip            23.0.1

platformdirs   3.5.3

pluggy         1.0.0

pre-commit     3.3.2

pytest         7.3.2

pytest-asyncio 0.21.0

PyYAML         6.0

setuptools     65.5.0

tomli          2.0.1

traitlets      5.9.0

virtualenv     20.23.0

websockets     11.0.3

y-py           0.6.0

ypy-websocket  0.8.4
@bnavigator bnavigator added the bug Something isn't working label Jun 13, 2023
@welcome
Copy link

welcome bot commented Jun 13, 2023

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
welcome
You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

@bnavigator
Copy link
Author

If I kill the node processes in between the pytest calls, the test suite passes all times:

for i in 1 2 3 4 5 6; do
  testenv/bin/python -m pytest -vv
  pkill -f yjs_client_0.js -e
done
================================================================ test session starts =================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                    

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                             [100%]

================================================================= 2 passed in 0.53s ==================================================================
node18 killed (pid 9145)
node18 killed (pid 9156)
================================================================ test session starts =================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                    

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                             [100%]

================================================================= 2 passed in 0.54s ==================================================================
node18 killed (pid 9187)
node18 killed (pid 9198)
================================================================ test session starts =================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                    

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                             [100%]

================================================================= 2 passed in 0.53s ==================================================================
node18 killed (pid 9229)
node18 killed (pid 9240)
================================================================ test session starts =================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                    

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                             [100%]

================================================================= 2 passed in 0.53s ==================================================================
node18 killed (pid 9271)
node18 killed (pid 9282)
================================================================ test session starts =================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                    

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                             [100%]

================================================================= 2 passed in 0.53s ==================================================================
node18 killed (pid 9313)
node18 killed (pid 9324)
================================================================ test session starts =================================================================
platform linux -- Python 3.10.11, pytest-7.3.2, pluggy-1.0.0 -- /tmp/jupyter_ydoc-1.0.2/testenv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/jupyter_ydoc-1.0.2
configfile: pytest.ini
plugins: asyncio-0.21.0
asyncio: mode=auto
collected 2 items                                                                                                                                    

tests/test_ypy_yjs.py::test_ypy_yjs_0[0] PASSED                                                                                                [ 50%]
tests/test_ypy_yjs.py::test_plotly_renderer PASSED                                                                                             [100%]

================================================================= 2 passed in 0.53s ==================================================================
node18 killed (pid 9355)
node18 killed (pid 9366)

@bnavigator bnavigator changed the title Flaky test failure: Duplicate cells Flaky test failure: Duplicate cells due to lingering node processes Jun 13, 2023
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Jun 14, 2023
@bmwiedemann
Update python-jupyter-ydoc to version 1.0.2 / rev 4 via SR 1092986
244cd2a 
https://build.opensuse.org/request/show/1092986
by user bnavigator + dimstar_suse
- Kill nodejs in between pytest runs
  gh#jupyter-server/jupyter_ydoc#168
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@bnavigator