Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Intermittent test_evaluate_javascript failure in Cocoa CI #2512

Open
mhsmith opened this issue Apr 18, 2024 · 3 comments
Open

Intermittent test_evaluate_javascript failure in Cocoa CI #2512

mhsmith opened this issue Apr 18, 2024 · 3 comments
Labels
bug A crash or error in behavior. macOS The issue relates to Apple macOS support.

Comments

@mhsmith
Copy link
Member

mhsmith commented Apr 18, 2024

This has happened twice in the last two days:

tests/widgets/test_webview.py::test_load_empty_url PASSED                [ 98%]
=== Messages dropped during live streaming (use `log show` to see what they were)
=== Messages dropped during live streaming (use `log show` to see what they were)
=== Messages dropped during live streaming (use `log show` to see what they were)
tests/widgets/test_webview.py::test_static_content PASSED                [ 98%]
=== Messages dropped during live streaming (use `log show` to see what they were)
tests/widgets/test_webview.py::test_evaluate_javascript FAILED           [ 99%]
=== Messages dropped during live streaming (use `log show` to see what they were)
tests/widgets/test_webview.py::test_evaluate_javascript_error_without_handler PASSED [100%]

=================================== FAILURES ===================================
___________________________ test_evaluate_javascript ___________________________
Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 155, in _multicall
    teardown[0].send(outcome)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/unraisableexception.py", line 88, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/unraisableexception.py", line 78, in unraisable_exception_runtest_hook
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
pytest.PytestUnraisableExceptionWarning: Exception ignored on calling ctypes callback function: <rubicon.objc.api.objc_method object at 0x10f0978b0>

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/rubicon/objc/api.py", line 335, in __call__
    result = self.py_method(py_self, *args)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/toga_cocoa/widgets/webview.py", line 28, in webView_didFinishNavigation_
    self.interface.on_webview_load()
AttributeError: 'NoneType' object has no attribute 'on_webview_load'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/runner.py", line 339, in from_call
    result: Optional[TResult] = func()
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/runner.py", line 260, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 159, in _multicall
    _warn_teardown_exception(hook_name, teardown[1], e)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 49, in _warn_teardown_exception
    warnings.warn(PluggyTeardownRaisedWarning(msg), stacklevel=5)
pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: unraisableexception, Hook: pytest_runtest_call
PytestUnraisableExceptionWarning: Exception ignored on calling ctypes callback function: <rubicon.objc.api.objc_method object at 0x10f0978b0>

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/rubicon/objc/api.py", line 335, in __call__
    result = self.py_method(py_self, *args)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/toga_cocoa/widgets/webview.py", line 28, in webView_didFinishNavigation_
    self.interface.on_webview_load()
AttributeError: 'NoneType' object has no attribute 'on_webview_load'

For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
@mhsmith mhsmith added bug A crash or error in behavior. macOS The issue relates to Apple macOS support. labels Apr 18, 2024
@freakboy3742
Copy link
Member

This is a weird error - it shouldn't be possible for self.interface to be None. I can only assume that it happens as some sort of garbage collection edge case - the test case is erroring out, causing the cleanup of the interface widget, but the native delegate is persisting just long enough to handle one last signal.

That suggests the actual problem is a failure of the javascript execution, but that an intermittent failure that shouldn't be occurring at all, so it's not clear how to reproduce the problem (or even instrument the problem for reporting purposes).

@mhsmith
Copy link
Member Author

mhsmith commented Apr 27, 2024

https://github.com/beeware/toga/actions/runs/8861280348 may be related, but in this case it involves the MapView rather than the WebView:

=================================== FAILURES ===================================
________________________________ test_add_pins _________________________________
Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 156, in _multicall
    teardown[0].send(outcome)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/unraisableexception.py", line 88, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/unraisableexception.py", line 78, in unraisable_exception_runtest_hook
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
pytest.PytestUnraisableExceptionWarning: Exception ignored on calling ctypes callback function: <rubicon.objc.api.objc_method object at 0x10682b580>

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/rubicon/objc/api.py", line 335, in __call__
    result = self.py_method(py_self, *args)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/toga_cocoa/widgets/mapview.py", line 35, in mapView_regionDidChangeAnimated_
    region = self.impl.native.region
AttributeError: 'NoneType' object has no attribute 'native'

@freakboy3742
Copy link
Member

Another variation on theme from this run

_______________________ ERROR at setup of test_load_url ________________________
Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 156, in _multicall
    teardown[0].send(outcome)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/unraisableexception.py", line 83, in pytest_runtest_setup
    yield from unraisable_exception_runtest_hook()
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/unraisableexception.py", line 78, in unraisable_exception_runtest_hook
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
pytest.PytestUnraisableExceptionWarning: Exception ignored on calling ctypes callback function: <rubicon.objc.api.objc_method object at 0x107cf11b0>

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/rubicon/objc/api.py", line 335, in __call__
    result = self.py_method(py_self, *args)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/toga_cocoa/widgets/webview.py", line 30, in webView_didFinishNavigation_
    if self.impl.loaded_future:
AttributeError: 'NoneType' object has no attribute 'loaded_future'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/runner.py", line 339, in from_call
    result: Optional[TResult] = func()
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/_pytest/runner.py", line 260, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 160, in _multicall
    _warn_teardown_exception(hook_name, teardown[1], e)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/pluggy/_callers.py", line 50, in _warn_teardown_exception
    warnings.warn(PluggyTeardownRaisedWarning(msg), stacklevel=5)
pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: unraisableexception, Hook: pytest_runtest_setup
PytestUnraisableExceptionWarning: Exception ignored on calling ctypes callback function: <rubicon.objc.api.objc_method object at 0x107cf11b0>

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/rubicon/objc/api.py", line 335, in __call__
    result = self.py_method(py_self, *args)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/toga_cocoa/widgets/webview.py", line 30, in webView_didFinishNavigation_
    if self.impl.loaded_future:
AttributeError: 'NoneType' object has no attribute 'loaded_future'

For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
=========================== short test summary info ============================
ERROR tests/widgets/test_webview.py::test_load_url - pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: unraisableexception, Hook: pytest_runtest_setup
PytestUnraisableExceptionWarning: Exception ignored on calling ctypes callback function: <rubicon.objc.api.objc_method object at 0x107cf11b0>

Traceback (most recent call last):
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/rubicon/objc/api.py", line 335, in __call__
    result = self.py_method(py_self, *args)
  File "/Users/runner/work/toga/toga/testbed/build/testbed/macos/app/Toga Testbed.app/Contents/Resources/app_packages/toga_cocoa/widgets/webview.py", line 30, in webView_didFinishNavigation_
    if self.impl.loaded_future:
AttributeError: 'NoneType' object has no attribute 'loaded_future'

The common theme appears to be that methods on the delegate are being invoked at a point where the widget shouldn't exist.

A workaround for these cases would be to add an if self.impl check anywhere that self.impl is being referenced... but that will result in branches that can't be reliably reached. That's not a huge issue as we can work around it with no-covers, but it's not a very satisfying response.

In the Webview case it's interesting to note that the error is manifesting when the impl is referenced - even though 2 lines earlier, there is a reference to the interface.

I have a vague hunch that the manifestation of this bug might be a side effect of beeware/rubicon-objc#256 - a weakly held object property (impl) is being disposed of, but the delegate is still firing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug A crash or error in behavior. macOS The issue relates to Apple macOS support.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@freakboy3742 @mhsmith