Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix intermittents tests for macos desktop #76

Open
jonasfj opened this issue Jan 10, 2024 · 5 comments
Open

Fix intermittents tests for macos desktop #76

jonasfj opened this issue Jan 10, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@jonasfj
Copy link
Member

jonasfj commented Jan 10, 2024

.github/workflows/test.yml contains:

      # TODO: Enable macos desktop when supported
      #- run: flutter test integration_test/webcrypto_test.dart -d macos
      #  working-directory: ./example

We really should enable integration tests on macos desktop. If anyone is interested in figuring how to get this working on Github Actions that would be great.

Contributions are highly appreciated, I tried in #74, but there is probably a tiny thing not working.

@jonasfj jonasfj added the bug Something isn't working label Jan 10, 2024
@CoreNion
Copy link
Contributor

On the actual machine, the test appears to get stuck on the step 'test app can compute a hash' if the example app is not the active window.
The test passes when the app is open, which is why I did not notice the issue.
This part runs testWidgets(), but it is likely not being handled by the app GUI on GitHub Action and is causing the test to hang?

@jsiedentop
Copy link
Contributor

I can confirm that the test app can compute a hash hangs on my mac if the example application is in the background during the test (Mac M1, macOS 14.1 23B74 darwin-arm64). However I was able to run the test.yml Action successfully on GitHub, without any relevant change:

image

@jonasfj
Copy link
Member Author

jonasfj commented Jan 11, 2024

Merged #77 and closing this for now.

If someone is interested in trying to figure out if this is an flaky, and/or what we could do to make it work better, please go ahead.

Contributions are welcome, please ping me @jonasfj for reviews (I'm sometimes slow to notice PRs).

@jonasfj jonasfj closed this as completed Jan 11, 2024
@jsiedentop
Copy link
Contributor

At least it seems, that the library itself seems to work like expected.

If the cause of the flaky test is really, that the application looses focus, then I would investigate:

  • if the test should work without focus?
  • why the app looses focus at all?
  • is there a way to rewrite the test to make it more stable.

I would argue, that it’s a separate topic. I keep it in mind and maybe will come back to this, if it turns out to be a big problem. However, sporadic errors that occur in the pipeline could be quite difficult to isolate.

@jonasfj jonasfj reopened this Jan 11, 2024
@jonasfj jonasfj changed the title Enable tests for macos desktop Fix intermittents tests for macos desktop Jan 11, 2024
@jonasfj
Copy link
Member Author

jonasfj commented Jan 11, 2024

I just landed #78 and will publish.

But I had to manually rerun the tests to get it to pass.

  • First time it timed out after 12 minutes.
  • Second time it finished in 2 minutes.

Ideas for what we can do to make flutter test integration_test/webcrypto_test.dart -d macos reliable are welcome.

The flutter part of the example app we have here are really dumb and simple. Nothing fancy, so this could be a bug in Flutter somewhere. Or it could be a bug in flutter integration test logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants