Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken tests on Windows due to EOF error and other minor bugs #543

Merged
merged 4 commits into from
Oct 16, 2023

Conversation

wookayin
Copy link
Member

@wookayin wookayin commented Oct 15, 2023

πŸŽ‰βœ… TESTS on WINDOWS ARE NOW ALL GREEN βœ…πŸŽ‰

Note: This PR consists of several commits and bugfixes, each of which is all required to fix the broken tests on windows platform for such a long time. I would like to request @justinmk that please merge this PR without squashing (either merge commit or rebasing), retaining all the individual commits.

fix: EOF error on piped stderr being closed on Windows

Most importantly, this PR should fix #505. Pipe (subprocess) based pynvim was broken since neovim 0.5.0, probably due to neovim/neovim@c86d5fa (neovim/neovim#11390): on Windows, stderr is closed for an embedded nvim.

See the commit message for more details.

fix: do not leak resources across tests so as to prevent side effects

See the commit message for more details. asyncio event loops were not closing properly during tests, and were leaking in subsequent EventLoop, Session, Nvim instances.

@wookayin
Copy link
Member Author

wookayin commented Oct 15, 2023

I dropped one commit (fix: monkey-patch _ProactorBasePipeTransport to suppress warning) from this PR. Tests are all passing anyway (but with many warnings).

The warnings on _ProactorBasePipeTransport.__del__, despite "fix: do not leak resources across tests so as to prevent side effects", means that there are still leaking and unclosed transports. Indeed, there are a few places where transports are leaking, which I already fixed in my working branches with a major refactoring on AsyncioEventLoop but didn't include in this PR because I didn't want to make this PR too big to review easily. (will follow after this is merged) -> #545

@wookayin wookayin changed the title Fix bugs and broken tests on Windows. Fix broken tests on Windows due to EOF error and other minor bugs Oct 15, 2023
@wookayin wookayin force-pushed the fix-windows branch 3 times, most recently from 9475be5 to 9e3d958 Compare October 15, 2023 12:40
Problem:
Across unit tests, custom path_hook reigstered globally by
ScriptHost globally was not being cleared up properly. This results in
leakage of internal resources and dangling access to them (such as
asyncio event loops that was already closed and no longer valid in other
test case instances).

More specifically, the asyncio EventLoop were never closed, which can
result in "Event loop is closed" errors upon garbage collection of
internal transport objects (during cleaning up pytest sessions).

Solution:
(1) Always call ScriptHost.teardown() when done using it.
(2) Make sure all internal resources (event loops, transport channels,
    etc.) are closed by closing the embedded neovim instance.
Problem: An EOF error happens when creating a subprocess Nvim instance
on Windows. All CI tests are failing, and `attach('child', ...)` cannot
be run.

Solution: Ignore pipe_connection_lost error, and do not close the
asyncio event loop. Since embedded nvim only expects to use stdin and
stdout only as a msgpack-RPC channel, it's fine to ignore broken pipes
on the stderr.

Fixes neovim#505
Problem:

```
  >       os.unlink(fname)
  E       PermissionError: [WinError 32] The process cannot access the
            file because it is being used by another process:
            'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpugb75_rw'
```
Copy link
Member

@justinmk justinmk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful, thank you!

@justinmk justinmk merged commit abc7547 into neovim:master Oct 16, 2023
19 checks passed
@wookayin wookayin deleted the fix-windows branch October 16, 2023 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OSError: EOF when trying to run example code on Windows
2 participants