Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No stdout from beforeEach/afterEach when failing test and --no-isolate --no-file-parallelism #6682

Open
5 tasks done
Janpot opened this issue Oct 10, 2024 · 9 comments · May be fixed by #6749
Open
5 tasks done

No stdout from beforeEach/afterEach when failing test and --no-isolate --no-file-parallelism #6682

Janpot opened this issue Oct 10, 2024 · 9 comments · May be fixed by #6749
Labels
p3-minor-bug An edge case that only affects very specific usage (priority)

Comments

@Janpot
Copy link

Janpot commented Oct 10, 2024 •
edited by AriPerkkio
Loading

Describe the bug

Logs from beforeEach/afterEach are missing when I run my tests with --no-isolate --no-file-parallelism and one of the suites has a failing test

Reproduction

  1. Open https://stackblitz.com/edit/vitest-dev-vitest-88szjo?file=README.md
  2. Run pnpm test run
  3. Notice twice printed 
    before
after
  4. Run pnpm test run -- --no-isolate --no-file-parallelism
  5. Notice only once printed 
    before
after

System Info

System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 18.20.3 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.2.3 - /usr/local/bin/npm
    pnpm: 8.15.6 - /usr/local/bin/pnpm
  npmPackages:
    @vitest/ui: latest => 2.1.2 
    vite: latest => 5.4.8 
    vitest: latest => 2.1.2

Used Package Manager

pnpm

Validations
@AriPerkkio
Copy link
Member

AriPerkkio commented Oct 10, 2024 •
edited
Loading

Setup file is run only once for the test runner due to --no-isolate. A single test runner is spawned due to --no-file-parallelism. The fact that there is a failing test case here doesn't affect anything - even when all tests pass, there is only a single log output.

But *each handlers should still be called 🤔

@AriPerkkio AriPerkkio added p3-minor-bug An edge case that only affects very specific usage (priority) and removed pending triage labels Oct 10, 2024
@Janpot
Copy link
Author

Janpot commented Oct 10, 2024

Worth mentioning this mode also breaks @testing-library/dom, see testing-library/dom-testing-library#1337.

@AriPerkkio
Copy link
Member

How is that related? You need isolation for tests that are manipulating the DOM.

That setup is not using *each hooks at all. The problem in this issue to me seems to be that afterEach is treated as afterAll, same for before* hooks.

@Janpot Janpot changed the title No stdout from before/afterEach when failing test and --no-isolate --no-file-parallelism No stdout from beforeEach/afterEach when failing test and --no-isolate --no-file-parallelism Oct 10, 2024
@Janpot
Copy link
Author

Janpot commented Oct 10, 2024

That setup is not using *each hooks at all.

It's (should be) called internally by @testing-library/react.

You need isolation for tests that are manipulating the DOM.

Why? Shouldn't it work if you disable file parallelism? Isn't that essentially how e.g. mocha works?

@AriPerkkio
Copy link
Member

AriPerkkio commented Oct 10, 2024 •
edited
Loading

this mode also breaks @testing-library/dom

It's (should be) called internally by @testing-library/react.

Oh right, you are using @testing-library/react, not @testing-library/dom. The issue is reported in wrong Github project.

See #1430 for the root cause there.

@Janpot
Copy link
Author

Janpot commented Oct 10, 2024

The issue is reported in wrong Github project.

They don't accept issues related to the dom query API in the @testing-library/react repo 🤷

@hi-ogawa
Copy link
Contributor

Surprisingly, the behavior is different if you have setupFiles: ['./setup.ts'] instead of setupFiles: ['./setup'] https://stackblitz.com/edit/vitest-dev-vitest-hps2bc?file=vite.config.ts

This isn't probably intended. Module cache invalidation here needs to consider resolution

vitest/packages/vitest/src/runtime/runners/test.ts

Lines 35 to 40 in 5e6de27

importFile(filepath: string, source: VitestRunnerImportSource): unknown {
if (source === 'setup') {
this.workerState.moduleCache.delete(filepath)
}
return this.__vitest_executor.executeId(filepath)
}

For the issue with testing-library's cleanup hooks #1430, that one might be already fixed by #6689

@hi-ogawa hi-ogawa linked a pull request Oct 20, 2024 that will close this issue
Draft
6 tasks
@Janpot
Copy link
Author

Janpot commented Oct 20, 2024 •
edited
Loading

btw, testing-library/dom-testing-library#1337 is not caused by the @testing-library/react cleanup hooks, I'm seeing the same behavior without react. I updated the reproduction.

@hcampos-professional
Copy link

hcampos-professional commented Dec 5, 2024 •
edited
Loading

We just ran into the same thing where afterEach runs a single time for the first test that completes, and then not anymore. A workaround for now is to add afterEach(() => cleanup()) to each file, but it's annoying.

Edit: In the end, we managed to work around it with: https://vitest.dev/api/#ontestfinished

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
p3-minor-bug An edge case that only affects very specific usage (priority)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(vitest): invalidate setupFiles with extension-less path hi-ogawa/vitest
4 participants
@Janpot @hi-ogawa @AriPerkkio @hcampos-professional