Run is stoped after a fail/error in any before() method

[olkaz]

Description of the bug/issue

When run get a NightwatchAssertError (or any other fail/error) in before() method the all run a stopped. No report is created.

I'm expecting that only failed test will be skipped, not an entire suite.

Steps to reproduce

Clone an example project
Run npm install
run >> npx nightwatch ./test/mobile-app-tests/ --env app.android.real --reporter=html --open

Sample test

Check an example project

Command to run

npx nightwatch ./test/mobile-app-tests/ --env app.android.real --reporter=html --open

Verbose Output

No response

Nightwatch Configuration

Check an example project

Nightwatch.js Version

3.8.0

Node Version

No response

Browser

No response

Operating System

No response

Additional Information

Example test > https://github.com/pretorian121/test-proj

Problem in a discord chat > https://discord.com/channels/618399631038218240/935468415731253258/1298987835650605206

[garg3133]

More context: https://discord.com/channels/618399631038218240/935468415731253258/1319252702563270679

[garg3133]

@olkaz Sorry for getting here late, but this issue is happening because you're using the done() call in the before() hook. Since you're already writing your tests using async/await syntax, done() is not required.

If you instead write your before hook as below, the tests will work fine.

    before(async function (app: NightwatchAPI) {
        await app.waitForElementPresent({locateStrategy: 'id', selector: 'org.wikipedia:id/fragment_onboarding_skip_button2'})
            .click('id', 'org.wikipedia:id/fragment_onboarding_skip_button2');
    });

The issue happening in your case is that the before hook gets temporarily stuck after the NightwatchAssertError because the first statement in before hook keeps on awaiting a result from the click() command but because the earlier waitForElementPresent command failed, the click command is never run. So, because the await in the first statement is never resolve, the following done() is also never called and so the before hook gets stuck.

Now, normally the hook would be stuck for 20 secs after which the asyncHookTimeout would come into play and the before hook would timeout, and so the rest of the tests would continue. But because you've set the asyncHookTimeout to 170000, the before hook remains stuck for 170 secs in your case, after which the tests continue normally.

But all in all, this issue is easily resolvable by removing the use of done in the hook.

Another alternative to resolve this is to use await with just the waitForElementPresent command and write browser.click() command as a separate statement instead of chaining them together.

[garg3133]

Closing this issue. Feel free to reopen in case this is still not resolved on your end.