unhandledRejection when waitUntil condition times out in custom command, crashes Nightwatch

[reallymello]

Description of the bug/issue

When I use waitUntil in an async custom command to wait for an event to occur and that event takes longer than the provided timeout I expect the test to fail gracefully, but instead an unhandledRejection occurs due to the reject property being null causing Nightwatch to exit before ending the session normally (leaving the browser session open).

Steps to reproduce

In general, to reproduce, I have a custom command that uses .waitUntil to wait for a loading panel to complete between postback operations. Sometimes the wait takes longer than the max timeout provided to the waitUntil command which results in an error such as

Error
   unhandledRejection: Cannot read properties of null (reading 'reject')
TypeError: Cannot read properties of null (reading 'reject')
    at C:\Users\dm\projects\example\node_modules\nightwatch\lib\api\_loaders\_base-loader.js:467:35

When this occurs the after/afterEach test hooks and the global session quit never runs. This causes severe issues in our BrowserStack CI runs because it gets interpreted as a timeout and we don't get results for our test run.

Sample test

// The test is very long, but here is a snippet

await desktopParticipantPage
      .click('@coverageMatchLink') // This action triggers a loading screen to appear to "lock" the page
      .waitForInProgressOverlay(
        500, // Set very short to ensure we timeout to reproduce the issue
        'clicked coverage match link in desktop participant page'
      )

// ---- Below is the custom command WaitForInProgressOverlay.ts ----

import { NightwatchClient } from 'nightwatch';

export default class WaitForInProgressOverlay {

async command(
    this: NightwatchClient,
    maxWaitInMs: number,
    consoleNote?: string
  ) {
    if (consoleNote && consoleNote.length > 0) {
      console.info('  🕒 ' + consoleNote);
    }

    let doneLoading = false;

    const startTime = performance.now();

    await this.api.waitUntil(
      () => {
        browser.executeScript(
          () => {
            const loadingPanel = (window as any)?.loadingPanel;
            const pageIsLocked = (window as any)?.pageIsLocked;

            return {
              doneLoading: !pageIsLocked && !loadingPanel,
              pageIsLocked: pageIsLocked,
              loadingPanel: loadingPanel,
            };
          },
          [],
          (result) => {
            doneLoading = (result?.value as any)?.doneLoading;
          }
        );

        if (doneLoading) {
          console.info(
            `  🕒 Waited ${Math.round(
              performance.now() - startTime
            )} milliseconds for the loading overlay to disappear`
          );
        }
        return doneLoading;
      },
      maxWaitInMs,
      100
    );

    const result = this.api.window.getSize(); // This is here to work around a different suspected issue in waitUntil returning the wrong type sometimes

    return result;
  }
}

Command to run

No response

Verbose Output

This isn't the verbose but this is what is reported. Verbose is too long given the size of the test.

  🕒 clicked coverage match link in desktop participant page
    Error   Error while running .wait() protocol action: Wait timed out after 543ms

  TimeoutError
   Wait timed out after 543ms
An error while waiting for the page locked status to return. Did you provide a long enough timeout? Error
    at CommandInstance.command (C:\Users\dm\projects\nightwatch-page-objects\nightwatch\commands\waitForInProgressOverlay.ts:28:22)
    at C:\Users\dm\projects\example\node_modules\nightwatch\lib\api\_loaders\command.js:182:29
    at processTicksAndRejections (node:internal/process/task_queues:95:5) {
  name: 'TimeoutError',
  remoteStacktrace: '',
  abortOnFailure: true,
  namespace: undefined
}

  TimeoutError
   Error while running "waitUntil" command: [TimeoutError] Wait timed out after 543ms
    Error location:
    C:\Users\dm\projects\nightwatch-page-objects\nightwatch\commands\waitForInProgressOverlay.ts:28
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

     26 |
     27 |     try {
     28 |       await this.api.waitUntil(
     29 |         () => {
     30 |           browser.executeScript(

    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

 

  Error

   unhandledRejection: Cannot read properties of null (reading 'reject')
TypeError: Cannot read properties of null (reading 'reject')
    at C:\Users\dm\projects\example\node_modules\nightwatch\lib\api\_loaders\_base-loader.js:467:35

Nightwatch Configuration

No response

Nightwatch.js Version

3.7

Node Version

20.11

Browser

Chrome 128

Operating System

Windows 11 / Linux

Additional Information

Appears to manifest as a Timeout error when this occurs in BrowserStack TurboScale without any trace information

[reallymello]

Some extra context I discovered this morning. It seems that the issue occurs when the waitForInProgressOverlay command waitUntil times out and I have a downstream click command with nested .findByText that isn't awaited.

// Broken example
await desktopParticipantPage
      .click('@coverageMatchLink') // This action triggers a loading screen to appear to "lock" the page
      .waitForInProgressOverlay(
        500, // Set very short to ensure we timeout to reproduce the issue
        'clicked coverage match link in desktop participant page'
      )
      .click(browser.element.findByText('Ok'))
      .assert.textContains('@coverageMatch', 'Covered')

// Working example
await desktopParticipantPage
      .click('@coverageMatchLink') // This action triggers a loading screen to appear to "lock" the page
      .waitForInProgressOverlay(
        500, // Set very short to ensure we timeout to reproduce the issue
        'clicked coverage match link in desktop participant page'
      )
      .click(await browser.element.findByText('Ok'))
      .assert.textContains('@coverageMatch', 'Covered')

[garg3133]

I tried to reproduce this but I couldn't. Then I realized you are using the custom command on a page object, which clears one thing that this issue is only present for the page objects (although I'm yet to reproduce it, which I'll try in the morning) and the commands work as intended when called on the browser object.

[garg3133]

Tried this on page object as well but still unable to reproduce the unhandledRejection error.

Would it be possible for you to provide us with a minimal reproducible project?

[reallymello]

I'll work on it

[AutomatedTester]

@reallymello have you had chance to get a small reproducible test case for us here?

[reallymello]

While its not a end-to-end runnable test case here is an example of a test that failed an assertion and left the browser open and exited early. In red is the before where it was exhibiting the issue and the green was the refactor that allowed the test to fail and teardown normally/close out the browser + session in the post test hook.

The underlined line, where the policy holder name is asserted, there was a failure because the element text did not match the test expectation. In the original version these assertions were chained with a downstream click and custom async wait command. When I separated the assertions from those events it failed as expected.

While not a full test case I'm hoping this maybe narrows things down.

[AutomatedTester]

Unfortunately, it doesn't narrow it down...

We're still struggling to recreate the issue

[garg3133]

@reallymello I tried nearly everything but could not reproduce the issue locally. That said, it would be really helpful if you could provide us with the verbose logs for the sessions where you face this issue, that would give us a good idea what Nightwatch is doing under-the-hood and where things went wrong.

Also, you're using the normal Nightwatch test runner and not the Nightwatch Programmatic APIs, right?

[garg3133]

Although we are still not able to reproduce this issue locally, it is anyways a good idea to introduce a null check in that part of the code hoping it could solve part of your problem for now.

Created #4278 to add the null checks.

[reallymello]

I emailed verbose logs from 3.8 to @garg3133
This was a test that was working in 3.7, but its exhibiting the timeout in 3.8. It happens after switching frame contexts and immediately using waitForElementPresent.