Reject custom commands node promise on abortOnFailure. #4314
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a command results in an error with
abortOnFailure
set totrue
, the node promise of the command inside the test case still gets resolved instead of getting rejected, while the queue is anyway cleared because ofabortOnFailure
.This leads to a situation where if we chain some more commands after this command in the test case, the test would get stuck (because the queue is emptied) and hence will abort successfully. But, if the await is directly used on the command, that await will resolve and the test will continue normally since the following commands will be added to the queue after the queue is cleared.
The command promise inside the test case should instead be rejected so that the test case throws an error and thus aborting itself instead of continuing normally.
Impact
Only affects custom-commands, which will now reject inside testcase on abort failures instead of getting resolved.
Future TODOs
Moreover, since the queue is emptied on a command failure, the commands between the failed command and resolved assertion never get executed but the commands after resolved assertion will start to run normally.
client-commands
andprotocol
commands while making sure that commands like.getTitle()
also work correctly if there's an abort failure inside thetitle()
command (child-command of the.getTitle()
command).done()
function of queue is reached before the main command could be resolved, leading to runnable getting resolved/rejected before the testcase is actually wrapped up. So, resolve the custom command treenode in testcase before moving todone()
or delay the call ofdone()
.^
done()
is being called earlier due to the failure of a sub-command of the custom-command.The main issue this causes is when inside the test case the custom-command gets resolved (after the call of
done()
and hence resolution/rejection of the runnable), the following commands in the testcase get added to the queue which then overlap with the commands of further tests (like saveScreenshot, afterEach) because the runnable was settled earlier.^ This main issue should be resolved in this PR because we are now rejecting the custom command node inside the test instead of resolving it.
Note: On the third point, even when this PR solves the issue partially, if the user decides to use
try/catch
to catch the custom command error inside the testcase, thedone()
would have still been called earlier due to which the following commands inside the test case will start to overlap with future runnables. So, to fix this fully, we should probably not calldone()
for the sub-command failure. On doing this, the main command will get rejected in the testcase before the call of queuedone()
(arising from the custom-command failure), it will be caught and other commands will be added to the queue. This would result in the queuedone()
call getting skipped in this case because at that point there will be some nodes existing inside the tree.