Fix #4183: Adjust Overall Performance runtime in HTML test report

[Vaibhavsahu2810]

Description

This pull request addresses issue #4183, where the Overall Performance runtime displayed in the Nightwatch HTML report differed from the Suite runtime for single-suite runs. The discrepancy arose due to differing calculation methods for these runtime metrics, causing confusion for users analyzing test execution data.

Changes Made

Updated the aggregateStats Method:
The method now calculates the Overall Performance runtime by summing the actual execution times (timeMs) of test cases instead of relying solely on start and end timestamps.

Proof of Work

The report shows consistent runtimes across sections for both Overall Performance and Suite runtime.

• Before marking your PR for review, please test and verify your changes by making appropriate modifications to any of the Nightwatch example tests (present in examples/tests directory of the project) and running them. ecosia.js and duckDuckGo.js are good examples to work with.
• Create a new branch from master (e.g. features/my-new-feature or issue/123-my-bugfix);
• If you're fixing a bug also create an issue if one doesn't exist yet;
• If it's a new feature explain why do you think it's necessary. Please check with the maintainers beforehand to make sure it is something that we will accept. Usually we only accept new features if we feel that they will benefit the entire community;
• Please avoid sending PRs which contain drastic or low level changes. If you are certain that the changes are needed, please discuss them beforehand and indicate what the impact will be;
• If your change is based on existing functionality please consider refactoring first. Pull requests that duplicate code will most likely be ignored;
• Do not include changes that are not related to the issue at hand;
• Follow the same coding style with regards to spaces, semicolons, variable naming etc.;
• Always add unit tests - PRs without tests are most of the times ignored.

[garg3133]

@Vaibhavsahu2810 Thanks for this PR. But, were you able to figure out why the subtraction of the above two variables is giving the wrong time?

Ideally, we should try to find the root cause of the problem and try to tackle that instead of just introducing a fix. If the this.results.endTimestamp is wrong above, we should fix that because it might be causing problems at other places as well.

Also, if the tests are running in parallel, we wouldn't just want to add up the time of all the tests but instead show the time the entire test run actually took from the user's perspective (if I'm not missing something here?).

[Vaibhavsahu2810]

@garg3133 The current time calculation in aggregateStats() has a problem that , It's calculating the total time using the global start and end timestamps . This includes time between test runs and setup/teardown time that isn't part of the actual test execution.
Meanwhile, individual suite times are calculated from the actual execution time of the tests (stored in module.timeMs), which is aggregated in the createInitialResult() and aggregateEnvironments() methods.
That's why i changed the time calculation method in aggregateStats() according to aggregateEnvironments() method.

[garg3133]

@Vaibhavsahu2810 That's fine. We are supposed to show the overall time (including setup/teardown and the time spent in between the test runs) in the Overall Performance section. This section should basically show the total run time of the Nightwatch process and not the sum of the run-time of individual test cases.

Let's say you run three tests in parallel using Nightwatch and the individual run time of the tests are as follows:

• Test 1: 30 secs
• Test 2: 75 secs
• Test 3: 40 secs

Then the total run time of the Nightwatch process need not be same as the sum of the run time of the three test cases (since the tests are running in parallel) but it should always be greater then the run time of the longest running test (75 secs).

This is exactly the problem in the issue where the overall performance shows a time less than the time of the longest running test, which is not possible.