Fix static assertions / node assertion messages

[garg3133]

Description of the bug/issue

The pass/fail messages emitted by the node assertions (referred to as static assertions in Nigtwatch) are not ideal and should be fixed.

This is essential to get the correct assertion messages in the new Element API assertions.

Let's take the example of .ok() assertion:

• await browser.assert.ok(true) -- seems okayish but could be improved
  
• await browser.assert.ok(false) -- needs to be improved. Instead of the whole block inside (), something simple could be used like: "Testing the value to be truthy". Also, instead of expected "true", this could be expected "a truthy value".
  
• await browser.assert.not.ok(false) -- needs to be improved. Instead of the whole block inside (), something simple could be used like: "Testing the value to be not truthy".
  
• browser.assert.not.ok(true) -- MOST IMPORTANT - the message should show the correct expected and actual values instead of "null".
  

Additional information

The code for the static assertions is present in lib/_loaders/static.js file.

All available Node Assertions in Nightwatch can be found here:

nightwatch/types/assertions.d.ts

Line 544 in 8f4a330

export interface NightwatchNodeAssertions<ReturnType> {

[ansh7432]

Can i work on this issue ? could you please assign me..

[garg3133]

Sure @ansh7432, go ahead! I'll assign this issue to you once you come up with some solution to this problem.

[ansh7432]

okay !

[lokii2024]

how to generate above messages, or assertion condition cause i'm running npm run but there are lot of tests that, running how to track that, can you tell guide i'm complete beginner.

[ansh7432]

Heyyy @garg3133 i had checked the assertions i understood the problem of defining the error message and customizing the assertions message so can i work on this can you assign this issue to me ?

[ansh7432]

I had made changes in the code and modified the assertions how to run the testcase now if its working now or not please guide

[Vineet1101]

@garg3133 I think we can make changes in the static.js file by writing more detailed messages for assertions
Fix the getMessage method to include more detail messages
Refactor the assert method to include some error handling logic

Here is the code that i have written can you check and tell if any changes have to be made in this code and also assign the issue to me

`const util = require('util');
const chai = require('@nightwatch/chai');
const assertModule = require('assert');
const EventEmitter = require('events');
const Utils = require('../../utils');
const Element = require('../../element');
const chaiExpect = chai.expect;
const { flag } = chai.util;
const { AssertionRunner } = require('../../assertion');
const namespacedApi = require('../../core/namespaced-api.js');

let last_deferred = null;

module.exports = class StaticAssert {
static get assertOperators() {
return {
ok: ['ok', 'not ok'],
equal: ['==', '!='],
notEqual: ['!=', '=='],
deepEqual: ['deepEqual', 'not deepEqual'],
notDeepEqual: ['not deepEqual', 'deepEqual'],
strictEqual: ['===', '!=='],
notStrictEqual: ['!==', '==='],
deepStrictEqual: ['deep strict equal', 'not deep strict equal'],
throws: ['throws', 'doesNotThrow'],
doesNotThrow: ['doesNotThrow', 'throws'],
match: ['matches', 'does not match'],
fail: 'fail',
ifError: 'ifError',
};
}

static get lastDeferred() {
return last_deferred;
}

static set lastDeferred(value) {
last_deferred = value;
}

constructor(nightwatchInstance) {
this.nightwatchInstance = nightwatchInstance;
}

static formatValue(value) {
if (value === null) return 'null';
if (value === undefined) return 'undefined';
if (typeof value === 'object') return util.inspect(value, { depth: 2 });
return value.toString();
}

static formatMessage(propName, { passed, negate, actual, expected }) {
const operator = negate
? StaticAssert.assertOperators[propName]?.[1] || 'not'
: StaticAssert.assertOperators[propName]?.[0] || '';

return `Assertion ${passed ? 'passed' : 'failed'} [${propName}]: ${
  negate ? 'NOT ' : ''
}Expected: ${StaticAssert.formatValue(expected)}, Actual: ${StaticAssert.formatValue(actual)}, Operator: ${operator}`;

}

createStaticAssertion(commandName, abortOnFailure, apiToReturn) {
class Assertion extends EventEmitter {
constructor({ negate, args }) {
super();
StaticAssert.lastDeferred = null;
this.negate = negate;
this.args = args;
this.passed = null;
this.expected = args[1];
this.actual = args[0];
const lastArgument = args[args.length - 1];
const isLastArgString = Utils.isString(lastArgument);
this.message =
isLastArgString && args.length > 2 ? lastArgument : '[Assertion]';
}

  assert(propName) {
    try {
      assertModule[propName](...this.args);
      this.passed = !this.negate;
      this.message = StaticAssert.formatMessage(propName, {
        passed: this.passed,
        negate: this.negate,
        actual: this.actual,
        expected: this.expected,
      });
    } catch (error) {
      this.passed = !!this.negate;
      this.message = StaticAssert.formatMessage(propName, {
        passed: this.passed,
        negate: this.negate,
        actual: error.actual,
        expected: error.expected,
      });


      this.actual = error.actual;
      this.expected = error.expected;
      this.stackTrace = error.stack;
    }
  }
}

return function assertFn({ negate, args }) {
  const assertion = new Assertion({ negate, args });
  const { reporter, nightwatchInstance } = this;
  assertion.assert(commandName);
  const startTime = new Date();

  const namespace = abortOnFailure ? 'assert' : 'verify';
  const commandFn = () => {
    const { passed, expected, actual, message, stackTrace } = assertion;
    const elapsedTime = new Date() - startTime;

    this.runner = new AssertionRunner({
      passed,
      err: { expected, actual },
      message,
      calleeFn: assertFn,
      abortOnFailure,
      stackTrace,
      reporter,
      elapsedTime,
    });

    return this.runner.run();
  };

  const isES6Async =
    nightwatchInstance.isES6AsyncTestcase ||
    nightwatchInstance.settings.always_async_commands;
  const deferred = Utils.createPromise();

  const node = this.commandQueue.add({
    commandName,
    commandFn,
    context: this.api,
    args: [],
    stackTrace: assertFn.stackTrace,
    namespace,
    deferred,
    isES6Async,
  });

  if (isES6Async || node.isES6Async) {
    StaticAssert.lastDeferred = deferred;

    Object.assign(node.deferred.promise, apiToReturn || this.api);


    node.deferred.promise.catch((err) => {
      return StaticAssert.lastDeferred.reject(err);
    });

    return node.deferred.promise;
  }

  return apiToReturn || this.api;
}.bind(this);

}

loadStaticAssertions(parent = null) {
Object.keys(assertModule).forEach((prop) => {
this.nightwatchInstance.setApiMethod(
prop,
parent ? parent.assert || {} : 'assert',
this.createStaticAssertion(prop, true)
);

  if (this.nightwatchInstance.startSessionEnabled) {
    this.nightwatchInstance.setApiMethod(
      prop,
      parent ? parent.verify || {} : 'verify',
      this.createStaticAssertion(prop, false)
    );
  }
});

return this;

}
};
`

[ritankarsaha]

@garg3133 I think the approach to this solution would be to

Enhance the Assertion Messages
For .ok(), messages should test for a "truthy value" rather than explicitly expecting true.
For .not.ok(), messages should test for a "non-truthy value."
and ensuring the messages clearly state the actual and expected values.

Example change

ok(value, message = 'Testing the value to be truthy') {
  this.assert(
    value,
    message,
    `Expected value to be a truthy value, but got ${value}`,
    `Expected value not to be a truthy value`
  );
}

Also we need to ensure that the NightwatchNodeAssertions interface in nightwatch/types/assertions.d.ts reflects any updates to the method signatures.

interface NightwatchNodeAssertions<ReturnType> {
  ok(value: any, message?: string): ReturnType;
  notOk(value: any, message?: string): ReturnType;
}

Is this approach correct or are there any other things that needs to be taken into consideration?

[ansh7432]

assert(propName) {
try {
if (propName === 'ok') {
// Modify to handle truthy values check
if (!this.args[0]) {
throw new Error('Expected a truthy value');
}
this.passed = !this.negate;
this.message = ${this.negate ? 'Failed' : 'Passed'} [${propName}]: Testing the value to be truthy;

by changing the assertion code to this and others methods we can solve this @garg3133

[ritankarsaha]

@garg3133 I think the approach to this solution would be to

Enhance the Assertion Messages For .ok(), messages should test for a "truthy value" rather than explicitly expecting true. For .not.ok(), messages should test for a "non-truthy value." and ensuring the messages clearly state the actual and expected values.

Example change

ok(value, message = 'Testing the value to be truthy') {
  this.assert(
    value,
    message,
    `Expected value to be a truthy value, but got ${value}`,
    `Expected value not to be a truthy value`
  );
}

Also we need to ensure that the NightwatchNodeAssertions interface in nightwatch/types/assertions.d.ts reflects any updates to the method signatures.

interface NightwatchNodeAssertions<ReturnType> {
  ok(value: any, message?: string): ReturnType;
  notOk(value: any, message?: string): ReturnType;
}

Is this approach correct or are there any other things that needs to be taken into consideration?

@garg3133 If this approach is all right I would like to carry my work on with this issue.

[ansh7432]

heyy @garg3133 should i start working on this issue.. just a friendly reminder...