Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON loaded from external file doesn't unwrap when passed to programmatic runner #3179

Open
jfo opened this issue Nov 22, 2023 · 0 comments
Open

Comments

@jfo
Copy link

jfo commented Nov 22, 2023

When a collection is contained in an external JSON file, the behavior wrt how it is processed differs between the CLI interface and the programmatic interface in a surprising way.

When passed as a filepath, the collection can be loaded if it's in a JSON object OR if it's in a JSON object under the key collection, but when passed as JSON directly, this unwrapping doesn't happen. The unwrapping happens here:

!source.values && _.isObject(source[type]) && (source = source[type]);

This leads to surprising behavior, as my expectation was that running a json file that works on the CLI would also work when passed as part of a script. There is no feedback that the formatting of the json could be the issue and be causing the difference.

  1. Newman Version (can be found via newman -v): 6.0.0

  2. OS details (type, version, and architecture): Mac Ventura M2

  3. Are you using Newman as a library, or via the CLI? Both, see issue

  4. Did you encounter this recently, or has this bug always been there: Idk

  5. Expected behaviour: Consistency between CLI and programmatic use

  6. Command / script used to run Newman:

  7. Sample collection, and auxiliary files (minus the sensitive details):
    reproduction

Reproduction can be found in this diff in this branch

Wrapped JSON:  {
  iterations: { total: 0, pending: 0, failed: 0 },
  items: { total: 0, pending: 0, failed: 0 },
  scripts: { total: 0, pending: 0, failed: 0 },
  prerequests: { total: 0, pending: 0, failed: 0 },
  requests: { total: 0, pending: 0, failed: 0 },
  tests: { total: 0, pending: 0, failed: 0 },
  assertions: { total: 0, pending: 0, failed: 0 },
  testScripts: { total: 0, pending: 0, failed: 0 },
  prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
Bare JSON:  {
  iterations: { total: 1, pending: 0, failed: 0 },
  items: { total: 3, pending: 0, failed: 0 },
  scripts: { total: 1, pending: 0, failed: 0 },
  prerequests: { total: 3, pending: 0, failed: 0 },
  requests: { total: 3, pending: 0, failed: 0 },
  tests: { total: 3, pending: 0, failed: 0 },
  assertions: { total: 2, pending: 0, failed: 0 },
  testScripts: { total: 1, pending: 0, failed: 0 },
  prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
Wrapped: Filepath {
  iterations: { total: 1, pending: 0, failed: 0 },
  items: { total: 3, pending: 0, failed: 0 },
  scripts: { total: 1, pending: 0, failed: 0 },
  prerequests: { total: 3, pending: 0, failed: 0 },
  requests: { total: 3, pending: 0, failed: 0 },
  tests: { total: 3, pending: 0, failed: 0 },
  assertions: { total: 2, pending: 0, failed: 0 },
  testScripts: { total: 1, pending: 0, failed: 0 },
  prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
Bare Filepath:  {
  iterations: { total: 1, pending: 0, failed: 0 },
  items: { total: 3, pending: 0, failed: 0 },
  scripts: { total: 1, pending: 0, failed: 0 },
  prerequests: { total: 3, pending: 0, failed: 0 },
  requests: { total: 3, pending: 0, failed: 0 },
  tests: { total: 3, pending: 0, failed: 0 },
  assertions: { total: 2, pending: 0, failed: 0 },
  testScripts: { total: 1, pending: 0, failed: 0 },
  prerequestScripts: { total: 0, pending: 0, failed: 0 }
}

I don't have the context to know what the expected behavior should be, other than I think passing a path and passing a JSON object should work consistently. I can see there is a lot of backwards compatibility stuff going on too. I'm filing this issue so that there is some visibility into the surprising behavior.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant