Merge pull request #20 from aviaviavi/imports

specs are top level objects or arrays. imports in specs!

README.md

@@ -46,41 +46,51 @@ There are few options to install:
 
 ### Writing a test specification
 
-Curl runnings tests are just data! A test spec is a top level array of test
-cases, where each item represents a single curl and set of assertions about the
-response. Write your tests specs in a yaml or json file.
+Curl runnings tests are just data! A test spec is an object containing an array
+of `cases`, where each item represents a single curl and set of assertions about
+the response. Write your tests specs in a yaml or json file. Note: the legacy
+format of a top level array of test cases is still supported, but may not be in
+future releases.
+
 
 ```yaml
 ---
 # example-test.yaml
 #
-# the top level of the file is an array of test cases
-- name: A curl runnings test case
-  url: http://your-endpoint.com/status
-  requestMethod: GET
-  # Specify the json payload we expect here
-  expectData:
-    # The 1 key in this object specifies the matcher we want
-    # to use to test the returned payload. In this case, we
-    # require the payload is exactly what we specify.
-    exactly:
-      okay: true
-      msg: 'a message'
-  # Assertions about the returned status code. Pass in
-  # an acceptable code or list of codes
-  expectStatus: 200
+# specify all your test cases as an array keys on `cases`
+cases:
+  - name: A curl runnings test case
+    url: http://your-endpoint.com/status
+    requestMethod: GET
+    # Specify the json payload we expect here
+    expectData:
+      # The 1 key in this object specifies the matcher we want
+      # to use to test the returned payload. In this case, we
+      # require the payload is exactly what we specify.
+      exactly:
+        okay: true
+        msg: 'a message'
+    # Assertions about the returned status code. Pass in
+    # an acceptable code or list of codes
+    expectStatus: 200
 
 ```
 
 See /examples for more example curl runnings specifications, which walk
-through some of the other features that can be encoded in your tests.
+through some of the other features that can be encoded in your tests such as:
+- reference data from previous responses of previous test cases
+- reference environment variables
+- various easy-to-use json matchers
+- support for importing data from other yaml files in your spec
 
 ### Running
 
 Once you've written a spec, simply run it with:
 
 ```curl-runnings -f path/to/your/spec.yaml ```
 
+(hint: try using the --verbose flag for more output)
+
 If all your tests pass, curl-runnings will cleanly exit with a 0 code. A code of
 1 will be returned if any tests failed.
 

curl-runnings.cabal

@@ -2,10 +2,10 @@
 --
 -- see: https://github.com/sol/hpack
 --
--- hash: 1c28f9d3ccc7a7803ced196b66692ba0b0a50b0916199a1f8cb5c5f12474aca3
+-- hash: 3a156cb60b209631289f2d6cdc0fa0104b58c64f85d86802ba51cc6df440e537
 
 name:           curl-runnings
-version:        0.7.1
+version:        0.8.0
 synopsis:       A framework for declaratively writing curl based API tests
 description:    Please see the README on Github at <https://github.com/aviaviavi/curl-runnings#readme>
 category:       Testing
@@ -22,6 +22,7 @@ cabal-version:  >= 1.10
 extra-source-files:
     examples/example-spec.json
     examples/example-spec.yaml
+    examples/importable.yaml
     examples/interpolation-spec.yaml
     README.md
 

examples/example-spec.yaml

@@ -1,113 +1,115 @@
 ---
-# The top level of the file is an array of test cases
+# A curl runnings spec is an object with an array of `cases`.
+# Note: the legacy format of a top level array of cases is still supported at this time, but it's recommended
+# to migrate to the new format, as it may not be supported in later versions.
+cases:
+  - name: test 1 # required
+    url: http://your-endpoint.com/status # required
+    requestMethod: GET # required
+    # [Optional] Specify the json payload we expect here
+    # The 1 key in this block should be either:
+    # exactly | contains
+    expectData:
+      # The 1 key in this object specifies the matcher we want
+      # to use to test the returned payload. In this case, we
+      # require the payload is exactly what we specify.
+      exactly:
+        okay: true
+        msg: 'a message'
+    # [Required] Assertions about the returned status code. Pass in
+    # an acceptable code or list of codes
+    expectStatus: 200
 
-- name: test 1 # required
-  url: http://your-endpoint.com/status # required
-  requestMethod: GET # required
-  # [Optional] Specify the json payload we expect here
-  # The 1 key in this block should be either:
-  # exactly | contains
-  expectData:
-    # The 1 key in this object specifies the matcher we want
-    # to use to test the returned payload. In this case, we
-    # require the payload is exactly what we specify.
-    exactly:
-      okay: true
-      msg: 'a message'
-  # [Required] Assertions about the returned status code. Pass in
-  # an acceptable code or list of codes
-  expectStatus: 200
+  - name: test 2
+    url: http://your-endpoint.com/path
+    requestMethod: POST
+    expectStatus:
+    - 200
+    - 201
+    # [Optional] json data to send with the request
+    requestData:
+      hello: there
+      num: 1
 
-- name: test 2
-  url: http://your-endpoint.com/path
-  requestMethod: POST
-  expectStatus:
-  - 200
-  - 201
-  # [Optional] json data to send with the request
-  requestData:
-    hello: there
-    num: 1
+  - name: test 3
+    url: http://your-url.com/other/path
+    requestMethod: GET
+    expectData:
+      # In the `contains` case of data validation, a list of matchers is specified. Currently,
+      # possible types are `keyMatch` | `valueMatch` | `keyValueMatch`.
+      contains:
+      # `keyValueMatch` looks for the key/value pair anywhere in the payload
+      # here, {'okay': true} must be somewhere in the return payload
+      - keyValueMatch:
+          key: okay
+          value: true
+      # `valueMatch` searches for a value anywhere in the payload (note: _not_ a key).
+      # Here, we look for the value `true` anywhere in the payload.
+      # This can be useful for matching against values where you don't know the key ahead of time,
+      # or for values in a top level array.
+      - valueMatch: true
+      # `keyMatch` searches for a key anywhere in the payload
+      - keyMatch: okay
+    expectStatus: 200
 
-- name: test 3
-  url: http://your-url.com/other/path
-  requestMethod: GET
-  expectData:
-    # In the `contains` case of data validation, a list of matchers is specified. Currently,
-    # possible types are `keyMatch` | `valueMatch` | `keyValueMatch`.
-    contains:
-    # `keyValueMatch` looks for the key/value pair anywhere in the payload
-    # here, {'okay': true} must be somewhere in the return payload
-    - keyValueMatch:
-        key: okay
-        value: true
-    # `valueMatch` searches for a value anywhere in the payload (note: _not_ a key).
-    # Here, we look for the value `true` anywhere in the payload.
-    # This can be useful for matching against values where you don't know the key ahead of time,
-    # or for values in a top level array.
-    - valueMatch: true
-    # `keyMatch` searches for a key anywhere in the payload
-    - keyMatch: okay
-  expectStatus: 200
+  - name: test 4
+    url: http://your-url.com/other/path
+    requestMethod: GET
+    expectData:
+      # In the `notContains` case of data validation, a list of matchers is specified. If any
+      # matcher is found in the response payload, the test will fail. Currently,
+      # possible matchers are `valueMatch` | `keyValueMatch`.
+      notContains:
+      - keyValueMatch:
+          key: okay
+          value: true
+      - valueMatch: true
+      # notContains + keyMatch works great for asserting no errors came back
+      - keyMatch: error
+    expectStatus: 200
 
-- name: test 4
-  url: http://your-url.com/other/path
-  requestMethod: GET
-  expectData:
-    # In the `notContains` case of data validation, a list of matchers is specified. If any
-    # matcher is found in the response payload, the test will fail. Currently,
-    # possible matchers are `valueMatch` | `keyValueMatch`.
-    notContains:
-    - keyValueMatch:
-        key: okay
-        value: true
-    - valueMatch: true
-    # notContains + keyMatch works great for asserting no errors came back
-    - keyMatch: error
-  expectStatus: 200
+  - name: test 5
+    url: http://your-url.com/other/path
+    requestMethod: GET
+    expectData:
+      # you can have both a contains and a notContains block in your expectData
+      contains:
+      - keyValueMatch:
+          key: okay
+          value: true
+      notContains:
+      - valueMatch: false
+    expectStatus: 200
 
-- name: test 5
-  url: http://your-url.com/other/path
-  requestMethod: GET
-  expectData:
-    # you can have both a contains and a notContains block in your expectData
-    contains:
-    - keyValueMatch:
-        key: okay
-        value: true
-    notContains:
-    - valueMatch: false
-  expectStatus: 200
+  - name: test 6
+    url: http://your-url.com/other/path
+    requestMethod: GET
+    # Specify the headers you want to sent, just like the -H flag in a curl command
+    # IE "key: value; key: value; ..."
+    headers: "Content-Type: application/json"
+    expectStatus: 200
+    # The response must contain at least these headers exactly.
+    # Header strings again match the -H syntax from curl
+    expectHeaders: "Content-Type: application/json; Hello: world"
 
-- name: test 6
-  url: http://your-url.com/other/path
-  requestMethod: GET
-  # Specify the headers you want to sent, just like the -H flag in a curl command
-  # IE "key: value; key: value; ..."
-  headers: "Content-Type: application/json"
-  expectStatus: 200
-  # The response must contain at least these headers exactly.
-  # Header strings again match the -H syntax from curl
-  expectHeaders: "Content-Type: application/json; Hello: world"
+  - name: test 7
+    url: http://your-url.com/other/path
+    requestMethod: GET
+    headers: "Content-Type: application/json"
+    expectStatus: 200
+    # You can also specify a key and/or value to look for in the headers
+    expectHeaders: 
+    -
+      key: "Key-With-Val-We-Dont-Care-About"
 
-- name: test 7
-  url: http://your-url.com/other/path
-  requestMethod: GET
-  headers: "Content-Type: application/json"
-  expectStatus: 200
-  # You can also specify a key and/or value to look for in the headers
-  expectHeaders: 
-  -
-    key: "Key-With-Val-We-Dont-Care-About"
+  - name: test 8
+    url: http://your-url.com/other/path
+    requestMethod: GET
+    headers: "Content-Type: application/json"
+    expectStatus: 200
+    # Specify a mix of full or partial header matches in a list like so:
+    expectHeaders: 
+    - "Hello: world"
+    -
+      value: "Value-With-Key-We-Dont-Care-About"
 
-- name: test 8
-  url: http://your-url.com/other/path
-  requestMethod: GET
-  headers: "Content-Type: application/json"
-  expectStatus: 200
-  # Specify a mix of full or partial header matches in a list like so:
-  expectHeaders: 
-  - "Hello: world"
-  -
-    value: "Value-With-Key-We-Dont-Care-About"
-

examples/importable.yaml

@@ -0,0 +1,4 @@
+---
+# you can define any variables in a yaml file, which can be imported in your specs via the
+# !import <filename> directive. This is especially useful for defining aliases for data you use frequently
+some_key: &ping ping