The IBM OpenAPI Validator lets you validate OpenAPI 3.0.x and OpenAPI 3.1.x documents for compliance with the OpenAPI specifications, as well as IBM-defined best practices.
- Node 16.0.0+
- NPM 8.3.0+
The validator analyzes your API definition and reports any problems within. The validator is highly customizable, and supports OpenAPI 3.0.x and 3.1.x documents. The tool also supports a number of rules from Spectral. You can easily extend the tool with custom rules to meet your specific needs and ensure compliance to your standards.
Get started by installing the tool, then run the tool on your API definition.
By default, the validator will use the IBM Cloud Validation Ruleset (npm package @ibm-cloud/openapi-ruleset).
However, if the validator detects the presence of any of the standard Spectral ruleset files (spectral.yaml, spectral.yml, spectral.json,
or spectral.js) in the current directory (from which the validator is being run) or in any containing directory within the file system,
then that ruleset file will be used instead.
To explicitly specify an alternate ruleset, you can use the -r/--ruleset option (or the ruleset configuration property)
to specify the name of your custom ruleset file.
If one of the standard Spectral ruleset files are present and you'd like to force the use of the IBM Cloud Validation Ruleset instead,
you can use -r default or --ruleset default (or set the ruleset configuration property to the value 'default').
Details about these options are provided below in the Usage section.
You can modify the behavior of the validator for your project to meet your preferred standards. See the customization documentation for more information.
There are three ways to install the validator: using NPM, downloading a platform-specific binary, or building from source.
npm install -g ibm-openapi-validator
The -g flag installs the tool globally so that the validator can be run from anywhere in the file system. Alternatively, you can pass no flag or the --save-dev flag to add the validator as a dependency to your project and run it from your NPM scripts or JavaScript code.
Platform-specific binary files are packaged with each release for MacOS, Linux, and Windows. See the releases page to download the executable for your platform. These do not depend on Node.JS being installed.
- Clone or download this repository
- Navigate to the root directory of this project.
- Install the dependencies using
npm install - Build the command line tool by running
npm run link.
If you installed the validator using npm install -g ibm-openapi-validator, you will need to run npm uninstall -g ibm-openapi-validator before running npm run link.
It is also possible to build platform specific binaries from the source code by running npm run pkg in the project root directory. The binaries (lint-openapi-macos, lint-openapi-linux, lint-openapi-windows.exe) will be in the project's packages/validator/bin directory.
Run the validator with the container image by mounting your API definition.
If it is named openapi.yaml in the current directory, then run:
docker run \
--rm --tty \
--volume "$PWD:/data:ro" \
ibmdevxsdk/openapi-validator:latest \
openapi.yamlYou should replace latest with a specific tagged version to avoid any surprises when new releases are published.
Flag and argument syntax is the same as described in Usage, but file paths are relative to /data.
To use a custom ruleset named ruleset.yaml in the current directory, run:
docker run \
--rm --tty \
--volume "$PWD:/data:ro" \
ibmdevxsdk/openapi-validator:latest \
--ruleset ruleset.yaml \
openapi.yamlIf the existing image doesn't suit your needs, you could extend it and build your own.
For example, to build a validator image with your own custom ruleset package installed, make a Dockerfile like this:
FROM ibmdevxsdk/openapi-validator:latest
RUN npm install -g ${your-ruleset-pkg-here}Usage: lint-openapi [options] [file...]
Run the validator on one or more OpenAPI 3.x documents
Options:
-c, --config <file> use configuration stored in <file> (*.json, *.yaml, *.js)
-e, --errors-only include only errors in the output and skip warnings (default is false)
-i, --ignore <file> avoid validating <file> (e.g. -i /dir1/ignore-file1.json --ignore /dir2/ignore-file2.yaml ...) (default is []) (default: [])
-j, --json produce JSON output (default is text)
-l, --log-level <loglevel> set the log level for one or more loggers (e.g. -l root=info -l ibm-schema-description-exists=debug ...) (default: [])
-n, --no-colors disable colorizing of the output (default is false)
-r, --ruleset <file> use Spectral ruleset contained in `<file>` ("default" forces use of default IBM Cloud Validation Ruleset)
-s, --summary-only include only the summary information and skip individual errors and warnings (default is false)
-q, --impact-score compute scores representing the API impact of rule violations and include with the results (default is false)
-m, --markdown-report generate a Markdown file with a report on all validator results (default is false)
-w, --warnings-limit <number> set warnings limit to <number> (default is -1)
--version output the version number
-h, --help display help for commandwhere [file...] is a space-separated list containing the filenames of one or more OpenAPI 3.x documents to be validated.
The validator supports OpenAPI documents in either JSON or YAML format, using these supported file extensions:
.json
.yaml
.yml
Assuming your command shell supports the use of wildcards, you can use wildcards when specifying the names of files to be validated.
For example, to run the validator on all .yaml files contained in the /my/apis directory, you could use
this command:
lint-openapi /my/apis/*.yamlNote that the -i/--ignore option can be particularly useful when using wildcards because it allows you to skip the
validation of specific files which might otherwise be included in a validation run.
For example, to validate all .yaml files in the /my/apis directory, except for /my/apis/broken-api.yaml use the command:
lint-openapi /my/apis/*.yaml -i /my/apis/broken-api.yamlIn addition to command-line options, the validator supports the use of a configuration file containing options as well.
A configuration file can be in JSON, YAML or Javascript format, using these supported extensions: .json, .yaml, .yml, and .js.
Its structure must comply with this JSON schema.
You can specify the name of your configuration file with the -c/--config option. Here's an example:
lint-openapi -c my-config.yaml my-api.jsonwhere my-config.yaml might contain the following:
errorsOnly: true
limits:
warnings: 25
outputFormat: 'json'
summaryOnly: trueThis would be equivalent to this command:
lint-openapi --errors-only --warnings-limit=25 --json --summary-only my-api.jsonWhen using both a configuration file and various command-line options, be aware that the options specified via the command-line will take precendence and override any corresponding properties specified in the configuration file.
This section provides information about each of the properties that are supported within a configuration file.
| Description | Default |
The colorizeOutput configuration property corresponds to the -n/--no-colors
command-line option. If set to true, then the validator will colorize its output. |
true |
.yaml/.yml |
.json |
.js |
colorizeOutput: false |
{
"colorizeOutput": false
}
|
module.exports = {
colorizeOutput: false
};
|
| Description | Default |
The errorsOnly configuration property corresponds to the -e/--errors-only command-line option.
If set to true, the validator will include only errors in its output, while messages of severity warning,
info or hint will be skipped. |
false |
.yaml/.yml |
.json |
.js |
errorsOnly: true |
{
"errorsOnly": true
}
|
module.exports = {
errorsOnly: true
};
|
| Description | Default |
The files configuration property corresponds to positional command-line arguments (i.e. [file...]).
You can set this property to the names of the OpenAPI documents to be validated. If any filenames are also entered as positional arguments
on the command-line, they will override any values specified in this configuration property. |
[](empty list) |
.yaml/.yml |
.json |
.js |
files: - file1.json - file2.yaml |
{
"files": [
"file1.json",
"file2.yaml"
]
}
|
module.exports = {
files: [
'file1.json',
'file2.yaml'
]
};
|
| Description | Default |
The ignoreFiles configuration property corresponds to the -i/--ignore command-line option.
Set this property to the fully-qualified filenames of OpenAPI documents to be excluded from validation.
This property can be particularly useful when using wildcards for specifying the OpenAPI documents to be validated,
because it allows you to skip the validation of specific files which might otherwise be included in a validation run.
For example, to validate all .yaml files in the /my/apis directory, except for
/my/apis/broken-api.yaml use the command:
lint-openapi /my/apis/*.yaml --ignore /my/apis/broken-api.yaml |
[](empty list) |
.yaml/.yml |
.json |
.js |
ignoreFiles: - /my/apis/file1.yml |
{
"ignoreFiles": [
"/my/apis/file1.yml"
]
}
|
module.exports = {
ignoreFiles: [
'/my/apis/file1.yml'
]
};
|
| Description | Default |
The limits configuration property corresponds to the -w/--warnings-limit command-line option.
Use this property to set the warnings limit. When validating an OpenAPI document, the validator will compare the number of warnings
it encounters with the warnings limit. If the number of warnings exceeds the limit, then an error will be logged and the
validator will return exitCode 1, similar to if actual errors were found. If the warnings limit is set to a negative number,
then no warnings limit check will be performed by the validator. |
{ warnings: -1 }(warnings limit check disabled) |
.yaml/.yml |
.json |
.js |
limits: warnings: 25 |
{
"limits": {
"warnings": 25
}
}
|
module.exports = {
limits: {
warnings: 25
}
};
|
| Description | Default |
The logLevels property is an object that specifies the logging level
(error, warn, info, or debug)
associated with each logger within the validator. It corresponds to the -l/--log-level command-line option. |
{ root: 'info' } |
.yaml/.yml |
.json |
.js |
logLevels: root: error ibm-schema-description-exists: debug |
{
"logLevels": {
"root": "error",
"ibm-schema-description-exists": "debug"
}
}
|
module.exports = {
logLevels: {
root: 'error',
'ibm-schema-description-exists': 'debug'
}
};
|
| Description | Default |
You can set the outputFormat configuration property to either text or json
to indicate the type of output you want the validator to produce.
This property corresponds to the -j/--json command-line option. |
text |
.yaml/.yml |
.json |
.js |
outputFormat: json |
{
"outputFormat": "json"
}
|
module.exports = {
outputFormat: 'json'
};
|
| Description | Default |
You can use the ruleset configuration property to specify a custom ruleset to be used by the validator.
This corresponds to the -r/--ruleset command-line option.
By default, the validator will look for the standard Spectral ruleset files ( If you want to force the use of the IBM Cloud Validation Ruleset even if one of the standard Spectral ruleset files are present,
you can specify |
null, which implies that a standard Spectral ruleset file will be used (if present),
otherwise the IBM Cloud Validation Ruleset will be used. |
.yaml/.yml |
.json |
.js |
ruleset: my-custom-rules.yaml |
{
"ruleset": "my-custom-rules.yaml"
}
|
module.exports = {
ruleset: 'my-custom-rules.yaml'
};
|
| Description | Default |
The summaryOnly configuration property corresponds to the -s/--summary-only command-line option.
If set to true, the validator will include only the summary section in its output.
|
false |
.yaml/.yml |
.json |
.js |
summaryOnly: true |
{
"summaryOnly": true
}
|
module.exports = {
summaryOnly: true
};
|
| Description | Default |
The produceImpactScore configuration property corresponds to the
-q/--impact-score command-line option. If set to true,
the validator will, in addition to reporting individual rule violations, use the
rule violation data to produce API impact scores based on the categories of usability,
security, robustness, and cost of evolution. By default, the data demonstrating how
the scores are calculated from each rule is displayed. If this option is combined with
the "summary only" configuration option, only the categorized impact scores are displayed.
These scores are useful for "Automated Quality Screening". See [this documentation](docs/automated-quality-screening.md)
for more information about the purpose of these scores and how they are computed.
|
false |
.yaml/.yml |
.json |
.js |
produceImpactScore: true |
{
"produceImpactScore": true
}
|
module.exports = {
produceImpactScore: true
};
|
| Description | Default |
The markdownReport configuration property corresponds to the
-m/--markdown-report command-line option.
If set to true, the validator will generate a Markdown file containing a report on all of the validator results,
including the individual rule violations, the impact scores, and the data used to compute the impact scores.
It provides a single location to see all of the information produced by the validator.
A default filename is always used and is based on the name of the API definition file provided to the validator.
If a file of the same name already exists, it will be overwritten by default. This is because the primary use-case
of this feature is publishing reports during CI/CD builds, where a single report will need to be updated frequently.
|
false |
.yaml/.yml |
.json |
.js |
markdownReport: true |
{
"markdownReport": true
}
|
module.exports = {
markdownReport: true
};
|
While the validator does not expose an API for usage within a Node.js program, you can achieve programmatic behavior consistent with the CLI by using the open-source tool Spectral's Node API and the IBM OpenAPI Ruleset package. Here is a simple example of what that might look like:
const ibmOpenapiRuleset = require('@ibm-cloud/openapi-ruleset');
const { Spectral } = require('@stoplight/spectral-core');
function async runSpectral(openapiDocument) {
const spectral = new Spectral();
spectral.setRuleset(ibmOpenapiRuleset);
results = await spectral.run(openapiDocument);
console.log(results);
}The validator can produce output in either text or JSON format. The default is text output, and this can be
controlled with the -j/--json command-line option or outputFormat configuration property.
NOTE The text (i.e. "human readable") output is not a part of the tool's contract and may change in a minor or patch release. If you are building automation or scripts with this tool, use the JSON output, which is stable and subject to semantic versioning.
Here is an example of text output:
IBM OpenAPI Validator (validator: 0.97.5; ruleset: 0.45.5), @Copyright IBM Corporation 2017, 2023.
Validation Results for /my/directory/my-api.yaml:
Errors:
Message : Path contains two or more consecutive path parameter references: /v1/clouds/{cloud_id}/{region_id}
Rule : ibm-no-consecutive-path-parameter-segments
Path : paths./v1/clouds/{cloud_id}/{region_id}
Line : 332
Warnings:
Message : Operation summaries should not have a trailing period
Rule : ibm-summary-sentence-style
Path : paths./v1/clouds.post.summary
Line : 46
Message : Operation summaries should not have a trailing period
Rule : ibm-summary-sentence-style
Path : paths./v1/clouds.get.summary
Line : 93
Summary:
Total number of errors : 1
Total number of warnings : 2
Errors:
1 (100%) : Path contains two or more consecutive path parameter references
Warnings:
2 (100%) : Operation summaries should not have a trailing period
As you can see, any errors detected by the validator are listed first, then warnings, and finally a summary section.
- The
-s/--summary-onlycommand-line option or thesummaryOnlyconfiguration property causes only the summary to be displayed. - The
-e/--errors-onlyoption orerrorsOnlyconfiguration property causes only error-level violations to be displayed. - The
-q/--impact-scoreoption orproduceImpactScoreconfiguration property causes the validator to show aggregated impact scores. See the example below:
Example of impact score tables that are appended to the standard output:
┌────────────────┬───────────┐
│ category │ max score │
├────────────────┼───────────┤
│ usability │ 98 /100 │
│ security │ 100 /100 │
│ robustness │ 97 /100 │
│ evolution │ 67 /100 │
│ overall (mean) │ 91 /100 │
└────────────────┴───────────┘
┌──────────────────────────────┬───────┬─────────────────┬──────────────────┬─────────────────┬───────────────────┬──────────────────┬────────────┐
│ rule │ count │ func │ usability impact │ security impact │ robustness impact │ evolution impact │ rule total │
├──────────────────────────────┼───────┼─────────────────┼──────────────────┼─────────────────┼───────────────────┼──────────────────┼────────────┤
│ operation-operationId-unique │ 1 │ 1×3÷operations │ 1 │ │ 2 │ 3 │ 6 │
│ ibm-no-array-responses │ 2 │ 2×10÷operations │ │ │ │ 20 │ 20 │
│ no-$ref-siblings │ 1 │ 1×1÷operations │ 0.33 │ │ │ │ 0.33 │
└──────────────────────────────┴───────┴─────────────────┴──────────────────┴─────────────────┴───────────────────┴──────────────────┴────────────┘
When displaying JSON output, the validator will produce a JSON object which complies with this JSON schema. The JSON data will include information about all rule violations, as well as all impact score information computed from the rule violations. Here is an example of JSON output:
{
"error": {
"results": [
{
"message": "Path contains two or more consecutive path parameter references: /v1/clouds/{cloud_id}/{region_id}",
"path": [
"paths",
"/v1/clouds/{cloud_id}/{region_id}"
],
"rule": "ibm-no-consecutive-path-parameter-segments",
"line": 332
}
],
"summary": {
"total": 1,
"entries": [
{
"generalizedMessage": "Path contains two or more consecutive path parameter references",
"count": 1,
"percentage": 100
}
]
}
},
"warning": {
"results": [
{
"message": "Operation summaries should not have a trailing period",
"path": [
"paths",
"/v1/clouds",
"post",
"summary"
],
"rule": "ibm-summary-sentence-style",
"line": 46
},
{
"message": "Operation summaries should not have a trailing period",
"path": [
"paths",
"/v1/clouds",
"get",
"summary"
],
"rule": "ibm-summary-sentence-style",
"line": 93
}
],
"summary": {
"total": 2,
"entries": [
{
"generalizedMessage": "Operation summaries should not have a trailing period",
"count": 2,
"percentage": 100
}
]
}
},
"info": {
"results": [],
"summary": {
"total": 0,
"entries": []
}
},
"hint": {
"results": [],
"summary": {
"total": 0,
"entries": []
}
},
"hasResults": true
"impactScore": {
"categorizedSummary": {
"usability": 94,
"security": 100,
"robustness": 100,
"evolution": 100,
"overall": 99
},
"scoringData": [
{
"rule": "ibm-no-consecutive-path-parameter-segments",
"count": 1,
"func": "1×10÷operations",
"demerits": {
"usability": 3.33,
"total": 3.33
}
},
{
"rule": "ibm-summary-sentence-style",
"count": 2,
"func": "2×1÷operations",
"demerits": {
"usability": 0.67,
"total": 0.67
}
}
]
}
}The JSON output is also affected by the -s/--summary-only and -e/--errors-only options as well as the summaryOnly and errorsOnly
configuration properties. It is not affected by the -q/--impact-score option or produceImpactScore property.
The validator uses a logger for displaying messages on the console.
The core validator uses a single logger named root, while each of the rules contained in the
IBM Cloud Validation Ruleset uses their own unique logger whose name will match the rule's id
(e.g. ibm-accept-header, ibm-schema-description-exists, etc.).
Each logger has a logging level associated with it: error, warn, info, and debug.
Each of these levels implicitly includes the levels that precede it in the list.
For example, if you set the logging level of a logger to info, then all messages of type
info, warn, and error will be displayed, but debug messages will not.
To set the level of the root logger to info, you could use this option: --log-level root=info.
To set the level of the logger used by the ibm-accept-header rule to debug,
you could use this option: -l ibm-accept-header=debug.
You can also use a glob-like value for a logger name to set the level on multiple loggers.
For example, to set the level for all loggers whose name starts with ibm-property, try this:
-l ibm-property*=debug.
Enabling debug logging for a specific rule might be useful in a situation where the rule is
reporting violations which seem to be inexplicable. In this case, additional debug information
might be helpful in determining why the violations are occurring, and could possibly lead to a solution.
For example, suppose the ibm-pagination-style rule is reporting several violations,
but yet at first glance it's not obvious why these violations are occurring.
To enable debug logging for this rule, use a command like this:
lint_openapi -l ibm-pagination-style=debug my-new-api.yaml
The default log level for the root logger is info, while the default log level for
rule-specific loggers is warn.
See CONTRIBUTING.
This project is licensed under Apache 2.0. Full license text is available in LICENSE.