Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Fails to download jars #680

Open
ventsyv opened this issue Jan 12, 2023 · 23 comments
Open

[BUG] Fails to download jars #680

ventsyv opened this issue Jan 12, 2023 · 23 comments
Assignees
Labels
bug Something isn't working

Comments

@ventsyv
Copy link

ventsyv commented Jan 12, 2023

⚠️ Important Notice

Please differentiate the bug


🐛 Bug Report:

Describe the bug

When we run the generator for the first time expecting to get resolved from the PATH (openapi-generator-cli version) it hangs and after a while gives the following error:

node_modules/@openapitools/openapi-generator-cli/main.js:1045
            error.response.data.on('data', data => this.logger.log(data.toString('utf8')));
                                ^
TypeError: error.response.data.on is not a function
    at VersionManagerService.printResponseError (/GMSS/node_modules/@openapitools/openapi-generator-cli/main.js:1045:33)
    at /GMSS/node_modules/@openapitools/openapi-generator-cli/main.js:924:18
    at /GMSS/node_modules/rxjs/dist/cjs/internal/operators/catchError.js:13:51
    at OperatorSubscriber._this._error (/GMSS/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:43:21)
    at Subscriber.error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)

Steps to Reproduce

Install like this

cd ~
mkdir testdir
cd ~/testdir/
npm install -g npm@latest
npm install @openapitools/openapi-generator-cli

sudo ln -s /home/$USER/testdir/node_modules/@openapitools/openapi-generator-cli/main.js /usr/bin/openapi-generator-cli

sudo chmod 755 node_modules/@openapitools/openapi-generator-cli/main.js

Now try running:

~/testdir/$ which openapi-generator-cli
/usr/bin/openapi-generator-cli
~/testdir$ openapi-generator-cli version
Unable to query repository, because of: "Request failed with status code 504"

Response:
date Thu, 12 Jan 2023 17:45:56 GMT
content-type text/html
content-length 160
connection keep-alive
server nginx

/home/user/testdir/node_modules/@openapitools/openapi-generator-cli/main.js:1045
            error.response.data.on('data', data => this.logger.log(data.toString('utf8')));
                                ^

TypeError: error.response.data.on is not a function
    at VersionManagerService.printResponseError (/home/user/testdir/node_modules/@openapitools/openapi-generator-cli/main.js:1045:33)
    at /home/user/testdir/node_modules/@openapitools/openapi-generator-cli/main.js:924:18
    at /home/user/testdir/node_modules/rxjs/dist/cjs/internal/operators/catchError.js:13:51
    at OperatorSubscriber._this._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:43:21)
    at Subscriber.error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)

Strangely enough, changing the directory seems to work

cd ..
~$ openapi-generator-cli version
Download 6.2.1 ...
Downloaded 6.2.1
Did set selected version to 6.2.1
6.2.1

You can break it again:

cd testdir
rm -rf node_modules/@openapitools/openapi-generator-cli/versions/
openapi-generator-cli version

The above will give you the same error

Expected behavior

openapi-generator-cli version
Download 6.2.1 ...
Downloaded 6.2.1
Did set selected version to 6.2.1
6.2.1

Screenshots

If applicable, add screenshots to help explain your problem.

Operation System (please complete the following information):

  • OS: Ubuntu
  • Version: 22

Package System (please complete the following information):

  • Node.js v19.3.0
  • npm - 9.2.0

Additional context

Add any other context about the problem here.

@ventsyv ventsyv added the bug Something isn't working label Jan 12, 2023
@JoePotentier
Copy link

Seeing the same. Blocking deployments for my team.

@saivamsilolla
Copy link

Even we are blocked

@risotto-master
Copy link

Blocked as well when installing through NPM per instructions from https://openapi-generator.tech/docs/installation/

@githubeto
Copy link

Is there an alternative plan?

@githubeto
Copy link

I confirmed that it works fine when using 'openapitools/openapi-generator-cli' docker images.

@PSchubertMHP
Copy link

Hi, we're having the same issue. Also blocks deployments. Seems like the repo is offline.

@talesAtTilt
Copy link

Same here too.

@DavidBiesack
Copy link

DavidBiesack commented Jan 13, 2023

We use

npx openapi-generator-cli version-manager set 5.4 stable

as there are bugs in openapi-generator 6 that affect us, but now we cannot download 5.4 due to this outage

Unable to query repository, because of: "Request failed with status code 502"

Response:

server awselb/2.0
date Fri, 13 Jan 2023 13:08:10 GMT
content-type text/html
content-length 122
connection close

and openapi-generator uses 6.2.1 instead of 5.4

As a workaround, use a static copy of the 5.4.0 openapi-generator jar and access that directly via

--custom-generator=/path/to/openapi-generator-5.4.0.jar

One can download the needed jar from maven, such as
https://repo1.maven.org/maven2/org/openapitools/openapi-generator/5.4.0/openapi-generator-5.4.0.jar
or https://repo1.maven.org/maven2/org/openapitools/openapi-generator/6.2.1/openapi-generator-6.2.1.jar

@ventsyv
Copy link
Author

ventsyv commented Jan 13, 2023

Creating an openapitools.json file in the current working directory with the following content seems to fix it for me:

{
  "$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "6.2.1"
  }
}

It appears that sometimes the file is not generated for some reason. When it is being generated everything works fine. I can't find any obvious reason (permissions, etc) for the file not to be generated.
Another strange thing is that the schema path is not valid from the current working directory - so I can't see a reason why it would fix the issues, but it seems that it does. I tried 4 - 5 times and every time I added the file (different working dirs) it works like a charm.

This started happening in the last couple of days.

@dougkeen
Copy link

I looked into this a bit, and it seems related to the not-so-great uptime of search.maven.org:
image
image

This will cause failures when:

  1. Defaulting to using latest openapi-generator version, since that search.maven.org endpoint is called to figure out what the latest published version is.
  2. Explicitly specifying the openapi-generator version, since the set method on the version manager calls the search.maven.org endpoint to validate the version param.

In other words, you're in trouble in all cases if search.maven.org is down.

It seems like validation via the search step while doing a openapi-generator-cli version-manager set <versionTags...> may be causing more problems than it solves, given the coupling to search.maven.org.

@dougkeen
Copy link

For contrast, here's the uptime of the CDN from where the openapi-generator jars are actually downloaded:
image

It would be great if there was a way to specify the exact openapi-generator version to use, and skip the query step so your usage of the tool is decoupled from search.maven.org.

@GustavoKatel
Copy link

the situation is aggravated because the command does not return a non-zero code, so it does not stop in CI and it silently fails in some cases.

@alanwguo
Copy link

alanwguo commented Jan 18, 2023

We use

npx openapi-generator-cli version-manager set 5.4 stable

as there are bugs in openapi-generator 6 that affect us, but now we cannot download 5.4 due to this outage

Unable to query repository, because of: "Request failed with status code 502"

Response:

server awselb/2.0
date Fri, 13 Jan 2023 13:08:10 GMT
content-type text/html
content-length 122
connection close

and openapi-generator uses 6.2.1 instead of 5.4

As a workaround, use a static copy of the 5.4.0 openapi-generator jar and access that directly via

--custom-generator=/path/to/openapi-generator-5.4.0.jar

One can download the needed jar from maven, such as https://repo1.maven.org/maven2/org/openapitools/openapi-generator/5.4.0/openapi-generator-5.4.0.jar or https://repo1.maven.org/maven2/org/openapitools/openapi-generator/6.2.1/openapi-generator-6.2.1.jar

@DavidBiesack,

The --custom-generator param doesn't seem to work for me. It still tries to use the version set via npx openapi-generator-cli version-manager set 5.4 stable. Is this the way you used it?

$ wget https://repo1.maven.org/maven2/org/openapitools/openapi-generator/4.3.0/openapi-generator-4.3.0.jar -O /tmp/openapi-generator-4.3.0.jar
$ npx openapi-generator-cli generate \
		--custom-generator=/tmp/openapi-generator-4.3.0.jar \
		-i ../../backend/gen/openapi.json -g typescript-fetch \
		-o src/gen/openapi -c openapiconfig.yaml

@enceem
Copy link

enceem commented Jan 18, 2023

we are encountering same issue "Request failed with status code 502" in our corporate network. I think that it is a common scenario to haven nexus configured as mirror for maven. So i would suggest:
If maven is installed then maven should be used for the download the jar file instead of hard-coded url.
mvn dependency:copy -Dartifact=org.openapitools:openapi-generator-cli:6.2.1 -DoutputDirectory=./

@DavidBiesack
Copy link

alanwguo yes, that is how I'm using it. I'm using this
instead of calling npx openapi-generator-cli version-manager set 5.4 stable.
The other solution, creating the openapitools.json tool, also worked for me. I use

openapiGeneratorVersion=5.4.0
cat <<EOF >| openapitools.json
{
  "\$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "${openapiGeneratorVersion}"
  }
}
EOF
That is, I think the `openapi-generator-cli` will use the version in the `openapitools.json` file, 
but calling `openapi-generator-cli version-manager set 5.4 stable` will still force it to
lookup versions in Maven repo to verify it, so to fix this issue, you need to not use that command.

@LKreutzer
Copy link

LKreutzer commented Jan 19, 2023

Seeing the same issue magma/magma#14860, but for us it seems to be flaky, see workflow runs. It started ca. 1 week ago.

@stephanecodes
Copy link

I have the same problem when validating an openapi schema in an Azure pipeline.
The --custom-generator option is only supported for the generate command, not for the validate command

On the other hand, it is possible to specify a storageDir which openapi-generator-cli uses to download the JAR and check if it is already downloaded.
The solution I have chosen is to store a recent version of the openapi-generator-x.y.z.jar JAR in my Git repository and use it if search.maven.org is not responding.

The openapitools.json file is not committed in my repo, it forces openapi-generator-cli to download the latest version available at each build.

Here is my validate:openapi script in package.json

{
  "scripts": {
    "validate:openapi": "openapi-generator-cli validate -i ../path/to/openapi.yaml",
  },
}

And the part of the CI shell script in the pipeline that validates the openapi schema

# If search.maven.org is down the openapi-generator-cli-x.y.z.jar 
# won't be downloaded and the command will fail
# The issue is described at https://github.com/OpenAPITools/openapi-generator-cli/issues/680

set +e;
# The next command will automatically check and download the latest version of openapi-generator-cli
# Let's capture the command exit status. 
# If exit status is not zero then fallback on a static version of the jar (may not be the latest version)
npm run validate:openapi
exit_status=$?

# Exit immediately if a command returns a non-zero status
set -e;

if (( exit_status != 0 )); then
    echo ""
    echo "Failed to fetch version of openapi-generator-cli (exit status=$exit_status)"
    # Use the last modified openapi-generator jar stored in static dir as fallback
    fallback_jar_version=$(basename -s .jar $(ls -t ./static/openapi-generator-cli/*.jar | head -n1))
    echo "Using fallback library openapi-generator-cli-$fallback_jar_version.jar"
    # Create or replace openapitools.json file and force the use of the fallback version

cat <<EOF >| openapitools.json
{
  "\$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "storageDir": "./static/openapi-generator-cli",
    "version": "${fallback_jar_version}"
  }
}
EOF

    # Retry
    npm run validate:openapi
fi

# Remove openapitools.json
rm -f openapitools.json

@talesAtTilt
Copy link

It is back.

openapi-generator-cli version-manager set 6.4.0
Unable to query repository, because of: "Request failed with status code 502"
Response:
date Tue, 27 Jun 2023 14:53:24 GMT
content-type text/html
content-length 150
connection close
server nginx
/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/main.js:1036
error.response.data.on('data', data => this.logger.log(data.toString('utf8')));
^
TypeError: error.response.data.on is not a function
at VersionManagerService.printResponseError (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/main.js:1036:33)
at /usr/local/lib/node_modules/@openapitools/openapi-generator-cli/main.js:915:18
at /usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/operators/catchError.js:13:51
at OperatorSubscriber._this._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:43:21)
at OperatorSubscriber.Subscriber.error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
at OperatorSubscriber.Subscriber._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
at OperatorSubscriber.Subscriber.error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
at OperatorSubscriber.Subscriber._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
at OperatorSubscriber.Subscriber.error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
at OperatorSubscriber.Subscriber._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)

@arnaud-deprez
Copy link

When going to https://search.maven.org, I get this:

image

It is probably related...

@swcm-mnestler
Copy link

We're seeing this issue very often in the last few days, a rerun of the build mostly works. Would it make sense to implement a basic retry of the maven search call in the openapi-generator-cli?

@rettgerst
Copy link

rettgerst commented Jan 29, 2024

happening again today. this project needs to stop depending on search.maven.org, which is described as "legacy" by its maintainers and has shown a history of being unreliable.

the offending function is here.

I'm unfamiliar with the java ecosystem - would if be bad practice to bypass the search and make a request directly to https://repo1.maven.org/maven2/org/openapitools/openapi-generator/ to download the jar?

@swcm-mnestler
Copy link

@rettgerst what helped us was running npx openapi-generator-cli version directly before the generation - it will download the version that is set in the openapitools.json without a maven search lookup

@heyline
Copy link

heyline commented May 22, 2024

Anyone struggling with this, I chose to submit the jar package to git-lfs and manually copy the jar to the corresponding folder after npm install, so that I can skip the download. Here is a gradle task that can help me copy the file there.

tasks.register<Copy>("openapi-generate-preprocessing") {
  from(layout.projectDirectory.file("lib/openapi-generator-cli/7.5.0.jar"))
  into(layout.projectDirectory.dir("node_modules/@openapitools/openapi-generator-cli/versions"))
}

At the same time, I also have openapitools.json in the project root directory to fix the openapi-generator-cli version.

{
  "$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "7.5.0"
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests