Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

npm publish fails / npm publish <folder> publishes from cwd folder #504

Open
unlight opened this issue Jul 31, 2022 · 14 comments · May be fixed by #531
Open

npm publish fails / npm publish <folder> publishes from cwd folder #504

unlight opened this issue Jul 31, 2022 · 14 comments · May be fixed by #531

Comments

@unlight
Copy link

unlight commented Jul 31, 2022

I have configured semantic-release/npm with pkgRoot = './dist'
On CI/CD command is npx -w my-package semantic-release

CI run 28 Logs:

[1:20:01 PM] [semantic-release] [@semantic-release/git] › ℹ  Prepared Git release: my-package-v1.1.2
[1:20:03 PM] [semantic-release] › ✔  Created tag my-package-v1.1.2
[1:20:03 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/npm"
[1:20:03 PM] [semantic-release] [@semantic-release/npm] › ℹ  Publishing version 1.1.2 to npm registry on dist-tag latest
+ [email protected]
[1:20:04 PM] [semantic-release] [@semantic-release/npm] › ℹ  Published [email protected] to dist-tag @latest on https://registry.npmjs.org/

Note: [email protected] is a result of npm publish

CI run 29 Logs:

2022-07-31T13:06:50.035Z semantic-release:plugins options for @semantic-release/npm/verifyConditions: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.040Z semantic-release:plugins options for @semantic-release/npm/prepare: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.040Z semantic-release:plugins options for @semantic-release/npm/publish: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.041Z semantic-release:plugins options for @semantic-release/npm/addChannel: {
  pkgRoot: '/root/packages/my-package/dist'
}
[1:06:51 PM] [semantic-release] [@semantic-release/npm] › ℹ  Write version 1.1.2 to package.json in /root/packages/my-package/dist
v1.1.2
[1:06:52 PM] [semantic-release] › ✔  Completed step "prepare" of plugin "@semantic-release/npm"
[1:06:52 PM] [semantic-release] [@semantic-release/git] › ℹ  Found 1 file(s) to commit
2022-07-31T13:06:52.112Z semantic-release:git commited files: [ 'CHANGELOG.md' ]
[1:06:52 PM] [semantic-release] [@semantic-release/git] › ℹ  Prepared Git release: my-package-v1.1.2
[1:06:53 PM] [semantic-release] › ✔  Created tag my-package-v1.1.2
[1:06:53 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/npm"
[1:06:53 PM] [semantic-release] [@semantic-release/npm] › ℹ  Publishing version 1.1.2 to npm registry on dist-tag latest
npm ERR! code E403
npm ERR! 403 403 Forbidden - PUT https://registry.npmjs.org/my-package - You cannot publish over the previously published versions: 0.0.0-dev.1.
  shortMessage: 'Command failed with exit code 1: npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/',
  command: 'npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/',
  escapedCommand: 'npm publish "/root/packages/my-package/dist" --userconfig "/tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc" --tag latest --registry "https://registry.npmjs.org/"',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: 'npm ERR! code E403\n' +
    'npm ERR! 403 403 Forbidden - PUT https://registry.npmjs.org/my-package - You cannot publish over the previously published versions: 0.0.0-dev.1.\n' +
    'npm ERR! 403 In most cases, you or one of your dependencies are requesting\n' +
    'npm ERR! 403 a package version that is forbidden by your security policy, or\n' +
    'npm ERR! 403 on a server you do not have access to.\n' +
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false,
  pluginName: '@semantic-release/npm'
}
Error: Command failed with exit code 1: npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/
##[error]Process completed with exit code 1.

As you can see from logs version 1.1.2 is correctly written to package.json in dist folder, but following command is failing

npm publish "/root/packages/my-package/dist" --userconfig "/tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc" --tag latest --registry "https://registry.npmjs.org/"

It's trying to publish 0.0.0-dev.1 which is in package.json in /root/packages/my-package (equals to process.cwd())

But if I execute this command locally, it works without errors.

Corresponding npm publish command:

const result = execa(
  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd, env, preferLocal: true}
);

Looks like that execa has issues with cwd or preferLocal (or its combination).
Update: Some additional research results here in https://github.com/semantic-release/npm/issues/504#issuecomment-1201540336

@NullVoxPopuli
Copy link

NullVoxPopuli commented Jul 31, 2022

I just ran in to this as well.

my stack:

   at makeError (<repo>/node_modules/.pnpm/[email protected]/node_modules/execa/lib/error.js:60:11)
    at handlePromise (<repo>/node_modules/.pnpm/[email protected]/node_modules/execa/index.js:118:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async module.exports (<repo>/node_modules/.pnpm/@[email protected][email protected]/node_modules/@semantic-release/npm/lib/prepare.js:21:3)}

This fails when running not through execa as well:

❯ npm version 6.1.2 --userconfig /tmp/28e1f8c78e58a4699a9d64a8774e2e50/.npmrc --no-git-tag-version --allow-same-version
ember-statechart-component
v6.1.2
npm ERR! Cannot read properties of null (reading 'matches')

execa is on version 6 something -- could it be relateD?

@travi
Copy link
Member

travi commented Aug 1, 2022

execa is on version 6 something -- could it be relateD?

the range defined for this project's dependency on execa does not allow v6, so something is misconfigured in your project. you will need to resolve that first before further investigation could be done within the context of semantic-release

@unlight
Copy link
Author

unlight commented Aug 1, 2022

I made some research.
I patched node_modules/@semantic-release/npm/lib/publish.js

const command = `npm publish ${basePath} --userconfig ${npmrc} --tag ${distTag} --registry ${registry}`;
require('child_process').execSync(command);

Same result, it maybe issue with npm itself (my version is 8.11.0), also I do no see any related bug fixes in changelog of execa (after v5)

I've tried another trick:

// npm publish --userconfig /tmp/xxx/.npmrc --tag latest --registry http://localhost:4873/ # {cwd: /root/packages/my-package/dist}
const result = execa(  'npm',
   ['publish', '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}
 );

And it works!

So it definitly something wrong with npm and npm publish <folder> command, npm publish for current folder works ok.

@unlight unlight changed the title npm publish through execa fails npm publish through execa fails / npm publish <folder> publishes from cwd folder Aug 1, 2022
@travi
Copy link
Member

travi commented Aug 1, 2022

But if I execute this command locally, it works without errors.

in your local test, did you execute directly from your dist directory, or with the same command that semantic-release is running and from the effective cwd?

i'm wondering if what you are describing is either a breaking change in npm or a bug that should be reported to them.

my version is 8.11.0

the npm plugin for semantic-release defines a direct dependency on npm, so the version installed because of that would have more impact on this situation than your globally installed version. you can determine which version is locally installed with the command npm ls npm. if you are using our recommendation of executing with npx instead of installing locally to your project, that would mean that the latest version within the defined range will be used.

@unlight
Copy link
Author

unlight commented Aug 1, 2022

But if I execute this command locally, it works without errors.

in your local test, did you execute directly from your dist directory, or with the same command that semantic-release is running and from the effective cwd?

same command that semantic-release is running and from the effective cwd

my version is 8.11.0

Yes, sorry, it's my global version. npm ls npm shows [email protected]

@unlight
Copy link
Author

unlight commented Aug 1, 2022

i'm wondering if what you are describing is either a breaking change in npm or a bug that should be reported to them.

In 1st comment I described behavior of underlying npm ([email protected])
In research global npm was used.
Same results.

@Badisi
Copy link

Badisi commented Oct 25, 2022

Just got the exact same issue yesterday and took me the whole day to figure it out.

This has nothing to do with execa and is more related to an npm issue while using workspaces.
You can see the detailed explanation and reproducible repo here : npm/cli#5745.

Nonetheless, this can be fixed in @semantic-release/npm by replacing cwd with cwd: basePath here: publish.js#L26.
As I don't see why the publish command should be executed from a different cwd anyway...

@unlight unlight changed the title npm publish through execa fails / npm publish <folder> publishes from cwd folder npm publish fails / npm publish <folder> publishes from cwd folder Oct 25, 2022
@Badisi
Copy link

Badisi commented Nov 1, 2022

@travi, would you accept a PR to fix that ? (ie. replace cwd with cwd: basePath)

@travi
Copy link
Member

travi commented Nov 4, 2022

As I don't see why the publish command should be executed from a different cwd anyway...

are you considering the pkgRoot option in that statement?

This has nothing to do with execa and is more related to an npm issue while using workspaces.

@unlight @NullVoxPopuli are you using npm workspaces in the projects where you encountered this situation?

@NullVoxPopuli
Copy link

pnpm workspaces, but ya

@Badisi
Copy link

Badisi commented Nov 6, 2022

@travi, yes I'm considering it.

What I meant is:

Looking at the code here (publish.js#L26):

const basePath = pkgRoot ? path.resolve(cwd, pkgRoot) : cwd;
[...]
const result = execa(
  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd, env, preferLocal: true}
);

We can see that :

  • basePath is the final dist location (nb: it is taking pkgRoot into account)
  • execa is executed from cwd
  • npm publish command is given a basePath to publish

But why would execa be executed from a different path than basePath ?


A fix to this issue (also suggested by @unlight) would be to make sure that execa is executed directly from the dist path.

Something like:

  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}

or simply:

  'npm',
  ['publish', '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}

Badisi added a commit to Badisi/npm that referenced this issue Dec 3, 2022
@Badisi
Copy link

Badisi commented Dec 3, 2022

As it is a blocking issue for me, I have made a PR to fix it.

@travi, I would be very pleased if you could approve it. Thanks 🙏

@donaldxdonald
Copy link

any progress? 💦

@Badisi
Copy link

Badisi commented Apr 12, 2023

For anyone interested

As no action were taken here and it was blocking for me, I'm using this temporary fix :

package.json#L28

{
  "scripts": {
    "postinstall": "node patch-semantic-release-npm.js"
  }
}

patch-semantic-release-npm.js#L1-L24

/**
 *  TODO: this whole patch can be removed when https://github.com/semantic-release/npm/pull/531 is fixed
 */

const { readFileSync, writeFileSync, existsSync } = require('fs-extra');
const { green, red, gray } = require('@colors/colors/safe');
const { dirname, join } = require('path');

const filePath = join(dirname(require.resolve('@semantic-release/npm')), '/lib/publish.js');
if (existsSync(filePath)) {
  let data = readFileSync(filePath, { encoding: 'utf8' });

  if (!data.match(/cwd: basePath/gm)) {
    data = data.replace("'publish', basePath,", "'publish',");
    data = data.replace("cwd, env,", "cwd: basePath, env,");
    writeFileSync(filePath, data, { encoding: 'utf8' });
    console.log(green('success'), '@semantic-release/npm patched.');
  } else {
    console.log(green('success'), '@semantic-release/npm already patched.');
  }
} else {
  console.error(red('error'), 'cannot patch @semantic-release/npm.');
  console.error(gray(`"${filePath}" not found`));
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants