Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subpath imports in ESM mode #688

Open
stazz opened this issue Feb 27, 2023 · 5 comments
Open

Subpath imports in ESM mode #688

stazz opened this issue Feb 27, 2023 · 5 comments
Milestone

Comments

@stazz
Copy link

stazz commented Feb 27, 2023

🐛 Bug report

Please notice that this problem actually spans a wide range of -ts packages. At least the following are affected

  • fp-ts,
  • io-ts,
  • io-ts-types,
  • monocle-ts, and
  • newtype-ts

Current Behavior

If using the following subpath import:

import * as errorReport from "io-ts/PathReporter";

And then running using ts-node with --esm flag, or ESM-based bundler (e.g. Vite), while things look OK in IDE, at runtime, there will be error:

xyz/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:352
     throw new ERR_MODULE_NOT_FOUND(
           ^
 CustomError: Cannot find module '/xyz/node_modules/io-ts/PathReporter' imported from /xyz/my-file.ts

This happens because there is no matching entry for PathReporter subpath in package.json file of io-ts package.

Note that everything works fine when in CJS mode.

Expected behavior

I expect subpath imports of io-ts behave successfully at both compile- as well as runtime, both with CJS modules and ESM modules.
Without any additional setup.

Reproducible example

Create package.json with io-ts dependency:

{
  "name":  "io-ts-bug-repro",
  "private": true,
  "version": "1.0.0",
  "dependencies": {
    "fp-ts": "2.13.1",
    "io-ts": "2.2.20"
  },
  "devDependencies": {
    "ts-node": "10.9.1",
    "typescript": "4.9.5"
  }
}

Add basic TS configuration to tsconfig.json:

{
  "compilerOptions": {
    "strict": true,
    "module": "es2022",
    "moduleResolution": "node",
    "lib": [
      "ES2022"
    ],
    "target": "ES2022"
  },
  "ts-node": {
    "esm": true,
    "experimentalSpecifierResolution": "node",
  }
}

Create file src/index.ts with the following code:

import * as errorReport from "io-ts/PathReporter";

const testing = errorReport.success();

and try to execute it:

ts-node src

Suggested solution(s)

It looks like a whole family of -ts packages should be migrated to ESM era. The current setup is extremely weird, lacking "type": "module" entry from package.json files, and duplicating .d.ts files, which also use /lib or /es6 imports interchangeably (sometimes ES6 things including from xyz-ts/lib/abc). This results subpath imports for -ts packages being completely unuseable in ESM.

There are I guess many approaches to solve this. I think it is important to retain CJS functionality still, but also enable everything work for ESM without additional setup. I personally flavor this pattern, which I found to be working for all my packages (note that order of "types", "import", and "require" in the file is meaningful!). I keep .d.ts files in their own dist-ts folder, all the CJS-flavored .js files in their own dist-cjs folder, and finally all ESM-flavored .js files in their own dist-esm folder. I ended up with this setup after long experiments with subpath imports, both outside of module, and within.

{
  "name": "package-usable-by-both-cjs-and-esm",
  "type": "module",
  "main": "./dist-cjs/index.js",
  "module": "./dist-esm/index.js",
  "types": "./dist-ts/index.d.ts",
  "exports": {
    ".": {
      "types": "./dist-ts/index.d.ts",
      "import": "./dist-esm/index.js",
      "require": "./dist-cjs/index.js"
    }
  }
}

If I want to enable subpath export for e.g. my-subpath, I need to first modify the exports in package.json:

{
  "exports": {
    ...,
    "./my-subpath": {
        "types": "./dist-ts/my-subpath.d.ts",
        "import": "./dist-esm/my-subpath.js",
        "require": "./dist-cjs/my-subpath.js"
    }
  }
}

And then create the following stub package file to ./my-subpath/package.json path within distributed NPM package (this is only for CJS support):

{
  "type": "module",
  "main": "../dist-cjs/my-subpath.js",
  "module": "../dist-esm/my-subpath.js",
  "types": "../dist-ts/my-subpath.d.ts"
}

Technically, only main property is needed, but the others are just in case.

This setup allows the same subpath imports always to work, no matter whether they are in TS, CJS, or ESM mode. Furthermore, it adhers to DRY principle, so that it doesn't duplicate .d.ts files.

Additional context

Your environment

  • Which versions of io-ts are affected by this issue?
    • All of them, AFAIK.
  • Did this work in previous versions of io-ts?
    • No.
Software Version(s)
io-ts 2.2.20
fp-ts 2.13.1
TypeScript 4.9.5
@stazz
Copy link
Author

stazz commented Feb 27, 2023

FWIW, I got my own application to work after I did modifications summarized by this gist. The rabbithole went pretty deep.

I hope this will be useful for anyone stumbling upon this until ESM mode starts working properly for -ts packages.

Notice that my usecase might be different from yours and there might be additional setup for those exports fields mentioned in the gist to make things work.

@stazz stazz changed the title Relative imports in ESM mode Subpath imports in ESM mode Feb 27, 2023
@stazz
Copy link
Author

stazz commented Feb 27, 2023

Here are the scripts that I use to produce the file layout described in suggested solution part (entrypoint is build:ci script):

{
  "scripts": {
    "build:ci": "yarn run clear-build-artifacts && yarn run compile-d-ts-files && yarn run tsc --outDir ./dist-esm && yarn run tsc --module CommonJS --outDir ./dist-cjs && yarn run format-output-files && yarn run fix-subpath-exports",
    "clear-build-artifacts": "rm -rf dist dist-ts dist-cjs dist-esm build",
    "compile-d-ts-files": "yarn run tsc --removeComments false --emitDeclarationOnly --declaration --declarationDir ./dist-ts && yarn run copy-d-ts-files && yarn run tsc:plain --project tsconfig.out.json",
    "copy-d-ts-files": "find ./src -mindepth 1 -maxdepth 1 -name '*.d.ts' -exec cp {} ./dist-ts +",
    "fix-subpath-exports": "mkdir <SUBPATH> && cp package.json.<SUBPATH> <SUBPATH>/package.json",
    "format-output-files": "find dist-ts -name '*.ts' -type f -exec sh -c \"echo '/* eslint-disable */\n/* eslint-enable prettier/prettier */'\"' | cat - $1 > $1.tmp && mv $1.tmp $1' -- {} \\; && eslint --no-eslintrc --config '.eslintrc.output.ts.cjs' --fix './dist-ts/**/*.ts' && eslint --no-eslintrc --config '.eslintrc.output.cjs' --fix 'dist-cjs/*js' 'dist-esm/*js'",
    "lint": "eslint ./src --ext .ts,.tsx",
    "tsc": "tsc --project tsconfig.build.json",
    "tsc:plain": "tsc",
  }
}

Notice that when dealing with large number of supported subpath exports, like all -ts packages seem to do, one probably wants fix-subpath-exports script to be more generic. One option would be to read subpaths from file, and then iterate each line with for loop, and create directory + copy/generate the stub package.json file.

Here are the TSConfig files used by script:

// tsconfig.out.json
{
  // TS config file for formatting the resulting .d.ts files (<project name>/dist-ts/**/*.d.ts) that end up in NPM package for typing information.
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "rootDir": "./dist-ts",
    "noEmit": true
  },
  "include": [
    "./dist-ts/**/*"
  ]
}
// tsconfig.build.json
{
  // TS config file to use to compile <project name>/src/**/*.ts files in CI.
  "extends": "./tsconfig.json",
  "compilerOptions": {
    // We don't want dangling // eslint-disable-xyz comments, as that will cause errors during formatting output .[m]js files.
    "removeComments": true
  },
}

And then the normal tsconfig.json file as your usual business. The motivation for these is that we would still have nicely auto-formatted (by ESLint, I personally use the Prettier plugin) .d.ts and .js files, instead of barely readable garbage produced by TS compiler and such.

P.S. No need to take this whole process into use. Just pick the parts which you think suits best to whatever approach you are using to build & publish packages. To be fair, I am happy with anything that makes subpath imports work in ESM mode. :)

@gcanti gcanti added this to the 3.0 milestone Feb 27, 2023
@stazz
Copy link
Author

stazz commented Feb 28, 2023

@gcanti I took a look at the package.json of this project, and I think I could contribute a PR related to this issue. Would you like me to do that, or are you handling this via some internal ideas/pipelines?

@gcanti
Copy link
Owner

gcanti commented Feb 28, 2023

I'm working on a new iteration of io-ts https://github.com/fp-ts/schema that will be soon included in the @effect org.
As soon as the new packages will be stable I will backport what's possible to the fp-ts ecosystem.
After that I will consider to release major versions of many packages, io-ts included, which will support esm.

@stazz
Copy link
Author

stazz commented Feb 28, 2023

Oh, I see, really interesting to hear about the new iteration, as well as it being included to @effect umbrella! Will check it out for sure. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants