Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Could not read from file …" error thrown when using ?url and ?worker suffixes inside 3rd party module #10838

Open
7 tasks done
wojtekmaj opened this issue Nov 8, 2022 · 4 comments · May be fixed by #16418
Open
7 tasks done

"Could not read from file …" error thrown when using ?url and ?worker suffixes inside 3rd party module #10838

wojtekmaj opened this issue Nov 8, 2022 · 4 comments · May be fixed by #16418
Labels
pending triage

Comments

@wojtekmaj
Copy link

wojtekmaj commented Nov 8, 2022 •
edited

Describe the bug

Getting module URL in our own code like so:

import module2Url from "module-2/index?url";

works fine, but if that very same line is placed in node_modules/module-1/index.js, build crashes:

wmaj@MacBook-Pro vite-url-suffix-bug % yarn dev          
Port 5173 is in use, trying another one...

  VITE v3.2.3  ready in 83 ms

  ➜  Local:   http://localhost:5174/
  ➜  Network: use --host to expose
✘ [ERROR] Could not read from file: /vite-url-suffix-bug/node_modules/module-2/index.js?url

    node_modules/module-1/index.js:1:23:
      1 │ import module2Url from "module-2/index?url";
        ╵                        ~~~~~~~~~~~~~~~~~~~~

/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1566
  let error = new Error(`${text}${summary}`);
              ^

Error: Build failed with 1 error:
node_modules/module-1/index.js:1:23: ERROR: Could not read from file: /vite-url-suffix-bug/node_modules/module-2/index.js?url
    at failureErrorWithLog (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1566:15)
    at /vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1024:28
    at runOnEndCallbacks (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1438:61)
    at buildResponseToResult (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1022:7)
    at /vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1134:14
    at responseCallbacks.<computed> (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:671:9)
    at handleIncomingPacket (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:726:9)
    at Socket.readFromStdout (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:647:7)
    at Socket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12) {
  errors: [
    {
      detail: undefined,
      id: '',
      location: {
        column: 23,
        file: 'node_modules/module-1/index.js',
        length: 20,
        line: 1,
        lineText: 'import module2Url from "module-2/index?url";',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'Could not read from file: /vite-url-suffix-bug/node_modules/module-2/index.js?url'
    }
  ],
  warnings: []
}

Node.js v18.12.0

At the same time, I was not able to find any mention regarding these suffixed working only on limited subset of modules when reading the following page: https://vitejs.dev/guide/assets.html

Reproduction

https://github.com/wojtekmaj/vite-url-suffix-bug

Steps to reproduce

  • yarn
  • yarn copy-packages (to replace symlinks in node_modules with actual modules, since they come from internal workspace)
  • yarn dev
  • 💥

System Info

System:
    OS: macOS 13.0
    CPU: (8) arm64 Apple M1 Pro
    Memory: 457.75 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.12.0 - /usr/local/bin/node
    Yarn: 3.2.4 - /usr/local/bin/yarn
    npm: 8.19.2 - /usr/local/bin/npm
  Browsers:
    Chrome: 107.0.5304.87
    Firefox: 106.0.3
    Safari: 16.1
  npmPackages:
    vite: ^3.2.3 => 3.2.3

Used Package Manager

yarn

Logs

No response

Validations
@wojtekmaj wojtekmaj changed the title Unable to use ?url and ?worker suffixes inside 3rd party module "Could not read from file …" error thrown when using ?url and ?worker suffixes inside 3rd party module Nov 8, 2022
@wojtekmaj wojtekmaj mentioned this issue Nov 9, 2022
Closed
7 tasks
@vinodnimbalkar vinodnimbalkar mentioned this issue Feb 26, 2023
Closed
@seo-rii
Copy link

seo-rii commented Nov 16, 2023

Is there any update about this?

@TheMoonDawg
Copy link

In my case, I was updating to React PDF v7 and ran into this issue. What I ended up doing was including the transitive dependency / 3rd party module in our Vite config to also optimize it when using the ?url suffix. Worked like a charm!

export default defineConfig({
  ...
  optimizeDeps: { include: ['pdfjs-dist'] },
})

@Danielku15
Copy link

I dived a bit into the bug and the problem is in this line:

vite/packages/vite/src/node/plugins/workerImportMetaUrl.ts

Line 154 in e526573

file = tryFsResolve(file, fsResolveOptions) ?? file

If the file system resolve fails here, we just stay on the artificially produced path like: 'D:\Dev\myproj\src\vite-react\node_modules\.vite\deps\mylib.worker.mjs'. This leads to two problems:

  1. The worker file is considered an "optimized dependency" because its file path starts with .vite\deps

    vite/packages/vite/src/node/optimizer/index.ts

    Lines 912 to 921 in f8e0791

    function getDepsCacheDirPrefix(config: ResolvedConfig): string {
    return normalizePath(path.resolve(config.cacheDir, 'deps'))
    }
    export function createIsOptimizedDepFile(
    config: ResolvedConfig,
    ): (id: string) => boolean {
    const depsCacheDirPrefix = getDepsCacheDirPrefix(config)
    return (id) => id.startsWith(depsCacheDirPrefix)
    }
  2. This path is then encoded and decoded to URLs and used by modules but will end up with files which cannot be found.

Hence a fix would mean: resolve at this location the correct path to the worker file of the 3rd party module in the node_modules directory.

It seems a code bit like this will work:

file = tryFsResolve(file, fsResolveOptions) ?? tryOptimizedDepResolve(url, id) ?? file;


function tryOptimizedDepResolve(url, depId, fsResolveOptions) {
	if (depsOptimizer?.isOptimizedDepFile(depId)) {
		const depFile = cleanUrl(depId);
		const info = optimizedDepInfoFromFile(depsOptimizer.metadata, depFile);
		const depSrc = info.src;
		if (depSrc) {
			const resolvedFile = path.resolve(path.dirname(depSrc), url)
            return tryFsResolve(file, fsResolveOptions);
		}
	}
	
	return undefined;
}

It is a similar flow like the existing tryOptimizedResolve but simplified.

I'll check if I can make a change to Vite and test it locally.

Danielku15 added a commit to Danielku15/vite that referenced this issue Apr 13, 2024
@Danielku15
fix: broken worker urls in 3rd party modules (fix vitejs#10838)
89ce8d8
@Danielku15 Danielku15 linked a pull request Apr 13, 2024 that will close this issue
Open
5 tasks
@Danielku15
Copy link

I prepared a fix which works nicely for my library at #16418
Would be great if some more people could give it a test. To test it locally:

  1. Clone and built vite from https://github.com/Danielku15/vite/tree/fix/workers-in-3rd-party-modules (pnpm install && pnpm build)
  2. Copy the local vite build to your project where you use vite with a 3rd party library.
  3. Run and test your application and check if the worker loads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pending triage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: broken worker urls in 3rd party modules (fix #10838) Danielku15/vite
4 participants
@Danielku15 @wojtekmaj @TheMoonDawg @seo-rii