Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webpack module federation breaks with multiple entrypoints #2692

Open
1 task done
codepunkt opened this issue Aug 3, 2020 · 90 comments
Open
1 task done

Webpack module federation breaks with multiple entrypoints #2692

codepunkt opened this issue Aug 3, 2020 · 90 comments

Comments

@codepunkt
Copy link

This time it's @sokra who told me this is probably the place to open an issue.

  • Operating System: Windows 10
  • Node Version: 12.16.2
  • NPM Version: 6.14.4
  • webpack Version: 5.0.0-beta.22
  • webpack-dev-server Version: 3.11.0
  • Browser: Chrome 84.0.4147.105, Firefox 79.0, Edge 84.0.522.48
  • This is a bug

Code

https://github.com/codepunkt/module-federation-examples/tree/dynamic-host-remote

Expected Behavior

Using webpack-dev-server instead of webpack should still support module federation with additional content tacked onto the remoteEntry by defining it as an additional entry.

Actual Behavior

Running "yarn start" to start webpack-dev-server breaks module federation and thus breaks the app in development mode.

For Bugs; How can we reproduce the behavior?

  1. clone repository
  2. ensure you're on branch "dynamic-host-remote"
  3. run yarn on repo root
  4. go into "dynamic-host-remote" directory
  5. run yarn start in "dynamic-host-remote" directory
  6. open localhost:3001 in the browser
  7. encounter an error in the browser console that happens when executing app2/remoteEntry.js with the additional contents that were added to this entrypoint by webpack-dev-server
  8. OPTIONAL: run yarn build && yarn serve and revisit localhost:3001 to see production build working just fine.
@alexander-akait
Copy link
Member

alexander-akait commented Aug 3, 2020

@webpack-bot move to webpack/webpack-dev-server My mistake

@alexander-akait
Copy link
Member

Feel free to send a fix

@codepunkt
Copy link
Author

If I had a clue what the problem is, i'd send a PR right away. Spent a few hours with this, don't have the slightest idea 🤔

@alexander-akait
Copy link
Member

/cc @sokra Any ideas on this?

@ScriptedAlchemy
Copy link
Member

ScriptedAlchemy commented Aug 4, 2020

Okay, so the issue here is that WDS is likely appending its own entrypoint to the array / injecting a "require" to the hmr client. That typically serves as the entry module of an application. When ModuleFederation emits a remoteEnty with the same name as the entrypoint, they are merged.

This lets us attach initialization code, to federated remotes. something like changing publicPath.

webpack_public_path = new URL(document.currentScript.src).origin + "/";

module-federation/module-federation-examples#277

image

The remote becomes 400kb big (usually like 5kb) and the module returned to the global is not __webpack_require__("webpack/container/entry/app1"); - therefore window.app1 will not be __webpack_require__("webpack/container/entry/app1"); and the federated API is never set to the global

@sokra i believe that either the WDS entry needs to return the container - which may lead to HMR capabilities? or the WDS entry module should not be applied to federated entry points when combined.

module.exports = {
  entry: {
    app1: "./src/setPublicPath",
    main: "./src/index",
  },
plugins: [
    new ModuleFederationPlugin({
      name: "app1",
      remotes: {
        app2: "app2@http://localhost:3002/remoteEntry.js",
      },
      shared: { react: { singleton: true }, "react-dom": { singleton: true } },
    }),
    new HtmlWebpackPlugin({
      template: "./public/index.html",
    }),
  ],

@sokra
Copy link
Member

sokra commented Aug 4, 2020

The wds entrypoint should be prepended instead of appended. The last value is exported.

@ScriptedAlchemy
Copy link
Member

Will take a look, thanks @sokra

@ScriptedAlchemy
Copy link
Member

Looks like we want to modify this

 const prependEntry = (originalEntry, additionalEntries) => {
      if (typeof originalEntry === 'function') {
        return () =>
          Promise.resolve(originalEntry()).then((entry) =>
            prependEntry(entry, additionalEntries)
          );
      }

      if (typeof originalEntry === 'object' && !Array.isArray(originalEntry)) {
        /** @type {Object<string,string>} */
        const clone = {};

        Object.keys(originalEntry).forEach((key) => {
          // entry[key] should be a string here
          const entryDescription = originalEntry[key];
          if (typeof entryDescription === 'object' && entryDescription.import) {
            clone[key] = Object.assign({}, entryDescription, {
              import: prependEntry(entryDescription.import, additionalEntries),
            });
          } else {
            clone[key] = prependEntry(entryDescription, additionalEntries);
          }
        });

@sokra
Copy link
Member

sokra commented Aug 5, 2020

It already prepends... So there is somethings else wrong

@sokra
Copy link
Member

sokra commented Aug 6, 2020

The bug is here:

/** @type {Entry} */
const entriesClone = additionalEntries.slice(0);
[].concat(originalEntry).forEach((newEntry) => {
if (!entriesClone.includes(newEntry)) {
entriesClone.push(newEntry);
}
});

Desprite the function name prepend it actually appends...

Change it to:

      /** @type {Entry} */
      const entriesClone = [].concat(originalEntry);
      additionalEntries.forEach((newEntry) => {
        if (!entriesClone.includes(newEntry)) {
          entriesClone.push(newEntry);
        }
      });

Could somebody send a PR for that? As test case you can use output.library = "TEST" and check if this created the correct global.

@alexander-akait
Copy link
Member

Great, feel free to send a PR

@snitin315
Copy link
Member

I will send a PR.

@codepunkt
Copy link
Author

I'm not sure this will actually fix HMR with module federation - but i'm looking forward to try!

@ScriptedAlchemy
Copy link
Member

Opening PR, will need to work on / help on test case

ScriptedAlchemy added a commit to ScriptedAlchemy/webpack-dev-server that referenced this issue Aug 9, 2020
this resolves an issue with module federation

re webpack#2692
ScriptedAlchemy added a commit to ScriptedAlchemy/webpack-dev-server that referenced this issue Aug 10, 2020
this resolves an issue with module federation

re webpack#2692
@knagaitsev
Copy link
Collaborator

knagaitsev commented Aug 27, 2020

I looked into this more closely, and here is the full problem:

In most cases, the dev server is trying to add some variation of these 2 entries (unless you disable their injection): webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js.

Current behavior (webpack 5):

Our config has a single entry: main.js.

The 2 dev server entries are appended to the entry list (as explained above), resulting in something like:

main: {
  import: {
    'main.js',
    'webpack-dev-server/client/default/index.js',
    'webpack/hot/dev-server.js'
  }
}

Then we call the entryOption hook with all of these entries here:

compiler.hooks.entryOption.call(config.context, config.entry);

This results in an entry list on the webpack side that looks like this:
main.js, main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js

So it would seem that prepending the 2 entries would fix the issue:
main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js, main.js

However, it is not that simple because webpack essentially filters out the duplicate entries, taking only the first of the duplicates (https://github.com/webpack/webpack/blob/bdf4a2b942bed9d78815af828f7935ddfcd3d567/lib/Compilation.js#L1764), so we get: main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js (regardless of if we append or prepend in the webpack step). The very last in this list is currently the one that is exported, so in no case is it ever main.js while there are other entries being injected.

Thus, the solution is to either change how this filtering works, or change the selection of entries by webpack such that the very first entry from the entryOption hook is used, rather than the very last.

I think the latter is preferred, because I realize now that it is not ideal to be pushing duplicate entries into the entryOption hook (as we do currently). Instead, we should only be pushing webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js into the entryOption hook, as they have not yet been registered as entries. On the webpack side, these entries should not be considered as the module exports, as they were added later after the initial configuration entries.

@sokra
Copy link
Member

sokra commented Aug 28, 2020

Then we call the entryOption hook with all of these entries here:

compiler.hooks.entryOption.call(config.context, config.entry);

Why does webpack-dev-server do that? The hooks is not owned by webpack-dev-server. It should not call it. It's already called by webpack itself. That's probably also the reason why the entries are added twice...

Seems like this seem to be some kind of hack to reapply the entry option, because it has been modified after the options has been applied, which is too late. It doesn't make sense to modify the options after they have been applied to the compiler (converted to plugin). You need to use a plugin instead.

A plugin to add these entries for webpack-dev-server could be like that:

compiler.hooks.make.tapAsync({
    name: "webpack-dev-server",
    stage: -100
}, (compilation, callback) => {
    const options = {
        name: undefined // global entry, added before all entries
    };
    const dep = EntryPlugin.createDependency("webpack-dev-server/client/...", options);
    compilation.addEntry(context, dep, options, err => {
        callback(err);
    });
});

Note the hack works in webpack 4 as each entrypoint can only have a single module (arrays are wrapped in a artificial module), so reapplying the entryOption also adds it twice, but the second one overrides the first one.

There is no global entry in webpack 4, so the above plugin doesn't work for webpack 4. I would keep the hack as legacy code, as a clean solution is more complex.

@codepunkt
Copy link
Author

Thanks everyone for chiming in.

What confuses me a little about this is that webpack-dev-server, which i always thought of as a "major part of webpack", actually seems to be an afterthought when it comes to new developments.

I don't understand a lot of the details everyone wrote in this issue, but i'm glad this problem is being ironed out and fixed.

@tianyingchun
Copy link

could you give suggestions?

@alexander-akait
Copy link
Member

No idea, maybe non official plugin break, hash should not changed on initial run, please create reproducible test repo

@tianyingchun
Copy link

ok i have created demo repo for reproduce this issue.

@alexander-akait
Copy link
Member

Great, can you provide a link?

@tianyingchun
Copy link

tianyingchun commented Aug 19, 2021

wait me moment, it seem that i can't commit github now, authentication failed.
https://github.com/tianyingchun/webpack-dev-server-demo-2692

@tianyingchun
Copy link

tianyingchun commented Aug 19, 2021

i have updated repo demo here please check it .
https://github.com/tianyingchun/webpack-dev-server-demo-2692

  1. npm install
  2. node serve.js
  3. open url: http://localhost:5000

you can see the page infinite loop reload now

@alexander-akait
Copy link
Member

Thanks, I will look at this in near time

@alexander-akait
Copy link
Member

oh, you have multiple hmr with different hashes, it is weird usage, to be honestly, ideally you should run dev server on each compiler

@alexander-akait
Copy link
Member

we plan for [email protected] implement plugin support so you will put plugins: [new DevServer(options)] for the each compiler

@alexander-akait
Copy link
Member

Or use multiple entries...

@alexander-akait
Copy link
Member

i.e. you have only one hmr code and web socket connection, but handle them different compilers (so web socket gets hashes from first and from second compilers), it is not safe generally, before it works only because we have bug in runtime logic, after fix it you got the problem

@alexander-akait
Copy link
Member

Can you clarify why do you use single dev server for multi compiler mode? What is use case? Why do not run two dev servers?

@tianyingchun
Copy link

yes, we have huge complex project with multiple modules in one repo, we setup configurations to allow us start serve for these individual modules one or more ,because these modules is individual SPA application;

  1. we want to startup one server with the same host && port like http://localhost:5000/
  2. we attached customized middleware in dev Server pipeline /pages/*
  3. we don't want to start serve for each module with different dev server, because it has different access url.
  4. also has more reason, because webpack it has support multi compilers, i think dev-server should support we pass multi compilers with hot

@tianyingchun
Copy link

we plan for [email protected] implement plugin support so you will put plugins: [new DevServer(options)] for the each compiler

for this, do dev-server start only one or multi dev server for multi modules?

@alexander-akait
Copy link
Member

also has more reason, because webpack it has support multi compilers, i think dev-server should support we pass multi compilers with hot

Sorry, it is impossible by design, you can't have the same HMR code for multi compilers, it is invalid and weird, before you don't have bug due our invalid logic for initial, in fact you should have gotten this problem much earlier, we support multi compiler mode, but your case has multiple independent builds with the same HMR, even more new features like lazyCompilation, buildless mode (future) will not work too

@alexander-akait
Copy link
Member

alexander-akait commented Aug 20, 2021

Even more, you have only one web socket connection, therefore hash from one build will be already rewrite hash from other build, in theory we can pass unique name of compiler and implement multiple statuses, but it is more work then you think, also many plugins (for dev server) can break your HMR

@alexander-akait
Copy link
Member

yes, we have huge complex project with multiple modules in one repo, we setup configurations to allow us start serve for these individual modules one or more ,because these modules is individual SPA application;

in this case you should use multi entry mode (it will optimize your code even better) or module federation

@tianyingchun
Copy link

ok, i have rewrite my codebase to startup multi dev-server with multi port and dynamic organize them into express server page proxy.

BTW it seems that we can't disabled logger.info() message via dev-server config, or webpack stats.logging= false.

@tianyingchun
Copy link

And does webpack-dev-server provider @types for v4.0.0?

@snitin315
Copy link
Member

And does webpack-dev-server provider @types for v4.0.0?

No, but it is in our roadmap.

@alexander-akait
Copy link
Member

@tianyingchun for logger we have the special option https://webpack.js.org/configuration/other-options/#level, we decide don't have a lot of different logger options as were before and focus only on one

@IgorStetsiuk
Copy link

IgorStetsiuk commented Feb 9, 2022

This time I'll be that guy... do we have any updates on this?

Have one question regarding Hot Module Replacement. Seems it doesn’t work properly. I used CRA with react-scripts v 5.0.0 and CRACO to override the webpack config in order to set up my module federation and that’s everything is fine (build, test, start commands)… Thought when it comes to the local development I need manually hard reload pages to reflect changes, and it’s really inconvenient.

I'm thinking to build React app from a scratch and playing around with webpack config but that is not what I wanted to do as we already have a pretty well config provided by CRA only need to override it for Module Federation config is that correct?

Does somebody know whether CRA (latest version 5) is able to work with hot module reload? What configuration do I miss?
Thank you!

@ScriptedAlchemy
Copy link
Member

This time I'll be that guy... do we have any updates on this?

Have one question regarding Hot Module Replacement. Seems it doesn’t work properly. I used CRA with react-scripts v 5.0.0 and CRACO to override the webpack config in order to set up my module federation and that’s everything is fine (build, test, start commands)… Thought when it comes to the local development I need manually hard reload pages to reflect changes, and it’s really inconvenient.

I'm thinking to build React app from a scratch and playing around with webpack config but that is not what I wanted to do as we already have a pretty well config provided by CRA only need to override it for Module Federation config is that correct?

Does somebody know whether CRA (latest version 5) is able to work with hot module reload? What configuration do I miss?

Thank you!

This is a very hard problem to solve. An easy workaround I use is I'll fetch all remote urls on a poll and hash them. If any hash based on the contents changes, I window reload the page. Simple but easy way to get live reloading.

HMR is not easy since webpack needs to know both where something exists and where it's consumed. All graphs would need to be interlinked

@IgorStetsiuk
Copy link

@ScriptedAlchemy Hello! Thank you for your reply. Wouldn't you mind sharing a piece of that config where you "fetch all remote URLs on a poll and hash them"? I'm not super aware of how to set up such config in webpack and it sounds a bit difficult to me to implement that so maybe some samples would be really helpful to me to understand that configuration and make live reloading in my case. Thank you again!

@ScriptedAlchemy
Copy link
Member

Fetch(jsUrl).then(text).then(md5hashFronString)

Store it on some object and put the fetch in a setInterval.

@xujie-phper
Copy link

any updates, i face the same issue when use module federation

@rdenman
Copy link

rdenman commented Apr 12, 2023

Since this issue is still open, I'll link to my (pretty damn hacky) "solution" for HMR I posted in another thread: webpack/webpack#11240 (comment)

It's very far from perfect, but it does the job in a pinch

@ssuvorov
Copy link

20 October 2023 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests