New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node WorkerChannel native addon #1394

Draft

jmillan wants to merge 82 commits into v3 from node-worker-channel

Member

jmillan commented May 9, 2024 •

edited

Loading

Substitute spawning a process with mediasoup for running mediasoup in a thread created by this addon.

The communication is now done via message passing rather than using unixsocket, avoiding the kernel networking stack for each and every message passed from Node to C++ and viceversa.

Performance gains are enhanced by approximately 8 times. Tested by sending a request from Node to C++ and waiting for the response N times in a loop.

Caveats:

Since we don't use a separate process, a crash in mediasoup would crash Node, meaning it must be handled by node process signal handling.
- At the same time, the coredumps will be named something-node-something
- In order to debug coredumps the worker-channel.node binary can be used along with the coredump file and gdb/lldb

TODO:

Document removed 'died' and 'subprocessclose' Worker.ts events.
Document removed 'died()' and 'subprocessClosed()' Worker.ts methods.
Missing all the prebuilt binary upload/download stuff.

jmillan added 2 commits

May 9, 2024 18:07


WIP: Node WorkerChannel native addon

9a9409d

Substitute spawning a process with mediasoup for running mediasoup in a
thread created by this addon.

The communication is now done via message passing rather than using
unixsocket, avoiding the kernel networking stack for each and every
message passed from Node to C++ and viceversa.

Performance gains are enhanced by approximately 8 times. Tested by
sending a request from Node to C++ and waiting for the response N times
in a loop.


remove worker-channel from dependencies in package.json

1e59102

ibc reviewed

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

Member

ibc commented May 9, 2024

.prettierignore and .eslintignore should include /node/workerChannel/lib.

ibc reviewed

View reviewed changes

node/src/Worker.ts Show resolved Hide resolved

ibc reviewed

View reviewed changes

node/src/Worker.ts Show resolved Hide resolved

ibc reviewed

View reviewed changes

node/src/Worker.ts Show resolved Hide resolved

ibc reviewed

View reviewed changes

node/workerChannel/src/workerChannel.hpp Outdated Show resolved Hide resolved

ibc reviewed

View reviewed changes

node/workerChannel/src/workerChannel.hpp Outdated Show resolved Hide resolved

ibc reviewed

View reviewed changes

npm-scripts.mjs Outdated Show resolved Hide resolved

ibc reviewed

View reviewed changes

npm-scripts.mjs Outdated Show resolved Hide resolved

ibc reviewed

View reviewed changes

package.json Outdated Show resolved Hide resolved

jmillan added 8 commits

May 10, 2024 10:09


provide version

ee18617


add workerChannels to .eslintignore and .prettierignore

2c80482


move header file to include folder

f925bfd


remove blank line

18971e0


package.json: remove worker:build-lib script

02d95d8


install workerChannel via 'npm ci'

f6c222b


properly build Debug/Release build types

77a67b4


lint

23df194

Member

ibc commented May 10, 2024

Please merge v3 and check prettier and eslint path arrays in npm-scripts.mjs

jmillan added 9 commits

May 10, 2024 15:19


remove .eslintignore and .prettierignore

65ae95e


Merge remote-tracking branch 'origin/v3' into node-worker-channel

5d7ca09


add workerChannel to eslint and prettier

6ed5f88


fix name for libmediasoup-worker library in linux

fbe2120


add *.ts files inside src/ folder

6cec438


workerChannel: npm-scripts.ts

69bc91f


eslint

66261f5


npm-scripts: build worker lib before typescript


npm-scripts: remove typo

906d3ba

jmillan added 3 commits

May 20, 2024 12:42


do not use a separate TS or npm project for WorkerChannel

9c170da


lint

4037cab


npm-scripts: test, --forceExit

1b248e8

Node will clean the native addon once it garbage collects the
corresponding JS instance.

ibc reviewed

View reviewed changes

npm-scripts.mjs Outdated Show resolved Hide resolved

jmillan added 25 commits

May 21, 2024 10:54


workerChannel: add into node/src

f437881


WorkerChannel: .gitignore

d11d964


workflow: enable all SO

e22cace


lint

147b08d


workflow: fix tabs

a258ea6


clang-format

b892741


clang-format: typo

19d5242


test: remove --forceExit

738acea

assign JS instance of the addon to 'undefined' when closed, to indicate
the Node GC that it can be freed.

Otherwise the tests won't finish due to the correponsing handles being
opened.


.gitignore

0c15f42


worker: format


windows: set RuntimeLibrary for Debug and Release

85784ad


workflow: enable all OS in mediasoup-node

b3044ad


tests: add comment of addon tests

e80a59e


WIP: prebuild

c3e5d67


lint

95d9cd0


WIP: prebuilt

cbf9b30


WIP: prebuilt

f83ca80


WIP: prebuilt

376bdcf


prebuilt working

b559e4d


comment

9f24c50


workerChannel: Do not copy the addon binary into node/lib

a003c63


package.json: files, add workerChannel files

9fd5dfb


.gitignore

c82c605


cleanup

e469bf2


package.json: engines, set a minimum npm version

d95db9d

Lower npm versions than 10.3.0 fail building the addon.

Error: No module named 'distutils'

jmillan changed the title ~~WIP: Node WorkerChannel native addon~~ Node WorkerChannel native addon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment