Importing node-api-dotnet into node into a dotnet app

[camnewnham]

The use case here is to create a runtime "script editor" that uses javascript/typescript as the language.

So the goal is:

1. Running in dotnet host program
2. User writes javascript that can interop with net objects
3. Javascript is executed by the dotnet side.

I'm having issues calling the javascript import of node-api-dotnet when it's running inside the host process. Here's a reproducible example, where the TEST_MS flag just makes sure it does work with the simple ms package.

//#define TEST_MS

// See https://aka.ms/new-console-template for more information
using Microsoft.JavaScript.NodeApi;
using Microsoft.JavaScript.NodeApi.Runtime;
using System.Reflection;

Console.WriteLine("Hello, World!");

string baseDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)!;

File.WriteAllText(Path.Combine(baseDir, "package.json"), $"{{\"type\":\"module\"}}");

NodejsPlatform platform = new NodejsPlatform(Path.Combine(baseDir, "native", "win-x64", "libnode.dll"));

NodejsEnvironment env = platform.CreateEnvironment(baseDir);

await env.RunAsync(async () =>
{
#if TEST_MS
    JSValue console = JSValue.CreateObject();
    console.SetProperty("info", JSValue.CreateFunction("info", (args) =>
    {
        for (int i = 0; i < args.Length; i++)
        {
            Console.WriteLine(args[i].GetValueExternalOrPrimitive()!.ToString());
        }
        return JSValue.Undefined;

    }));
    JSValue.Global.SetProperty("console", console);

    JSValue.RunScript("const ms = require(\"C:/node_modules/ms\"); " +
        "console.info(\"data\", ms(1231234));");
#else
    JSValue.RunScript("const dotnet = require(\"C:/node_modules/node-api-dotnet\");");
    Console.WriteLine("Finished.");
#endif

});

The exception I get here is:

Unhandled Exception: System.EntryPointNotFoundException: Arg_EntryPointNotFoundExceptionParameterizedNoLibrary, napi_create_string_utf16
   at System.Runtime.InteropServices.NativeLibrary.GetSymbol(IntPtr, String, Boolean) + 0x57
   at Microsoft.JavaScript.NodeApi.Runtime.NodejsRuntime.CreateString(JSRuntime.napi_env, ReadOnlySpan`1, JSRuntime.napi_value&) + 0x62
   at Microsoft.JavaScript.NodeApi.JSValue.CreateStringUtf16(String) + 0x72
   at Microsoft.JavaScript.NodeApi.JSValue.op_Implicit(String) + 0x15
   at Microsoft.JavaScript.NodeApi.DotNetHost.NativeHost.InitializeModule(JSRuntime.napi_env, JSRuntime.napi_value) + 0x435

Which is the same when using await env.ImportAsync("C:/node_modules/node-api-dotnet/net8.0", null, true);

It did occur to me that the Microsoft.JavaScript.NodeApi.dll exists in both the dotnet project and the node module here. Any ideas on how to resolve this?

[jasongin]

This is an interesting scenario, but not something we have tried to enable so far.

JSValue.RunScript("const dotnet = require("C:/node_modules/node-api-dotnet");");

Unhandled Exception: System.EntryPointNotFoundException: napi_create_string_utf16
at Microsoft.JavaScript.NodeApi.DotNetHost.NativeHost.InitializeModule(JSRuntime.napi_env, JSRuntime.napi_value) + 0x435

This fails because NativeHost.InitializeModule() tries to use NativeLibrary.GetMainProgramHandle() to get the module containing the napi_* native entry-points used by the interop layer. In a normal Node.js application the main program module would be the node executable. But in this case the main program is your .NET application, that is why the entry-point is not found. What it actually needs there is the libnode module handle, since that would have the requested entry-points.

That might be possible to fix, but then there would be another problem: it will try to initialize the .NET runtime, which I think will fail because .NET is already loaded in the current process. (And you don't want it to use a separate instance of the .NET runtime anyway.)

Instead of that, I think it should be possible to initialize Microsoft.JavaScript.NodeApi.DotNetHost.ManagedHost from the .NET app, then provide its JS exports object to the JavaScript code, rather than trying to load the node-api-dotnet module from JS. (Maybe eventually the node-api-dotnet module could detect this scenario and handle it automatically.) I'll experiment with these ideas and get back to you.

[camnewnham]

Thanks Jason. This seems to work with the system classes, is it what you had in mind?

NodejsPlatform platform = new("libnode.dll");
NodejsEnvironment env = platform.CreateEnvironment();

await env.RunAsync(async () =>
{
    JSObject exports = [];
    JSValue.Global.SetProperty("dotnet", exports);
    ManagedHost mh = new ManagedHost(exports);
    JSValue.RunScript($"dotnet.System.Console.WriteLine(\"tada\")");
});

How do I go about importing a third party dll into this? (ManagedHost.LoadAssembly is private)

[camnewnham]

To load an external dll "correctly", the following seemed to work:

1. Override ManagedHost._loadContext via reflection to use the load context of the main program.
2. Invoke ManagedHost.LoadAssembly via reflection to load the assembly

 private static void LoadDefaultDotNetTypes(NodejsEnvironment env)
 {
     ManualResetEventSlim mre = new ManualResetEventSlim();
     env.RunAsync(async () =>
     {
         try
         {
             JSObject exports = [];
             JSValue.Global.SetProperty("dotnet", exports);
             ManagedHost managedHost = new ManagedHost(exports);

             FieldInfo field = typeof(ManagedHost).GetField("_loadContext", BindingFlags.NonPublic | BindingFlags.Instance);
             field.SetValue(managedHost, AssemblyLoadContext.GetLoadContext(Assembly.GetExecutingAssembly()));

             LoadDotNetLibrary(managedHost, typeof(MyLib).Assembly.Location);
         }
         finally
         {
             mre.Set();
         }
     });
     mre.Wait();
 }

and

private static void LoadDotNetLibrary(ManagedHost managedHost, string libFullPath)
{
    MethodInfo method = typeof(ManagedHost).GetMethod("LoadAssembly", BindingFlags.Instance | BindingFlags.NonPublic);
    method.Invoke(managedHost, new object[] { libFullPath, false });
}

Though the load context should probably be assigned before running the constructor?

[camnewnham]

After playing with #340 for a while, I realise I have been a little silly. The dll is already loaded into the dotnet context, so it doesn't need to be loaded again by the ManagedHost.

Here's a little hack that works for me, which does the type exporting functionality of the ManagedHost without the assembly loading.

  // Set JSMarshaller.Current because it is usually set by ManagedHost.
  typeof(JSMarshaller)
      .GetField("s_current", BindingFlags.Static | BindingFlags.NonPublic)
      .SetValue(null, new JSMarshaller()
  {
      AutoCamelCase = false
  });
  
  JSObject managedTypes = (JSObject)JSValue.CreateObject();
  JSValue.Global.SetProperty("dotnet", managedTypes);

  TypeExporter te = new TypeExporter(JSMarshaller.Current, managedTypes);
  te.ExportAssemblyTypes(typeof(MyClass).Assembly);
  te.ExportType(typeof(MyClass));

So, finally:

1. Is there (or can there be) a way to configure the above snippet without using reflection? I could provide a different Marshaller, but without setting it to Current I worry I would lose some other functionality.
2. Is there a way to export all the types? Or is this avoided for some other reason? I had expected ExportAssemblyTypes to do this, but without using ExportType I get Class not registered for JS export: MyClass
3. Can JSMarshaller.ToJS<T>(Object o) also have a JSMarshaller.ToJS(Type t, Object o) signature? Calling this via reflection works. Or similarly JSRuntimeContext.GetOrCreateObjectWrapper.

[jasongin]

Is there (or can there be) a way

Certainly there can be a better way. I plan to improve the API to make this scenario easier, but I have not had time yet to focus on this.

Is there a way to export all the types? I had expected ExportAssemblyTypes to do this, but without using ExportType I get Class not registered for JS export: MyClass

Currently ExportAssemblyTypes() skips non-public and un-namespaced types. Is that the reason?

Can JSMarshaller.ToJS(Object o) also have a JSMarshaller.ToJS(Type t, Object o) signature?

Yes, I was thinking of adding that after the discussion in #346 where there was a need to call it with a Type object instead of type parameter.

[camnewnham]

Is there a way to export all the types? I had expected ExportAssemblyTypes to do this, but without using ExportType I get Class not registered for JS export: MyClass

Currently ExportAssemblyTypes() skips non-public and un-namespaced types. Is that the reason?

From some testing my understanding is that I need ExportAssemblyTypes to be able to call a dotnet constructor on the JS side, and I need ExportType to be able to marshal. Is that correct?

If so, this becomes a non-issue as ExportAssemblyTypes can be called initially, then ExportType can be called on-demand prior to wrapping/unwrapping via #346