Add env object to WASIContext to pass an externally defined Memory object into the WebAssembly context

[dmarcos]

My first typescript code. My type-fu is not great. Feel free to suggest alternatives.

[taybenlor]

Hey mate! Thanks for the PR.

This is a great suggestion! Being able to specify the Memory is important.

I don't want to mix up the env variables for WASI with the memory to give to WebAssembly.

I think a refactor is in order to make the interface better match the NodeJS WASI interface.

If you'd like to give that a shot I can help you with that. Or I'm happy to give it a go in my next available time.

[dmarcos]

I think a refactor is in order to make the interface better match the NodeJS WASI interface.

I'm happy to refactor. Can you be a bit more specific? Not sure I'm understand. I'm not familiar with the NodeJS WASI interface. is that any different than the wasi sdk?

I'm building C to WASM and running in a browser environment. Can see how I'm building and linking. It looks the --import-memory flag of the linker somehow binds to the env.memory but still learning how all the moving parts work

[taybenlor]

So I think the confusion here is between env in WASI and env in WebAssembly. In WASI env represents the environment variables to a CLI command (like a dotenv file). In WebAssembly env is things like memory (what you're passing through).

The NodeJS API makes this more clear because you create a WASI instance and then pass it to the WebAssembly instantiate method. https://nodejs.org/api/wasi.html

So in that API you would be able to specify the memory during the WebAssembly Instantiate and not how Runno forces you to do it (from the WASI layer).

My goal with a refactor would be to better match that NodeJS API.

[taybenlor]

After looking at the changes properly, I'm a bit confused about what you're trying to achieve here. I thought you were trying to specify a custom memory import, but it looks like you're overriding the memory that Runno writes to.

What problem are you trying to solve?

[taybenlor]

Could you please look at the documentation in my refactor: #278

Does that achieve what you want?

[taybenlor]

Thanks Diego, could you answer my questions above? If you still want to go ahead with this we can discuss a plan.

[dmarcos]

After looking at the changes properly, I'm a bit confused about what you're trying to achieve here. I thought you were trying to specify a custom memory import, but it looks like you're overriding the memory that Runno writes to.

What problem are you trying to solve?

The application might wanna define and have control over the memory. Might be JS code that wants to manipulate it.

[taybenlor]

Okay so with the changes I just released in 0.6.0 you can specify your own memory instance when initialising the WebAssembly runner. Check out packages/wasi/README.md for instructions.

If you'd just like to access the memory object exported by the binary, that is available on the WASI instance when it is instantiated with new WASI().

Have a look at the PR referenced above for what I mean.

Does that solve your problem?

[dmarcos]

Could you please look at the documentation in my refactor: #278

Does that achieve what you want?

Tried the PR and get the error:

Uncaught TypeError: WebAssembly.instantiate(): Import #0 module="env" error: module is not an object or function

My WASM code expect the memory passed through the env object. I don't have control over it since that comes imposed from the wasi-sdk and compiler / linker (clang - llvm) when using the --target=wasm32-wasi option. Don't see a way to change that.

Ideally and for max flexibility I would want control on what it's passed to WebAssembly.instantiateStreaming (another need I'll have soon is to pass an object with JS functions to be called from wasm. that object can have any arbitrary name) but I imagine It might not fit with the current runno abstraction. Not sure what the right solution is. Still in the process of understanding all the moving parts.

[taybenlor]

And that's using the method I've outlined in the README?

// Then instantiate your binary with the imports provided by the wasi object
const wasm = await WebAssembly.instantiateStreaming(
  fetch("/binary.wasm"),
  {
    ...wasi.getImportObject(),

    // Your own custom imports (e.g. memory)
    memory: new WebAssembly.Memory({ initial: 32, maximum: 10000 });
  }
);

I don't understand why the memory would be passed through the env variable in the context. That is for environment variables (https://en.wikipedia.org/wiki/Environment_variable#:~:text=An%20environment%20variable%20is%20a,in%20which%20a%20process%20runs.) which can only be strings.

[dmarcos]

And that's using the method I've outlined in the README?

// Then instantiate your binary with the imports provided by the wasi object
const wasm = await WebAssembly.instantiateStreaming(
  fetch("/binary.wasm"),
  {
    ...wasi.getImportObject(),

    // Your own custom imports (e.g. memory)
    memory: new WebAssembly.Memory({ initial: 32, maximum: 10000 });
  }
);

I don't understand why the memory would be passed through the env variable in the context. That is for environment variables

I tried your refactor and two problems.

1) In the docs you gotta do:

     const wasm = await WebAssembly.instantiateStreaming(
     ...
     env: {memory: new WebAssembly.Memory({ initial: 32, maximum: 10000 }
    .... )

Looking at my wasm code I see:

memory $env.memory (;0;) (import "env" "memory") 

I see the same error in the context of emscripten so not exclusive to the wasm-sdk toolchain. the env object looks like a convention / standard but couldn't find much info.

2) Also need a reference to the passed memory (as I did in my PR). Notice that if the memory is externally defined it's not longer part of the instance.exports object:

runno/packages/wasi/lib/wasi/wasi.ts

Line 88 in 1fe1062

this.memory = this.instance.exports.memory as WebAssembly.Memory;

Opened a PR to fix 1. Not sure how you wanna handle 2. Open to suggestions

[taybenlor]

Ahhh okay, that makes sense. Thanks for the extra context, super helpful!

I've just merged another PR which should fix those issues for you: #281

Feel free to file a bug or discussion topic if you have any other problems.

Cheers!

[dmarcos]

Thanks for the patience. It works now! 👏

[taybenlor]

Yay!! Thanks for the feature request.