Feature request: official rust SDK

[Mossaka]

Hey 👋,

I am a core maintainer of the CNCF containerd/runwasi project, which facilitates running wasm workloads managed by containerd and Kubernetes. It can be used as a rust library to implement containerd shims who are responsible for managing Wasm containers. We've been asked to add the WebAssembly Micro Runtime support in runwasi as a shim to ensure a wide coverage of runtimes. See issues containerd/runwasi#337 and #2127.

Problem Statement

While we're eager to add support for WebAssembly Micro Runtime in runwasi, the lack of an official, up-to-date Rust crate for wamr is hindering progress. I am writing to propose a feature request to publish an official rust SDK for wasm micro runtime

Prior Work

There is a community-created rust crate wamr-sys by @rahul-thakoor on crates.io for #230 . However, this crate is not up to date and lacks active maintenance. A fork by @0xE282B0 adds WASI support, which can be found here: https://github.com/0xE282B0/wamr-sys.

Preferred Approach

We prefer a safe Rust SDK over just providing unsafe bindings. This would significantly streamline the integration process and ensure better safety guarantees.

Thank you for considering this feature request. Looking forward to your feedback. 🙏

[lum1n0us]

Thanks for contacting with us.

IMU, the key point is about "a safe Rust SDK over just providing unsafe bindings". I am going to do some research to figure out what does it really means beside using unsafe marcos. 😶‍🌫️

If there are shareable BKMs and recommanded reading lists, feel free 🤝 .

[Mossaka]

I would recommend first to generate the ffi bindings rust bindgen https://github.com/rust-lang/rust-bindgen and then build a SDK on top of it. Here is a safe and ergonomic Rust SDK for Sqlite: https://github.com/rusqlite/rusqlite

[0xE282B0]

A minimal bindgen project could look like this: https://github.com/0xE282B0/wasm-micro-runtime/tree/rust-sdk/wamr-sdk-rust moved to https://github.com/0xE282B0/wasm-micro-runtime/tree/rust-sdk/language-bindings/rust

@lum1n0us is it ok to place the rust-sdk in the wamr repo?
Where can I find good examples on how to use the C SDK?

[lum1n0us]

@0xE282B0 thanks for your contribution.

please place it into language-bindings/rust/.

There are two sets of C SDK. Generally, it isn't allowed to use both in one program.

• core/iwasm/include/wasm_export.h. it is a WAMR custimized private API. Samples are samples/basic, product-mini/platforms/posix/main.c, and doc/embed_wamr.md
• core/iwasm/include/wasm_c_api.h. it follows the wasm-c-api proposal. Samples are samples/wasm-c-api.

[wenyongh]

@0xE282B0 thanks a lot for the advice. For examples on how to use the C SDK, maybe you can refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/embed_wamr.md

https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/samples/basic/src/main.c#L94-L150

And you can also enable libc-wasi, refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/product-mini/platforms/posix/main.c#L974-L977

[0xE282B0]

Hi all,
I'd like to open a PR for the rust language binding to give others the opportunity to contribute.
Since it is still very raw, should I open a PR to main or do you want to create a feature branch?

[lum1n0us]

a feature branch is better.

@wenyongh please create a dev branch for rust-sdk. thanks

[wenyongh]

@0xE282B0 I have created dev/rust_sdk branch, could you please submit the PR to the branch? Thanks.

[lum1n0us]

@Mossaka aftering having a minimal Rust binding, I am thinking about next moves on "build a Rust SDK on top of it.".

I went through the rusqlite. It seems to create another set of high-level abstract programming concepts in Rust. I am not sure the same story is able to be reproduced on WAMR side and if it is necessary. So, I guess what's left for us is about "a safe Rust SDK".

I found some interesting material about creating a safe interface. Does it explain the concept of "a safe Rust SDK", you mentioned before?

[Mossaka]

I think the main thing is to create a Rust SDK that follows Rust idioms and best practices. The article you pasted is a good first step, but at the same time, I think it's important to create a SDK that provides a good developer experience. To name a few points:

• I feel nature to use a WAMR Runtime struct that implements init(), load()``set_wasi() and instantiate() and execute() methods or similar.
• I would assume each function will return a Result type that user can decide how to handle the Error type.
• Use destructors to drop resources and keep a safe interface that has minimal or does not have any unsafe APIs.

You can also see examples of how wasmtime rust SDK works:

https://docs.wasmtime.dev/api/wasmtime/

[lum1n0us]

Going to use this comment to track all doing and todo related.

Done

Next problems are

• shall we publish wamr-sys and wamr-rust-sdk on crates.io? At least, User can refer it like wamr-rust-sdk = { git = "https://github.com/lum1n0us/wamr-rust-sdk" } in their Cargo.toml
• And how to do it? IIUC, wamr-sys isn't a useable crate until it doesn't require compilation from source code before using.
• Start to consider how to integrate WAMR with RunWASI

ToDo

• add [features] to control functionalities
• support importing host defined functions
• support thread basement
• support wasi-threads
• support socket?
• control log level
• enable SIMD
• setup a separate repo about wamr-rust-sdk. https://github.com/bytecodealliance/wamr-rust-sdk

Done

• a rough idea about the Rust API #2984
• [rust sdk] support host functions #3173
• [rust sdk] fix a bug in singleton pattern #3167
• [rust sdk]enable llvm jit by default #3152

[Mossaka]

Hey there, thanks for putting out a rust-sdk for wamr and I am very excited for it. I went ahead and implement a wamr shim in runwasi: containerd/runwasi#508.

However, when I ran the shim, I've got an error "thread signal env initialized failed" in Instance creation. Could you please help me with this?

[lum1n0us]

@Mossaka Sure. I'll help.

Btw, I've created an issue (bytecodealliance/wamr-rust-sdk#6) to trace all your comments. please take a look in case I've left something.

[danielstuart14]

One thing that I didn't like about https://github.com/bytecodealliance/wamr-rust-sdk is that it uses the platform code already in WAMR. (Which is highlighted by how ESP support was added bytecodealliance/wamr-rust-sdk@ceeffbe)

For me a better way of integrating with Rust would be:
1 - Use the Rust standard library for the WAMR platform implementation, making it portable to any project targeting std rust.
2 - Allow for custom platforms to be implemented through Rust, making it possible to port it to no_std targets while still using rust libraries (Embassy, libm, etc) and custom allocators.

Option 1 (std) would be the default, with a cbindgen platform implementation that uses the standard library and the language alloc feature.

Option 2 (no_std) could have the cbindgen API (that uses a "Platform" trait) implemented and the math functions from libm. It would be the developer's responsibility to then implement the "Platform" trait with the needed functionalities. We could even go beyond and have a "PlatformVMCore" and a "PlatformExtension" trait, with the extension trait being optional, but for no_std targets I think just the VMCore functionality is enough.