add C/C++ guide #134

poemm · 2018-12-06T23:17:43Z

No description provided.

c_cpp_guide.md

jakelang · 2018-12-06T23:20:01Z

c_cpp_guide.md

@@ -0,0 +1,68 @@
+# C/C++ to ewasm contracts


It seems the de facto consensus is to use Ewasm with the capital E.

I thought it was eWASM, no?
Based on this first line from the PR ethereum/go-ethereum#16957

c_cpp_guide.md

axic · 2018-12-06T23:38:54Z

c_cpp_guide.md

+
+The output is `main.wasm`, but it needs some cleanup to remove some of it's extra import and export junk. For this, we use PyWebAssembly. 
+
+Aside: Alternatively, one can use a rust implementation of `wasm_chisel`. But that requires a rust compiler.


For python one needs a python interpreter :)

(and it is broken on default mac installation)

Hmmm. I want to add a link to wasm chisel in rust. Do you have a link?

https://github.com/wasmx/wasm-chisel

Installation instruction is though as simple as cargo install chisel.

poemm · 2018-12-06T23:47:42Z

Wow, you guys are lightning quick!

axic · 2018-12-06T23:57:46Z

I'd prefer if the two C/C++ documentations could be merged into a single file, because they have a lot common (binaryen, chisel, etc.)

axic · 2018-12-07T00:18:29Z

c_cpp_guide.md

+cd ..
+git clone https://github.com/poemm/pywebassembly.git
+cd pywebassembly/examples/
+python3 ewasm_chisel.py ../../cwrc20/main.wasm


Just checked, https://github.com/poemm/pywebassembly/blob/master/examples/ewasm_chisel.py doesn't behave like wasm-chisel (well, chisel is configurable through a config file, but talkinb about default behaviour).

chisel translates the form env:ethereum_useGas to ethereum:useGas, but only on recognised import names. ewasm_chisel translates any env namespace to ethereum. As a result it has a different naming requirement.

Yes, ewasm_chisel.py is minimalist. It would be easy for ewasm_chisel.py to check against a list of eei method names, but I don't want to update it every time the eei changes. I have a similar problem with main.syms, which I instructed to user to check. It would be wise to just update a parsable list of current eei methods which can automate this.

My solution is to change the guide to reflect that they are different.

The main difference is although the two tools are named similarly, they expect a different naming scheme from the user.

chisel expects the user to use functions like ethereum_useGas in C, while ewasm_chisel expects the user to use useGas.

Good point. Changed ewasm_chisel.py to ewasmify.py.

poemm · 2018-12-07T00:43:42Z

My C/C++ guide is focused on ewasm tips and concrete examples. Perhaps the other one might be an advanced section in my guide.

Feel free to copy whatever you want from mine and close this PR, or push to this PR.

Reformat into bulleted lists. Tweak some wording.

Make some things clearer, and add a few more links for reference

lrettig

This definitely needs to be reconciled/merged with #46 since the two present similar, somewhat contradictory sets of instructions. That doc is pretty thin so I don't think it should be too much work to merge them. Let me know if you'd like a hand.

lrettig · 2018-12-27T12:00:27Z

c_cpp_guide.md

@@ -0,0 +1,111 @@
+# Compiling C/C++ to Ewasm
+
+First an introduction, then a basic step-by-step guide, then advanced things. Warning: the Ewasm spec and tools below are subject to change.


Warning: the Ewasm spec and tools below are subject to change.

I understand why you'd add this caveat here, but I don't find it constructive or helpful on its own, i.e., it just makes the reader worry that the instructions aren't going to work. Consider making this more constructive by saying something like, "Every effort is made to keep this document up to date, but if you notice anything wrong please feel free to submit a PR or an issue to report and/or fix it."

Agreed. It is understood that things are subject to change. I completely removed it so that the guide is more concise.

c_cpp_guide.md

lrettig · 2018-12-27T12:10:51Z

c_cpp_guide.md

+
+1. The output of compilers is a `.wasm` binary which may have imports and exports which do not meet Ewasm requirements. We have tools to fix the imports and exports.
+
+1. There are no tutorials for debugging/testing a contract. Hera supports extra Ewasm helper functions to print things, which have helped in writing test cases. A tutorial is needed to allow early adopters to debug/test their contracts without having to do it on the testnet.


Hera supports extra Ewasm helper functions to print things, which have helped in writing test cases

Could you link to a doc on these, or otherwise make it more explicit here? Assume you are talking about things like printmemhex?

We should mention the state of debugging tools since it is important for developers. I changed it to say that early adoptors can debug on the testnet for now. I am left feeling that there is a great need for tools.

Agree, was not suggesting removing this, but instead linking to docs we have elsewhere in this repo on debug tools.

Can you suggest a link? Do you think that it is reasonable to link instructions on how to write test fillers?

c_cpp_guide.md

lrettig · 2018-12-27T12:13:02Z

c_cpp_guide.md

+
+1. WebAssembly is still primitive and lacks features. For example, WebAssembly lacks support for exceptions and we have no way to do system calls in Ewasm. Compilers and libraries are still primitive. For example, we have a patched version of libc to allow `malloc`, but the patches are not yet enough for `std::vector` because other memory managment calls are unavailable. But perhaps any memory management beyond memory allocation may be unwanted for Ewasm contracts since it costs gas. This situation will improve as WebAssembly, compilers, and libraries mature.
+
+1. In the current Ewasm design, all communication between the contract and the client is done through the module's memory. For example, the message data ("call data") sent to the contract is accessed by calling `callDataCopy()`, which puts this data to WebAssembly memory at a location given by a pointer. This pointer must be to either to a statically allocated array, or to dynamically allocated memory using `malloc`. For example, before calling `callDataCopy()`, one may use `getCallDataSize()` to see how many bytes of memory to `malloc`.


callDataCopy()
getCallDataSize()

Consider linking to the EEI specs for these two methods.

Also, this would all be much clearer with an example using code. Could you maybe link to the wrc20 example code in C++, or even include it inline here?

I just tried including an example, but ended up having to explain too many things. This may overwhelm a first-time user. I think it is better to just give a concise high-level explanation, and allow the user to explore concrete examples when they know the basics. I overhauled this paragraph.

lrettig · 2018-12-27T12:18:41Z

c_cpp_guide.md

+
+When writing Ewasm contracts in C/C++, one should bear in mind the following caveats:
+
+1. WebAssembly is still primitive and [lacks features](https://github.com/WebAssembly/design/blob/master/FutureFeatures.md). For example, WebAssembly lacks support for exceptions and we have no way to do system calls in Ewasm. Compilers and libraries are still primitive. For example, we have a patched version of libc to allow `malloc`, but the patches are not yet enough for `std::vector` because other memory managment calls are unavailable. But perhaps any memory management beyond memory allocation may be unwanted for Ewasm contracts since it costs gas. This situation will improve as WebAssembly, compilers, and libraries mature.


But perhaps any memory management beyond memory allocation may be unwanted for Ewasm contracts since it costs gas.

I don't think this statement belongs here, it leaves the reader with too many unanswered questions. I think the best thing to do would be to compile a list of open design questions in one place (not specific to C/C++) and provide a link to it somewhere in this doc. A link to open issues on the ewasm/design repo might suffice for now if we don't have a more mature doc.

To make this more helpful and constructive, it would be nice to conclude this section by saying something along the lines of, "For now, to work around these issues, ensure that you only use basic structs and malloc calls" (or whatever the advice should be). "To join the conversation and contribute to ongoing Ewasm interface design, see X" (link to another doc).

This situation will improve as WebAssembly, compilers, and libraries mature.

Consider dropping, I don't think this is critical.

Agreed that a guide is not a place for design discussions. I overhauled this paragraph.

lrettig · 2018-12-27T12:27:16Z

c_cpp_guide.md

+
+```sh 
+# checkout LLVM, clang, and lld
+svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm


Can we use git instead of svn? The instructions from Jake's doc seem reasonable no?

I mentioned that the official guide http://llvm.org/docs/GettingStarted.html uses svn. But changed our guide to use git.

I am still compiling the git version to test it. Will revert to svn if there is a problem.

The git version successfully compiles wrc20. Compiling the git version of LLVM had a few errors along the way, but restarted each time and finally it finished.

Didn't realize svn was in the official guide. Glad to hear it works with git too!

lrettig · 2018-12-27T12:38:17Z

c_cpp_guide.md

+
+Check whether the command line output of `ewasmify.py` above lists only valid Ewasm imports and exports. To troubleshoot, you may wish to also inspect `main.wasm` in its text representation, so proceed to the next step with binaryen or wabt.
+
+We can convert from the `.wasm` binary format to the `.wat` (or `.wast`) text format (these are equivalent formats and can be converted back-and-forth). This conversion can be done with Binaryen's `wasm-dis`.


.wat (or .wast) text format (these are equivalent formats and can be converted back-and-forth)

Strictly speaking, this is not true. wast defines a "scripted" format with certain extensions to the grammar, and my understanding is that it's a superset of the wat format. See https://github.com/WebAssembly/design/blob/master/TextFormat.md#text-format. We've tended to elide over this in our docs thus far but I think it might be helpful to be more explicit about it. What do you think?

Agreed. I removed .wast. I think that we should start using the correct extension .wat.

Cool. It's clearly not critical but we can bring it up on a call and try to get everyone on the same page.

lrettig · 2018-12-27T12:40:37Z

c_cpp_guide.md

+
+We can convert from the `.wasm` binary format to the `.wat` (or `.wast`) text format (these are equivalent formats and can be converted back-and-forth). This conversion can be done with Binaryen's `wasm-dis`.
+
+Aside: Alternatively one can use Wabt's `wasm2wat`. But Binaryen's `wasm-dis` is recommended because Ewasm studio uses Binaryen internally, and Binaryen can be quirky and fail to read a `.wat` generated by another program. Another tip: if Binaryen's `wasm-dis` can't read the `.wasm`, try using Wabt's `wasm2wat` then `wat2wasm` before trying again with Binaryen.


Let's link to @hugo-dc's new doc on binaryen and wabt here. Maybe we could rebase this against #141?

I think that this binaryen section should be removed and replaced with a link to Hugo's doc once it is merged.

lrettig · 2018-12-28T20:27:12Z

@poemm and I discussed yesterday and I understand that he wrote this doc from scratch and feels that it covers everything covered in #46. I just reviewed them both again and I agree with the caveat that the compilation steps are more explicit in #46, whereas this doc basically says:

make sure to modify the Makefile to clang++, use extern "C" around the helper function declarations.
run make

Basically, if the content in Compile C/C++ to WebAssembly could be brought into this doc, or else if it's no longer relevant (@axic said s2wasm was removed), then I think we're good to go with this one instead.

@axic, you suggested you wanted to cherry pick from #46 -- are we good to merge this and then bring anything required over from that one?

poemm · 2018-12-28T22:30:02Z

make sure to modify the Makefile to clang++, use extern "C" around the helper function declarations.
run make

I agree that this is awkward, so I replaced this text with a link to a C++ example.

@lrettig Do you think that we should leave compilation steps in the makefiles, explicitly write compilation steps in the doc, or put compilation steps in both the makefiles and doc? I personally prefer only makefiles, but I trust your judgement.

poemm · 2018-12-28T22:36:51Z

Did anyone try following the steps in this doc? Would be nice to have at least one confirmation that it works.

poemm · 2019-01-03T13:19:07Z

Included downloading LLVM 7 from repositories, since LLVM 7 supports Wasm by default, so no need to compile your own. Also put compilation commands in the doc instead of just in the makefile.

jakelang reviewed Dec 6, 2018

View reviewed changes

axic reviewed Dec 6, 2018

View reviewed changes

axic reviewed Dec 7, 2018

View reviewed changes

add C/C++ wasmception guide with example

5265b6d

poemm force-pushed the ccpp_guide_wasmception branch from da24bdd to 5265b6d Compare December 7, 2018 01:27

poemm added 4 commits December 6, 2018 19:49

C/C++ guide fixes

1f64706

C/C++ guide spellcheck and cleanup

9650e38

C/C++ guide clarifications and advanced section

7aecffd

C/C++ guide removed clang stuff

85343a2

poemm mentioned this pull request Dec 8, 2018

Add docs for using LLVM toolchain with Webassembly #46

Open

poemm added 5 commits December 9, 2018 13:02

C/C++ guide changed ewasm_chisel.py to ewasmify.py

b1471d7

C/C++ guide clarify ewasmify.py

a52a723

C/C++ guide clarifications

696768c

C/C++ guide added basic LLVM guiden

c229d16

C/C++ guide typos and clarifications

0556196

poemm requested a review from lrettig December 21, 2018 03:41

poemm and others added 6 commits December 20, 2018 21:51

C/C++ guide clarifications

19253ee

Add link on non-determinism

9585d5c

Minor tweaks

1e5ee1b

Reformat into bulleted lists. Tweak some wording.

Add link to future Wasm features

63d742c

Add section header

5c6a92b

Misc cleanup

c6ca269

Make some things clearer, and add a few more links for reference

lrettig requested changes Dec 27, 2018

View reviewed changes

poemm added 2 commits December 27, 2018 12:08

C/C++ guide fixes for Lane's comments

912f6ed

C/C++ Guide spellcheck

3ae2678

C/C++ guide add C++ example

4dab415

C/C++ Guide get llvm 7 from repositories

e86d665


		The output is `main.wasm`, but it needs some cleanup to remove some of it's extra import and export junk. For this, we use PyWebAssembly.

		Aside: Alternatively, one can use a rust implementation of `wasm_chisel`. But that requires a rust compiler.

		@@ -0,0 +1,111 @@
		# Compiling C/C++ to Ewasm

		First an introduction, then a basic step-by-step guide, then advanced things. Warning: the Ewasm spec and tools below are subject to change.


		1. The output of compilers is a `.wasm` binary which may have imports and exports which do not meet Ewasm requirements. We have tools to fix the imports and exports.

		1. There are no tutorials for debugging/testing a contract. Hera supports extra Ewasm helper functions to print things, which have helped in writing test cases. A tutorial is needed to allow early adopters to debug/test their contracts without having to do it on the testnet.


		1. WebAssembly is still primitive and lacks features. For example, WebAssembly lacks support for exceptions and we have no way to do system calls in Ewasm. Compilers and libraries are still primitive. For example, we have a patched version of libc to allow `malloc`, but the patches are not yet enough for `std::vector` because other memory managment calls are unavailable. But perhaps any memory management beyond memory allocation may be unwanted for Ewasm contracts since it costs gas. This situation will improve as WebAssembly, compilers, and libraries mature.

		1. In the current Ewasm design, all communication between the contract and the client is done through the module's memory. For example, the message data ("call data") sent to the contract is accessed by calling `callDataCopy()`, which puts this data to WebAssembly memory at a location given by a pointer. This pointer must be to either to a statically allocated array, or to dynamically allocated memory using `malloc`. For example, before calling `callDataCopy()`, one may use `getCallDataSize()` to see how many bytes of memory to `malloc`.


		When writing Ewasm contracts in C/C++, one should bear in mind the following caveats:

		1. WebAssembly is still primitive and [lacks features](https://github.com/WebAssembly/design/blob/master/FutureFeatures.md). For example, WebAssembly lacks support for exceptions and we have no way to do system calls in Ewasm. Compilers and libraries are still primitive. For example, we have a patched version of libc to allow `malloc`, but the patches are not yet enough for `std::vector` because other memory managment calls are unavailable. But perhaps any memory management beyond memory allocation may be unwanted for Ewasm contracts since it costs gas. This situation will improve as WebAssembly, compilers, and libraries mature.


		Check whether the command line output of `ewasmify.py` above lists only valid Ewasm imports and exports. To troubleshoot, you may wish to also inspect `main.wasm` in its text representation, so proceed to the next step with binaryen or wabt.

		We can convert from the `.wasm` binary format to the `.wat` (or `.wast`) text format (these are equivalent formats and can be converted back-and-forth). This conversion can be done with Binaryen's `wasm-dis`.


		We can convert from the `.wasm` binary format to the `.wat` (or `.wast`) text format (these are equivalent formats and can be converted back-and-forth). This conversion can be done with Binaryen's `wasm-dis`.

		Aside: Alternatively one can use Wabt's `wasm2wat`. But Binaryen's `wasm-dis` is recommended because Ewasm studio uses Binaryen internally, and Binaryen can be quirky and fail to read a `.wat` generated by another program. Another tip: if Binaryen's `wasm-dis` can't read the `.wasm`, try using Wabt's `wasm2wat` then `wat2wasm` before trying again with Binaryen.

add C/C++ guide #134

Are you sure you want to change the base?

add C/C++ guide #134

Conversation

poemm commented Dec 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

poemm commented Dec 6, 2018

axic commented Dec 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

poemm commented Dec 7, 2018

lrettig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lrettig commented Dec 28, 2018

poemm commented Dec 28, 2018

poemm commented Dec 28, 2018

poemm commented Jan 3, 2019