use kerl package if installed

[scottbelden]

Resolves #161

This adds an optional C extension for the kerl functionality. It can be installed by itself with pip install kerl but I've updated the setup.py so that it can also be installed with pip install pyota[kerl]. Additionally I created a group for both C extensions so that one can do pip install pyota[speedups].

When creating the extension I changed the names of convertToTrits and convertToBytes to bytes_to_trits and trits_to_bytes. I thought these names were more descriptive and I don't think this should affect anyone as I don't imagine many (if any) people were importing these names.

Finally, the numbers...

I used the following script as my benchmark:

from iota import Iota
import time
from iota.commands.extended.utils import iter_used_addresses
from iota.crypto.addresses import AddressGenerator

seed = 'A' * 81

api = Iota('http://foo.bar:14265', seed)
adapter = api.adapter

start = time.time()
addresses = []
for addy in AddressGenerator(api.seed).create_iterator(0):
    addresses.append(addy)
    if len(addresses) == 10:
        break
print(addresses)
print('total time: {}'.format(time.time() - start))

With the python implementation, it takes me 4.759015798568726 seconds. Using the C extension, it takes 0.9080002307891846 seconds (over five times faster).

Just for fun, I also timed how long it takes to run the tests with both versions. Under the python version it takes 112.564s while the C extension takes 72.736s.

[todofixthis]

Hey Scott! This looks awesome!!

Out of curiosity, is it too late to call the package ckerl, so that we can have ccurl and ckerl to complement PyCurl and PyKerl? 😸

Since the package is hosted on an external repo, I'll need to get clearance from the IOTA Foundation to include it as an installable extra. I did a quick once-over, and your code looks great; will do some more testing this weekend.

[scottbelden]

Yeah, I can change the name to ckerl. I'll update this PR once that is done.

[scottbelden]

Updated.

[todofixthis]

Rockstar (: Thanks mate! I'll follow up as soon as I get the OK from the IF.

[todofixthis]

Hey Scott. Sorry for the delay; I am still waiting to get approval to merge this.

@alon-e could you have a look at this PR and the linked ckerl extension, and merge if it looks OK?

[scottbelden]

No worries. I haven't had much time to dedicate to Iota things recently (and won't for at least the next month or so), so I'm in no rush. However, I do think it would be useful.

When I get more time I had started a cython implementation of the pearldiver. I had gotten it to work in the single threaded case and it was much faster than the python implementation that someone else had ported, but I had not yet gotten it working in a multi threaded case (which would probably be necessary to make it worthwhile)

[scottbelden]

Oops. Didn't mean to close.

[dwjorgeb]

pip3 install pyota[ckerl] still says no package available, on what release will that be included?

@scottbelden if I do pip3 install ckerl is the library ready to use the C lib? Or do I have to wait until it's released?

[todofixthis]

Hey @dwjorgeb we're just waiting for sign off from an IOTA Foundation member.

In the meantime, you can install the ckerl package separately (pip3 install pyota ckerl).

[dwjorgeb]

I see, but I don't think it will help until it's merged, the library won't use it. @todofixthis

[todofixthis]

Ah... right 😁

[scottbelden]

@dwjorgeb The ckerl library is all ready to go. If you want to use it locally before this is merged you would have to patch your local pyota install with the changes in this PR.