RouterInfo and Lease Set queries

[sxlissxn]

Hello, i'm conducting a simulation of I2pd in a self-contained environment. I'm having a few issues...
Currently it requires the simulation of the reseed server for boostraping and floodfill routers. Those entities requires the su3 files, which can be generated using some available tools. However, the su3 is constructed using the routerinfo.dat files, but the i2pd routers only generates the router.info files. Based on my research the routerinfo.dat file contains additional metadata.Thus are there ways to generate the router.info files and lease sets for a given number of simulated routers?

[Vort]

I don't know exact answer to your question.
However, as far as I know, I2P network can work without reseeds, but you need to manually copy some starting routerinfos to netDb directories.
Here is some example code: https://codeberg.org/weko/i2pd-test-tools/src/commit/4663af6da3b511af1c066940a8535ea32cb15eaf/start_testnet.py.
Also you can look how reseeds work here:
https://github.com/PurpleI2P/pyseeder

[sxlissxn]

thanks :) i did the reseeding manually, however, i am unsure about whether its the reseed which is causing this:


Are there ways to solve this?

[Vort]

Not enough nodes maybe?
You can look if nodes make connections at all:
/debug - SSU2: Connecting to ip:port (hash)
/debug - NTCP2: Connecting to ip:port (hash)

Also you need some nodes to have floodfill option enabled.

[sxlissxn]

i have simulated 15 i2pd routers, however, is the self-contained characteristic of my simulation tool causing the peer discovery issue? i have filled the NetDB, have floodfill and reseed server but i still get the same error.

[Vort]

How IP addresses for your nodes look like?
Also it worth checking for connections as I said before.

There may be bugs in simulation tool of course, but first of all, more probable variants should be checked.

[sxlissxn]

for the ip addresse , now i am getting those errors, the nodes are not making connection at all:

[Vort]

Try using "real" addresses like 1.1.1.1 - 1.1.1.254 for example.
I think simulation environment should allow it.

Not sure if errors will go away, there may be more problems, but changing IP addreses is worth to try.

[sxlissxn]

It does not work :(
For the peer selection, does it take it from the NetDB?
Since i am not connecting to the internet for the simulation, i had to manually add files to the netDB and create the su3 files, is the fact that i changed the extension from router.info to routerInfo-Id.dat probably creating this issue?

[Vort]

It does not work :(

Maybe it worth to try something simpler than i2pd first - web server (nginx) and web client (curl) for example.
Or custom made tcp server and client.

For the peer selection, does it take it from the NetDB?

Yes, but problem in your case appears much earlier:
Node can't open its own ports.
It makes no sense trying to connect to other nodes in such case.
This problem should be solved first.

[sxlissxn]

Maybe it worth to try something simpler than i2pd first - web server (nginx) and web client (curl) for example. Or custom made tcp server and client.

I've already tried those early this year.

Try using "real" addresses like 1.1.1.1 - 1.1.1.254 for example. I think simulation environment should allow it.
I was able to solve the binding problem by assigning ip addresses using the simulation configuration file instead of the i2pd configuration file.

However, i'm still having issues with the peer connection and now i am facing a new issue:

[Vort]

However, i'm still having issues with the peer connection

Do you see Connecting messages like I mentioned eariler?

and now i am facing a new issue:

I don't think this error is much important.
And it affects only SSU2.
NTCP2 should be unaffected.

[Vort]

Also I suggest to look at routerinfos inside of netdb directory for your nodes.
Do they have host and port specified? What caps they have?

[sxlissxn]

[Vort]

PfG caps are not right. Why transit is disabled? It should not be like that.
Also where this RI was located? In netdb of node with other IP (like 100.0.0.2)?

upd: It should be possible to use this node as floodfill, but no tunnels through it will be possible.
upd2: Another thing to check: node can be banned because of unreachability if you started it little later than it was contacted by other node. It is better to clear peerProfiles directory just in case. And start first node slightly earlier than others.

[sxlissxn]

this is the router.info found in the data directory for that specific i2pd router.I haven't enabled the no transit

[Vort]

this is the router.info found in the data directory for that specific i2pd router

It should appear in netdb directory of other routers, in some way, either manually (which I suggest to try first) or with the help of reseeds. With different name, like this one: netDb\r6\routerInfo-6NcDCoHU0H2TRBEANjcZaonkqPtYYn2wnCIHAfILZPE=.dat. This name is made by hashing part of RI contents.

I haven't enabled the no transit

Ok, maybe this situation will correct itself somehow (with simulation of longer duration for example).

[sxlissxn]

i added the routerInfo manually to the netDB.However, i though that "6NcDCoHU0H2TRBEANjcZaonkqPtYYn2wnCIHAfILZPE=" was the router ID and not hashed part of the RI content... I've set the simulation for 30 min, however, due to the errors in tunnel creations it lasts only 15-20 seconds

[Vort]

Router Ident is calculated with hashing.
I was saying it because if you have RI file of "unknown" router, you can calculate such ID without additional information.

i added the routerInfo manually to the netDB

So other routers have this file?
They should try to make connections to this router then.

[Vort]

I made test by myself, with two virtual machines.
First machine, 100.0.0.1, hosted floodfill with such config:

bandwidth = P
floodfill = true
loglevel = debug

host = 100.0.0.1
port = 1024

[reseed]
urls =
yggurls =

Second virtual machine, 100.0.0.2, hosted regular node:

bandwidth = P
floodfill = false
loglevel = debug

[reseed]
urls =
yggurls =

After I placed floodfill (node1) RI into correct place, i2pd_node2.zip\netdb\rK\routerInfo-k3Kki0MLyPKK-uhKZEosY80CM8AF~r8mupJxwsbMXQ8=.dat, second node was able to start making connection:

17:03:59@332/debug - SSU2: Connecting to 100.0.0.1:1024 (k3Kk)
...
17:03:59@332/debug - SSU2: Block type 0 of size 4
17:03:59@332/debug - SSU2: Datetime
17:03:59@332/debug - SSU2: Block type 13 of size 6
17:03:59@332/debug - SSU2: Address
17:03:59@332/info - SSU2: Our external address is 100.0.0.2:10849
...

You can look at full logs in these archives:
i2pd_node1.zip
i2pd_node2.zip
Binaries are omitted to keep files small.

[sxlissxn]

Hello, I tried it tool on virtual machines, it works. I think the issue lies within the simulation tool itself and not within the i2pd configuration.

[Vort]

You may also check for time related problems.
If simulation have zero timestamps at the start, it may cause problems.
I don't know for sure if that's the case, but many programs have special meaning for zero values.

upd. Also you need to be sure that your RouterInfos are not from future and not too old.
If timestamps in RouterInfos are too different from local time, i2pd will throw them away.

---

Also I suggest to compare debug logs made in virtual machine with logs made in simulation.
There is high chance to have visible differences giving a clues.

[sxlissxn]

The timestamp in my simulation seems fine.
The i2pd routers in my simulation are set up in such a way that it connects to the i2p network, however, is there a way to ensure that it does not connect to the i2p network but rather communicate with each other only (private network)?

[Vort]

is there a way to ensure that it does not connect to the i2p network but rather communicate with each other only

By clearing extra RouterInfos in netdb directories and setting

[reseed]
urls =
yggurls =

on each node.

My example with two nodes in virtual machines was set up this way.

You may also disable addressbook, but it should not prevent nodes from working anyway.

[sxlissxn]

Hello, I ended up dropping the simulation for vms simulation. Thanks a lot! However, i have a queation: Can i use the java version of the I2CP Api with a i2pd router or it works only on the java version of i2p?

[Vort]

I know that I2PSnark uses I2CP and can work both with i2pd and java i2p.
But I have no experience with them.

[sxlissxn]

For the routerInfo-Ident=.dat to be manually sent to netDbs, did you rename it mannualy or? Because, now that i have added more nodes into the simulation doing it this way, I am getting this error:

I2pd is not starting.

[Vort]

I made it manually for my tests, but such sorting can be automated by writing tool for this task.

If you have reproduction steps for getting that crash, it may make sense to modify i2pd to prevent it from happening, even if crash happen because of wrong file placement.

[sxlissxn]

can you elaborate on this?

[sxlissxn]

If I use https://github.com/PurpleI2P/pyseeder on another vm, will it bootstrap them and place them in their correct place with their correct ident in the netDb?

[Vort]

can you elaborate on this?

1. Router idents are obtained by hashing. Do you want to know what exactly needs to be hashed?
2. From your "core-dump" result I make conclusion that i2pd crashed. It should not happen. If you can explain how such crash can be reproduced on other computers, it may help to improve code.

[sxlissxn]

It crashed after i filled in the netDb

[Vort]

If I use https://github.com/PurpleI2P/pyseeder on another vm, will it bootstrap them and place them in their correct place with their correct ident in the netDb?

Reseed works with already launched network.
If you are starting network from scratch, I doubt it will help much.

It crashed after i filled in the netDb

netDb is filled regularly during node operation and no crashes happen.
Can you attach zipped contents of such netdb?

[sxlissxn]

i2pd.zip

[Vort]

I already see mistake.
routerInfo--lGZb... should be in r- directory, not in rI.

upd. I see no crashes however. Probably because I use Windows and not Linux.

[sxlissxn]

I changed it... thanks for pointing it out. However, i still can't create tunnels , it is stuck at this:

[sxlissxn]

It activated then back to failed...

[Vort]

it is stuck at this:

Absence of message is, most likely, because of default loglevel.
With debug level there will be more info.

It activated then back to failed...

By activated you mean established?
Tunnel with how many hops was created?

Also how many connections to other nodes this particular node was able to establish?
With low amount of connected nodes there will be problems with creation of long (many hops) tunnels.

[sxlissxn]

I am getting this issue

For the routerInfo-Ident=.dat to be manually sent to netDbs, did you rename it mannualy or? Because, now that i have added more nodes into the simulation doing it this way, I am getting this error: I2pd is not starting.

and this issue as well

[sxlissxn]

For the routerInfo-Ident=.dat to be manually sent to netDbs, did you rename it mannualy or? Because, now that i have added more nodes into the simulation doing it this way, I am getting this error: I2pd is not starting.

Im still getting this issue

[Vort]

I though you meant failed tunnels.

failed service means that crash is still there and it is not related to wrong location of RouterInfo in netdb.

There are methods of obtaining stack trace with gdb from specially made builds with debug information (symbols), it may help to understand where exactly in code i2pd is crashing.
But it is easier to revert back to the state when no crash happened and to slowly add changes until crash appear (or produce intermediate states with "half-split" method).