Validators who want to participate in a Tableland Network must register themselves in a smart contract

[joewagner]

This is a proposal is in response to tension #240. I'm posting as a means to get ideas flowing, and it would be great if other folks want to create alternatives or add to this.

I'll propose that for each Tableland Network there will be a smart contract that has a mapping that maps a uint256 to a struct that contains an RFC 3986 DNS address and a wallet address. Every entry of this mapping contains the DNS address a Validator can be reached at and the public key of the wallet they are using to interact with the Registry contract (and maybe how much wei they have staked?).

pseudocode:

struct Validator {
    address wallet;
    string host;
    uint256 stake;  // maybe...
}

mapping(uint256 => Validator) private _validators;

This mapping will be readable via a few different public functions, and writable with a protected function who's protections can match whatever the criteria for being a registered Validator are, e.g. Staking some kind of token or coin, asking to be part of our early external Validator program, etc... We may also want a means to ensure that only one entry per wallet and/or host is allowed.

When a client wants to connect to Tableland but doesn't already have a specific Validator in mind, they can use their EVM chain provider to call one of the public read functions with a timestamp (maybe least significant 10^7 digits of ms since epoch?). The read function can do a modulo with number of Validators to pick which Validator info the client should use and send that info to the client. This accomplishes a simple sort of load balancing that doesn't give any kind of preference to any specific Validators (Is this true?).

pseudocode:

function getValidator(ts) {
  num = ts % numberOfValidators
  return _validators[num]
}

getValidator(Date.now() % 10000000) // => { host: 'https://testnet.example.com', wallet: '0xasdf...qwerty', stake: 1000000000000000 }

There's some trickiness in implementation here because we need to track how many Validators we have, and we need to enable removing Validators that no longer should/want to participate. This removal could be something Validators do themselves, or in the near term while the core team manages external validators we could explicitly do it. In medium/long term if some kind of staking is included, the Validator's stake going to zero might trigger a removal. All that is TBD, just mentioning as food for thought.

Pushing aside a few implementation details, now clients have a way to get the info they need to connect to a Validator without putting preference on any specific one. There's another potential use this might solve. evm-tableland issue 33 describes enabling approving accounts to act on behalf of others. This could be as simple as if an address is approved, then let it act. However we could build in more protections against malicious addresses if mechanism to check the approved address includes enforcement that the address is a Validator in good standing.

Notes:

• This contract could be combined with TablelandTables.sol, or be made into a library contract that TablelandTables.sol can use.
• The suggestion of significant 10^7 digits of ms since epoch is just any idea for how clients might accomplish an even balance between registered validators, really any way to get a number less than the max uint256 is fine.
• The term Validator here could be replaced by Gateway depending on how we decide those two concepts are related in Tableland. Whichever term we go with, that thing will need it's own DNS entry.

[asutula]

Nice writeup and interesting idea. I actually have no idea what the answer is, but just some reactions and thoughts.

• I think there are some higher level protocol design and economics that needs to be solved first before we get a clear answer here.
• Is it a goal of Tableland to provide payments to nodes providing write/read services as part of the protocol? Is so, that is pretty cool and new. This is in comparison to the way EVM "providers" like Infura and Alchemy work where they provide the standard API, but have their own billing infrastructure.
• The Infura/Alchemy approach seems to work, should we go in that direction where it's just up to users to find a Tableland provider and use it?
• Do nodes providing read/write services need filesystem access to the sqlite db? (Do they need to be materializing Tableland events?)
• Are there other roles on the network for nodes that only materialize data or only provide fisherman behavior etc?

I know many of these questions have been typed about elsewhere, but these are the types of things that started swirling around in my mind when reading.

[joewagner]

I think there are some higher level protocol design and economics that needs to be solved first before we get a clear answer here.

This is good point. Maybe there's some kind of work around to solve the "how to connect?" problem in the short term? I thought a little bit about a single gateway we control, but I ended up running into issues with a need for some kind of sticky session when a client wants to get a confirmation that something has been materialized.
Another simple thought is we have some kind of list of Validator info hosted somewhere, and clients just pick whatever one they want. The problem I potentially saw here is that everyone ends up picking the first one in the list. Maybe that's ok though?
Some other proposals for #240 might be good as a way to think about comparisons.

[joewagner]

• Is it a goal of Tableland to provide payments to nodes providing write/read services as part of the protocol? If so, that is pretty cool and new. This is in comparison to the way EVM "providers" like Infura and Alchemy work where they provide the standard API, but have their own billing infrastructure.
• The Infura/Alchemy approach seems to work, should we go in that direction where it's just up to users to find a Tableland provider and use it?

The Infura/Alchemy approach definitely works, I find it off-putting but that's not a good reason to ignore the fact it works. I would love us to find a way to incentivize read services as a part of the protocol without a fisherman concept, but that might not be realistic.

• Do nodes providing read/write services need filesystem access to the sqlite db? (Do they need to be materializing Tableland events?)
• Are there other roles on the network for nodes that only materialize data or only provide fisherman behavior etc?

This is a very handwavy idea but I could see some kind of indexer node that only provides read services for indexed queries. This could be paid by whoever wants to ensure the reads are readily available, e.g. Yuga Labs, or a DAO, etc...
I don't want to distract from the goal of this proposal, but as a tangent, we could have some kind of SC fund that trickles payments to read services. It would be similar, but different, to how an IPFS pinning service works. Instead of me paying pinata to pin on IPFS and then ceasing to pay if they don't pin, a read service would request funds and be paid if they do provide reads. There's still going to be a two part two party trusted transaction, but it would expose reputations.
Anyway that's all still just an idea in my head, and if we want to elaborate on it maybe worth a different tpd.

[jsign]

I'm curious why you came up with the idea of doing the "timestamp modulo" logic at the contract level instead of the SDK.

My approach to selection would be to let the contract return all the validators and let the SDK manage that (being the modulo idea, or other since a validator with less stake maybe is less trusted than others with higher stake?). The SDK can also cache the result for some period to avoid on-chain calls (even if they're read, they can have a cost).

[joewagner]

The modulo thought was an idea that to lookup and send every Validator would cost more than number % number then lookup and send one validator. I don't really know what is cheaper, I just read somewhere that looping though all of a mapping was expensive.

I don't have a lot of the implementation details worked out, but I was thinking the mapping would be readable via a few different public functions. If we want to explore the overall SC tracking validators idea more, we could change the structure of the mapping and/or the struct to better suit the reads we want to enable, as well as reduce costs.

[jsign]

Oh, maybe I explained incorrectly. My question is even simpler.

Why do we need to load balance validator selection in the contract compared to simply having a contract function that returns all validators and let the SDK or client choose in any way they want?

As in, what are we gaining with an on-chain selection?

[joewagner]

Ah, I see what you mean. Maybe we don't need to load balance, but I was thinking that if we don't load balance most clients would end up using the same Validator. That could be ok though, what do you think?
The larger thing we gain with an on-chain selection would be the longer term ability to have the selection based on a predefined set of public rules that potentially incorporate staking or slashing. Maybe this is getting ahead of ourselves with the goal of enabling connecting to a network of Validators?

[jsign]

My point is only about where load balancing happens: contract vs SDK. The modulo idea can be done both in the contract or just in the SDK. So my question was more about: why one vs the other, since the SDK sounds more flexible and cheaper.

Moving out of "where" it happens, the "load balancing" per see could be a good idea. But I can also imagine it can make UX more unpredictable since:

• Not all validators might offer a good service.
• If the user does actions A, B, and C; if they hit different validators that can have bad consequences. e.g: the three of them aren't synced at the same point, you could get incoherent results compared to only targeting one. (If that one is lagged, at least A, B and C results would be more coherent).

Don't take this as a strong opinion; just exploring a bit deeper.

[joewagner]

Very good points! I'm not totally convinced this is the right way to go either, but seems good to explore it as an option.

RE exploring modulo vs. all:
This stackexchange answer https://ethereum.stackexchange.com/a/23663 shows some of the complexity in sending back all of the entries of a mapping. Specifically look at the function named iterator, it loops through the mapping and collects the entries in an in memory list then returns it. If there's a way to just do a direct lookup one and send, it would seem less expensive to me. But this is a guess, and comes out of the fact the data is stored as a mapping, maybe we could find a different way to store it?

RE loadbalancing vs. not:

Not all validators might offer a good service.

This will probably be the case at some point. Exploring a few options:

1. We don't load balance, and the developer chooses a Validator, or list of Validators and bakes them into their App. This could introduce an issue where new Validators aren't discovered and old Validators that change from good to bad service are treated as good indefinitely. The App developer could try to build a means to manage the reputation of Validators and update their list, but this seems like we are passing the problem off to them.
2. The SC has a mechanism to rank Validator service, and Apps are told to use Validators based on that list. This seems like it could be very complicated, and I'm not entirely sure how to accomplish this. I will say, it makes me think about some kind of staking and fisherman concept.

If the user does actions A, B, and C; if they hit different validators that can have bad consequences. e.g: the three of them aren't synced at the same point, you could get incoherent results compared to only targeting one. (If that one is lagged, at least A, B and C results would be more coherent).

The solution that comes to mind for me is, that once a client gets connection info, they are responsible for connecting to the same Validator as long as needed. In the Browser context this could be a simple entry in localStorage. Not sure this is a good answer since we are again passing the problem off to the dev. The only other solutions I can come up with would require something like a session, which I don't like the idea of.