Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: track filters in state for eth_getFilterChanges #448

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

eshaan7
Copy link
Contributor

@eshaan7 eshaan7 commented Nov 25, 2024

Closes #447.


  1. Transaction hashes:
image
  1. Block hashes:
image
  1. Logs:
image

@eshaan7 eshaan7 marked this pull request as ready for review November 25, 2024 13:43
Copy link
Collaborator

@ncitron ncitron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start, but we want to be verifying all data returned by the filter is valid. The current standard we hold ourselves to is that an RPC should be able to withhold parts of the data, but should not be able to insert additional incorrect data (we'd like to check for exhaustiveness at some point too but there are some unfortunate blockers there for a few of the rpc methods like log fetching).

It looks like we aren't checking validity for the non-log filter return types. We should be able to do this by checking if any returned block, blockhash, transaction, or transaction hash has actually been seen by Helios. All of the data to perform these checks should be accessible from the State type.

@eshaan7
Copy link
Contributor Author

eshaan7 commented Nov 28, 2024

It looks like we aren't checking validity for the non-log filter return types. We should be able to do this by checking if any returned block, blockhash, transaction, or transaction hash has actually been seen by Helios

So the eth_newBlockFilter filter returns block hashes, while the eth_newPendingTransactionFilter filter returns transaction hashes. In both cases, the response is an array of hashes without any indication of whether they are block hashes or transaction hashes. This makes verification quite tricky because Helios cannot verify the new transaction hashes — these are just transactions in the mempool and have not yet been included in any block. For block hashes, I can check if they exist in the state, but if they don’t, I can’t simply error out because they might actually be transaction hashes, and there’s no way to distinguish between the two.

For now, have added comments in the code with these notes.

@ncitron
Copy link
Collaborator

ncitron commented Nov 29, 2024

Oh this is a good point.

How that I think about it we should probably be managing more of the filter logic ourselves rather than farming it out to the RPC.

We should keep track of what filters we have, and only use the RPC to manage log and pending transaction filters. For new block filters we don't really need to use the rpc at all. We can just keep track of those filters in Helios and implement the logic directly for returning all the heads that helios has seen.

@eshaan7
Copy link
Contributor Author

eshaan7 commented Dec 3, 2024

We can just keep track of those filters in Helios and implement the logic directly for returning all the heads that helios has seen.

That makes sense. We can maintain a list of filter IDs for eth_newBlockFilter filter in Helios' state and don't need RPC for it then. However, one small edge case that still remains is that if Helios sees an unknown filter ID, it will fallback to RPC for 3 all filter types. This is, again, because the filter ID or its response does not provide information on the filter type.

@ncitron
Copy link
Collaborator

ncitron commented Dec 3, 2024

If Helios sees an unknown filter ID I think we would want to return an error? All filters that a client asks for should have been previously created through Helios before. The same would be true when using an RPC.

If we maintain a list of all filters and their types internally in Helios we should be able to understand whether an inbound request for a filter had been correctly registered prior, and whether it needs to be forwarded off to the rpc or handled entirely internally in the newHeads case.

@eshaan7 eshaan7 changed the title fix: get_filter_changes return type feat: track filters in state for eth_getFilterChanges Dec 5, 2024
@eshaan7
Copy link
Contributor Author

eshaan7 commented Dec 5, 2024

@ncitron; Implemented tracking filter and changes (for eth_newBlockFilter) in state. PTAL. LMK if you think I missed anything.

@eshaan7 eshaan7 requested a review from ncitron December 5, 2024 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[bug] eth_getFilterChanges only supports log return type
2 participants