[pallet-revive] add tracing support (3/3)

[pgherveou]

To record nested call traces and call events similar to what Geth does, I am introducing a new rpc in Substrate state_debugBlock.

It's role is to fetch the runtime at "parent_hash" and call the desired runtime API method with the Block struct of "hash"

#[method(name = "state_debugBlock", blocking, with_extensions)]
fn debug_block(&self, name: String, block: Hash, bytes: Bytes) -> Result<Bytes, Error>;

We will use this endpoint in pallet_revive Eth RPC to turn on the call tracer and replay the block to capture the traces.

This is what the implement our runtime APIs look like:

polkadot-sdk/substrate/bin/node/runtime/src/lib.rs

Lines 3305 to 3328 in a4ce01c

	fn trace_block(
	block: Block,
	config: pallet_revive::evm::TracerConfig
	) -> Result<Vec<(u32, pallet_revive::evm::EthTraces)>, sp_runtime::DispatchError> {
	let mut tracer = pallet_revive::debug::Tracer::from_config(config);
	let mut traces = vec![];
	pallet_revive::using_tracer(&mut tracer, || {
	Executive::initialize_block(block.header());
	for (index, ext) in block.extrinsics().into_iter().enumerate() {
	let _ = Executive::apply_extrinsic(ext.clone());
	pallet_revive::with_tracer(|tracer| {
	if !tracer.has_traces() {
	let tx_traces = tracer.collect_traces().as_eth_traces::<Runtime>();
	traces.push((index as u32, tx_traces));
	}
	});
	}
	});
	Ok(traces)
	}

Example:

Capture all traces for a block

curl  'http://localhost:9944' \
--header 'Content-Type: application/json' \
--data '{ "jsonrpc":"2.0", "id":1, "method":"state_debugBlock", "params": ["ReviveApi_trace_block", "0x<hash>"]
}'

Capture trace for a single transaction in a block

curl  'http://localhost:9944' \
--header 'Content-Type: application/json' \
--data '{ "jsonrpc":"2.0", "id":1, "method":"state_debugBlock", "params": ["ReviveApi_trace_tx", "<block_hash>", "<tx_index>"]
}'

[kianenigma]

I don't get why you should pass the name: String here? If this RPC is only to call ReviveApi_trace_tx, it should hardcode it?

[kianenigma]

also asking better docs as to what each param is in general, RPC docs are important and get propagated into PJS/PAPI.

[kianenigma]

ah maybe this is not the one that is meant to be used by end users?

[pgherveou]

+1 for the docs, added more details.

I don't get why you should pass the name: String here?

A few reason, We have 2 runtime methods that make use of this. One that replay an entire block, the other that replay a single transaction. Replaying a single transaction involve that you replay all the transactions that precede
to ensure that it's replayed from the same state that when it was executed.

Also potentially other Runtime API could make use of this, so hard coding it does not make much sense here.

[kianenigma]

I cannot review the revive parts but I do find some confusion around the end-user RPC.

[pgherveou]

I cannot review the revive parts but I do find some confusion around the end-user RPC.

Let me break it down into 2 PR so that you don't need to deal with the revive changes here.

[pgherveou]

I cannot review the revive parts but I do find some confusion around the end-user RPC.

Remove all the parts that weren't relevant to the RPC API review. My main goal was to validate the approach of exposing new APIs that can replay a block or a transaction within a block, allowing us to toggle tracing features to extract and return traces in our case.

[niklasad1]

Do you know "ballpark size" of the response for ReviveApi_trace_block?

The reason why I'm asking is because --rpc-max-response-limit==10MB by default and maybe worth adding a comment about it whether one needs a bigger limit for this RPC

[pgherveou]

Good point I will mention it in the doc
People running this will probably do it on a dedicated "tracing node" where they can tune this up.

Similarly, can the wasm heap memory @kianenigma be tuned up for this kind of "offchain" call?
For validators, I understand this is defined on-chain, but I want people running such "tracing node" to be able to adjust it.

[niklasad1]

I wonder whether these could be pallet specific RPCs/runtime APIs for pallet-revive instead of "state_debug_block" because from my understanding it supposed to be used for "ReviveApi_trace_tx" and "ReviveApi_trace_block"?

[pgherveou]

I thought it was pretty generic and potentially useful for other pallets that could setup tracing features in a similar way.

It can't just be a runtime API as it need to load the runtime and the blocks before passing it over to the runtime api that will handle this data

[niklasad1]

In general I'm quite reluctant to add new stuff to the legacy RPCs but this is fine by me.

You didn't look into implementing this on-top of https://paritytech.github.io/json-rpc-interface-spec/api/archive_unstable_body.html?

[lexnv]

Could we build this entirely with the new RPC APIs instead? We plan to revisit the legacy ones soon and deprecate as much as possible.

One way that this could work without introducing the debug_block RPC:

• Step 1 (maybe optional): Let the user call into archive_unstable_header. This maybe needed because the implementation below calls a runtime API at the parent hash of the desired block. If the user knows the parent block (ie because they might have subscribed to chainHead_follow API), then this call is entirely optional.

• Step 2: Let the user call archive_unstable_call with the newly added ReviveApi_trace_tx runtime method

Overall the code looks good 🙏 I feel like debug_block is a tiny wrapper that could live in a separate repo, or the users can rely on archive_unstable_call entirely. Would love to get your thoughts on this

[pgherveou]

Hey @lexnv thanks for the review, sorry for the slow response, I was trying to test this end to end and it took more time than expected.

I wasn't aware of the new "unstable" APIs.
So all I need is a mean to call my runtime api from "parent_hash" and pass it the block so I can replay the block with the executive pallet inside an environmental context that will set the tracer first so that we can capture traces.

If we can port the "debug_block" fn to the new unstable APIs then it looks like this should be a simple port.

I feel like debug_block is a tiny wrapper that could live in a separate repo, or the users can rely on archive_unstable_call entirely. Would love to get your thoughts on this

Do you mean polkadot-sdk crate / rpc folder ? This will be a core feature of Plaza and needs to live in the mono-repo

[niklasad1]

No, the idea is that someone could implement this on the "client-side" like CLI tool which under the hood would do two RPC calls (not adding a new server API called debug_block):

1. archive_unstable_header
2. archive_unstable_call

For example you could add it eth revive proxy instead since it already is using subxt... but we need to add the "archive API" in subxt which is missing but that should be trivial

[paritytech-workflow-stopper]

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12830826913
Failed job name: cargo-clippy