Add TableScan Replayer #11205
Add TableScan Replayer #11205
Conversation
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
duanmeng
force-pushed
the
split_tracer
branch
5 times, most recently
from
cb32bbb to
8bc94c4
Compare
duanmeng
force-pushed
the
split_tracer
branch
7 times, most recently
from
cdba012 to
1e45029
Compare
duanmeng
changed the title
duanmeng
force-pushed
the
split_tracer
branch
5 times, most recently
from
73387d0 to
dfdedd0
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
duanmeng
force-pushed
the
split_tracer
branch
4 times, most recently
from
cbe9dee to
507c1e6
Compare
| } | ||
|
|
||
| void OperatorTraceSplitWriter::write(const exec::Split& split) const { | ||
| VELOX_CHECK(!split.hasGroup(), "Do not support grouped execution"); |
There was a problem hiding this comment.
I guess we could support group later once we have driver id mapping in trace node? Leave a TODO here? thanks!
velox/exec/TableScan.cpp
Outdated
| @@ -117,6 +118,9 @@ RowVectorPtr TableScan::getOutput() { | |||
| if (!split.hasConnectorSplit()) { | |||
| noMoreSplits_ = true; | |||
| dynamicFilters_.clear(); | |||
| if (splitTracer_ != nullptr) { | |||
There was a problem hiding this comment.
Let's just call this in Operator::finishTrace()? Let the framework handle the tracing calls? Thanks!
duanmeng
force-pushed
the
split_tracer
branch
11 times, most recently
from
2cc5150 to
6dc8fe8
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
duanmeng
force-pushed
the
split_tracer
branch
from
6dc8fe8 to
ebb7e09
Compare
duanmeng
force-pushed
the
split_tracer
branch
from
ebb7e09 to
35a67df
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@xiaoxmeng merged this pull request in 12b52e7. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Add a split tracer to record and load the input splits from a tracing 'TableScan'
operator, and for getting the traced splits when replaying 'TableScan'.
Currently, it only works with 'HiveConnectorSplit'. In the future, it will
be extended to handle more types of splits, such as 'IcebergHiveConnectorSplit'.
part of #9668