Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HashJoinReplayer #11271

Closed
wants to merge 1 commit into from
Closed

Conversation

duanmeng
Copy link
Collaborator

@duanmeng duanmeng commented Oct 16, 2024

Add HashJoinReplayer to support replaying of hash join, and add
a valid list of operator types supporting tracing.

Part of #9668

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 16, 2024
@duanmeng duanmeng requested review from xiaoxmeng and removed request for majetideepak and assignUser October 16, 2024 11:46
Copy link

netlify bot commented Oct 16, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit d265d13
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/6729bfa66a74fd00089ebfba

velox/exec/QueryTraceTraits.h Outdated Show resolved Hide resolved
velox/tool/trace/HashJoinReplayer.cpp Outdated Show resolved Hide resolved
@duanmeng duanmeng force-pushed the join_replay branch 2 times, most recently from c0a056b to 98696c1 Compare October 17, 2024 06:29
velox/exec/Operator.cpp Outdated Show resolved Hide resolved
velox/exec/HashProbe.cpp Outdated Show resolved Hide resolved
velox/exec/HashBuild.cpp Outdated Show resolved Hide resolved
@duanmeng duanmeng force-pushed the join_replay branch 7 times, most recently from 69e87a7 to 7d5a051 Compare October 19, 2024 15:30
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duanmeng thanks for the change!

velox/exec/HashBuild.cpp Outdated Show resolved Hide resolved
velox/exec/Operator.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duanmeng thanks for the update % one comment on hash build

velox/exec/QueryTraceTraits.h Outdated Show resolved Hide resolved
velox/exec/QueryTraceTraits.h Outdated Show resolved Hide resolved
@duanmeng duanmeng force-pushed the join_replay branch 4 times, most recently from 4dcada9 to 3ac8fe5 Compare October 25, 2024 06:36
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duanmeng LGTM % minors on test. thanks!

velox/tool/trace/tests/HashJoinReplayerTest.cpp Outdated Show resolved Hide resolved
velox/tool/trace/tests/HashJoinReplayerTest.cpp Outdated Show resolved Hide resolved
velox/tool/trace/tests/HashJoinReplayerTest.cpp Outdated Show resolved Hide resolved
velox/tool/trace/tests/HashJoinReplayerTest.cpp Outdated Show resolved Hide resolved
velox/tool/trace/tests/HashJoinReplayerTest.cpp Outdated Show resolved Hide resolved
@duanmeng duanmeng force-pushed the join_replay branch 7 times, most recently from a66f733 to 101c528 Compare November 1, 2024 04:35
velox/exec/Operator.cpp Outdated Show resolved Hide resolved
velox/tool/trace/HashJoinReplayer.cpp Show resolved Hide resolved
@duanmeng duanmeng force-pushed the join_replay branch 3 times, most recently from a2bc745 to 6747743 Compare November 1, 2024 12:16
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duanmeng LGTM. Thanks!

using namespace facebook::velox::common::hll;

namespace facebook::velox::tool::trace::test {
class HashJoinReplayerTest : public HiveConnectorTestBase {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider to have TraceReplayTestBase : public HiveConnectorTestBase in followup to handle the common env setup. Thanks!

@facebook-github-bot
Copy link
Contributor

@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@duanmeng duanmeng force-pushed the join_replay branch 3 times, most recently from b7540f8 to aa07ab1 Compare November 2, 2024 12:03
@facebook-github-bot
Copy link
Contributor

@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@xiaoxmeng merged this pull request in 1e22de8.

Copy link

Conbench analyzed the 1 benchmark run on commit 1e22de82.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants