Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Eagerly initialize RapidsShuffleManager for SPARK-45762 [databricks] #11904

Merged
merged 4 commits into from
Dec 24, 2024
Merged

Eagerly initialize RapidsShuffleManager for SPARK-45762 [databricks] #11904

merged 4 commits into from
Dec 24, 2024
+126 −17

Conversation

gerashegalov
Copy link
Collaborator

@gerashegalov gerashegalov commented Dec 24, 2024 •
edited
Loading

Fixes #11107

  • Initialize RapidsShuffleManager on construction for Spark 4.0 and Databricks 14.3
  • Disable lazy initialization and conf validation in the case above

@gerashegalov
Copy link
Collaborator Author

build

@gerashegalov gerashegalov merged commit 2b7a0e2 into NVIDIA:branch-25.02 Dec 24, 2024
50 of 51 checks passed
@gerashegalov gerashegalov deleted the eager-rsm branch December 24, 2024 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jlowe jlowe jlowe approved these changes

@abellina abellina Awaiting requested review from abellina

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Rework RapidsShuffleManager initialization for Apache Spark 4.0.0
2 participants
@gerashegalov @jlowe