Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(db): Sort data for IN before chunking #45058

Merged
merged 1 commit into from May 7, 2024

Conversation

ChristophWurst
Copy link
Member

Summary

Nextcloud sends chunks of DELETE FROM oc_filecache WHERE parentid IN (....long list...) and sometimes MariaDB does not seem to pick up the index. Apparently this may happen when the query analyzer thinks it's cheaper to scan the table than to traverse the index. If we put the data of the chunks closer together, we can hopefully convince the db that the index is the faster choice.

@juliushaertl @nickvergessen as it was suggested to us.

I don't know how to trigger this code with lots of data so this is more or less untested.

Checklist

Copy link
Collaborator

@Altahrim Altahrim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't knew the trick to sort items. The other one is to use smaller chunks.

Copy link
Contributor

@come-nc come-nc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Altahrim Is that related to the big folder deletion&restore we looked into together? Would it help with the deadlock there?

@nickvergessen
Copy link
Member

Would it help with the deadlock there?

Could, as that is also the direction where we are coming from

@ChristophWurst ChristophWurst added 4. to release Ready to be released and/or waiting for tests to finish and removed 3. to review Waiting for reviews labels May 6, 2024
@ChristophWurst
Copy link
Member Author

/backport to stable29

@ChristophWurst
Copy link
Member Author

/backport to stable28

@ChristophWurst
Copy link
Member Author

/backport to stable27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4. to release Ready to be released and/or waiting for tests to finish feature: files feature: filesystem performance 🚀
Projects
Status: ☑️ Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants