Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Corruption of table due to silent error in large table ALTER engine query from myisam #1249

Open
Qubitium opened this issue Oct 22, 2022 · 0 comments

Comments

@Qubitium
Copy link

This bug maybe isolated to Percona mysql 5.7.latest stable. Did not test other mysql distro with rocksdb engine.

mysql error log:


2022-10-22T02:32:32.656099Z 5 [ERROR] Chunk size is too small to process merge.
2022-10-22T02:32:32.656137Z 5 [ERROR] Error during preparation of heap.

Large table migration from MYISAM to ROCKSDB via ALTER TABLE engine=rocksdb can lead to successful query but silently corrupted (empty) table that will never auto-recover.

After testing with various param/env changes, increasing rocksdb_merge_buf_size confirmed resolved this issue and bypasses this bug.

rocksdb_merge_buf_size = 2GB #64MB default
rocksdb_merge_combine_read_size = 8GB #1GB default
In summary, there are two critical bugs in this report that may cause silent table corruption via (ALTER engine=rocksdb).
  1. BUG one: Rdb_index_merge::merge_heap_prepare() error is not correctly propagated to the ALTER sql query engine.

int Rdb_index_merge::merge_heap_prepare() {

  1. BUG two: default rocksdb_merge_buf_size and rocksdb_merge_combine_read_size can cause fatal rocksdb migration/merging error.

  2. Minor issue: L307 should also print the actual size that exceeded buffer to aid in debugging.

Cross posting bug from Percona jira which contains more details:

https://jira.percona.com/browse/PS-8440

@Qubitium Qubitium changed the title Silent corruption of table due to silent error in large table ALTER engine query from myisam Corruption of table due to silent error in large table ALTER engine query from myisam Oct 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant