merge cf_id and index_id to uint64 full_id #1196
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: RocksDB 7.3 added a strict SingleDelete contract enforcement that does not allow to mix Delete and SingleDelete for the same key. As of now, MyRocks Compaction Filter has not complied with the new contract, even though the contract was not necessary since our Compaction Filter usage was for purging unused data. This diff sets enforce_single_del_contracts to false until we fully follow the strict SingleDelete contract. update-submodule: rocksdb Reviewed By: luqun Differential Revision: D36566118 fbshipit-source-id: 39925ada0c0b1f6db528a22d5e036f8dde15058f
Summary: There is a regression in 8.0 where the query sent to the audit plugin contains the whole request packet, instead of just the query currently being executed. This means that for multiqueries, the audit plugin will log the same query multiple times. The fix is to restore what was done in 5.6, and pass actual query length down to the logging code. Fixes https://bugs.mysql.com/bug.php?id=107390 Reviewed By: hermanlee Differential Revision: D36648219 fbshipit-source-id: 061e4b8c800ee23f5d5d3250d65d75987a9e47d4
Summary: This diff will add a test bypass rpc plugin and a new mtr test for checking the result coming from bypass rpc plugin. This works the following way. (1) The test script writes queries to be replayed by the plugin in test_bypass_rpc_plugin_input.txt file, one query per line. (2) The test script then installs the test bypass rpc plugin. (3) The plugin, when initialized, opens the input file, reads each line of query, parses and converts it into myrocks_select_from_rpc format. Note that this conversion comes with many restrictions. For example, - in the where clause, "IN" operator has to come earlier than any other ">|>=|=|<|<=" operators - there shouldn't be unnecessary whitespaces - order by clause shouldn't omit its order keyword (ASC or DESC) (4) Then, the plugin calls "bypass_select()" function with the converted struct. (5) "send_row()", the callback function called by bypass engine, writes the value of each column into the output file. (6) Finally, the test script writes the output file to the result file with "cat_file", so that its result can be compared with *.result file. Reviewed By: yizhang82 Differential Revision: D36355974 fbshipit-source-id: 6b844c81d5ae627fe7a921636f91a0897ac8712c
Summary: This diff ports other existing mtr tests for bypass sql, bypass_select_range_pk, bypass_select_range_sk, bypass_select_scenarios, bypass_select_unsupported, to bypass rpc, excluding some test queries that are not relevant to bypass rpc. Also, the following changes were made in nosql_access.cc * Fixed a bug for a case where unrequested columns are included in the where clause. The correct index pointer to be used was m_field_list.size(), not m_field_index.size(). * Fixed a bug when calculating the length of blob field data in rpc_protocol. The correct one was field_blob->data_length(), not field_blob->get_length_bytes(). * Added support for MYSQL_TYPE_STRING type * Made bypass rpc return the exact same error message with bypass sql when returning "not supported" error. This will be necessary for running bypass_rpc_unsupported test. Finally, the following changes were made in the test plugin: * Added support for parsing where clause in which its value type is string * Added the implementation of sql_handle_error() to output the error message. Reviewed By: yizhang82 Differential Revision: D36695567 fbshipit-source-id: e0e114ba4c7f369cb969bb60fd60b5c2296a4912
Summary: Enable read default value from DD data for instant columns. During read path, instant column and non-instant column use same decoders to read the rocksdb data, The only difference for instant column is that if decoder reach end of rocksdb value slice and there are still some instant fields data aren't filled, it will fetch default data stored in m_encoder_arr[i] which contains default value for instant column. Details: - add three new fields into Rdb_field_encoder: - m_is_instant_field: whether the field/column is instant - m_instant_default_value: default value for instant field/column - m_instant_default_value_len: default value length for instant field/column - add DD:Table argument for Rdb_converter::setup_field_encoders to setup instant columns/fields - Update Rdb_value_field_iterator::next to support instant cols LIMITATION: - Don't support partition table - Don't support add columns which will cause null bit flag change Reviewed By: yizhang82 Differential Revision: D35919716 fbshipit-source-id: 7eba6793b1ee24a4ebf25de384ec2bef3309693b
…g dir Summary: When log and wal dir are pointing at the same dir the event logger framework can end up creating some dirs in the same path even before raft is initialized. When raft is inited it sees that the dir is not empty so it assumes that there is an existing FS and fails while trying to read it. Reviewed By: yichenshen Differential Revision: D36832721 fbshipit-source-id: 1ea536347398fb64de4d5991d000cc46d2d59eac
Summary: Added changes so that we can pass in the thread_id directly rather than having to make a system call. Reviewed By: george-reynya Differential Revision: D36727300 fbshipit-source-id: 2f418eeca8ea6b522b813274fbd9e0af98d2dbc8
…d removed Summary: We hit a low frequency race condition in testing, where running "stop shardbeater; start shardbeater" can cause the server to crash. This was happening because we set the state of the SHARDBEATER to STOPPED a bit too early. This unblocked the stop shardbeater SQL command and allowed it to pick up the start shardbeater SQL next, while the generate_shardbeats thread was still running and would next destroy sb_thd and remove it from global threads. This led to the start shardbeats SQL to start a THD which was immediately deleted causing the crash. Reviewed By: abhinav04sharma Differential Revision: D36879642 fbshipit-source-id: c9da99d4f5a2b5f767cdd1b3c27e84925669f02f
Summary: In MySQL bypass, if we decide to fallback *after* lock_table (typically when we see complex conditions such as multiple AND field > N AND field > M, etc), the table will be locked again which and it'll leak lock and actually triggers assert in debug build. MySQL relies on per-handler THR_LOCK_DATA (you can think it as the lock request) and often per table share THR_LOCK (the actual lock that tracks the read/read_wait/write/write_wait lists) to track more granular table level locks in addition to metadata lock. If the lock gets leaked, what will happen is THR_LOCK_DATA on the table/handler (such as ha_rocksdb::m_db_lock) will be left on in the THR_LOCK (such as Rdb_table_handler::m_thr_lock), and this can cause many different issues: * the THR_LOCK_DATA::next will point to itself if the same table is locked again on read lock * the THR_LOCK_DATA::next will be updated to the write lock if the table is locked for write, essentially linking the read list and write list together, leading to further corruptions * when the table and its THR_LOCK_DATA gets released, the THR_LOCK_DATA instance will obviously become dangling pointer In practice this often leads to deadlock in replication thread and instance gets flagged by automation due to lagging behind too much. Interestingly 5.6 doesn't have this issue because it has a check `if (is_query_tables_locked)` before the lock, so we are doing the same in 8.0 as well. We should also see if the THR_LOCK are actually needed in MyRocks as InnoDB doesn't use it anymore since 5.7. See https://dev.mysql.com/worklog/task/?id=6671 Reviewed By: luqun Differential Revision: D36765784 fbshipit-source-id: d6930d80f80425c6e8ff04e6bb934191d60c7595
Summary: Previously, I added a definition for a new service, rpc_plugin_service, to include/mysql/service_rpc_plugin.h file, but not in libservices. As a result, if a test plugin calling rpc_plugin_service is built with MYSQL_DYNAMIC_PLUGIN option, it fails to find it. This diff fixes this issue by adding rpc_plugin_service to libservices. Reviewed By: abal147 Differential Revision: D36888689 fbshipit-source-id: e73a86384815d8f3db409896ab25a95d8b07dc70
Summary: There is a new warning(38) generated when calling realpath with "./#innodb_temp" in container Temporarily ignore warning 38 before container fix Reviewed By: Pushapgl Differential Revision: D36943225 fbshipit-source-id: 7d40d5be4712f601da63a65110ff92f4f229e23b
Summary: Fixes the issue where adding 'COMPRESSED' keyword to 'INTO OUTFILE' crashes. The fix is to set the IO_CACHE structure properly for the compressed case. Reviewed By: hermanlee Differential Revision: D36937776 fbshipit-source-id: 059c78b7e65cdc43c980089ac8939aa8ff12625c
Summary: While working on adding a new mtr test previously, I mistakenly deleted one line in test_service_privacy. This diff recovers the line. Reviewed By: jasson15 Differential Revision: D37008888 fbshipit-source-id: 7e3a6546f919cd8ae4fe16aac6b32a558d925ae4
Summary: Currently SE_PRIVATE_DATA field only exists in mysql.tables /mysql.columns and it contains useful/critical information about instant DDL. Innodb expose these SE_PRIVATE_DATA information from mysql.tables /mysql.columns into I_S.innodb_tables/ I_S.innodb_columns. For myrocks, Currently it doesn't have its own I_S.rocksdb_tables and I_S.rocksdb_columns. thus, expose SE_PRIVATE_DATA into I_S.tables/I_S.columns instead Reviewed By: yizhang82 Differential Revision: D36579195 fbshipit-source-id: 270824a2d84a041e03a0f163b0abec4d2fe473ea
Summary: dynamic_load_error testcase simulate error during dynamic load wsenv library. Sometime dynamic_load_error MTR will fail due to different errno: 11(EAGAIN), ``` -ERROR HY000: Can't open shared library 'librocks_object_factory.so' (errno: 0 (null)) +ERROR HY000: Can't open shared library 'librocks_object_factory.so' (errno: 11 (null)) ``` update testcase to compare "Can't open shared library" only. Reviewed By: lth Differential Revision: D37026294 fbshipit-source-id: 5d466340a200c1a5238fb4ee02155b24f77590b6
Summary: As titled. Reviewed By: luqun Differential Revision: D37044391 fbshipit-source-id: 3996d39075bd7f9d3589b8b88a7373ce09cb706d
hermanlee
force-pushed
the
fb-mysql-8.0.23
branch
from
5e1eeb3
to
13a6f16
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
use union to merge cf_id and index_id to uint64 full_id, because uint64 compare is faster than two uint32 compare