-
Notifications
You must be signed in to change notification settings - Fork 717
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Commit sst in dedicated threads #1275
base: fb-mysql-8.0.28
Are you sure you want to change the base?
Conversation
@@ -392,13 +393,23 @@ int Rdb_sst_info::open_new_sst_file() { | |||
} | |||
|
|||
void Rdb_sst_info::commit_sst_file(Rdb_sst_file_ordered *sst_file) { | |||
m_commiting_threads_mutex.lock(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
consider std::lock_guardstd::mutex? we use RDB_MUTEX_LOCK_CHECK
macros elsewhere in this file
@@ -479,6 +490,11 @@ int Rdb_sst_info::finish(Rdb_sst_commit_info *commit_info, | |||
close_curr_sst_file(); | |||
} | |||
|
|||
for (auto& thr : m_commiting_threads) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to grab a log here too?
could you please provide some rough calculations about the time savings? |
In our private branch, when |
Rdb_index_merge and Rdb_sst_file_ordered both use cf's comparator, when Rdb_index_merge write to Rdb_sst_file_ordered, the keys should be in cf's increasing order, m_use_stack should be false in this case. |
We removed |
Execute
Rdb_sst_info::commit_sst_file
in dedicated threads, this improves performance:Rdb_sst_file_ordered::commit
may use stack to reverse input data, this is time consumingm_sst_file_writer->Finish
may be consuming, at least it need to callfsync
Execute
Rdb_sst_info::commit_sst_file
in dedicated threads increase parallelization with mimimal code changes.