Skip to content

Commit

Permalink
feat: Add 2 options to limit thread count (#2011)
Browse files Browse the repository at this point in the history
There are 2 options stats_dump_period_sec and stats_persist_period_sec in
RocksDB to adjust the period to dump/persist stat, each RocksDB instance
creates 2 dependent threads for this aim.

A replica server is possible to serve thousands of replicas, it means
thousands of RocksDB instances are possible to be running in one process,
then thousands of threads will be created will may consume much resouce.

This patch introduces 2 options, then it's possible to disable the threads
by setting them to 0.
  • Loading branch information
acelyc111 authored May 22, 2024
1 parent d7087b6 commit a5498a9
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions src/server/pegasus_server_impl_init.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -568,6 +568,14 @@ DSN_DEFINE_string(pegasus.server,
DSN_DEFINE_validator(rocksdb_filter_type, [](const char *value) -> bool {
return dsn::utils::equals(value, "common") || dsn::utils::equals(value, "prefix");
});
DSN_DEFINE_uint64(pegasus.server,
stats_dump_period_sec,
600, // 600 is the default value in RocksDB.
"If not zero, dump rocksdb.stats to RocksDB LOG every stats_dump_period_sec");
DSN_DEFINE_uint64(pegasus.server,
stats_persist_period_sec,
600, // 600 is the default value in RocksDB.
"If not zero, dump rocksdb.stats to RocksDB every stats_persist_period_sec");

namespace dsn {
namespace replication {
Expand Down Expand Up @@ -663,6 +671,8 @@ pegasus_server_impl::pegasus_server_impl(dsn::replication::replica *r)
_db_opts.listeners.emplace_back(new pegasus_event_listener(this));
_db_opts.max_background_flushes = FLAGS_rocksdb_max_background_flushes;
_db_opts.max_background_compactions = FLAGS_rocksdb_max_background_compactions;
_db_opts.stats_dump_period_sec = FLAGS_stats_dump_period_sec;
_db_opts.stats_persist_period_sec = FLAGS_stats_persist_period_sec;
// init rocksdb::ColumnFamilyOptions for data column family
_data_cf_opts.write_buffer_size = FLAGS_rocksdb_write_buffer_size;
_data_cf_opts.max_write_buffer_number = FLAGS_rocksdb_max_write_buffer_number;
Expand Down

0 comments on commit a5498a9

Please sign in to comment.