Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(logging): Introduce spdlog and remove simple_logger and screen_logger #2083

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

acelyc111
Copy link
Member

@acelyc111 acelyc111 commented Jul 31, 2024

Behavior changes:

  • The meaning of [tools.simple_logger]fast_flush has been changed to Whether to flush logs every second rather than flush on every log.
  • All configs in section [tools.screen_logger] have been removed.
  • The log severity level leading character in logs F (Fatal) is changed to C (Critical)
  • Symlink to the current log file is removed.
  • Log file name is changed from yyyyMMdd_hhmmss_SSS to log.[d+]*.txt
  • Log content is changed:
    • The nanoseconds timestamp is removed, it's not necessary and may cause performance issue.
    • The new print thread id can be found by ps -T command, it's helpful for debugging.
    • For example, from I2024-08-01 19:45:16.8 (1722512716008970768 1c65af) replica.replica18.0200656f0072d04f: replica_stub.cpp:1303:on_group_check(): <message> to I2024-08-01 12:01:24.710 53193 replica.replica1.216400909301791341 replica_stub.cpp:1115:on_group_check(): <message>
  • Rotate files algorithm changed, for example:
    log.txt -> log.1.txt
    log.1.txt -> log.2.txt
    log.2.txt -> log.3.txt
    log.3.txt -> delete
    

Performance:
The LoggingTest.SimpleBenchmark shows that it can reduce about 50% time cost when
set logging_test_simple_benchmark_loops to 1,000,000.

empiredan pushed a commit that referenced this pull request Aug 1, 2024
This is a previous work of #2083.

This patch introduces [spdlog](https://github.com/gabime/spdlog) as a thirdparty
to prepare the docker image, then the #2083 can use it to run CI.

spdlog is a very fast C++ logging library, you can see the benchmarks:
https://github.com/gabime/spdlog?tab=readme-ov-file#benchmarks. spdlog using
`libfmt` to format message content, it is fast and compatible with the current
logging practice in Pegasus. It's a good choice to replace the current
`simple_logger` or `screen_logger` implementation in Pegasus.
@acelyc111 acelyc111 marked this pull request as ready for review August 2, 2024 10:08
@acelyc111 acelyc111 force-pushed the add_spdlog branch 2 times, most recently from c1d12ff to 025a2d1 Compare August 20, 2024 09:01
@acelyc111 acelyc111 force-pushed the add_spdlog branch 3 times, most recently from 6911ca1 to 3b732c1 Compare September 19, 2024 03:59
@acelyc111 acelyc111 force-pushed the add_spdlog branch 4 times, most recently from cc824c6 to 09c908a Compare September 23, 2024 15:31
@acelyc111 acelyc111 requested a review from empiredan September 24, 2024 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant