Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug(duplication):some nodes never start GC plog after computer room failure #2015

Open
ninsmiracle opened this issue May 22, 2024 · 2 comments
Labels
type/bug This issue reports a bug.

Comments

@ninsmiracle
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    The computer room which service for our duplication master cluster meet an accidents. And most of the node in this room shutdown in a short time.
    When all the nodes alive , we found some partition of the duplication table never GC private log (plog) again.

  2. What did you expect to see?
    All the partition can GC it's plog correctly.

  3. What did you see instead?
    stdout (error log):

// stdout
90146:E2024-05-14 15:59:52.512 (1715673592512665104 67086) replica.default8.040005fe0319646c: nfs_server_impl.cpp:221:on_get_file_size(): {nfs_service} get stat of file /home/work/ssd2/pegasus/alsgsrv-monetization-master/replica/reps/8.53.pegasus/plog/log.18129.608864535790 failed, err = No such file or directory

We can see this replica request a old plog.
image

Because the partition can not clear plog as nomarl,so the disk always fully. We need to clear the plog sometimes.

  1. What version of Pegasus are you using?
    Pegasus v2.4
@acelyc111
Copy link
Member

acelyc111 commented Jul 24, 2024

Is the file /home/work/ssd2/pegasus/alsgsrv-monetization-master/replica/reps/8.53.pegasus/plog/log.18129.608864535790 actually exists or not?

@ninsmiracle
Copy link
Contributor Author

Is the file /home/work/ssd2/pegasus/alsgsrv-monetization-master/replica/reps/8.53.pegasus/plog/log.18129.608864535790 actually exists or not?

When coredump happened, the file actually not exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug This issue reports a bug.
Projects
None yet
Development

No branches or pull requests

2 participants