Hang (Spinlock) When There is Too Many Servers

[gyroninja]

When I do /discord connect, weechat indefinitely hangs (I gave it overnight). It does show at least one server, (that server had so many channels that it went offscreen). I also got some messages about muted channels IIRC.
While I don't know the true cause, I am guessing it is because I have so many servers / channels. I have ~40 servers and each server has ~10 channels each. I'm not sure why it does this, but perhaps its running into a rate limit?
If you are not able to reproduce this, feel free to send me a separate branch / diff for me to test.

[khyperia]

Probably duplicate of #19 and #28. There's a (bad) workaround in 28. Thanks for the report!

[gyroninja]

@khyperia Using gdb it looks like it is getting stuck near the same place. The workaround in #28 did not work for me. I gave it a good 20 minutes and still nothing. I commented out lines 42-50. I also tried commenting out all the self.buffer.set calls.
If I remove all of those the backtrace is.

#0  0x00007f0dc477009a in PyEval_EvalFrameEx (f=f@entry=0x7f0dbdb7c7f0, 
    throwflag=throwflag@entry=0) at Python/ceval.c:1213
#1  0x00007f0dc4773c7c in PyEval_EvalCodeEx (co=<optimized out>, 
    globals=<optimized out>, locals=locals@entry=0x0, 
    args=args@entry=0x7f0dbda87628, argcount=1, kws=kws@entry=0x0, kwcount=0, 
    defs=0x0, defcount=0, closure=0x7f0dbda87390) at Python/ceval.c:3584
#2  0x00007f0dc47e53ac in function_call (func=0x7f0dbdaab410, 
    arg=0x7f0dbda87610, kw=0x0) at Objects/funcobject.c:523
#3  0x00007f0dc46f26d3 in PyObject_Call (func=func@entry=0x7f0dbdaab410, 
    arg=arg@entry=0x7f0dbda87610, kw=kw@entry=0x0) at Objects/abstract.c:2547
#4  0x00007f0dc46f32c7 in PyObject_CallFunctionObjArgs (
    callable=0x7f0dbdaab410) at Objects/abstract.c:2774
#5  0x00007f0dc470e8c0 in listsort (self=0x7f0dbd88bb00, 
    args=<optimized out>, kwds=<optimized out>) at Objects/listobject.c:2100
#6  0x00007f0dc4770a3f in do_call (nk=<optimized out>, na=<optimized out>, 
    pp_stack=0x7fff749fb470, func=0x7f0dbda8b710) at Python/ceval.c:4566
#7  call_function (oparg=<optimized out>, pp_stack=0x7fff749fb470)
    at Python/ceval.c:4374
#8  PyEval_EvalFrameEx (f=f@entry=0x7f0dbda8d240, throwflag=throwflag@entry=0)
    at Python/ceval.c:2989
#9  0x00007f0dc4773c7c in PyEval_EvalCodeEx (co=<optimized out>, 
    globals=<optimized out>, locals=locals@entry=0x0, 
    args=args@entry=0x7f0dbdbd71a8, argcount=3, kws=kws@entry=0x0, kwcount=0, 
    defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3584
#10 0x00007f0dc47e53ac in function_call (func=0x7f0dbda8a230, 
    arg=0x7f0dbdbd7190, kw=0x0) at Objects/funcobject.c:523
#11 0x00007f0dc46f26d3 in PyObject_Call (func=func@entry=0x7f0dbda8a230, 
    arg=arg@entry=0x7f0dbdbd7190, kw=kw@entry=0x0) at Objects/abstract.c:2547
#12 0x00007f0dc46f282b in call_function_tail (args=0x7f0dbdbd7190, 
    callable=0x7f0dbda8a230) at Objects/abstract.c:2579
#13 PyObject_CallFunction (callable=0x7f0dbda8a230, format=<optimized out>)
    at Objects/abstract.c:2603
#14 0x00007f0dc46c0ed1 in weechat_python_exec ()
   from /usr/local/lib/weechat/plugins/python.so
#15 0x00007f0dc46d4dc6 in weechat_python_api_hook_signal_cb ()
   from /usr/local/lib/weechat/plugins/python.so
#16 0x0000000000453cb5 in hook_signal_send ()
#17 0x000000000047e4df in gui_buffer_set_short_name ()
#18 0x000000000047fa8f in gui_buffer_set ()
#19 0x00007f0dc55fd629 in weecord::event_proc::open_and_sync_buffers::h7f5aafea45c5998a () from /root/.weechat/plugins/libweecord.so
#20 0x00007f0dc55f7a8b in weecord::connection::MyConnection::new::_26612u7b26612u7d$::hea7fef5078749918 ()
   from /root/.weechat/plugins/libweecord.so
#21 0x00007f0dc595e5ad in panic_unwind::__rust_maybe_catch_panic ()
    at /checkout/src/libpanic_unwind/lib.rs:98
#22 0x00007f0dc55f238f in weecord::ffi::PokeableFd::new::callback_fn::hd80be88840defc1d () from /root/.weechat/plugins/libweecord.so
#23 0x0000000000451c62 in hook_fd_exec ()
#24 0x00000000004af606 in gui_main_loop ()
#25 0x0000000000421b95 in main ()

[khyperia]

Thanks for the stack trace - that specific stack trace is in this line. However, I see that you have a Python hook on short_name running - which may explain a large chunk of the performance problem. Perhaps try running a completely-clean weechat to see if the problem goes away (make test creates a fresh temp weechat config dir).

However, the bug is still the same - open_and_sync_buffers is still an incredibly terrible function that does way more work than it should (so it's calling the Python hook way more often than necessary). I have unpushed changes on my laptop that will hopefully resolve this, I'll hopefully get around to validation and pushing that soon.

[gyroninja]

Odd, I could have sworn I commented out that line. I even restarted weechat before testing it again like the README says. Hopefully your changes resolve this issue.

[khyperia]

Oh, yeah, no - I never said commenting out that line was a thing you should do, it (and surrounding lines) are rather important, unlike the nicklist population (what the for loop is). Sorry about the confusion.