Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: issue #2251 Memory Overflow During Data Insertion with DorisWriter, Task Stuck in Blocked State #2252

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Yida-Zhangfb
Copy link

resolve sync block on memory overflow in Doris

  • 当检测到内存溢出时,改进了进程的终止行为,避免了同步操作的长时间阻塞。
  • 修复了在内存溢出情况下,Doris 数据同步进程出现阻塞的问题。
  • startAsyncFlushing()为异步线程消费flushQueue的数据,如出现内存溢出catch为Exception时则不能捕获到,flushException为null,checkFlushException()则不能检测到异常;异步线程内存溢出主动退出,由于flushQueue数据不能被消费,则导致写入的主线程仍阻塞在flushQueue.put()
    Closes Memory Overflow During Data Insertion with DorisWriter, Task Stuck in Blocked State #2251

- 当检测到内存溢出时,改进了进程的终止行为,避免了同步操作的长时间阻塞。
- 修复了在内存溢出情况下,Doris 数据同步进程出现阻塞的问题。
- startAsyncFlushing()为异步线程消费flushQueue的数据,如出现内存溢出catch为Exception时则不能捕获到,flushException为null,checkFlushException()则不能检测到异常;异步线程内存溢出主动退出,由于flushQueue数据不能被消费,则导致写入的主线程仍阻塞在flushQueue.put()
Closes alibaba#2251
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory Overflow During Data Insertion with DorisWriter, Task Stuck in Blocked State
1 participant