Bookie OutOfDirectMemoryError when using DirectEntryLogger #4134
Labels
Comments
hangc0276
pushed a commit
that referenced
this issue
Jan 8, 2024
) ### Motivation When we use DirectIO and a catch up read occurs, a memory leak occurs because the ByteBuf is not released. Because although in the `internalReadEntry` method in `DirectEntryLogger`, even if the upper layer `fillReadAheadCache` uses the `ReferenceCountUtil.release(entry)` method in the finally block to release ByteBuf, it will still cause an exception thrown by `internalReadEntry`, and the `entryLogger.readEntry` method happens to be in try. ..catch unexpected, causing ByteBuf to not be released correctly ### Changes Master Issue: #4134
hangc0276
pushed a commit
to hangc0276/bookkeeper
that referenced
this issue
Jan 18, 2024
…ache#4135) ### Motivation When we use DirectIO and a catch up read occurs, a memory leak occurs because the ByteBuf is not released. Because although in the `internalReadEntry` method in `DirectEntryLogger`, even if the upper layer `fillReadAheadCache` uses the `ReferenceCountUtil.release(entry)` method in the finally block to release ByteBuf, it will still cause an exception thrown by `internalReadEntry`, and the `entryLogger.readEntry` method happens to be in try. ..catch unexpected, causing ByteBuf to not be released correctly ### Changes Master Issue: apache#4134 (cherry picked from commit cc1c2e0)
|
closed by #4135 |
Ghatage
pushed a commit
to sijie/bookkeeper
that referenced
this issue
Jul 12, 2024
…ache#4135) ### Motivation When we use DirectIO and a catch up read occurs, a memory leak occurs because the ByteBuf is not released. Because although in the `internalReadEntry` method in `DirectEntryLogger`, even if the upper layer `fillReadAheadCache` uses the `ReferenceCountUtil.release(entry)` method in the finally block to release ByteBuf, it will still cause an exception thrown by `internalReadEntry`, and the `entryLogger.readEntry` method happens to be in try. ..catch unexpected, causing ByteBuf to not be released correctly ### Changes Master Issue: apache#4134
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
BUG REPORT
Describe the bug
2023-11-25T14:29:27,343+0800 [BookieReadThreadPool-OrderedExecutor-5-0] ERROR org.apache.bookkeeper.server.service.BookieService - Unable to allocate memory, exiting bookie io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 8392704 byte(s) of direct memory (used: 15024140317, max: 15032385536)To Reproduce
This issue occurs when using high throughput for writing and reading (e.g., 1GB/s), and when a catch up read occurs.
Bookie Version: 4.16.2
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: