Memory usage on ra_xxx_log_ets tables

[albertored]

We have an application that sometimes applies commands with a big payload in the raft machine.
In such situations we get a lot of memory "wasted" in the raft machine process itself but we are able to solve this by returning the :garbage_collect effect. We still have some ETS tables (owned by ra_xxx_log_ets) that consume a lot of memory and it is not clear how/when they are freed or if there are parameters to configure this behaviour.
Do you have some advice on how to reduce this memory consumption?
Thanks

[Kation]

Same problem.
We have a message delive in dead loop because a bug that throw exception with manual ack.
Then ra_log_ets size increase very large.
Even we handled this message for a long time, ra_log_ets never relase memory.

[Kation]

We are warpping the message handler that exception throw by business code will use BasicNack.
So that message will deliver again and again.

When we found there a business code bug case by invalid message body, we stop application and purge messages in this quorum queue.
And ra_log_ets does not release.

[michaelklishin]

@Kation requeueing a message over and over is the same as not acknowledging it at all, and exactly why https://www.rabbitmq.com/docs/quorum-queues#poison-message-handling exists. We need an executable way to reproduce. We do not guess in this community.

[michaelklishin]

In fact, it is extremely likely to be a classic consumer "crash-requeue-crash-requeue" scenario and nothing else. Compaction will be triggered as it should be when the crash loop is eliminated. But let's see what the executable way to reproduce will look like.

[Kation]

The point is that message maybe importance and should consume after consumer application update.
I will try to write a demo later.

[michaelklishin]

I don't see why an important message must be requeued may times over and over. That is the problematic scenario. Quorum queues design assumes that messages are acknowledged reasonably quickly, not after a 50 redelivery loop one day later.