catalog-web crashes with CPU usage spike

[FuhuXia]

catalog-web recently saw a CPU usage spike and crashes, as shown in the following stats, 3 out of 5 instances crashed, making catalog site responding to user requests really slow.

     state     since                  cpu      memory           disk       details
#0   running   2023-12-07T18:00:22Z   286.8%   572.2M of 850M   1G of 2G
#1   crashed   2023-12-07T18:00:08Z   242.0%   16.7M of 850M    1G of 2G
#2   running   2023-12-07T17:59:23Z   360.5%   759.4M of 850M   1G of 2G
#3   crashed   2023-12-07T18:00:00Z   380.5%   761.2M of 850M   1G of 2G
#4   crashed   2023-12-07T18:00:21Z   373.2%   429.3M of 850M   1G of 2G

When it crashes, there is error message, as noted in the O&M ticket.

Worker (pid:424) was sent SIGKILL! Perhaps out of memory?

How to reproduce

Noticed last week.

Sketch

[Notes or a checklist reflecting our understanding of the selected approach]