Log rotation is not functioning properly for multi-instance Splunk TAs.

[pdudhaiya-crest]

Description:
Log rotation is not functioning properly for multi-instance Splunk TAs.(This may be considered an ENHANCEMENT) and gets rotated multiple times and out-of-time by different inputs. I think this issue is happening since each input is a separate process in multi-instance mode while in single-instance the log instance is singleton between all threads (inputs). We've received some customer issues regarding the same.

Steps to reproduce:

1. Create multiple TA inputs that support multi-instance configurations (e.g., TA with multi-instance options like Snow, Remedy, Box). Let the log file threshold to reach (25 MB by default).
2. Check the log files:
   • If more than five inputs are enabled, five backup log files are created at the same time.
   • If fewer than five inputs are enabled, the number of backup log files created corresponds to the number of enabled inputs.

Expectation:
Only one backup log file (.log.1) should be created when the log file reaches the maxBytes limit.

Actual:
When the .log file first time reaches the maxBytes limit, each process/input tries to rotate the logs files individually (.log.1, .log.2, .log.3, .log.4, .log.5), instead of rotating just one log file (.log.1). If more than five inputs are enabled, five backup log files are created at the same time; if fewer than five inputs are enabled, the number of backup log files created corresponds to the number of enabled inputs.

[pdudhaiya-crest]

This issue can be resolved by keeping a separate log file for each of the inputs.

However, I would like the logger to be common between all inputs in the TA since dedicated log files for each of the inputs would mean the files would be lying around in Splunk even after the deletion of inputs and would be stale and useless.