Create a intermediate file format from a set of fio results

[harriscr]

This PR is the foundation for the work to get CBT to automatically produce a hockey-stick output curve of IOps or Bandwidth vs Latency.
It consists of a python class that will convert the json output from an fio benchmark run in CBT to a common intermediate format, the unit tests for that class, and a wrapper around the class allowing it to be run independently.

The vision is that eventually there will be a class or classes to convert the results from any CBT benchmark run into this intermediate format. This will then allow a single class/utility to be written to produce hockey-stick curves from any CBT benchmark run results.

The next steps are to write the class to plot the data, and then add the code to CBT so it gets run as part of a fio benchmark, possibly using an option in the CBT yaml file

Testing:

Both manual and automated testing were done

Automated:

Teuthology performance tests that run CBT. This is just a regression as no code or behaviour has changed:

perf-basic
https://pulpito.ceph.com/harriscr-2024-11-19_09:33:42-perf-basic-wip-harriscr-cbt-refactor-2024-10-17-0839-distro-default-smithi/

rados/perf
https://pulpito.ceph.com/harriscr-2024-11-19_09:23:32-rados:perf-wip-harriscr-cbt-refactor-2024-10-17-0839-distro-default-smithi/

• failed with missing test module:

No module named 'tasks' in run_tasks.py

crimson-rados/perf
https://pulpito.ceph.com/harriscr-2024-11-19_09:34:16-crimson-rados:perf-wip-harriscr-cbt-refactor-2024-10-17-0839-distro-default-smithi/

• failed 2 tests with job timeout:

hit max job timeout

Python unit tests:

======================== 326 passed, 3 skipped in 0.52s ========================
Finished running tests!

Python tools

black, mypy and ruff were all run to check for pep8 compliance:

harriscr@eilistraee:~/cbt $ source ~/python_environments/ceph_python/bin/activate
(ceph_python) harriscr@eilistraee:~/cbt $ ruff check common_output_formatter.py
All checks passed!
(ceph_python) harriscr@eilistraee:~/cbt $ mypy common_output_formatter.py
Success: no issues found in 1 source file
(ceph_python) harriscr@eilistraee:~/cbt $ black common_output_formatter.py
All done! ✨ 🍰 ✨
1 file left unchanged.

Manual

Ran the wrapper against a folder of fio test results:

common_formatter_wrapper_testing.txt

An example file:
4096B_randread.json

The associated hockey-stick curve: