PMM Framework #37
PMM Framework #37
Conversation
2) Add new pmm-framework python script wrapper
saikumar-vs
requested review from
yurkovychv,
puneet0191,
olexandr-havryliak and
sandraromanchenko
as code owners
pmm_qa/pmm-framework.py
Outdated
| # Check if the container is in the list of running containers | ||
| # and establish N/W connection with it. | ||
| if container_name in running_containers: | ||
| subprocess.run(['docker', 'network', 'create', 'pmm-qa']) |
There was a problem hiding this comment.
I see we are still calling bash scripts from python, so a good question to ask is:
If python is only used to call bash scripts, is it even needed here? What is the value added?
There was a problem hiding this comment.
thanks Alex, it is a valid point but we are first aiming to port the main script to Python, it is a huge work to migrate everything to Python, so we have broken it down a bit, the base pmm-framework first and slowly setup's over time.
There was a problem hiding this comment.
Alex thanks for your questions, below are some good reasons for considering Python,
Note: Most of our setups are Ansible based, and one main thing to keep in mind during our evaluation.
-
Python comes with a comprehensive standard library that includes modules for working with files, subprocesses, JSON, YAML, and more
-
Python's simple syntax and readability make it easy to learn and use, even for beginners. This makes Python a popular choice for automation tasks, including writing wrapper scripts for tools like Ansible.
-
Python has a vast ecosystem of third-party libraries and frameworks that can be leveraged to enhance your Ansible wrapper script. For example, you can use libraries like click or argparse for command-line argument parsing, logging for advanced logging capabilities, multiprocessing to achieve parallelism.
There was a problem hiding this comment.
Hey, I'm getting the same output from the search engines :)
But seriously, why would I give parallelism as an argument, if this is the exact opposite of stable? I don't see us leverage parallelism anytime soon unless we want to make the code unmaintainable.
What did we miss from the bash scripts in the first place? And why do we need wrappers for Ansible? Can't we just run ansible like we always did, i.e. ansible-playbook book.yml? What do those wrappers bring?
If the ansible tasks I see in the code are simple shell calls with ignore_errors all over the place, then it's even a bigger issue, because 1) we don't follow ansible's best practices, and 2) we don't even need ansible to make such calls. If this is the only thing that ansible does, than it's definitely the wrong tool. It would take one line to do the same with a bash script.
Don't get me wrong, I'm also a big fan of python, but I truly believe that we used a cannon where a pistol would be good enough. IMO, it already smells a higher maintenance tax.
Usage a) sudo -E ./pmm-framework.py --database "mysql=5.7,QUERY_SOURCE=perfschema,CLIENT_VERSION=3-dev-latest" --database "pdpgsql=16,USE_SOCKET=1,CLIENT_VERSION=3.0.0"
b) sudo -E ./pmm-framework.py --v --database pgsql --database pdmysql